Decoding method, encoding method, decoder, and encoder

Abstract

Embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder. Embodiments of the present application relate to the technical field of video coding. In the decoding method provided by the embodiments of the present application, a first transformation is performed on the first transformation coefficient of the current block based on a transformation set corresponding to a first intra prediction mode. The first intra prediction mode is designed to include any one of an intra prediction mode derived from the DIMD mode used for the prediction block of the current block, an intra prediction mode derived from the DIMD mode used for the output vector of the optimal MIP mode for predicting the current block, an intra prediction mode derived from the DIMD mode used for the reconstructed samples in the first template area adjacent to the current block, and an intra prediction mode derived from the TIMD mode. Therefore, the decompression performance of the current block can be improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of video coding, and more specifically, to a decoding method, an encoding method, a decoder, and an encoder.

Background Art

[0002] Digital video compression technology is mainly a technology for compressing huge digital image video data for transmission, storage, etc. With the rapid increase in Internet videos and the growing demand of people for video clarity, existing digital video compression standards can implement video decompression technology, but in order to improve compression efficiency, better digital video decompression technology is required.

Summary of the Invention

[0003] Embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder, whereby the compression efficiency can be improved.

[0004] In a first aspect, the present application provides a decoding method. The method includes the following. Analyze the bitstream of the current sequence to obtain the FirstThe transformation coefficients are obtained. The first intra-prediction mode is determined. The first intra-prediction mode includes one of the following: an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for the prediction block of the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of the optimal matrix-based intra-prediction (MIP) mode for predicting the current block; an intra-prediction mode derived from a DIMD mode used for the reconstruction sample in the first template region adjacent to the current block; and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. Based on the transformation set corresponding to the first intra-prediction mode, the first transformation is performed on the first transformation coefficients to obtain the second transformation coefficients of the current block. The second transformation is performed on the second transformation coefficients to obtain the residual block of the current block. Based on the prediction block and the residual block of the current block, the reconstruction block of the current block is determined.

[0005] In a second aspect, the present application provides an encoding method, which includes: obtaining the residual block of the current block in the current sequence; performing a third transformation on the residual block of the current block to obtain a third transformation coefficient for the current block; determining a first intra-prediction mode, which includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for the prediction block of the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of an optimal, matrix-based intra-prediction (MIP) mode for predicting the current block; an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region adjacent to the current block; and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. Based on the transformation set corresponding to the first intra-prediction mode, a fourth transformation is performed on the third transformation coefficient to obtain a fourth transformation coefficient for the current block; and the fourth transformation coefficient is encoded.

[0006] In a third embodiment, the present application provides a decoder comprising an analysis unit, a conversion unit, and a reconstruction unit. The analysis unit analyzes the bitstream of the current sequence to convert the current block FirstThe system is configured to obtain transformation coefficients. The transformation unit is configured to determine a first intra-prediction mode, perform a first transformation on the first transformation coefficients based on the transformation set corresponding to the first intra-prediction mode to obtain a second transformation coefficient for the current block, and perform a second transformation on the second transformation coefficients to obtain a residual block for the current block. The first intra-prediction mode includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for the predicted block of the current block, an intra-prediction mode derived from a DIMD mode used for the output vector of an optimal, matrix-based intra-prediction (MIP) mode for predicting the current block, and one of an intra-prediction mode derived from a DIMD mode or an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode used for reconstruction samples in a first template region adjacent to the current block. The reconstruction unit is configured to determine the reconstruction block of the current block based on the predicted block and the residual block of the current block.

[0007] In a fourth aspect, the application provides an encoder comprising a residual unit, a transformation unit, and an encoding unit. The residual unit is configured to obtain the residual block of the current block in the current sequence. The transformation unit is configured to perform a third transformation on the residual block of the current block to obtain a third transformation coefficient of the current block, to determine a first intra-prediction mode, and to perform a fourth transformation on the third transformation coefficient based on the transformation set corresponding to the first intra-prediction mode to obtain a fourth transformation coefficient of the current block. The first intra-prediction mode includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for the prediction block of the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of an optimal, matrix-based intra-prediction (MIP) mode for predicting the current block; and one of an intra-prediction mode derived from a DIMD mode and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode used for reconstructed samples in a first template region adjacent to the current block. The encoding unit is configured to encode the fourth conversion coefficient.

[0008] In a fifth embodiment, the application provides a decoder comprising a processor and a computer-readable storage medium. The processor is configured to execute computer instructions. Computer instructions are stored in the computer-readable storage medium, and the computer instructions are loaded by the processor and the decoding method of the first embodiment or each embodiment thereof is executed.

[0009] In one embodiment, the processor is one or more, and the memory is one or more.

[0010] In one embodiment, the computer-readable storage medium may be integrated with the processor, or it may be installed separately from the processor.

[0011] In a sixth embodiment, the application provides an encoder comprising a processor and a computer-readable storage medium. The processor is configured to execute computer instructions. Computer instructions are stored in the computer-readable storage medium, and the computer instructions are loaded by the processor and the encoding method of the second embodiment or each embodiment thereof is executed.

[0012] In one embodiment, the processor is one or more, and the memory is one or more.

[0013] In one embodiment, the computer-readable storage medium may be integrated with the processor, or it may be installed separately from the processor.

[0014] In a seventh embodiment, the present application provides a computer-readable storage medium. The computer-readable storage medium is configured to store computer instructions, which, when read and executed by the processor of a computer device, cause the computer device to execute the decoding method according to the first embodiment or the encoding method according to the second embodiment.

[0015] In the eighth embodiment, the present application provides a bitstream, which is either the bitstream according to the first embodiment or the bitstream according to the second embodiment.

[0016] Based on the above proposed technology, by introducing a first intra-prediction mode and performing a first transformation on the first transformation coefficient of the current block based on a transformation set corresponding to the first intra-prediction mode, the decompression performance of the current block can be improved. In particular, when the decoder predicts the current block using a non-traditional intra-prediction mode, it is possible to avoid directly performing the first transformation using a transformation set corresponding to the planar mode, and the transformation set corresponding to the first intra-prediction mode can reflect the texture orientation of the current block to some extent, thereby improving the decompression performance of the current block. [Brief explanation of the drawing]

[0017] [Figure 1] Figure 1 is a block diagram showing an encoding framework according to an embodiment of this application. [Figure 2] Figure 2 is a schematic diagram showing the MIP mode according to an embodiment of this application. [Figure 3] Figure 3 is a schematic diagram showing how to derive a prediction mode based on DIMD according to the embodiment of this application. [Figure 4] Figure 4 is a schematic diagram showing the derivation of a prediction block based on DIMD according to the embodiment of this application. [Figure 5] Figure 5 is a schematic diagram showing a template used in TIMD according to the embodiment of this application. [Figure 6] Figure 6 is an example of an LFNST according to an embodiment of this application. [Figure 7] Figure 7 shows an example of an LFNST conversion set according to the embodiment of this application. [Figure 8] Figure 8 is a block diagram showing a decoding framework according to an embodiment of this application. [Figure 9] Figure 9 is a flowchart showing the decoding method according to an embodiment of this application. [Figure 10] Figure 10 is a flowchart showing the encoding method according to an embodiment of this application. [Figure 11] FIG. 11 is a block diagram showing a decoder according to an embodiment of the present application. [Figure 12] FIG. 12 is a block diagram showing an encoder according to an embodiment of the present application. [Figure 13] FIG. 13 is a block diagram showing an electronic device according to an embodiment of the present application.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, the technical solutions in the embodiments of the present application will be described with reference to the drawings.

[0019] The solution according to the embodiment of the present application can be applied to the technical field of digital video coding. For example, the technical field includes, but is not limited to, the fields of image coding, video coding, hardware video coding, dedicated circuit video coding, and real-time video coding. Also, the solution according to the embodiment of the present application can be combined with an audio video coding standard (AVS), the second-generation AVS standard (AVS2), or the third-generation AVS standard (AVS3). Examples include, but are not limited to, the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Further, with the solution according to the embodiment of the present application, lossy compression or lossless compression can be performed on an image. The lossless compression may be visually lossless compression or mathematically lossless compression.

[0020] A mixed coding framework based on blocks is used in video coding standards. Specifically, each image (frame) in video is divided into the largest coding unit (LCU) or coding tree unit (CTU) of the same size square (e.g., 128x128, 64x64, etc.). Each largest coding unit or coding tree unit can also be divided into rectangular coding units (CU) based on rules. Coding units can be further divided into prediction units (PU), transform units (TU), etc. The mixed coding framework includes modules such as prediction, transform, quantization, entropy coding, and loop filtering. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. Because there is a strong correlation between adjacent samples in video images, video coding techniques utilize intra-prediction methods to eliminate spatial redundancy between adjacent samples. In intra-prediction, sample information within the current segment block is predicted by referring only to image information from the same frame. Because there is a strong similarity between adjacent images in video, video coding techniques can improve coding efficiency by using inter-prediction methods to eliminate temporal redundancy between adjacent images. In inter-prediction, image information from different frames is referred to, and motion estimation is used to find the motion vector information that best matches the current segment block. Through transformation, the predicted image block is transformed into the frequency domain, and energy redistribution is performed.By combining transformation and quantization, information that is not sensitive to the human eye can be removed, and this is used to eliminate visual redundancy. Entropy coding can remove character redundancy based on the current context model and the probabilistic information of the binary bitstream.

[0021] In the digital video encoding process, the encoder first reads a grayscale or color image from the original video sequence, and then encodes the grayscale or color image. Here, the grayscale image may contain samples of the lumen component, and the color image may contain samples of the chroma component. Selectively, the color image may further contain samples of the lumen component. The color format of the original video sequence may be a luminance-chroma (YCbCr, YUV) format or a red-green-blue (RGB) format, etc. Specifically, after reading the grayscale or color image, the encoder divides it into blocks, generates a predicted block of the current block by performing intra-prediction or inter-prediction on the current block, obtains a residual block by subtracting the predicted block from the original block of the current block, transforms and quantizes the residual block to obtain a quantization coefficient matrix, and outputs the quantization coefficient matrix as an entropy code to a bitstream. In the digital video decoding process, the decoding side generates a predicted block of the current block by performing intra-prediction or inter-prediction on the current block. Furthermore, the decoding side decodes the bitstream to obtain the quantization coefficient matrix, inversely quantizes and inversely transforms the quantization coefficient matrix to obtain residual blocks, and adds the predicted blocks and residual blocks to obtain reconstructed blocks. The reconstructed blocks form the reconstructed image. The decoding side loop-filters the reconstructed image based on the image or blocks to obtain the decoded image.

[0022] The current block can be the current coding unit (CU) or the current prediction unit (PU), etc.

[0023] Furthermore, the encoding side also requires processing similar to that on the decoding side in order to obtain the decoded image. The decoded image can serve as a reference image for interpretation of subsequent images. Mode information or parameter information such as block partitioning information, prediction, transformation, quantization, entropy coding, and loop filtering, determined on the encoding side, is output to the bitstream as needed. The decoding side analyzes and processes the existing information to determine the same block partitioning information, prediction, transformation, quantization, entropy coding, and loop filtering mode information or parameter information as the encoding side. This ensures that the decoded image obtained on the encoding side is the same as the decoded image obtained on the decoding side. The decoded image obtained on the encoding side is usually also called the reconstructed image. During prediction, the current block may be divided into prediction units, and during transformation, the current block may be divided into transformation units. The division of the prediction units and transformation units may be the same or different. The above is the basic flow of video coding in a block-based mixed coding framework. As technology advances, some modules or some steps in the flow of this framework may be optimized. This application applies to the basic flow of a video codec in a mixed coding framework based on the said block.

[0024] To facilitate understanding, we will first briefly explain the encoding framework related to this application.

[0025] Figure 1 is a block diagram showing an encoding framework 100 according to an embodiment of this application.

[0026] As shown in Figure 1, the encoding framework 100 may include an intra-prediction unit 180, an inter-prediction unit 170, a residual unit 110, a transform and quantization unit 120, an entropy encoding unit 130, an inverse transform and inverse quantization unit 140, and a loop filtering unit 150. Optionally, the encoding framework 100 may further include a decoded image buffer unit 160. The encoding framework 100 is also called a mixed framework encoding mode.

[0027] The intra-prediction unit 180 or inter-prediction unit 170 can predict image blocks awaiting coding and output predicted blocks. The residual unit 110 can calculate residual blocks, i.e., the difference between predicted blocks and image blocks awaiting coding, based on the predicted blocks and image blocks awaiting coding. The transformation and quantization unit 120 is used to perform operations such as transformation and quantization on the residual blocks, thereby removing information that is not sensitive to the human eye and eliminating visual redundancy. Selectively, residual blocks before transformation and quantization by the transformation and quantization unit 120 may be called temporal residual blocks, and temporal residual blocks after transformation and quantization by the transformation and quantization unit 120 may be called frequency residual blocks or frequency-domain residual blocks. The entropy encoding unit 130 receives the quantized transform coefficient output by the transformation and quantization unit 120 and can output a bitstream based on the said transform quantization coefficient. For example, the entropy encoding unit 130 can remove character redundancy based on a target context model and probabilistic information of a binary bitstream. For example, the entropy encoding unit 130 can be used in context-based adaptive binary arithmetic coding (CABAC). The entropy encoding unit 130 may also be called a header information encoding unit. Optionally, in this application, the image block awaiting coding may also be called an original image block or a target image block. A prediction block may also be called a prediction image block or an image prediction block, and may also be called a prediction signal or prediction information.A reconstructed block may also be called a reconstructed image block or image reconstruction block, and may also be called a reconstructed signal or reconstructed information. Furthermore, on the encoding side, the image block awaiting coding may also be called an encoding block or encoded image block. On the decoding side, the image block awaiting coding may also be called a decoding block or decoded image block. The image block awaiting coding may be a CTU or a CU.

[0028] In the encoding framework 100, the difference between the predicted block and the image block awaiting coding is calculated to obtain a residual block. Processes such as transformation and quantization are performed on the residual block, and the residual block is transmitted to the decoding side. Accordingly, the decoding side receives the bitstream, decodes it, obtains a residual block through steps such as inverse transformation and inverse quantization, and obtains a reconstructed block by adding the residual block to the predicted block obtained by the decoding side.

[0029] Furthermore, the inverse transform and inverse quantization unit 140, the loop filtering unit 150, and the decoding image buffer unit 160 within the encoding framework 100 can be used to form a decoder. The intra-prediction unit 180 or inter-prediction unit 170 can predict image blocks awaiting coding based on existing reconstruction blocks, thereby ensuring that the encoding side can use the reference frame in the same manner as the decoding side. In other words, the encoder can replicate the decoder's processing loop, thereby generating the same predictions as the decoding side. Specifically, the quantized transformation coefficients are inversely transformed and inversely quantized by the inverse transform and inverse quantization unit 140 to replicate the approximate residual block on the decoding side. After the prediction block is added to this approximate residual block, the loop filtering unit 150 can be used to smooth out the effects of block-based processing and blocking artifacts due to quantization. The image blocks output from the loop filtering unit 150 can be stored in the decoding image buffer unit 160 for use in subsequent image prediction.

[0030] Figure 1 is merely an example of this application and should not be interpreted as a limitation of this application.

[0031] For example, the loop filtering unit 150 within the encoding framework 100 may include a deblocking filter (DBF) and a sample adaptive offset (SAO). The role of the DBF is to remove deblocking artifacts, and the role of the SAO is to remove ringing effects. In other embodiments of this application, a neural-network-based loop filtering algorithm can be used in the encoding framework 100 to improve the video compression efficiency. Alternatively, the encoding framework 100 may be a deep learning neural network-based video coding hybrid framework. In one embodiment, the results after sample filtering can be calculated using a convolutional neural network-based model based on the DBF and SAO. The network structure in the luminance component and the network structure in the saturation component of the loop filtering unit 150 may be the same or different. Given that the luminance component contains more visual information, the luminance component can be used to guide the filtering of the saturation component in order to improve the reconstruction quality of the saturation component.

[0032] The following explains the related aspects of intra-prediction and inter-prediction.

[0033] Interpretation can eliminate temporal redundancy by referencing image information from different frames and using motion estimation to find the motion vector information that best matches the image block awaiting coding. The images used in interpretation may be P-frames and / or B-frames. A P-frame represents a forward predicted picture, and a B-frame represents a bidirectional predicted picture.

[0034] Intra prediction predicts sample information within an image block awaiting coding by referencing only the information of the same image to eliminate spatial redundancy. The image used for intra prediction may be an I-frame. For example, following the coding order from left to right and top to bottom, the top-left image block, the image block above, and the left image block can be used as reference information to predict the image block awaiting coding. The image block awaiting coding is then used as reference information for the next image block. In this way, the entire image can be predicted. If the input digital video is in a color format such as YUV4:2:0, each of the four pixels in each image frame of the digital video consists of four Y components and two UV components. The encoding framework can encode the Y component (i.e., luminance block) and the UV component (i.e., chromaticity block), respectively. Similarly, the decoding side can decode according to the format.

[0035] For the intra-prediction process, in order to obtain a predicted block, the intra-prediction can predict image blocks awaiting coding using angle prediction mode and non-angle prediction mode. Based on rate distortion information calculated from the predicted block and the image block awaiting coding, the optimal prediction mode for the image block awaiting coding is selected and transmitted to the decoding side via bitstream. The decoding side analyzes the prediction mode, predicts and obtains the predicted block of the target decoding block, and obtains the reconstructed block by adding the time-domain residual block obtained via bitstream transmission.

[0036] Through the development of successive digital video coding standards, non-angle prediction modes have remained relatively stable, consisting of average mode and planar mode. The number of angle prediction modes has increased with the evolution of digital video coding standards. Taking the international digital video coding standard H-series as an example, the H.264 / AVC standard has 8 angle prediction modes and 1 non-angle prediction mode, while H.265 / HEVC expands to 33 angle prediction modes and 2 non-angle prediction modes. In H.266 / VVC, the intra-prediction mode is further expanded to include 67 conventional prediction modes, non-conventional prediction modes, and a matrix-weighted intra-frame prediction (MIP) mode for luminance blocks. The 67 conventional prediction modes include planar mode, DC mode, and 65 angle modes. Planar mode is typically used to process blocks where the texture has a gradient, DC mode, as the name suggests, is used to process flat areas, and angle prediction mode is typically used to process blocks where the angle texture is relatively sharp.

[0037] In this application, the current block used for intra prediction may be a square block or a rectangular block.

[0038] Furthermore, since all intra-prediction blocks are square, the probability of each angle prediction mode being used is equal. If the length and width of the current block differ, for horizontal blocks (width greater than height), the probability of using the upper reference sample is greater than the probability of using the left reference sample, and for vertical blocks (height greater than width), the probability of using the upper reference sample is less than the probability of using the left reference sample. When predicting rectangular blocks, the conventional angle prediction mode is converted to a wide-angle prediction mode. If the rectangular block can be predicted using the wide-angle prediction mode, the predicted angle range for the current block is greater than the predicted angle range when predicting a rectangular block using the conventional angle prediction mode. Selectively, when using the wide-angle prediction mode, the signal can still be transmitted using the index of the conventional angle prediction mode. Accordingly, the decoding side can convert the conventional angle prediction mode back to the wide-angle prediction mode after receiving the signal, so that neither the total number of intra-prediction modes nor the encoding method of the intra-modes changes.

[0039] Furthermore, based on the size of the current block, you can determine or select the intra-prediction mode to execute. For example, based on the size of the current block, you can determine or select the wide-angle prediction mode to intra-predict the current block. For example, if the current block is a rectangular block (with different width and height dimensions), you can intra-predict the current block using the wide-angle prediction mode. The aspect ratio of the current block can be used to determine the angle prediction mode to which the wide-angle prediction mode is replaced, and the angle prediction mode after the replacement. For example, when predicting the current block, you can select any intra-prediction mode with an angle that does not exceed the diagonal of the current block (from the bottom left corner to the top right corner of the current block) to be the angle prediction mode after the replacement.

[0040] The following describes other intra-prediction modes related to this application.

[0041] (1) Matrix-based Intra Prediction (MIP) mode

[0042] The MIP mode can also be called the Matrix-weighted intra-prediction mode. The process involved in the MIP mode can be divided into three main steps: the downsampling process, the matrix multiplication process, and the upsampling process. Specifically, first, in the downsampling process, spatially adjacent reconstructed samples are downsampled. Next, the obtained downsampled sample sequence is used as the input vector for the matrix multiplication process, that is, the output vector of the downsampling process is used as the input vector for the matrix multiplication process, the input vector of the matrix multiplication process is multiplied by a predetermined matrix, a bias vector is added to the result, and the calculated sample vector is output. The output vector of the matrix multiplication process is used as the input vector for the upsampling process, and the final prediction block is obtained by upsampling.

[0043] Figure 2 is a schematic diagram showing the MIP mode according to an embodiment of this application.

[0044] JPEG0007879954000001.jpg80168

[0045] In other words, to predict a block with width W and height H, MIP requires H reconstructed samples from the left column of the current block and W reconstructed samples from the row above the current block as input. MIP generates the predicted block based on three steps: averaging of reference samples, matrix-vector multiplication, and interpolation. The core of MIP is matrix-vector multiplication, which can be considered the process of generating a predicted block using input samples (reference samples) in a matrix-vector multiplication manner. Various matrices are provided for MIP, and differences in prediction methods can be reflected in differences in matrices, resulting in different results when different matrices are used for the same input samples. Furthermore, the processes of averaging and interpolation of reference samples are designed to consider the trade-off between performance and complexity. For blocks with large sizes, averaging of reference samples can achieve an effect that approximates downsampling, allowing the input to be adapted to a relatively small matrix. Interpolation provides an upsampling effect. Thus, it is no longer necessary to provide a MIP matrix for each block size; instead, only one or a few matrices of specific sizes need to be provided. As the need for compression performance increases and hardware performance improves, more complex MIPs may appear in next-generation standards.

[0046] In MIP mode, the MIP mode can be simplified from a neural network; for example, the matrix used in the MIP mode can be obtained based on training. Therefore, the MIP mode has relatively strong generalization ability and predictive effects that cannot be achieved by conventional predictive modes. The MIP mode is a model obtained by performing multiple simplifications of hardware and software complexity on an intra-predictive model based on a neural network. Based on a large number of training samples, multiple predictive modes can represent multiple models and parameters, better covering the texture of natural sequences.

[0047] MIP mode is somewhat similar to planar mode, but clearly, MIP mode is more complex and flexible than planar mode.

[0048] The number of MIP modes can differ for coding units of different block sizes. For example, for a 4x4 coding unit, MIP mode has 16 prediction modes. For an 8x8 coding unit, where the width is equal to 4 or the height is equal to 4, MIP mode has 8 prediction modes. For coding units of other sizes, MIP mode has 6 prediction modes. MIP mode also has a transpose function. For prediction modes that match the current size, MIP mode can attempt transpose calculations on the encoding side. Therefore, MIP mode requires a usage flag to indicate whether or not MIP mode is used for the current coding unit. If MIP mode is used for the current coding unit, a transpose flag and an index flag must also be transmitted to the decoder. The transpose flag is binarized using the Fixed Length (FL) coding scheme and has a length of 1. The index flag is binarized using the Truncated Binary (TB) coding scheme. For example, for a 4x4 coding unit, the MIP mode has 16 prediction modes. The index flag can be a 5-bit or 6-bit truncated binary flag.

[0049] (2) Decoder side Intra Mode Derivation (DIMD) mode

[0050] The core of DIMD mode is that the decoder derives the intra-predictive mode using the same method as the encoder. This avoids transmitting the intra-predictive mode index of the current coding unit in the bitstream, thereby saving bit overhead.

[0051] The specific process for DIMD mode can be divided into the following two main steps:

[0052] Step 1: Derive the prediction mode.

[0053] Figure 3 is a schematic diagram showing how to derive a prediction mode based on DIMD according to the embodiment of this application.

[0054] As shown in Figure 3(a), DIMD derives a prediction mode using samples in a template within the reconstruction region (reconstruction samples to the left and above the current block). For example, the template can include three adjacent rows of reconstruction samples above the current block, three adjacent columns of reconstruction samples to the left, and the corresponding adjacent reconstruction sample to the upper left. Based on this, the window (for example, in Figure 3( b ) or Figure 3 ( c According to the window shown in ( ), multiple gradient values ​​corresponding to multiple adjacent reconstruction samples are determined within the template. Each gradient value is used to obtain one intra prediction mode (IPM) that fits the direction of its gradient. Based on this, the encoder can derive the prediction modes corresponding to the largest gradient value and the second largest gradient value among the multiple gradient values. For example, as shown in Figure 3(b), for a 4x4 block, all adjacent reconstruction samples for which gradient values ​​need to be determined are analyzed to obtain the corresponding gradient histograms. For example, as shown in Figure 3(c), for blocks of other sizes, all adjacent reconstruction samples for which gradient values ​​need to be determined are analyzed to obtain the corresponding gradient histograms. Finally, the prediction modes corresponding to the largest gradient and the second largest gradient in the gradient histograms are derived as the prediction modes.

[0055] Of course, the gradient histogram in this application is merely an example for determining the derived prediction mode, and when implemented in practice, it can be implemented in various simple forms, and this application is not particularly limited thereto. Furthermore, this application is not limited to the method of obtaining the gradient histogram, and for example, it can be obtained using the Sobel operator or other methods. Moreover, in other alternative embodiments, the gradient value in this application may be replaced with a gradient amplitude value, and this application is not particularly limited thereto.

[0056] Step 2: Derive the predicted block.

[0057] Figure 4 is a schematic diagram showing the derivation of a prediction block based on DIMD according to the embodiment of this application.

[0058] As shown in Figure 4, the encoder can weight the predicted values ​​corresponding to three intra-prediction modes (planar mode and two intra-prediction modes derived based on DIMD). The codec obtains the predicted block for the current block using the same prediction block derivation scheme. Assuming that the prediction mode corresponding to the largest gradient value is prediction mode 1 and the prediction mode corresponding to the second largest gradient value is prediction mode 2, the encoder determines the following two conditions: 1. The gradient value for prediction mode 2 is not 0. 2. Neither Prediction Mode 1 nor Prediction Mode 2 are planar mode or direct current (DC) prediction modes.

[0059] If the above two conditions are not met simultaneously, the predicted sample value for the current block is calculated using only prediction mode 1, i.e., the normal prediction process is applied to prediction mode 1. If not, i.e., if the above two conditions are met simultaneously, the predicted block for the current block is derived using the weighted averaging method. The specific method is as follows: Planar mode accounts for 1 / 3 of the weight, and the remaining 2 / 3 is the combined weight of prediction mode 1 and prediction mode 2. For example, the gradient amplitude value of prediction mode 1 is divided by the sum of the gradient amplitude values ​​of prediction mode 1 and prediction mode 2, and the result is used as the weight for prediction mode 1. The gradient amplitude value of prediction mode 2 is divided by the sum of the gradient amplitude values ​​of prediction mode 1 and prediction mode 2, and the result is used as the weight for prediction mode 2. A weighted averaging is performed on the predicted blocks obtained based on the above three prediction modes, i.e., predicted block 1, predicted block 2, and predicted block 3 obtained based on planar mode, prediction mode 1, and prediction mode 2, respectively, to obtain the predicted block for the current coding unit. The decoder also obtains its predicted block in the same step.

[0060] In other words, the specific weights for step 2 above are calculated as follows: In JPEG0007879954000002.jpg30150, mode1 and mode2 represent prediction mode 1 and prediction mode 2, respectively, and amp1 and amp2 represent the gradient amplitude values ​​for prediction mode 1 and prediction mode 2, respectively. In DIMD mode, a flag must be transmitted to the decoder. This flag is used to indicate whether DIMD mode is being used for the current coder unit.

[0061] Of course, the weighted average method described above is merely one example of this application and should not be understood as a limitation of this application.

[0062] In summary, DIMD uses gradient analysis of the reconstructed sample to select an intra-prediction mode and weights the two intra-prediction modes and the planar mode according to the analysis results. The advantage of DIMD is that, once a DIMD mode is selected for the current block, there is no need to specifically indicate which intra-prediction mode is being used in the bitstream, as this is derived by the decoder itself through the process described above, thus saving some overhead.

[0063] (3) Template-based Intra Mode Derivation (TIMD) mode

[0064] The technical principle of TIMD mode is similar to that of DIMD mode described above, and in both cases, the codec derives the prediction mode using the same operation, saving the overhead of transmitting the mode index. TIMD prediction mode can be understood in two main parts. First, cost information for each prediction mode is calculated based on a template, and the prediction mode corresponding to the smallest cost and the prediction mode corresponding to the second smallest cost are selected. The prediction mode corresponding to the smallest cost is denoted as prediction mode 1, and the prediction mode corresponding to the second smallest cost is denoted as prediction mode 2. The ratio of the second smallest cost (costMode2) to the smallest cost (costMode1) is: If the data satisfies pre-defined conditions, such as JPEG0007879954000003.jpg15150, the prediction block corresponding to prediction mode 1 and the prediction block corresponding to prediction mode 2 are weighted and fused according to the weights corresponding to prediction mode 1 and prediction mode 2 to obtain the final prediction block.

[0065] For example, the weights corresponding to prediction mode 1 and prediction mode 2 are determined based on the following method. JPEG0007879954000004.jpg24150weight1 is the weight of the prediction block corresponding to prediction mode 1, and weight2 is the weight of the prediction block corresponding to prediction mode 2. If the ratio of the second smallest cost costMode2 to the smallest cost costMode1 does not satisfy a predetermined condition, weight fusion between prediction blocks will not be performed, and the prediction block corresponding to prediction mode 1 will become a TIMD prediction block.

[0066] Furthermore, when performing intraprediction on the current block using TIMD mode, if the reconstruction sample template for the current block does not contain any available adjacent reconstruction samples, TIMD mode will select planar mode and perform intraprediction on the current block, i.e., will not perform weighted fusion. Similar to DIMD mode, TIMD mode requires the transmission of a flag to the decoder. This flag is used to indicate whether TIMD mode is being used for the current coder unit.

[0067] The encoder or decoder primarily calculates cost information for each prediction mode as follows: Intra-mode prediction is performed on samples within the template region based on reconstructed samples adjacent to the upper and left sides of the template region, and the prediction process is the same as for the original intra-prediction mode. For example, when performing intra-mode prediction on samples within the template region using DC mode, the average value of the entire coding unit is calculated. As another example, when performing intra-mode prediction on samples within the template region using angle prediction mode, a corresponding interpolation filter is selected according to the mode, and predicted samples are obtained by interpolation according to the rules. In this case, based on the predicted samples and reconstructed samples within the template region, the distortion between the predicted samples and reconstructed samples within that region, i.e., the cost information of the current prediction mode, can be calculated.

[0068] Figure 5 is a schematic diagram showing a template used in TIMD according to the embodiment of this application.

[0069] As shown in Figure 5, if the current block is a coding unit with width equal to M and height equal to N, the codec can calculate the template for the current block by selecting a Reference of template for the current block from a coding unit with width equal to 2(M+L1)+1 and height equal to 2(N+L2)+1. If the template for the current block does not contain any available adjacent reconstruction samples, TIMD mode selects planar mode and performs intra-prediction for the current block. For example, the template for the current block may be the samples adjacent to the left and above the current CU in Figure 5, i.e., there are no available reconstruction samples in the shaded area. In other words, if there are no available adjacent reconstruction samples in the shaded area, TIMD mode selects planar mode and performs intra-prediction for the current block.

[0070] Except in the case of boundaries, when encoding the current block, theoretically, reconstructed values ​​can be obtained from the left and above the current block, i.e., the template of the current block contains available adjacent reconstructed samples. In a specific embodiment, the decoder predicts the template using a certain intra-prediction mode and compares the predicted value with the reconstructed value to obtain the cost of that intra-prediction mode on the template. Examples include the Sum of Absolute Differences (SAD), Sum of Absolute Transformed Difference (SATD), and Sum of Squared for Error (SSE). Since the template and the current block are adjacent to each other, the reconstructed samples in the template and the samples in the current block are correlated. Therefore, the behavior of the prediction mode on the template can be used to estimate the behavior of this prediction mode on the current block. TIMD predicts the template using several candidate intra-prediction modes to obtain the costs of the candidate intra-prediction modes on the template, and the predicted value of the one or two intra-prediction modes with the lowest cost is taken as the intra-prediction value of the current block. If the difference between the two costs corresponding to the two intra-prediction modes on the template is not large, the compression performance can be improved by performing a weighted average on the predicted values ​​of the two intra-prediction modes. Selectively, the weights of the predicted values ​​of the two prediction modes are related to the above costs, for example, the weights are inversely proportional to the costs.

[0071] In summary, TIMD allows for the selection of intra-prediction modes by leveraging their predictive effects on the template, and then weighting two intra-prediction modes according to their cost on the template. The advantage of TIMD is that, once a TIMD mode is selected for the current block, there is no need to specifically indicate which intra-prediction mode is being used in the bitstream; this is derived by the decoder itself through the process described above, thus saving some overhead.

[0072] Through a brief introduction to the above intra-prediction modes, the following can be observed: The technical principles of DIMD mode and TIMD mode are similar. Both utilize the fact that the decoder performs the same operation as the encoder to estimate the prediction mode of the current coding unit. In such prediction modes, if the complexity is acceptable, the transmission of the prediction mode index can be omitted, saving overhead and improving compression efficiency. However, due to the limitations of the information that can be referenced and the fact that they do not improve prediction quality in themselves, DIMD mode and TIMD mode are more effective in large areas where texture characteristics are consistent. When the texture changes slightly or the template area cannot be covered, the prediction effect of such prediction modes is inferior.

[0073] Furthermore, both DIMD and TIMD modes merge or weight prediction blocks obtained based on multiple conventional prediction modes. Merging prediction blocks can produce effects that cannot be achieved with a single prediction mode. In DIMD mode, a planar mode can be introduced as an additional weighted prediction mode to enhance the spatial relevance between adjacent reconstructed samples and predicted samples, thereby improving the prediction effect of intra-prediction. However, because the prediction principle of the planar mode is relatively simple, using the planar mode as an additional weighted prediction mode for prediction blocks where there is a clear difference between the upper right corner and the lower left corner may be counterproductive.

[0074] The following explains the process of converting residual blocks.

[0075] In the encoding process, the current block is first predicted. During prediction, spatial or temporal dependencies are used to obtain an image that is the same as or similar to the current block. For one block, the predicted block and the current block may be exactly the same, but it is difficult to guarantee that all blocks in a video will be the same. In particular, in natural video or video shot with a camera, factors such as the complexity of the image texture and the presence of noise in the image mean that although the predicted block and the current block are usually similar, there are actually differences. Also, due to irregular motion, distortion and deformation, occlusion, and brightness changes in the video, it is difficult to perfectly predict the current block. Therefore, in a mixed coding framework, the residual image is obtained by subtracting the predicted image from the original image of the current block, or the residual block is obtained by subtracting the predicted block from the current block. Since the residual block is usually much simpler than the original image, prediction can greatly improve compression efficiency. Rather than encoding the residual block directly, the residual block is usually transformed first. Transformation involves converting the residual image from the spatial domain to the frequency domain and removing the correlations of the residual image. After converting the residual image to the frequency domain, the energy is often concentrated in the low-frequency region, so the non-zero coefficients after conversion are often concentrated in the upper left corner, and then the residual block is further compressed by quantization. Selectively, since the human eye is not sensitive to high frequencies, a larger quantization step size can be used in the high-frequency region.

[0076] Image transformation technology transforms an original image so that it can be represented by orthogonal functions or matrices. This transformation is two-dimensional, linear, and reversible. Generally, the original image is called a spatial domain image, the transformed image is called a transformed domain image (also called a frequency domain image), and the transformed domain image can be transformed back into a spatial domain image. Image transformation allows for a more effective reflection of the image's own features while concentrating energy on a small amount of data, which is advantageous for image storage, transmission, and processing.

[0077] The following describes the technology related to the conversion described in this application.

[0078] In the field of video coding, an encoder can obtain a residual block and then transform that residual block. Transformation methods include linear and quadratic transformations. Linear transformation methods include, but are not limited to, the discrete cosine transform (DCT) and the discrete sine transform (DST). DCTs usable in video coding include, but are not limited to, DCT2 and DCT8, and DSTs usable in video coding include, but are not limited to, DST7. Because the DCT has strong energy concentration characteristics, after the original image is DCT transformed, non-zero coefficients exist only in a partial region (e.g., the upper left corner region). Of course, in video coding, images are divided into blocks for processing, so transformations are also performed based on these blocks.

[0079] It is important to note that since all images are two-dimensional, directly performing a two-dimensional conversion would result in computational complexity and memory overhead that would be unacceptable to the hardware. Therefore, the DCT2, DCT8, and DST7 conversions described above are usually divided into horizontal and vertical one-dimensional conversions, i.e., performed in two steps. For example, the horizontal conversion is performed first, followed by the vertical conversion, or vice versa. The above conversion method is effective for horizontal and vertical textures, but less effective for diagonal textures. Since horizontal and vertical textures are the most common, the above conversion method is very useful for improving compression efficiency.

[0080] Encoders can perform a secondary transformation on top of a primary transformation to further improve compression efficiency.

[0081] Linear transformations are used to process horizontal and vertical textures. Linear transformations are also called basic transformations. For example, linear transformations include, but are not limited to, the DCT2, DCT8, and DST7 transformations mentioned above. Quadratic transformations are used to process diagonal textures. For example, quadratic transformations include, but are not limited to, low-frequency non-separable transforms (LFNST). On the encoding side, quadratic transformations are performed after linear transformations and before quantization. On the decoder side, quadratic transformations are performed after inverse quantization and before inverse linear transformations.

[0082] Figure 6 is an example of an LFNST according to an embodiment of this application.

[0083] As shown in Figure 6, on the encoding side, the LFNST quadratically transforms the low-frequency coefficients in the upper left corner after the basic transformation. The first-order transformation concentrates energy in the upper left corner by removing correlation with the image. The second-order transformation again removes correlation with the low-frequency coefficients of the first-order transformation. On the encoding side, if 16 coefficients are input to a 4x4 LFNST, 8 coefficients are output, and if 64 coefficients are input to an 8x8 LFNST, 16 coefficients are output. On the decoder side, if 8 coefficients are input to a 4x4 inverse LFNST, 16 coefficients are output, and if 16 coefficients are input to an 8x8 inverse LFNST, 64 coefficients are output.

[0084] When an encoder performs a quadratic transformation on the current block in the current image, it can use one transformation core in the selected transformation set to transform the residual blocks of the current block. For example, if the quadratic transformation is LFNST, Conversion setA transformation core can refer to a set of transformation cores used to transform a particular diagonal texture, or a transformation set can include a set of transformation cores used to transform several similar diagonal textures. Of course, in other alternative embodiments, a transformation core may be referred to or replaced by similar or identical terms such as transformation matrix, transformation core type, or basis function, and a transformation set may be referred to or replaced by similar or identical terms such as transformation matrix set, transformation core type set, or basis function set, and the present application is not particularly limited.

[0085] Figure 7 shows an example of an LFNST conversion set according to the embodiment of this application.

[0086] As shown in Figures 7(a) to 7(d), LFNST can have four transformation sets, and the transformation cores of the same transformation set have similar diagonal textures. For example, the transformation set shown in Figure 7(a) can be the transformation set with index 0, the transformation set shown in Figure 7(b) can be the transformation set with index 1, the transformation set shown in Figure 7(c) can be the transformation set with index 2, and the transformation set shown in Figure 7(d) can be the transformation set with index 3.

[0087] The following describes a correlation scheme for applying LFNST to intra-encoded blocks.

[0088] Intra prediction predicts the current block using reconstructed samples around the current block as a reference, and since the current video is encoded from left to right and top to bottom, the reference samples available for the current block are typically on the left and top sides. Angle prediction predicts the value by tiling the reference samples onto the current block at a specified angle. This means that the predicted block has an obvious directional texture, and the residual of the current block after angle prediction also exhibits statistically obvious angular characteristics. Therefore, the set of transformations selected for LFNST can be bound to the intra prediction mode, i.e., after determining the intra prediction mode, LFNST can use a transform set where the texture direction corresponds to the angular features of the intra prediction mode to save bit overhead.

[0089] For illustrative purposes, let's assume that LFNST has four transformation sets, each with two transformation cores. Table 1 shows the correspondence between intra-prediction modes and transformation sets.

[0090] [Table 1]

[0091] As shown in Table 1, intra-prediction modes 0-81 can be associated with indices in four transformation sets.

[0092] It is important to note that the cross-component prediction modes used for chromatic intra-prediction are 81-83, while these modes are not available for luminance intra-prediction. The LFNST transformation set can handle more angles using a single transformation set by transposition. For example, both intra-prediction modes 13-23 and 45-55 correspond to transformation set 2, but intra-prediction modes 13-23 are clearly closer to horizontal modes, and intra-prediction modes 45-55 are clearly closer to vertical modes, and the transformations corresponding to intra-prediction modes 45-55 are adapted by transposition.

[0093] In practical implementation, since LFNST has four transformation sets, the encoding side can determine which transformation set to use for LFNST based on the intra-prediction mode used for the current block, and then determine which transformation core to use from that determined transformation set. By utilizing the correlation between the intra-prediction mode and the LFNST transformation set, the transmission of the selected LFNST transformation set in the bitstream is reduced. Whether the current block uses LFNST, and if so, whether to use the first or second transformation core of a single transformation set, can be determined by the bitstream and several other conditions.

[0094] Of course, considering that there are 67 normal intra-prediction modes and LFNST only has four transformation sets, and that each transformation set needs to occupy memory space to store the coefficients of the transformation cores in that set, multiple approximating angle prediction modes can only be supported by one transformation set, as a compromise between performance and complexity. As the demand for compression efficiency increases and hardware capabilities improve, LFNST can also be designed to be more complex. For example, by using larger transformation sets, more transformation sets, and each transformation set using more transformation cores.

[0095] For illustrative purposes, Table 2 shows another correspondence between intra-prediction modes and transformation sets.

[0096] [Table 2]

[0097] As shown in Table 2, 35 transformation sets were used, each using three transformation cores. The correspondence between the transformation sets and intra-prediction modes is as follows: Intra-prediction modes 0-34 correspond in the forward direction to transformation sets 0-34; that is, the larger the prediction mode number, the larger the index of the transformation set. Intra-prediction modes 35-67 correspond in the reverse direction to transformation sets 2-33 due to transposition; that is, the larger the prediction mode number, the smaller the index of the transformation set. The remaining prediction modes can be uniformly corresponded to the transformation set with index 2. In other words, if transposition is not considered, one intra-prediction mode corresponds to one transformation set, and with this design, the residuals corresponding to each intra-prediction mode can obtain a more appropriate transformation set, and compression performance is also improved.

[0098] Of course, theoretically, wide-angle prediction modes and transformation sets can also be mapped one-to-one, but such a design is less cost-effective, and this application will not describe it in detail. Furthermore, LFNST is merely one example of a quadratic transformation and should not be understood as a limitation of quadratic transformations. For example, LFNST is an inseparable quadratic transformation, and in other alternative embodiments, separable quadratic transformations can be used to improve the compression efficiency of the residuals of oblique textures, and this application is not specifically limited to such cases.

[0099] Figure 8 is a block diagram showing a decoding framework 200 according to an embodiment of this application.

[0100] As shown in Figure 8, the decoding framework 200 may include an entropy decoding unit 210, an inverse transform / inverse quantization unit 220, a residual unit 230, an intra-prediction unit 240, an inter-prediction unit 250, a loop filtering unit 260, and a decoding image buffer unit 270. The entropy decoding unit 210 receives and analyzes a bitstream to obtain a prediction block and a frequency-domain residual block. Steps such as inverse transform and inverse quantization can be performed on the frequency-domain residual block through the inverse transform / inverse quantization unit 220 to obtain a time-domain residual block. The residual unit 230 can obtain a reconstructed block by adding the prediction block obtained by the intra-prediction unit 240 or the inter-prediction unit 250 to the time-domain residual block obtained after inverse transform and inverse quantization has been performed through the inverse transform / inverse quantization unit 220.

[0101] Figure 9 is a flowchart of the decoding method 300 according to the present invention. The decoding method 300 can be executed by a decoder. For example, the decoding method 300 is shown in Figure 8 This applies to the decoding framework 200 shown. For the sake of clarity, the decoder will be used as an example below.

[0102] As shown in Figure 9, the decoding method 300 may include the following steps. S310: Analyze the bitstream of the current sequence and the current block First Obtain the conversion coefficient. S320: Confirm the first intra-prediction mode. The first intra-prediction mode includes one of the following: an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for predicting the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of the optimal, matrix-based intra-prediction (MIP) mode for predicting the current block; an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region adjacent to the current block; or an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. S330: Based on the transformation set corresponding to the first intra prediction mode, the first transformation is performed on the first transformation coefficient to obtain the second transformation coefficient for the current block. S340: Perform the second transformation on the second transformation coefficient to obtain the residual block of the current block. S350: Determine the reconstructed blocks of the current block based on the predicted blocks and residual blocks of the current block.

[0103] For example, when a decoder uses an intra-prediction mode derived from a DIMD mode for a reconstructed sample in a first template region, it can first calculate the gradient value of the reconstructed sample in the first template region, and then determine the intra-prediction mode that matches the gradient direction of the reconstructed sample with the largest gradient value in the first template region as the intra-prediction mode derived from the DIMD mode. Alternatively, the decoder can calculate the gradient value corresponding to each intra-prediction mode by traversing the intra-prediction modes based on the reconstructed sample in the first template region, and determine the intra-prediction mode with the largest gradient value as the intra-prediction mode derived from the DIMD mode.

[0104] For example, if the decoder uses an intra-prediction mode derived from the DIMD mode for the current block's predicted block (or the output vector of the optimal MIP mode), it can first calculate the gradient value of the predicted sample of the current block's predicted block (or the output vector of the optimal MIP mode), and then determine the intra-prediction mode that matches the gradient direction of the predicted sample with the largest gradient value in the current block's predicted block (or the output vector of the optimal MIP mode) as the intra-prediction mode derived from the DIMD mode. Alternatively, the decoder can calculate the gradient value corresponding to each intra-prediction mode by traversing the intra-prediction modes based on the predicted sample of the current block's predicted block (or the output vector of the optimal MIP mode), and determine the intra-prediction mode with the largest gradient value as the intra-prediction mode derived from the DIMD mode.

[0105] For example, the first transformation is used to handle diagonal textures in the current block.

[0106] For example, the second transformation is used to process the horizontal and vertical textures in the current block.

[0107] It should be understood that the first transformation is the inverse of the quadratic transformation on the encoding side, and the second transformation is the inverse of the base transformation on the encoding side. For example, the first transformation could be the inverse LFNST, and the second transformation could be the inverse DCT2, inverse DCT8, or inverse DCT7, etc.

[0108] Of course, the TMMIP technology and the method of adapting LFNST are also applicable to other quadratic transformation methods. For example, LFNST is an inseparable quadratic transformation, and in other alternative embodiments, the TMMIP technology is also applicable to separable quadratic transformations, and this application is not particularly limited to this.

[0109] It is important to note that when an encoder or decoder predicts the current block, it may perform LFNST using the transformation set corresponding to the PLANAR mode, for the following reasons: The transformation core used for LFNST is obtained by deep learning training on a dataset of conventional intra-prediction modes. Therefore, in a normal intra-prediction process, the transformation core used for LFNST is usually also a transformation core selected from the LFNST transformation set corresponding to the conventional intra-prediction mode. However, an encoder or decoder may predict the current block using a non-traditional intra-prediction mode. In this case, considering that planar mode is usually used to process blocks with gradients in textures, and LFNS is used to process diagonal textures, the texture information of the predicted block output in planar mode and the texture information of planar mode in conventional intra-prediction modes can usually be processed as the same type of texture. That is, when an encoder or decoder predicts the current block using a non-traditional intra-prediction mode, it will perform LFNST using the transformation set corresponding to planar mode. For example, when an encoder predicts the current block using MIP mode, it will perform LFNST using the transformation set corresponding to planar mode. However, the meaning of MIP mode differs from that of the conventional intra-prediction mode. While the conventional intra-prediction mode has a clear directionality, MIP mode is merely an index of matrix coefficients. Therefore, the planar mode is used to process blocks where a gradient exists in the texture, but it does not necessarily match the texture information of the current block. In other words, the texture direction of the transformation set used by LFNST does not necessarily match the texture direction of the current block, which degrades the decompression performance of the current block.

[0110] In view of this, the embodiment of the present invention can further improve the decompression performance of the current block by introducing a first intra-prediction mode and performing a first transformation on a first transformation coefficient of the current block based on a transformation set corresponding to the first intra-prediction mode. In particular, when the decoder predicts the current block using a non-traditional intra-prediction mode, it is possible to avoid directly performing the first transformation using a transformation set corresponding to the planar mode, and the transformation set corresponding to the first intra-prediction mode can reflect the texture orientation of the current block to some extent, thereby improving the decompression performance of the current block.

[0111] Below is the table 3 and table 4 Based on the test results, the beneficial effects of the technical solution provided in this application will be explained.

[0112] table 3 This is the result obtained by testing a test sequence when the current block is weighted and predicted using the optimal and suboptimal MIP modes, and the first intra-prediction mode is designed as an intra-prediction mode derived from the DIMD mode used for the prediction block of the current block. 4 This is the result obtained by testing a test sequence when the current block is weighted and predicted using the optimal MIP mode and an intra-prediction mode derived from the DIMD mode, which is used for reconstruction samples in a first template region adjacent to the current block, and the first intra-prediction mode is designed as an intra-prediction mode derived from the decoder-side intra-mode derivation (DIMD) mode, which is used for the prediction block of the current block.

[0113] [Table 3]

[0114] [Table 4]

[0115] table 3 and table 4 As shown, a negative incremental bitrate (BD-rate) indicates an improvement in performance of the solution provided by this application compared to the ECM2.0 test results. Based on the test results, under normal measurement conditions, the table shows... 3 and table 4 The test results all showed that an average brightness performance gain of 0.20% was achieved, demonstrating that 4K sequences were not vulgar. It should be noted that the TIMD prediction mode with ECM2.0 integration has a higher complexity than ECM1.0 and only achieves a performance gain of 0.4%. When current intra-encoding performance is difficult to improve, the solution provided by this application can yield good performance gains without increasing the complexity of the decoder, and the performance gain is particularly evident for 4K type video sequences. Furthermore, even if the encoding and decoding times fluctuate somewhat due to server load, theoretically the decoding time will hardly increase.

[0116] In some embodiments, the output vector of the optimal MIP mode is the vector before upsampling the output vector of the optimal MIP mode, or the output vector of the optimal MIP mode is the vector after upsampling the output vector of the optimal MIP mode.

[0117] In other words, the process of using an intra-prediction mode derived from the DIMD mode for the output vector of the optimal MIP mode may be performed before or after upsampling the output vector of the optimal MIP mode, and is not particularly limited in this application.

[0118] The decoder inputs the reference sample into the prediction matrix of the optimal MIP mode to obtain the output vector. The output vector of the optimal MIP mode has up to 64 prediction samples, compared to up to several thousand prediction samples in the upsampled prediction block. Before upsampling the output vector of the optimal MIP mode, the decoder derives from the DIMD mode. Ta By using an anticipatory mode, computational complexity can be reduced, and the decompression performance of the current block can be improved. For example, the decoder can effectively reduce computational complexity by calculating the gradient amplitude values ​​of each conventional anticipatory mode using DIMD before upsampling.

[0119] In some embodiments, S350 may include the decoder determining a first intra-prediction mode based on a prediction mode for predicting the current block.

[0120] For example, the decoder determines a first intra-prediction mode based on the mode type of the prediction mode for predicting the current block.

[0121] For example, the decoder determines a first intra-prediction mode based on the derivation mode of the prediction mode for predicting the current block.

[0122] Exemplary examples of derivation modes for predicting the current block include, but are not limited to, MIP mode, DIMD mode, and TIMD mode.

[0123] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode for predicting the current block, the decoder determines an intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or determines an intra-prediction mode derived from the DIMD mode to be used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0124] In other words, when the decoder weights and predicts the current block using the optimal and suboptimal MIP modes, it either determines the intra-prediction mode derived from the DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or determines the intra-prediction mode derived from the DIMD mode used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0125] In this embodiment, if the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode, the decoder first sets the intra-prediction mode derived from the DIMD mode used for the predicted block of the current block as the first intra-prediction mode, and the texture direction of the transformation set corresponding to the first intra-prediction mode is simultaneously adapted to the texture characteristics shown by the predicted block of the current block using the optimal MIP mode and the texture characteristics shown by the predicted block of the current block using the suboptimal MIP mode. doThis allows for further improvement of the current block's decompression performance. The decoder first uses an intra-prediction mode derived from the DIMD mode, which is used for the output vector of the optimal MIP mode, as the first intra-prediction mode. In the process of determining the optimal MIP mode, the output vector of the optimal MIP mode can be directly obtained, reducing the decompression complexity. Furthermore, the texture direction of the transformation set corresponding to the first intra-prediction mode can be matched to the texture characteristics shown by the predicted block of the current block by the optimal MIP mode, thereby improving the current block's decompression and compression performance as much as possible.

[0126] Of course, in other alternative embodiments, where the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode, the decoder may also determine an intra-prediction mode derived from a DIMD mode or an intra-prediction mode derived from a TIMD mode as the first intra-prediction mode used for the reconstructed sample in the first template region, and is not specifically limited herein.

[0127] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the TIMD mode, the decoder either determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode to be used for predicting the current block, or the decoder determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0128] In other words, if the second intra-prediction mode is an intra-prediction mode derived from the TIMD mode, the decoder will either determine the intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or the decoder will determine the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0129] In this embodiment, if the second intra-prediction mode includes an intra-prediction mode derived from the TIMD mode, the decoder first sets the intra-prediction mode derived from the DIMD mode, which is used for the prediction block of the current block, as the first intra-prediction mode, and the texture direction of the conversion set corresponding to the first intra-prediction mode is simultaneously matched to the texture characteristics shown by the prediction block of the current block by the optimal MIP mode and the texture characteristics shown by the prediction block of the current block by the intra-prediction mode derived from the TIMD mode. do This allows for further improvement of the current block's decompression performance. The decoder first sets the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode, and then directly sets the second intra-prediction mode as the first intra-prediction mode. This reduces the decompression complexity, and the texture direction of the transformation set corresponding to the first intra-prediction mode can be matched to the texture characteristics indicated by the predicted block of the current block using the optimal MIP mode, thereby improving the current block's decompression and compression performance as much as possible.

[0130] Of course, in other alternative embodiments, if the second intra-prediction mode is an intra-prediction mode derived from the TIMD mode, the decoder may also determine the first intra-prediction mode to be an intra-prediction mode derived from the DIMD mode used for the output vector of the optimal MIP mode, and an intra-prediction mode derived from the DIMD mode used for the reconstructed sample in the first template region, and the present invention is not particularly limited thereto.

[0131] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region, the decoder determines the intra-prediction mode derived from a DIMD mode used for predicting the current block as the first intra-prediction mode, or determines the intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region as the first intra-prediction mode.

[0132] In other words, if the second intra-prediction mode is an intra-prediction mode derived from the DIMD mode, the decoder either determines the intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or the decoder determines the intra-prediction mode derived from the DIMD mode to be used for the reconstructed sample in the first template region as the first intra-prediction mode.

[0133] In this embodiment, if the second intra-prediction mode includes an intra-prediction mode derived from a DIMD mode used for reconstructed samples in the first template region, the decoder first sets the intra-prediction mode derived from a DIMD mode used for the prediction block of the current block as the first intra-prediction mode, and the texture orientation of the conversion set corresponding to the first intra-prediction mode is simultaneously matched to the texture characteristics shown by the prediction block of the current block by the optimal MIP mode and the texture characteristics shown by the prediction block of the current block by the intra-prediction mode derived from a DIMD mode. doThis allows for further improvement of the current block's decompression performance. The decoder first sets the intra-prediction mode derived from the DIMD mode, which is used for the reconstruction sample in the first template region, as the first intra-prediction mode. Then, it can directly determine the second intra-prediction mode as the first intra-prediction mode, reducing the decompression complexity. Furthermore, the texture direction of the transformation set corresponding to the first intra-prediction mode can be matched to the texture characteristics shown by the predicted block of the current block using the optimal MIP mode, thereby improving the current block's decompression and compression performance as much as possible.

[0134] Of course, in other alternative embodiments, if the second intra-prediction mode is an intra-prediction mode derived from a DIMD mode used for the reconstructed sample in the first template region, the decoder may also determine the intra-prediction mode derived from a DIMD mode and the intra-prediction mode derived from a TIMD mode as the first intra-prediction mode, used for the output vector of the optimal MIP mode, and the present invention is not particularly limited thereto.

[0135] In some embodiments, the decoding method 300 may further include the following: Determine the second intra-prediction mode. The second intra-prediction mode includes a suboptimal MIP mode for predicting the current block, and one of the intra-prediction modes derived from the DIMD mode or the TIMD mode, which is used for the reconstructed sample in the first template region. Based on the optimal MIP mode and a second intra-prediction mode, the current block is predicted and the predicted block for the current block is obtained.

[0136] Exemplary, the process by which a decoder predicts the current block based on an optimal MIP mode and a second intra-prediction mode can also be abbreviated as Template Matching MIP (TMMIP) technology, TMMIP-based prediction mode derivation method, or TMMIP fusion enhancement technology. That is, after obtaining the residual block of the current block, the decoder can enhance the performance of the prediction process for the current block based on the derived optimal MIP mode and second intra-prediction mode. In other words, TMMIP technology can enhance the performance of the prediction process for the current block by utilizing at least one of a suboptimal MIP prediction mode, an intra-prediction mode derived from a TIMD mode, or an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region adjacent to the current block, along with an optimal MIP prediction mode.

[0137] In this embodiment, the decoder predicts the current block based on an optimal MIP mode and a second intra-prediction mode. The optimal MIP mode is determined based on the strain costs corresponding to multiple MIP modes and is designed to be the optimal MIP mode for predicting the current block. The second intra-prediction mode is designed to include at least one of a suboptimal MIP mode determined based on the strain costs corresponding to multiple MIP modes and for predicting the current block, an intra-prediction mode derived from a DIMD mode, and an intra-prediction mode derived from a TIMD mode, used for reconstruction samples in a first template region adjacent to the current block. This helps to avoid the decoder obtaining the MIP mode by analyzing the bitstream. Compared to conventional MIP techniques, this application can effectively reduce bit overhead at the coding unit level, thereby improving the decompression efficiency of the current block.

[0138] Specifically, the bit overhead of MIP mode is greater than that of other intra-prediction modes, requiring not only a flag to indicate whether MIP mode is used, but also a flag to indicate whether MIP mode is transposed, and finally, the largest part of the overhead is that truncated binary coding must be used to represent the MIP mode index. MIP mode is a simplified technique based on neural network technology and differs significantly from conventional interpolation filtering prediction techniques. For some special textures, MIP mode is more effective than conventional intra-prediction modes, but its large flag overhead is a drawback of MIP mode. For example, in a 4x4 coding unit, MIP mode has a total of 16 prediction modes, but its bit overhead includes one MIP mode use flag, one MIP mode transposition flag, and 5-bit or 6-bit truncated binary flags. In light of this, this application utilizes a method in which the decoder autonomously determines the optimal MIP mode for predicting the current block and determines the intra-prediction mode of the current block based on the optimal MIP mode, thereby saving up to 5 or 6 bits of overhead, effectively reducing the bit overhead at the coding unit level, and thereby improving decompression efficiency.

[0139] Furthermore, saving up to 5 or 6 bits of overhead per coding unit assumes that the template matching-based predictive mode derivation algorithm needs to be very accurate. If the accuracy of the template matching-based predictive mode derivation algorithm is too low, Decoding The MIP mode derived by the client differs from the MIP mode derived by the encoding client, further degrading coding performance. Alternatively, coding performance depends on the accuracy of the template matching-based prediction mode derivation algorithm.

[0140] However, both the template-based derivation algorithm in conventional intra-prediction mode and the template-matching-based derivation algorithm in inter-prediction mode fail to achieve the expected accuracy. While it can save bit overhead and improve compression efficiency, as the number of prediction modes in the template-matching-based prediction mode derivation algorithm increases, the additional bit overhead at the coding unit level introduced by the template-matching-based prediction mode derivation algorithm prevents subsequent technologies from improving compression efficiency by relying solely on the template-matching-based prediction mode derivation algorithm. Therefore, the template-matching-based prediction mode derivation algorithm needs to improve coding performance while also improving compression efficiency. As one possible embodiment, coding performance can be improved by saving bit overhead at the coding unit level and creating different novel prediction blocks, thereby ensuring prediction diversity and selection diversity. In light of this, this application combines the optimal MIP mode with a second intra-prediction mode, that is, performs fusion prediction on the current block based on the optimal MIP mode and the second intra-prediction mode. This avoids completely replacing the optimal prediction mode calculated based on rate distortion cost with the optimal MIP mode, thereby achieving both prediction accuracy and prediction diversity, and further improving decompression performance.

[0141] In particular, since TMMIP technology predicts the current block by combining an optimal MIP mode and a second intra-prediction mode, predicted blocks obtained by predicting the current block using different prediction modes may have different texture characteristics. Therefore, if the current block selects TMMIP technology, the predicted block of the current block will exhibit one texture characteristic due to the optimal MIP mode, and the predicted block of the current block will exhibit another texture characteristic due to the second intra-prediction mode. In other words, statistically speaking, after predicting the current block, the residual block of the current block can also exhibit two texture characteristics. That is, the residual block of the current block does not necessarily conform to a rule that can be embodied by a given prediction mode. In this case, for TMMIP technology, if the first intra-prediction mode is an intra-prediction mode derived from the DIMD mode used for the predicted block of the current block, the texture direction of the transformation set corresponding to the first intra-prediction mode simultaneously conforms to the texture characteristics shown by the predicted block of the current block due to the optimal MIP mode and the texture characteristics shown by the predicted block of the current block due to the second intra-prediction mode. do This improves the decompression performance of the current block. Furthermore, if the first intra-prediction mode is an intra-prediction mode derived from DIMD mode or an intra-prediction mode derived from TIMD mode used for the reconstruction sample in the first template region, the first intra-prediction mode can be directly determined in the process of determining the second intra-prediction mode, reducing decompression complexity, and the texture direction of the transformation set corresponding to the first intra-prediction mode is simultaneously matched to the texture characteristics shown by the predicted block of the current block by the optimal MIP mode and the texture characteristics shown by the predicted block of the current block by the second intra-prediction mode. do This allows for improved thawing efficiency.

[0142] In some embodiments, the decoder first predicts the current block based on the optimal MIP mode and obtains a first predicted block. Next, it predicts the current block based on a second intra-prediction mode and obtains a second predicted block. Then, it performs a weighting process on the first and second predicted blocks based on the weights of the optimal MIP mode and the second intra-prediction mode to obtain a predicted block for the current block.

[0143] In some embodiments, before obtaining the predicted block of the current block by weighting the first and second predicted blocks based on the weights of the optimal MIP mode and the weights of the second intra-prediction mode, the decoding method 300 may further include the following steps: If the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode for predicting the current block, or an intra-prediction mode derived from a TIMD mode, the weights of the optimal MIP mode and the second intra-prediction mode are determined based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the second intra-prediction mode. If the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for the reconstruction sample in the first template region, the weights of the optimal MIP mode and the second intra-prediction mode are determined to be preset values.

[0144] In some embodiments, the decoder predicts the current block based on the optimal MIP mode and obtains a first predicted block. It predicts the current block based on a second intra-prediction mode and obtains a second predicted block. The decoder then weights the first and second predicted blocks based on the weights of the optimal MIP mode and the second intra-prediction mode to obtain a predicted block for the current block.

[0145] For example, a decoder can directly intra-predict the current block based on the optimal MIP mode to obtain a first predicted block. Furthermore, the decoder can directly obtain the optimal and suboptimal predicted modes based on the TIMD mode, predict the current block, and obtain a second predicted block. For example, if neither the optimal nor suboptimal predicted mode is a DC mode (also called the mean mode) or a planar mode (also called the flat mode), and the distortion cost of the suboptimal predicted mode is less than the distortion cost of the optimal predicted mode twice as large, then a predicted block fusion operation is necessary. That is, the decoder can first intra-predict the current block based on the optimal predicted mode to obtain the optimal predicted block, then intra-predict the current block based on the suboptimal predicted mode to obtain the suboptimal predicted block, and then calculate the weight values ​​of the optimal and suboptimal predicted blocks using the ratio of the distortion costs of the optimal and suboptimal predicted modes, and finally, weight-fuse the optimal and suboptimal predicted blocks to obtain a second predicted block. Furthermore, if the optimal or suboptimal prediction mode is a planar mode or a DC mode, or if the distortion cost of the suboptimal prediction mode is greater than the distortion cost of twice the optimal prediction mode, then the prediction block fusion operation is not necessary. That is, only the optimal prediction block obtained based on the optimal prediction mode can be directly used as the second prediction block. After obtaining the first and second prediction blocks, the decoder performs a weighting process on the first and second prediction blocks to obtain the prediction block for the current block.

[0146] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode, a suboptimal MIP mode for predicting the current block, or an intra-prediction mode derived from a TIMD mode, the decoder determines the weights of the optimal MIP mode and the second intra-prediction mode based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the second intra-prediction mode. If the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region, the decoder determines that both the weights of the optimal MIP mode and the second intra-prediction mode are preset values.

[0147] In some embodiments, S320 may include the following: The decoder analyzes the bitstream of the current sequence and obtains a first flag. If the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using an optimal MIP mode and a second intra-prediction mode, the decoder determines the optimal MIP mode based on the distortion costs corresponding to multiple MIP modes.

[0148] For example, if the value of the first flag is a first value, the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode. If the value of the first flag is a second value, the first flag is used to indicate that it is not permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode. In one embodiment, the first value is 1 and the second value is 0. In another embodiment, the first value is 0 and the second value is 1. Of course, the first and second values ​​may be other values, and are not limited thereto in this application.

[0149] For example, if the first flag is true, it is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode. If the first flag is false, it is used to indicate that it is not permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode.

[0150] For example, the decoder analyzes block-level flags. If the intra-predictive mode is used for the current block, the decoder analyzes or retrieves a first flag. If the first flag is true, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0151] For example, if the first flag is denoted as sps_timd_enable_flag, the decoder parses or retrieves sps_timd_enable_flag. If sps_timd_enable_flag is true, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0152] For example, the first flag is a sequence-level flag.

[0153] The statement that the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode can be replaced with a statement having a similar or identical meaning. For example, in other alternative embodiments, the statement that the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode can be replaced with any of the following: The first flag is used to indicate that it is permitted to determine the intra-prediction mode of image blocks in the current sequence using TMMIP technology. The first flag is used to indicate that it is permitted to perform intra-prediction for image blocks in the current sequence using TMMIP technology. The first flag is used to indicate that it is permitted to use TMMIP technology for image blocks in the current sequence. The first flag is used to indicate that it is permitted to predict image blocks in the current sequence using an MIP mode determined based on multiple MIP modes.

[0154] Furthermore, in other alternative embodiments, when TMMIP technology is combined with other technologies, an enable flag for the other technology can indirectly indicate whether or not the use of TMMIP technology is permitted in the current sequence. For example, taking TIMD technology as an example, if the first flag is used to indicate that the use of TIMD technology is permitted in the current sequence, it also indicates that the use of TMMIP technology is permitted in the current sequence. In other words, if the first flag is used to indicate that the use of TIMD technology is permitted in the current sequence, it indicates that both TIMD technology and TMMIP technology are permitted in the current sequence. This further saves bit overhead.

[0155] In some embodiments, if a first flag is used to indicate that it is permitted to predict the image block in the current sequence using the optimal MIP mode and a second intra-prediction mode, the decoder analyzes the bitstream to obtain the second flag. If the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and a second intra-prediction mode, the decoder determines the optimal MIP mode based on the distortion costs corresponding to multiple MIP modes.

[0156] For example, the decoder analyzes block-level flags. If the current block uses an intra-predictive mode, the decoder analyzes or retrieves a first flag. If the first flag is true, the decoder analyzes or retrieves a second flag. If the second flag is true, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0157] For example, if the value of the second flag is the third value, the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode. If the value of the second flag is the fourth value, the second flag is used to indicate that it is not permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode. In one embodiment, the third value is 1 and the fourth value is 0. In another embodiment, the third value is 0 and the fourth value is 1. Of course, the third and fourth values ​​may be other values, and are not limited thereto in this application.

[0158] For example, if the second flag is true, it is used to indicate that predicting the current block using the optimal MIP mode and the second intra-prediction mode is permitted. If the second flag is false, it is used to indicate that predicting the current block using the optimal MIP mode and the second intra-prediction mode is not permitted.

[0159] For example, the first flag is denoted as sps_timd_enable_flag and the second flag as cu_timd_enable_flag, in which case the decoder parses or retrieves sps_timd_enable_flag. If sps_timd_enable_flag is true, the decoder parses or retrieves cu_timd_enable_flag. If cu_timd_enable_flag is true, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0160] For example, the second flag can be a block-level flag or a coding unit-level flag.

[0161] The statement that the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode can be replaced with a statement having a similar or identical meaning. For example, in other alternative embodiments, the statement that the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode can be replaced with any of the following: The second flag is used to indicate that it is permitted to determine the intra-prediction mode of the current block using TMMIP technology. The second flag is used to indicate that it is permitted to perform intra-prediction for the current block using TMMIP technology. The second flag is used to indicate that it is permitted to use TMMIP technology on the image block of the current block. The second flag is used to indicate that it is permitted to predict the current block using an MIP mode determined based on multiple MIP modes.

[0162] Furthermore, in other alternative embodiments, when TMMIP technology is combined with other technologies, the permission flag for the other technology can indirectly indicate whether or not the use of TMMIP technology is permitted in the current block. For example, taking TIMD technology as an example, if the second flag is used to indicate that the use of TIMD technology is permitted in the current block, it also indicates that the use of TMMIP technology is permitted in the current block. In other words, if the second flag is used to indicate that the use of TIMD technology is permitted in the current block, it indicates that both TIMD technology and TMMIP technology are permitted in the current block. This further saves bit overhead.

[0163] Furthermore, when the decoding side analyzes the second flag, it can analyze the second flag before analyzing the residual block of the current block, or it can analyze the second flag after analyzing the residual block of the current block. This application does not particularly limit this.

[0164] In some embodiments, the method 300 may further include the following: The decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. The strain costs corresponding to multiple MIP modes include strain costs obtained by using multiple MIP modes to predict samples in a second template region adjacent to the current block.

[0165] For example, before determining the optimal MIP mode for predicting the current block based on the strain costs of multiple MIP modes, the decoder needs to calculate the strain cost for each of the multiple MIP modes and sort the multiple MIP modes based on the strain cost for each MIP mode. The MIP mode with the lowest cost is the optimal prediction.

[0166] Furthermore, the distortion cost related to the decoder in this application is different from the rate-distortion cost (RDcost) related to the encoder. The rate-distortion cost is the distortion cost used when the encoding side determines a specific intra-prediction technique from among multiple intra-prediction techniques, and the rate-distortion cost can be the cost obtained by comparing the distorted image with the original image. Since the decoder cannot obtain the original image, the distortion cost related to the decoder can be the distortion cost between the reconstructed sample and the predicted sample, and examples include the SATD (Sum of Absolute Transformed Difference) cost between the reconstructed sample and the predicted sample, or a cost that can be used to calculate the difference between the reconstructed sample and the predicted sample.

[0167] Of course, in other alternative embodiments, the decoder first determines the sequence order of the multiple MIP modes based on the distortion costs corresponding to the multiple MIP modes, then determines the coding scheme to be used for the optimal MIP mode based on the sequence order of the multiple MIP modes, and finally decodes the bitstream of the current sequence to obtain the index of the optimal MIP mode based on the coding scheme to be used for the optimal MIP mode.

[0168] For example, the codeword length of the coding scheme used for the first n MIP modes in the sequence order is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the sequence order, and / or a variable-length coding scheme is used for the first n MIP modes and a truncated binary coding scheme is used for the MIP modes following the nth MIP mode. Exemplarily, n can be any value greater than or equal to 1. Note that in conventional MIP techniques, the MIP mode index is usually binarized by selecting a truncated binary scheme similar to equal probability coding. That is, in the truncated binary scheme, all prediction modes are divided into two segments, one segment represented by N codewords and the other segment represented by N+1 codewords. In view of this, in this application, the decoder can calculate the strain cost corresponding to each of the multiple MIP modes and sort the multiple MIP modes based on the strain cost corresponding to each MIP mode before determining the optimal MIP mode for predicting the current block based on the strain costs corresponding to the multiple MIP modes. Ultimately, the decoder selects and uses a more flexible variable-length coding scheme according to the array order of the multiple MIP modes. Compared to equiprobability coding, the flexible configuration of the MIP mode coding scheme can save bit overhead for the MIP mode indices.

[0169] The array order is obtained by the decoder arranging multiple MIP modes in ascending order of distortion cost. The smaller the distortion cost corresponding to a MIP mode, the higher the probability that the encoder will use that MIP mode to perform intra-prediction for the current block. Therefore, the coding scheme used for the first n MIP modes in the array order is designed to have a smaller codeword length than the coding scheme used for the MIP mode following the nth MIP mode, and / or the coding scheme used for the first n MIP modes is designed to be a variable-length coding scheme, and the coding scheme used for the MIP mode following the nth MIP mode is designed to be a truncated binary coding scheme. In this way, MIP modes that are used with a high probability by the encoder use relatively short codeword lengths or variable-length coding schemes. This saves bit overhead in the MIP mode index and improves decompression performance.

[0170] In some embodiments, the decoding method 300 may further include the following: If the second intra-prediction mode is a suboptimal MIP mode, the decoder determines whether to adopt the suboptimal MIP mode to predict the current block based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the suboptimal MIP mode. If it is determined not to adopt the suboptimal MIP mode, the decoder can directly predict the current block based on the optimal MIP mode. If it is determined to adopt the suboptimal MIP mode, the decoder can predict the current block based on the optimal MIP mode and the suboptimal MIP mode to obtain a predicted block for the current block.

[0171] For example, when the second intra-prediction mode is a suboptimal MIP mode, if the ratio of the strain cost corresponding to the optimal MIP mode to the strain cost corresponding to the suboptimal MIP mode is less than or equal to a preset ratio, the decoder can directly predict the current block based on the optimal MIP mode and obtain a predicted block for the current block. Alternatively, when the second intra-prediction mode is a suboptimal MIP mode, if the ratio of the strain cost corresponding to the suboptimal MIP mode to the strain cost corresponding to the optimal MIP mode is greater than or equal to a preset ratio, the decoder can directly predict the current block based on the optimal MIP mode and obtain a predicted block for the current block. For example, if the strain cost corresponding to the suboptimal MIP mode is a multiple (e.g., 2 times) or more of the strain cost corresponding to the optimal MIP mode, it can be interpreted that the suboptimal MIP mode already has a large strain and is unsuitable for the current block, i.e., it is possible to predict the current block using only the optimal MIP mode without fusion enhancement techniques.

[0172] In this embodiment, the decoder determining whether to adopt a suboptimal MIP mode to predict the current block based on the distortion cost of the optimal MIP mode and the distortion cost of the suboptimal MIP mode is equivalent to the decoder determining whether to adopt a suboptimal MIP mode to improve the performance of the optimal MIP mode based on the distortion cost of the optimal MIP mode and the distortion cost of the suboptimal MIP mode. This avoids carrying a flag in the bitstream to determine whether to adopt a suboptimal MIP mode to improve the performance of the optimal MIP mode, thereby saving bit overhead and improving decompression performance.

[0173] In some embodiments, the second template region and the first template region may be the same or different.

[0174] For example, the size of the second template region can be predefined according to the size of the current block. For instance, the width of the region adjacent to the top of the current block within the second template region is equal to the width of the current block, and its height is equal to the height of at least one row of samples. The height of the region adjacent to the left of the current block within the second template region is equal to the height of the current block, and its width is 2 column It is equal to the width of the sample. Of course, in other alternative embodiments, the second template region can be realized as a second template region of other sizes or dimensions, and this application is not specifically limited thereto.

[0175] In some embodiments, the method 300 may further include the following: The decoder predicts samples in a second template region based on a third flag and a plurality of MIP modes to obtain strain costs corresponding to the plurality of MIP modes under each state of the third flag. The third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP mode. The decoder determines the optimal MIP mode based on the strain costs corresponding to the plurality of MIP modes under each state of the third flag.

[0176] For example, the decoder predicts samples in a second template region based on a third flag and multiple MIP modes, before determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes, and obtains the strain costs corresponding to multiple MIP modes under each state of the third flag.

[0177] As mentioned above, conventional MIP technology has a higher bit overhead compared to other intra-prediction tools, requiring not only a flag to indicate whether MIP technology is used, but also a flag to indicate whether MIP is transposed, and finally, the largest overhead is that truncated binary coding must be used to represent the MIP prediction mode. MIP technology is a simplified technology based on neural network technology and differs significantly from conventional interpolation filtering prediction technology. For some special textures, the MIP prediction mode is more effective than conventional intra-prediction modes, but its large flag overhead is a drawback of MIP technology. For example, a 4x4 coding unit has a total of 16 prediction samples, and its bit overhead includes one MIP use flag, one MIP transpose flag, and a 5-bit or 6-bit truncated binary flag. In light of this, this application proposes that when determining the optimal MIP mode, traversing each state of a third flag can take into account the transpose function of the MIP mode, saving the overhead of one MIP transpose flag and further improving decompression efficiency.

[0178] For example, the decoder traverses each state of the third flag and multiple MIP modes, determines the distortion cost corresponding to each of the multiple MIP modes under each state of the third flag, and determines the optimal MIP mode based on the distortion costs corresponding to each of the multiple MIP modes under each state of the third flag. Alternatively, the decoder traverses each state of the third flag and multiple MIP modes, determines the distortion cost under each of the multiple MIP modes corresponding to each state of the third flag, and determines the optimal MIP mode based on the distortion costs under each of the multiple MIP modes corresponding to each state of the third flag. In other words, the decoding side may first traverse multiple MIP modes, or it may first traverse the states of the third flag.

[0179] For example, when the value of the third flag is the fifth value, the third flag is used to indicate that the input and output vectors corresponding to the MIP mode are transposed. When the value of the third flag is the sixth value, the third flag is used to indicate that the input and output vectors corresponding to the MIP mode are not transposed. In this case, each state of the third flag can be replaced by each value of the third flag. In one embodiment, the fifth value is 1 and the sixth value is 0. In another embodiment, the fifth value is 0 and the sixth value is 1. Of course, the fifth and sixth values ​​may be other values, and this application is not limited thereto.

[0180] For example, when the third flag is true, it is used to indicate that the input and output vectors corresponding to the MIP mode are transposed. When the third flag is false, it is used to indicate that the input and output vectors corresponding to the MIP mode are not transposed. In this case, both true and false of the third flag are states of the third flag.

[0181] For example, the third flag may be a sequence-level flag, a block-level flag, or a coding unit-level flag.

[0182] For example, the third flag may also be called the inverted message, inverted flag, or MIP inverted flag.

[0183] The statement that the third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP mode can be replaced with a similar or identical statement. For example, in other alternative embodiments, the third flag is used to indicate whether or not it is necessary to transpose the input and output corresponding to the MIP mode. The third flag is used to indicate whether or not the input and output vectors corresponding to the MIP mode are transposed vectors. The third flag is used to indicate whether or not to transpose.

[0184] In some embodiments, the decoding method 300 may further include the following: If the current block size is a preset size, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0185] For example, a predefined size may include a size where the width is a predefined width and the height is a predefined height. In other words, if the current block's width is a predefined width and the height is a predefined height, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0186] Exemplary, a pre-set size can be realized by pre-storing a corresponding code, table, or other method that can be used in a device (e.g., including a decoder and encoder) to indicate the relevant information. This application is not limited to specific embodiments. For example, a pre-set size may refer to a size defined in a protocol. Optionally, “protocol” may refer to a standard protocol in the coding art field and may include relevant protocols such as the VCC protocol or the ECM protocol.

[0187] Of course, in other alternative embodiments, the decoder may also determine, by other means, whether or not to determine the optimal MIP mode based on the strain cost corresponding to multiple MIP modes, based on a preset size. This application is not specifically limited thereto.

[0188] For example, a decoder can determine whether or not to determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes, based solely on the width or height of the current block. In one embodiment, if the width of the current block is a preset width, or if the height is a preset height, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. In another embodiment, the decoder can determine whether or not to determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes by comparing the size of the current block with a preset size. In one embodiment, if the size of the current block is greater than or less than the preset size, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. In another embodiment, if the width of the current block is greater than or less than the preset width, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. In yet another embodiment, if the height of the current block is greater than or less than the preset height, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0189] In some embodiments, the method 300 may include the following: If the frame in which the current block is located is an I-frame and the size of the current block is a predetermined size, the decoder determines the optimal MIP mode based on the strain costs corresponding to a plurality of MIP modes.

[0190] For example, if the frame in which the current block is located is an I-frame, the width of the current block is a preset width, and the height of the current block is a preset height, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. That is, only when the frame in which the current block is located is an I-frame, the decoder determines whether or not to determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes, based on the size of the current block.

[0191] In some embodiments, the method 300 may further include the following: If the frame in which the current block is located is a B frame, the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0192] For example, if the frame in which the current block is located is a B frame, the decoder can directly determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. That is, if the frame in which the current block is located is a B frame, regardless of the size of the current block, the decoder can directly determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0193] In some embodiments, prior to S320, the method 300 may further include the following: The decoder obtains the MIP mode to be used for the adjacent block adjacent to the current block. The decoder determines the MIP mode to be used for the adjacent block as a set of MIP modes.

[0194] Exemplary, an adjacent block may be an image block adjacent to at least one of the following: above, to the left, below left, above right, and above left of the current block. For example, the decoder may determine the image blocks acquired in the order of above, to the left, below left, above right, and above left of the current block as adjacent blocks. Selectively, multiple MIP modes may be used by the decoder to construct an available MIP mode or a list of available MIP modes to predict the current block and to determine the available MIP mode. Thereafter, the decoder determines the optimal MIP mode by predicting a sample in a second template region from the available MIP mode or list of available MIP modes.

[0195] In some embodiments, the method 300 may further include the following: The decoder performs reconstructed sample padding on a reference region adjacent to the outside of the second template region to obtain reference rows and reference columns of the second template region. The decoder takes the reference rows and reference columns as input and predicts samples in the second template region using each of a plurality of MIP modes to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes. Based on the plurality of predicted blocks and the reconstructed blocks in the second template region, the decoder determines the strain costs corresponding to the plurality of MIP modes.

[0196] For example, the decoder performs reconstructed sample padding on a reference region adjacent to the second template region, before determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes.

[0197] For example, the width of the region adjacent to the upper side of the second template region within the reference region is equal to the width of the second template region. The height of the region adjacent to the left side of the second template region within the reference region is equal to the width of the second template region. height This is equal to: If the width of the region adjacent to the upper side of the second template region within the reference region is greater than the width of the second template region, the decoder can obtain the reference row by downsampling or reducing the dimensionality of the region adjacent to the upper side of the second template region within the reference region. height If it is larger, the decoder can obtain the reference column by downsampling or reducing the dimensionality of the region adjacent to the left of the second template region within the reference region.

[0198] For example, the second template region may be the template region used for the TIMD mode described above, and the reference region may be the reference of template used for the TIMD mode. For example, referring to Figure 5, if the current block is a coding unit with a width equal to M and a height equal to N, the decoder pads a reference region consisting of coding units with a width equal to 2(M+L1)+1 and a height equal to 2(N+L2)+1 with reconstructed samples, performs downsampling or dimensionality reduction on the padded reference region to obtain reference rows and reference columns, and then constructs an input vector corresponding to the MIP mode based on the reference rows and reference columns.

[0199] Exemplary, after obtaining the reference row and reference column, the decoder uses the reference row and reference column as input to predict samples in the second template region using each of the multiple MIP modes, thereby obtaining multiple prediction blocks corresponding to the multiple MIP modes. That is, the decoder predicts samples in the second template region of the current block by traversing the multiple MIP modes based on the reconstructed samples in the reference template of the current block. Using the currently traversed MIP mode as an example, the decoder uses the reference row, reference column, the index of the currently traversed MIP mode, and the third flag mentioned above as input to obtain a prediction block corresponding to the currently traversed MIP mode. The reference row and reference column are used to construct the input vector corresponding to the currently traversed MIP mode. The index of the currently traversed MIP mode is used to determine the matrix and / or bias vector corresponding to the currently traversed MIP mode. The third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP mode. For example, if the third flag is used to indicate that the input and output vectors corresponding to the MIP mode are not transposed, the reference column is spliced ​​after the reference row to form the input vector corresponding to the currently traversed MIP mode. If the third flag is used to indicate that the input and output vectors corresponding to the MIP mode are transposed, the reference row is spliced ​​after the reference column to form the input vector corresponding to the currently traversed MIP mode. Accordingly, if the third flag is used to indicate that the input and output vectors corresponding to the MIP mode are transposed, the decoder transposes the output of the currently traversed MIP mode to obtain a prediction block in the second template region.The decoder, after obtaining multiple prediction blocks corresponding to multiple MIP modes by traversing multiple MIP modes, can then select the MIP mode with the lowest cost based on the strain cost between the multiple prediction blocks and the reconstructed sample in the second template region, following the principle of minimum strain cost, and determine it as the optimal MIP mode under template matching-based MIP modes for the current block.

[0200] In some embodiments, when the decoder predicts a sample in a second template region using each of several MIP modes, it first downsamples the reference row and reference column to obtain an input vector, then uses the input vector as input to predict a sample in the second template region by traversing the multiple MIP modes to obtain an output vector corresponding to the multiple MIP modes, and finally upsamples the output vector corresponding to the multiple MIP modes to obtain a prediction block corresponding to the multiple MIP modes.

[0201] For example, a reference row and / or reference column satisfy the input conditions for multiple MIP modes. If a reference row and / or reference column do not satisfy the input conditions for multiple MIP modes, first, the reference row and / or reference column are processed to become input samples that satisfy the input conditions for multiple MIP modes. Then, based on the input samples that satisfy the input conditions for multiple MIP modes, the input vectors corresponding to multiple MIP modes can be determined. For example, if the input condition is a specified number of input samples, and the reference row and / or reference column do not meet the number of input samples for the MIP mode, the decoder reduces the dimensionality of the reference row and / or reference column to the specified number of input samples by performing Haar-downsampling or similar methods, and then determines the input vectors corresponding to multiple MIP modes based on the dimensionality-reduced specified number of input samples.

[0202] In some embodiments, S320 may include the following: The decoder determines the optimal MIP mode based on the differential absolute sum (SATD) corresponding to multiple MIP modes in the second template region.

[0203] In this embodiment, when the decoder determines the optimal MIP mode based on the strain costs corresponding to multiple MIP modes within a second template region, it designs the strain costs corresponding to multiple MIP modes into SATDs corresponding to multiple MIP modes. Compared to directly calculating the rate strain costs corresponding to multiple MIP modes, this not only enables determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes within a second template region, but also simplifies the complexity of calculating the strain costs corresponding to multiple MIP modes, thereby improving the decompression performance of the decoder.

[0204] In summary, the scheme described in this application proposes the idea of ​​fusion enhancement based on the optimal MIP mode. That is, the decoder not only needs to determine the optimal MIP mode for predicting the current block, but also needs to fuse other prediction blocks to achieve different prediction effects. This not only saves bit overhead but also generates new prediction techniques. Regarding the fusion process, the optimal MIP mode cannot completely replace the optimal prediction mode calculated by the encoding side based on rate distortion cost, so the fusion method is used to achieve both prediction accuracy and prediction diversity.

[0205] For example, the main idea behind a decoder-based template matching method for deriving MIP modes can be divided into the following parts:

[0206] First, the reference region (for example, the reference template shown in Figure 5) is padded with reconstructed samples, i.e., with the reference reconstruction samples necessary to predict the samples in the second template region (for example, the template shown in Figure 5). Selectively, the width and height of the reference region do not need to exceed the width and height of the second template region. If the width and height of the sample-padded reference region exceed the width and height of the second template region, downsampling or other dimensionality reduction methods must be performed until the input dimension requirements of the MIP are met.

[0207] Next, the decoder takes the reference reconstruction sample in the reference region, the indices of multiple MIP modes, and the MIP transpose flag as input to predict samples in a second template region and obtain prediction blocks corresponding to multiple MIP modes. Selectively, the reference reconstruction sample in the reference region must satisfy the input conditions of the MIP mode, for example, by performing dimensionality reduction using half-down sampling up to a specified number of input samples. The indices of multiple MIP modes are used to determine the matrix index of the MIP technique and to obtain the MIP prediction matrix coefficients. The MIP transpose flag is used to indicate whether or not the input and output need to be transposed.

[0208] Next, for prediction blocks corresponding to multiple MIP modes, all combinations of MIP modes and whether or not to transpose the MIP can be traversed to obtain prediction samples in a second template region under each state of each MIP mode and MIP transposition flag. The strain between the prediction samples and reconstructed samples in the second template region is calculated, and its cost information is recorded. Finally, following the principle of least strain, the MIP mode with the lowest cost and its corresponding MIP transposition information are selected, and the MIP mode with the lowest cost is set as the optimal MIP mode under the template matching-based MIP prediction derivation mode for the current block.

[0209] Finally, the decoder uses the optimal MIP prediction mode and the second intra prediction mode to predict the current block, obtaining the first and second predicted blocks, respectively. Based on the weights of the optimal MIP prediction mode and the second intra prediction mode, the decoder performs weight calculations on the first and second predicted blocks to obtain the predicted block for the current block.

[0210] Furthermore, some of the calculations in this application can be replaced with lookup table or shift methods. While the lookup table method may result in some errors compared to performing division directly, it is beneficial for controlling hardware implementation and coding costs. Examples of the above-mentioned calculations include calculations related to strain cost or calculations related to determining the optimal MIP mode.

[0211] The decoding method according to the embodiment of this application has been described in detail above from the perspective of the decoder. 10 Referring to the encoder angle, the encoding method according to the embodiment of this application will be described.

[0212] Figure 10 is a flowchart illustrating an encoding method 400 according to an embodiment of this application. This encoding method 400 can be executed by an encoder. For example, this encoding method 400 is applied to the encoding framework 100 shown in Figure 1. For the sake of clarity, an encoder will be described below as an example.

[0213] As shown in Figure 10, the encoding method 400 may include the following steps. S410: Retrieves the residual block of the current block in the current sequence. S420: Perform a third transformation on the residual block of the current block to obtain the third transformation coefficient of the current block. S430: Determine the first intra-prediction mode. The first intra-prediction mode includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for the prediction block of the current block, an intra-prediction mode derived from a DIMD mode used for the output vector of the optimal matrix-based intra-prediction (MIP) mode for predicting the current block, and one of the following: an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region adjacent to the current block, and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. S440: Based on the transformation set corresponding to the first intra prediction mode, the fourth transformation is performed on the third transformation coefficient to obtain the fourth transformation coefficient for the current block. S450: Encode the fourth conversion coefficient.

[0214] It should be understood that the first transformation on the decoding side is the inverse transformation of the fourth transformation on the encoding side, and the second transformation on the decoding side is the inverse transformation of the third transformation on the encoding side. For example, the third transformation is the basic or linear transformation described above, and the fourth transformation is the quadratic transformation described above, and correspondingly, the first transformation is the inverse transformation of the quadratic transformation, and the second transformation is the inverse transformation of the basic or linear transformation. For example, the first transformation can be the inverse LFNST, and the second transformation can be the inverse DCT2, inverse DCT8, inverse DST7, etc., and correspondingly, the third transformation can be the DCT2, DCT8, DST7, etc., and the fourth transformation can be the LFNST.

[0215] In some embodiments, the output vector of the optimal MIP mode is the vector before upsampling the output vector of the optimal MIP mode, or the output vector of the optimal MIP mode is the vector after upsampling the output vector of the optimal MIP mode.

[0216] In some embodiments, S430 may include determining a first intra-prediction mode based on a prediction mode for predicting the current block.

[0217] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode for predicting the current block, then the intra-prediction mode derived from the DIMD mode used for predicting the current block is determined as the first intra-prediction mode, or the intra-prediction mode derived from the DIMD mode used for the output vector of the optimal MIP mode is determined as the first intra-prediction mode.

[0218] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the TIMD mode, then the intra-prediction mode derived from the DIMD mode is determined as the first intra-prediction mode to be used for predicting the current block, or the intra-prediction mode derived from the TIMD mode is determined as the first intra-prediction mode.

[0219] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region, then the intra-prediction mode derived from a DIMD mode used for predicting the current block is determined as the first intra-prediction mode, or the intra-prediction mode derived from a DIMD mode used for reconstructed samples in a first template region is determined as the first intra-prediction mode.

[0220] In some embodiments, the second template region and the first template region may be the same or different.

[0221] In some embodiments, S410 may further include the following: Confirm the second intra-prediction mode. The second intra-prediction mode includes a suboptimal MIP mode for predicting the current block, and one of the following intra-prediction modes used for reconstruction samples within the first template region: an intra-prediction mode derived from the DIMD mode and an intra-prediction mode derived from the TIMD mode. Based on the optimal MIP mode and a second intra-prediction mode, the current block is predicted and the predicted block for the current block is obtained. Based on the predicted block size of the current block, obtain the residual block size of the current block.

[0222] In some embodiments, the current block is predicted based on the optimal MIP mode to obtain a first predicted block. The current block is predicted based on a second intra-prediction mode to obtain a second predicted block. The first and second predicted blocks are weighted based on the weights of the optimal MIP mode and the second intra-prediction mode to obtain a predicted block for the current block.

[0223] In some embodiments, if the prediction mode for predicting the current block includes an optimal MIP mode, a suboptimal MIP mode for predicting the current block, or an intra-prediction mode derived from a TIMD mode, the weights of the optimal MIP mode and the second intra-prediction mode are determined based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the second intra-prediction mode. If the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for reconstruction samples in the first template region, the weights of the optimal MIP mode and the second intra-prediction mode are both determined to be preset values.

[0224] In some embodiments, the encoder obtains a first flag and determines a second intra-prediction mode if the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and a second intra-prediction mode. S450 may include encoding a fourth conversion coefficient and the first flag.

[0225] In some embodiments, if a first flag is used to indicate that it is permitted to predict an image block in the current sequence using an optimal MIP mode and a second intra-prediction mode, the current block is predicted based on the optimal MIP mode and the second intra-prediction mode to obtain a first rate distortion cost. The current block is predicted based on at least one intra-prediction mode to obtain at least one rate distortion cost. If the first rate distortion cost is less than or equal to the minimum of at least one rate distortion cost, the predicted block obtained by predicting the current block based on the optimal MIP mode and the second intra-prediction mode is confirmed as the predicted block of the current block. S450 may include the following: Encode the fourth conversion coefficient, the first flag, and the second flag. If the first rate distortion cost is less than or equal to the minimum of at least one rate distortion cost, the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode; if the first rate distortion cost is greater than the minimum of at least one rate distortion cost, the second flag is used to indicate that it is not permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode.

[0226] In some embodiments, the method 400 may further include: determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes; the strain costs corresponding to multiple MIP modes include strain costs obtained by using multiple MIP modes to predict samples in a second template region adjacent to the current block.

[0227] In some embodiments, the second template region and the first template region may be the same or different.

[0228] In some embodiments, samples within a second template region are predicted based on a third flag and multiple MIP modes to obtain strain costs corresponding to multiple MIP modes under each state of the third flag. The third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP modes. Based on the strain costs corresponding to multiple MIP modes under each state of the third flag, the optimal MIP mode is determined.

[0229] In some embodiments, before determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes, the method 400 may further include: obtaining the MIP mode used for adjacent blocks adjacent to the current block; and determining the MIP modes used for adjacent blocks as multiple MIP modes.

[0230] In some embodiments, before determining the optimal MIP mode based on the strain costs corresponding to multiple MIP modes, the method 400 may further include: Reconstruction sample padding is performed on a reference region adjacent to the outside of the second template region to obtain reference rows and reference columns of the second template region; samples in the second template region are predicted using each of the multiple MIP modes with the reference rows and reference columns as input to obtain multiple prediction blocks corresponding to the multiple MIP modes; strain costs corresponding to the multiple MIP modes are determined based on the multiple prediction blocks and the reconstruction blocks in the second template region.

[0231] In some embodiments, the reference row and reference column are downsampled to obtain an input vector. Using the input vector as input, samples in a second template region are predicted by traversing multiple MIP modes to obtain output vectors corresponding to multiple MIP modes. The output vectors corresponding to multiple MIP modes are upsampled to obtain prediction blocks corresponding to multiple MIP modes.

[0232] In some embodiments, the optimal MIP mode is determined based on the differential transformation absolute sum (SATD) corresponding to multiple MIP modes within a second template region.

[0233] Furthermore, the encoding method can be understood as the reverse process of the decoding method. Therefore, the specific scheme of the encoding method 400 can be found in the relevant section of the decoding method 300, and for the sake of clarity, it will not be described in detail in this application.

[0234] The proposed solution of this application will be described below in relation to specific embodiments.

[0235] <Example 1>

[0236] In this embodiment, the second intra prediction mode is a suboptimal prediction mode. That is, the encoder or decoder can perform intra prediction on the current block based on the optimal MIP mode and the suboptimal MIP mode to obtain a predicted block for the current block.

[0237] The encoder traverses the prediction mode. If intra-mode is used for the current block, the encoder obtains a sequence-level enable flag, such as sps_tmmip_enable_flag. Sequence-level enable flags are used to indicate whether template matching-based MIP mode derivation techniques are permitted for the current sequence. If all tmmip enable flags are true, it indicates that the encoder is currently permitted to use TMMIP techniques.

[0238] For example, the encoder process can be implemented as follows:

[0239] Step 1: If sps_tmmip_enable_flag is true, the encoder attempts the TMMIP technique, i.e., performs Step 2. If sps_tmmip_enable_flag is false, the encoder does not attempt the TMMIP technique, i.e., skips Step 2 and performs Step 3 directly.

[0240] Step 2: First, the encoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template area. The padding process is the same as the padding method of the original intra-prediction process. For example, the encoder can traverse from the bottom left corner to the top right corner to pad. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value for all of them. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the encoder traverses from the bottom left corner to the top right corner in order until the first available reconstructed sample appears, and then uses the first available reconstructed sample to pad for the previous unavailable position. Next, the encoder takes the reconstructed samples outside the padded second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0241] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0242] As an example, the specific prediction calculation process includes the following: First, the encoder performs half-downsampling on the reconstructed samples, for example, determining the down-sampling step size based on the block size. Next, the encoder adjusts the splicing order of the upper and left down-sampled reconstructed samples depending on whether transposition is required. If transposition is not required, the left down-sampled reconstructed sample is spliced ​​after the upper down-sampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper down-sampled reconstructed sample is spliced ​​after the left down-sampled reconstructed sample, and the resulting vector is used as input. Next, the encoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the encoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are arranged sequentially according to the horizontal direction and output as prediction blocks in the template area. If upsampling is required, first upsampling is performed horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0243] Next, the encoder calculates the strain cost based on the predicted blocks in the second template region obtained by traversing each MIP mode and the reconstructed samples in the second template region, and records the strain cost under each predicted mode and transpose information. After traversing all permitted predicted modes and transpose information, the encoder selects the optimal MIP mode and its corresponding transpose information, and the suboptimal MIP mode and its corresponding transpose information, according to the principle of least cost. The encoder determines whether fusion enhancement is necessary based on the relationship between the cost of the optimal MIP mode and the cost of the suboptimal MIP mode. If the cost of the suboptimal MIP mode is less than twice the cost of the optimal MIP mode, the optimal MIP predicted block and the suboptimal MIP predicted block need to be fused and enhanced. If the cost of the suboptimal MIP predicted mode is twice or more the cost of the optimal MIP mode, fusion enhancement is not necessary.

[0244] Finally, if fusion enhancement is required, the encoder obtains prediction blocks corresponding to the optimal MIP mode and prediction blocks corresponding to the suboptimal MIP mode based on the optimal MIP mode, the suboptimal MIP mode, the transpose information of the optimal MIP mode, and the transpose information of the suboptimal MIP mode. Specifically, first, the encoder may downsample the reconstructed samples adjacent to the top and left of the current block, splice them according to the transpose information to obtain the input vector, read the matrix coefficients under the current mode using the MIP mode as the index, and then obtain the output vector by calculating the input vector and matrix coefficients. The encoder transposes the output based on the transpose information, upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain the optimal MIP prediction block and the suboptimal MIP prediction block of the same size as the current block, and also performs a weighted average on the optimal MIP prediction block and the suboptimal MIP prediction block based on the calculated weight values ​​of the optimal MIP mode and the weight values ​​of the suboptimal MIP mode to obtain a new prediction block which becomes the final prediction block for the current block. If fusion enhancement is not required, the encoder can calculate the optimal MIP prediction block based on the optimal MIP mode and its transposition information, and the calculation process is the same as described above. Finally, the encoder sets the optimal MIP prediction block as the prediction block for the current block.

[0245] Furthermore, the encoder retrieves the rate distortion cost of the current block and records it as cost1.

[0246] The encoder either determines the intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or the encoder determines the intra-prediction mode derived from the DIMD mode to be used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0247] Step 3: The encoder continues to traverse other intra-prediction techniques and calculates the corresponding rate distortion costs, denoted as cost2...costN.

[0248] Step 4: If cost1 is the smallest of all rate distortion costs, use the TMMIP technique for the current block, set the TMMIP usage flag for the current block to true, and write it to the bitstream. If cost1 is not the smallest rate distortion cost, use another intra-prediction technique for the current block, set the TMMIP usage flag for the current block to false, and write it to the bitstream. Note that information such as flags or indices for other intra-prediction techniques is transmitted based on their definitions and is not described in detail here.

[0249] Step 5: The encoder determines the residual block of the current block based on the predicted block and the original block of the current block, performs a basic transformation on the residual block of the current block, and then performs a quadratic transformation on the transformation coefficients after the basic transformation based on the first intra-prediction mode, and then performs operations such as quantization, entropy coding, and loop filtering on the transformation coefficients after the quadratic transformation. The specific process of quantization can be found in the related content above and will not be described in detail here to avoid repetition.

[0250] The decoder-related scheme in this embodiment is described below.

[0251] The decoder parses block-level flags. If intra-mode is used for the current block, the decoder parses or retrieves sequence-level permission flags such as sps_tmmip_enable_flag. Sequence-level permission flags are used to indicate whether the current sequence is permitted to use template matching-based MIP mode derivation techniques. If all tmmip permission flags are true, it indicates that the decoder is currently permitted to use TMMIP techniques.

[0252] For example, the decoder process can be implemented as follows:

[0253] Step 1: If sps_tmmip_enable_flag is true, the decoder parses the TMMIP usage flag for the current block. Otherwise, the current decoding process does not need to decode the block-level TMMIP usage flag, and the block-level TMMIP usage flag is set to false by default. If the TMMIP usage flag for the current block is true, Step 2 is performed. Otherwise, Step 3 is performed.

[0254] Step 2: First, the decoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template area. The padding process is the same as the padding method of the original intra-prediction process. For example, the decoder can traverse from the bottom left corner to the top right corner to pad. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value for all of them. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the decoder traverses from the bottom left corner to the top right corner until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad for the previous unavailable position. Next, the decoder takes the reconstructed samples outside the padded second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0255] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0256] As an example, the specific prediction calculation process includes the following: First, the decoder performs half downsampling on the reconstructed samples. For example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splicing order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample depending on whether transposition is required. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is used as input. Next, the decoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the decoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are sequentially arranged horizontally and output as prediction blocks in the template region. If upsampling is required, upsampling is performed first horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0257] Next, the decoder calculates the strain cost based on the predicted blocks in the second template region obtained by traversing each MIP mode and the reconstructed samples in the second template region, and records the strain cost under each predicted mode and transpose information. After traversing all permitted predicted modes and transpose information, the decoder selects the optimal MIP mode and its corresponding transpose information, and the suboptimal MIP mode and its corresponding transpose information, according to the principle of least cost. The decoder determines whether fusion enhancement is necessary based on the relationship between the cost of the optimal MIP mode and the cost of the suboptimal MIP mode. If the cost of the suboptimal MIP mode is less than twice the cost of the optimal MIP mode, the optimal MIP predicted block and the suboptimal MIP predicted block need to be fused and enhanced. If the cost of the suboptimal predicted mode is twice or more the cost of the optimal MIP mode, fusion enhancement is not necessary.

[0258] Finally, if fusion enhancement is required, the decoder obtains prediction blocks corresponding to the optimal MIP mode and prediction blocks corresponding to the suboptimal MIP mode based on the optimal MIP mode, the suboptimal MIP mode, the transpose information of the optimal MIP mode, and the transpose information of the suboptimal MIP mode. Specifically, first, the decoder may downsample the reconstructed samples adjacent to the top and left of the current block, splice them according to the transpose information to obtain the input vector, read out the matrix coefficients under the current mode using the MIP mode as the index, and then obtain the output vector by calculating the input vector and matrix coefficients. The decoder transposes the output based on the transpose information, upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain the optimal MIP prediction block and the suboptimal MIP prediction block of the same size as the current block, and also performs a weighted average on the optimal MIP prediction block and the suboptimal MIP prediction block based on the calculated weight values ​​of the optimal MIP mode and the weight values ​​of the suboptimal MIP mode to obtain a new prediction block which becomes the final prediction block for the current block. If fusion enhancement is not required, the decoder can calculate the optimal MIP prediction block based on the optimal MIP mode and its transposition information, and the calculation process is the same as described above. Finally, the decoder sets the optimal MIP prediction block as the prediction block for the current block.

[0259] The decoder then determines either the intra-prediction mode derived from the DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or the intra-prediction mode derived from the DIMD mode used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0260] Step 3: The decoder continues to analyze information such as the use flags or indices of other intra-prediction techniques and determines the final predicted block for the current block based on the analyzed information.

[0261] Step 4: The decoder analyzes the bitstream to obtain the frequency-domain residual block of the current block (also referred to as frequency-domain residual information), and performs inverse quantization and inverse transformation on the frequency-domain residual block of the current block (first, perform inverse transformation on the secondary transformation based on the first intra prediction mode, and then perform inverse Exchange transformation on the base transformation or primary transformation), to obtain the residual block of the current block (also referred to as the time-domain residual block or time-domain residual information). Next, the decoder adds the prediction block of the current block to the residual block of the current block to obtain the reconstructed sample block.

[0262] Step 5: After performing techniques such as loop filtering on all the reconstructed sample blocks in the current image, the final reconstructed image is obtained.

[0263] Optionally, the reconstructed image may be used as a video output, or may be used as a reference for subsequent decoding.

[0264] In this embodiment, the size of the second template area used by the encoder or decoder using the TMMIP technology can be predefined according to the size of the current block. For example, the width of the area adjacent to the upper side of the current block within the second template area is equal to the width of the current block, and its height is equal to the height of two rows of samples. The height of the area adjacent to the left side of the current block within the second template area is equal to the height of the current block, and its width is 2 column sample widths. Of course, in other alternative embodiments, it can be realized as a second template area of other sizes, and this application does not specifically limit it.

[0265] In this embodiment, the TMMIP technology is adopted.

[0266] <Example 2>

[0267] In this embodiment, the second intra-prediction mode is an intra-prediction mode derived from the TIMD mode. That is, the encoder or decoder can perform an intra-prediction on the current block based on the optimal MIP mode and the intra-prediction mode derived from the TIMD mode to obtain a predicted block for the current block.

[0268] In other words, the template matching-based MIP mode derivation fusion enhancement technology can not only fuse two derived MIP prediction blocks, but also fuse them with prediction blocks generated by other template matching-based derivation technologies. This application provides a method for fusing derived conventional prediction blocks with matrix-based prediction blocks by fusing TMMIP technology and TIMD technology. TIMD technology uses the idea of ​​template matching on both the encoding and decoding sides to derive the optimal conventional intra-prediction mode, and can also perform offset expansion on this prediction mode to obtain an updated intra-prediction mode. TMMIP technology also uses the idea of ​​template matching on both the encoding and decoding sides to derive the optimal MIP mode. By fusing these two optimal prediction modes, it is possible to reconcile the directivity of conventional prediction blocks with the unique texture characteristics of MIP prediction, thereby generating entirely new prediction blocks and improving coding efficiency.

[0269] The encoder traverses the prediction mode. If intra-mode is used for the current block, the encoder obtains a sequence-level permission flag such as sps_tmmip_enable_flag. Sequence-level permission flags are used to indicate whether the current sequence is permitted to use template matching-based MIP mode derivation techniques. If all tmmip permission flags are true, it indicates that the encoder is currently permitted to use TMMIP techniques.

[0270] For example, the encoder process can be implemented as follows:

[0271] Step 1: If sps_tmmip_enable_flag is true, the encoder attempts the TMMIP technique, i.e., performs Step 2. If sps_tmmip_enable_flag is false, the encoder does not attempt the TMMIP technique, i.e., skips Step 2 and performs Step 3 directly.

[0272] Step 2: First, the encoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template region. The padding process is the same as the padding method in the original intra-prediction process. For example, the encoder can traverse from the bottom left corner to the top right corner to pad. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the encoder traverses from the bottom left corner to the top right corner until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad for the previous unavailable position. Next, the encoder uses the padded reconstructed samples outside the second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0273] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0274] As an example, the specific prediction calculation process includes the following: First, the encoder performs half-downsampling on the reconstructed samples. For example, the encoder determines the downsampling step size based on the block size. Next, the encoder adjusts the splicing order of the upper and left downsampled reconstructed samples depending on whether transposition is required. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is used as input. Next, the encoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the encoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are sequentially arranged horizontally and output as prediction blocks in the template area. If upsampling is required, upsampling is performed first horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0275] Furthermore, the encoder needs to attempt TIMD's template matching calculation process, obtaining different interpolation filters based on different prediction mode indices and retrieving predicted samples within the template by interpolating the reference samples.

[0276] Next, the encoder calculates the strain cost based on the predicted samples and reconstructed samples in the second template region obtained by traversing each MIP mode, and records the strain cost under each predicted mode and transpose information. Based on the strain cost under each predicted mode and transpose information, the encoder selects the optimal MIP mode and its corresponding transpose information according to the principle of least cost. Furthermore, the encoder must traverse all permitted intra-prediction modes in TIMD, calculate the predicted samples in the template, calculate the strain cost using the predicted samples and reconstructed samples in the template, and record the optimal predicted mode, suboptimal predicted mode, strain cost corresponding to the optimal predicted mode, and strain cost corresponding to the suboptimal predicted mode derived from the TIMD technique, according to the principle of least cost.

[0277] Finally, based on the obtained optimal MIP mode and transpose information, the encoder optionally downsamples the reconstructed samples adjacent to the top and left of the current block, splices them based on the transpose information to obtain the input vector, reads out the matrix coefficients under the current mode using the MIP mode as the index, and then obtains the output vector by calculating the input vector and matrix coefficients. The encoder transposes the output based on the transpose information, and upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain an output of the same size as the current block, which can be used as the optimal MIP prediction block for the current block.

[0278] For the optimal and suboptimal prediction modes derived from TIMD technology, if neither the optimal nor the suboptimal prediction mode is the mean value (DC) mode or the planar mode, and the strain cost corresponding to the suboptimal prediction mode is less than twice the strain cost corresponding to the optimal prediction mode, the encoder needs to merge the prediction blocks. First, the encoder obtains interpolation filtering coefficients based on the optimal prediction mode and performs interpolation filtering on the reconstructed samples adjacent to the upper and left to obtain prediction samples at all positions in the current block, which are recorded as the optimal prediction block. Next, the encoder obtains interpolation filtering coefficients based on the suboptimal prediction mode and performs interpolation filtering on the reconstructed samples adjacent to the upper and left to obtain prediction samples at all positions in the current block, which are recorded as the suboptimal prediction block. Furthermore, the encoder uses the ratio of the cost corresponding to the optimal prediction mode to the cost corresponding to the suboptimal prediction mode to calculate the weight values ​​of the optimal prediction block and the weight values ​​of the suboptimal prediction block. Finally, the encoder weight-merges the optimal and suboptimal prediction blocks to obtain the prediction block for the current block, which is output. Furthermore, if the optimal or suboptimal prediction mode is the mean mode (DC) or planar mode (PLANAR), or if the cost corresponding to the suboptimal prediction mode is greater than twice the cost corresponding to the optimal prediction mode, the encoder does not need to merge prediction blocks and uses only the optimal prediction mode to perform interpolation filtering on the reconstructed samples adjacent to the upper and left, and the resulting optimal prediction block is considered the optimal TIMD prediction block for the current block.

[0279] Finally, the encoder performs a weighted average on the optimal MIP prediction block and the optimal TIMD prediction block, based on the calculated optimal MIP mode weights and the prediction mode weights derived from the TIMD technique, to obtain a new prediction block. This new prediction block is the prediction block of the current block.

[0280] Furthermore, the encoder retrieves the rate distortion cost of the current block and records it as cost1.

[0281] The encoder either determines the intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or the encoder determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0282] Furthermore, the template region in TIMD technology and the second template region (i.e., the template region in TMMIP technology) can be set to be the same, meaning that the template regions for calculating strain costs are the same. Therefore, the cost information of the template region in TIMD technology and the cost information of the template region in TMMIP technology can be equivalent or at the same comparison level. In this case, it can be determined whether or not fusion strengthening is performed based on the cost information, and this application is not specifically limited.

[0283] Step 3: The encoder continues to traverse other intra-prediction techniques and calculates the corresponding rate distortion costs, denoted as cost2...costN.

[0284] Step 4: If cost1 is the smallest of all rate distortion costs, use the TMMIP technique for the current block, set the TMMIP usage flag for the current block to true, and write it to the bitstream. If cost1 is not the smallest rate distortion cost, use another intra-prediction technique for the current block, set the TMMIP usage flag for the current block to false, and write it to the bitstream. Note that information such as flags or indices for other intra-prediction techniques is transmitted based on their definitions and is not described in detail here.

[0285] Step 5: The encoder determines the residual block of the current block based on the predicted block of the current block and the original block of the current block, performs a base transform on the residual block of the current block, and performs a secondary transform on the transform coefficients after the base transform based on the first intra prediction mode, and then performs operations such as quantization, entropy coding, and loop filtering on the transform coefficients after the secondary transform. For the specific process of quantization, the relevant content above can be referred to, and for the sake of avoiding duplication, it will not be elaborated here.

[0286] The related scheme of the decoder in this embodiment will be described below.

[0287] The decoder analyzes the block-level flag. When the intra mode is used for the current block, the decoder analyzes or obtains the sequence-level usage permission flag such as sps_tmmip_enable_flag. The sequence-level usage permission flag is used to indicate whether the template matching-based MIP mode derivation technology is permitted to be used in the current sequence. When all the usage permission flags of tmmip are true, it indicates that currently the TMMIP technology is permitted to be used by the decoder.

[0288] Exemplarily, the process of the decoder can be realized as follows.

[0289] Step 1: When sps_tmmip_enable_flag is true, the decoder analyzes the TMMIP usage flag of the current block. Otherwise, in the current decoding process, it is not necessary to decode the block-level TMMIP usage flag, and the block-level TMMIP usage flag is default set to false. When the TMMIP usage flag of the current block is true, Step 2 is executed. Otherwise, Step 3 is executed.

[0290] Step 2: First, the decoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template region. The padding process is the same as the padding method in the original intra-prediction process. For example, the decoder can pad by traversing from the bottom left corner to the top right corner. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the decoder traverses from the bottom left corner to the top right corner until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad for the previous unavailable position. Next, the decoder takes the padded reconstructed samples outside the second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0291] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0292] As an example, the specific prediction calculation process includes the following: First, the decoder performs half downsampling on the reconstructed samples. For example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splicing order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample depending on whether transposition is required. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is used as input. Next, the decoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the decoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are sequentially arranged horizontally and output as prediction blocks in the template region. If upsampling is required, upsampling is performed first horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0293] Furthermore, the decoder needs to attempt TIMD's template matching calculation process, obtaining different interpolation filters based on different prediction mode indices and retrieving predicted samples within the template by interpolating the reference samples.

[0294] Next, the decoder calculates the strain cost based on the predicted samples and reconstructed samples in the second template region obtained by traversing each MIP mode, and records the strain cost under each predicted mode and transpose information. Based on the strain cost under each predicted mode and transpose information, the decoder selects the optimal MIP mode and its corresponding transpose information according to the principle of least cost. Furthermore, the decoder must traverse all permitted intra-prediction modes in TIMD, calculate the predicted samples in the template, calculate the strain cost using the predicted samples and reconstructed samples in the template, and record the optimal prediction mode, suboptimal prediction mode, strain cost corresponding to the optimal prediction mode, and strain cost corresponding to the suboptimal prediction mode derived from the TIMD technique, according to the principle of least cost.

[0295] Finally, based on the obtained optimal MIP mode and transpose information, the decoder optionally downsamples the reconstructed samples adjacent to the top and left of the current block, splices them based on the transpose information to obtain the input vector, reads out the matrix coefficients under the current mode using the MIP mode as the index, and then obtains the output vector by calculating the input vector and matrix coefficients. The decoder transposes the output based on the transpose information, upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain an output of the same size as the current block, which can then be used as the optimal MIP prediction block for the current block.

[0296] For the optimal and suboptimal prediction modes derived from TIMD technology, if neither the optimal nor the suboptimal prediction mode is the mean value (DC) mode or the planar mode, and the strain cost corresponding to the suboptimal prediction mode is less than twice the strain cost corresponding to the optimal prediction mode, the decoder needs to merge the prediction blocks. First, the decoder obtains interpolation filtering coefficients based on the optimal prediction mode and performs interpolation filtering on the reconstructed samples adjacent to the upper and left to obtain prediction samples at all positions in the current block, which are recorded as the optimal prediction block. Next, the decoder obtains interpolation filtering coefficients based on the suboptimal prediction mode and performs interpolation filtering on the reconstructed samples adjacent to the upper and left to obtain prediction samples at all positions in the current block, which are recorded as the suboptimal prediction block. Furthermore, the decoder uses the ratio of the cost corresponding to the optimal prediction mode to the cost corresponding to the suboptimal prediction mode to calculate the weight values ​​of the optimal prediction block and the weight values ​​of the suboptimal prediction block. Finally, the decoder weight-merges the optimal and suboptimal prediction blocks to obtain the prediction block for the current block, which is output. Furthermore, if the optimal or suboptimal prediction mode is the mean mode (DC) or planar mode (PLANAR), or if the cost corresponding to the suboptimal prediction mode is greater than twice the cost corresponding to the optimal prediction mode, the decoder does not need to merge prediction blocks and uses only the optimal prediction mode to perform interpolation filtering on the reconstructed samples adjacent to the upper and left, and the resulting optimal prediction block is considered the optimal TIMD prediction block for the current block.

[0297] Finally, the decoder obtains a new prediction block by performing a weighted average on the optimal MIP prediction block and the optimal TIMD prediction block, based on the calculated optimal MIP mode weights and the prediction mode weights derived from the TIMD technique. This new prediction block is the prediction block for the current block.

[0298] The decoder either determines the intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or the decoder determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0299] Step 3: The decoder continues to analyze information such as the use flags or indices of other intra-prediction techniques and determines the final predicted block for the current block based on the analyzed information.

[0300] Step 4: The decoder analyzes the bitstream to obtain the frequency domain residual block (also called frequency domain residual information) of the current block, and performs inverse quantization and inverse transformation on the frequency domain residual block of the current block (first, an inverse transformation is performed on the quadratic transformation based on the first intra-prediction mode, and then an inverse transformation is performed on the basic transformation or linear transformation). Exchange The decoder then obtains the residual block of the current block (also called the time-domain residual block or time-domain residual information). Next, the decoder obtains the reconstructed sample block by adding the predicted block of the current block to the residual block of the current block.

[0301] Step 5: After techniques such as loop filtering are performed on all reconstruction sample blocks in the current image, the final reconstructed image is obtained.

[0302] Selectively, the reconstructed image may be used as a video output or as a reference for subsequent decoding.

[0303] In this embodiment, the process for calculating weight values ​​for weighted fusion of TIMD prediction blocks can be found in the description of the TIMD technology described above, and will not be detailed here to avoid duplication. Furthermore, the encoder or decoder can determine whether or not fusion enhancement is performed based on the optimal prediction mode derived from TIMD. For example, if the optimal prediction mode derived from TIMD is DC mode or PLANAR mode, the encoder or decoder does not need to use fusion enhancement. That is, the encoder or decoder uses only prediction blocks generated by the optimal MIP mode derived from the TMMIP technology as the prediction blocks for the current block. Furthermore, the size of the second template region used by the encoder or decoder in the TMMIP technology can be predefined according to the size of the current block. For example, the definition of the second template region in the TMMIP technology may be the same as or different from the definition of the template region in the TIMD technology. For example, if the width of the current block is 8 or less, the height of the region adjacent to the top of the current block within the second template region is equal to the height of two rows of samples. Otherwise, the height is equal to the height of four rows of samples. Similarly, if the height of the current block is 8 or less, the width of the area adjacent to the left of the current block in the second template area is 2 columns of samples. width It is equal to . Otherwise, the width is 4 columns of samples width It is equal to.

[0304] <Example 3>

[0305] In this embodiment, the second intra-prediction mode described above is an intra-prediction mode derived from the DIMD mode. That is, the encoder or decoder can perform an intra-prediction on the current block based on the optimal MIP mode and the intra-prediction mode derived from the DIMD mode, which is used for the reconstruction sample in the first template region adjacent to the current block, to obtain a predicted block for the current block.

[0306] Similar to Example 2, TMMIP technology can also be integrated and enhanced with DIMD technology.

[0307] Note that while both the prediction modes derived from DIMD technology and TIMD technology are conventional intra-prediction modes, the derivation methods differ, so the prediction modes obtained by both are not necessarily the same. Also, the method of fusion enhancement between TMMIP technology and DIMD technology differs from the method of fusion enhancement between TMMIP technology and TIMD technology. For example, in TMMIP technology and TIMD technology, the size of the second template region is generally the same, and the calculated cost information is basically SATD (Sum of Absolute Transformed Difference), so it is also called a strain cost based on the Hadamard transform. In TMMIP technology and TIMD technology, the fusion weights can be calculated directly based on this cost information. However, the size of the second template region in DIMD technology is generally the same as in TMMIP technology (or TIMD The size of the second template region in the technology differs, and the rule for the DIMD derived prediction mode is based on the gradient amplitude value. Since the gradient amplitude value is not equivalent to the SATD cost, the weights cannot be easily calculated by referring to the scheme when fusing the TMMIP and TIMD technologies.

[0308] The encoder traverses the prediction mode. If intra-mode is used for the current block, the encoder obtains a sequence-level permission flag such as sps_tmmip_enable_flag. Sequence-level permission flags are used to indicate whether the current sequence is permitted to use template matching-based MIP mode derivation techniques. If all tmmip permission flags are true, it indicates that the encoder is currently permitted to use TMMIP techniques.

[0309] For example, the encoder process can be implemented as follows:

[0310] Step 1: If sps_tmmip_enable_flag is true, the encoder attempts the TMMIP technique, i.e., performs Step 2. If sps_tmmip_enable_flag is false, the encoder does not attempt the TMMIP technique, i.e., skips Step 2 and performs Step 3 directly.

[0311] Step 2: First, the encoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template region. The padding process is the same as the padding method in the original intra-prediction process. For example, the encoder can traverse from the bottom left corner to the top right corner to pad. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the encoder traverses from the bottom left corner to the top right corner until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad for the previous unavailable position. Next, the encoder uses the padded reconstructed samples outside the second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0312] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0313] As an example, the specific prediction calculation process includes the following: First, the encoder performs half-downsampling on the reconstructed samples. For example, the encoder determines the downsampling step size based on the block size. Next, the encoder adjusts the splicing order of the upper and left downsampled reconstructed samples depending on whether transposition is required. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is used as input. Next, the encoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the encoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are sequentially arranged horizontally and output as prediction blocks in the template area. If upsampling is required, upsampling is performed first horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0314] Furthermore, the encoder utilizes DIMD technology to derive the optimal intra-prediction mode, i.e., the optimal DIMD mode. In DIMD technology, the gradient value of the reconstructed sample in the first template region is calculated based on the Sobel operator, and the gradient value is transformed based on the angle value corresponding to the different prediction modes to obtain the amplitude value under the corresponding prediction mode.

[0315] Next, the encoder calculates the strain cost using the predicted blocks of the template obtained by traversing each MIP mode and the reconstructed samples within the template, and records the optimal MIP mode and transpose information according to the principle of least cost. Furthermore, the encoder traverses all intra-prediction modes that are permitted to be used, calculates the amplitude value under each intra-prediction mode, and records the optimal DIMD prediction mode according to the principle of maximum amplitude.

[0316] Finally, based on the obtained optimal MIP mode and transpose information, the encoder optionally downsamples the reconstructed samples adjacent to the upper and left sides of the current block, splices them based on the transpose information to obtain the input vector, reads the matrix coefficients under the current mode using the MIP mode as the index, and then obtains the output vector by calculating the input vector and matrix coefficients. The encoder transposes the output based on the transpose information, upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain an output of the same size as the current block, which can be the optimal MIP prediction block for the current block. Furthermore, for the optimal DIMD prediction mode, the encoder obtains the corresponding interpolation filtering coefficients, performs interpolation filtering on the reconstructed samples adjacent to the upper and left sides to obtain prediction samples at all positions within the current block, and records this as the optimal DIMD prediction block. The encoder obtains a new prediction block by weighted averaging each prediction sample in the optimal MIP prediction block and the optimal DIMD prediction block according to preset weights. The new prediction block is the prediction block for the current block.

[0317] Furthermore, the encoder retrieves the rate distortion cost of the current block and records it as cost1.

[0318] The encoder then determines either the intra-prediction mode derived from the DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or the intra-prediction mode derived from the DIMD mode used for the reconstruction sample in the first template region as the first intra-prediction mode.

[0319] Step 3: The encoder continues to traverse other intra-prediction techniques and calculates the corresponding rate distortion costs, denoted as cost2...costN.

[0320] Step 4: If cost1 is the smallest of all rate distortion costs, use the TMMIP technique for the current block, set the TMMIP usage flag for the current block to true, and write it to the bitstream. If cost1 is not the smallest rate distortion cost, use another intra-prediction technique for the current block, set the TMMIP usage flag for the current block to false, and write it to the bitstream. Note that information such as flags or indices for other intra-prediction techniques is transmitted based on their definitions and is not described in detail here.

[0321] Step 5: The encoder determines the residual block of the current block based on the predicted block and the original block of the current block, performs a basic transformation on the residual block of the current block, and then performs a quadratic transformation on the transformation coefficients after the basic transformation based on the first intra-prediction mode, and then performs operations such as quantization, entropy coding, and loop filtering on the transformation coefficients after the quadratic transformation. The specific process of quantization can be found in the related content above and will not be described in detail here to avoid repetition.

[0322] The following describes the decoder-related solutions in this embodiment.

[0323] The decoder parses block-level flags. If intra-mode is used for the current block, the decoder parses or retrieves sequence-level permission flags such as sps_tmmip_enable_flag. Sequence-level permission flags are used to indicate whether the current sequence is permitted to use template matching-based MIP mode derivation techniques. If all tmmip permission flags are true, it indicates that the decoder is currently permitted to use TMMIP techniques.

[0324] For example, the decoder process can be implemented as follows:

[0325] Step 1: If sps_tmmip_enable_flag is true, the decoder parses the TMMIP usage flag for the current block. Otherwise, the current decoding process does not need to decode the block-level TMMIP usage flag, and the block-level TMMIP usage flag is set to false by default. If the TMMIP usage flag for the current block is true, Step 2 is performed. Otherwise, Step 3 is performed.

[0326] Step 2: First, the decoder performs reconstructed sample padding on the rows and columns adjacent to the outside of the second template region. The padding process is the same as the padding method in the original intra-prediction process. For example, the decoder can pad by traversing from the bottom left corner to the top right corner. If all reconstructed samples are available, padding is performed sequentially with all available reconstructed samples. If all reconstructed samples are unavailable, padding is performed with the average value. If some reconstructed samples are available, padding is performed first with the available reconstructed samples, and for the remaining unavailable reconstructed samples, the decoder traverses from the bottom left corner to the top right corner until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad for the previous unavailable position. Next, the decoder takes the padded reconstructed samples outside the second template region as input and predicts the samples within the second template region using the permitted MIP mode.

[0327] For example, there are 16 MIP modes permitted for use with 4x4 blocks. There are 8 MIP modes permitted for use with blocks whose width or height is equal to 4, or with 8x8 blocks. There are 6 MIP modes permitted for use with blocks of other sizes. In addition, the MIP transpose function can be used with blocks of any size, and the TMMIP prediction modes described above are the same as those used in MIP technology.

[0328] As an example, the specific prediction calculation process includes the following: First, the decoder performs half downsampling on the reconstructed samples. For example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splicing order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample depending on whether transposition is required. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is used as input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is used as input. Next, the decoder obtains the MIP matrix coefficients using the traversed prediction mode as an index, and obtains the output vector by calculating the MIP matrix coefficients and the input vector. Finally, the decoder upsamples the output vector according to the number of samples in the output vector and the size of the current template. If upsampling is not required, the output vectors are sequentially arranged horizontally and output as prediction blocks in the template region. If upsampling is required, upsampling is performed first horizontally, and then vertically. Upsampling This process involves upsampling until the size matches that of the template, and then outputting the predicted blocks within the second template region.

[0329] Furthermore, the decoder uses DIMD technology to derive the optimal intra-prediction mode, i.e., the optimal DIMD mode. In DIMD technology, the gradient value of the reconstructed sample in the first template region is calculated based on the Sobel operator, and the gradient value is transformed based on the angle value corresponding to the different prediction modes to obtain the amplitude value under the corresponding prediction mode.

[0330] Next, the decoder calculates the strain cost using the prediction blocks of the template obtained by traversing each MIP mode and the reconstructed samples within the template, and records the optimal MIP mode and transpose information according to the principle of least cost. Furthermore, the decoder traverses all intra-prediction modes that are permitted to be used, calculates the amplitude value under each intra-prediction mode, and records the optimal DIMD prediction mode according to the principle of maximum amplitude.

[0331] Finally, based on the obtained optimal MIP mode and transpose information, the decoder optionally downsamples the reconstructed samples adjacent to the upper and left sides of the current block, splices them based on the transpose information to obtain the input vector, reads the matrix coefficients under the current mode using the MIP mode as the index, and then obtains the output vector by calculating the input vector and matrix coefficients. The decoder transposes the output based on the transpose information, upsamples the output vector based on the size of the current block and the number of samples in the output vector to obtain an output of the same size as the current block, which can be the optimal MIP prediction block for the current block. Furthermore, for the optimal DIMD prediction mode, the decoder obtains the corresponding interpolation filtering coefficients, performs interpolation filtering on the reconstructed samples adjacent to the upper and left sides to obtain prediction samples at all positions within the current block, and records this as the optimal DIMD prediction block. The decoder weights and averages each prediction sample in the optimal MIP prediction block and the optimal DIMD prediction block according to preset weights to obtain a new prediction block. The new prediction block is the prediction block for the current block.

[0332] The decoder either determines the intra-prediction mode derived from the DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or the decoder determines the intra-prediction mode derived from the DIMD mode used for the reconstruction sample in the first template region as the first intra-prediction mode.

[0333] Step 3: The decoder continues to analyze information such as the use flags or indices of other intra-prediction techniques and determines the final predicted block for the current block based on the analyzed information.

[0334] Step 4: The decoder analyzes the bitstream to obtain the frequency domain residual block (also called frequency domain residual information) of the current block, and performs inverse quantization and inverse transformation on the frequency domain residual block of the current block (first, an inverse transformation is performed on the quadratic transformation based on the first intra-prediction mode, and then an inverse transformation is performed on the basic transformation or linear transformation). Exchange The decoder then obtains the residual block of the current block (also called the time-domain residual block or time-domain residual information). Next, the decoder obtains the reconstructed sample block by adding the predicted block of the current block to the residual block of the current block.

[0335] Step 5: After techniques such as loop filtering are performed on all reconstruction sample blocks in the current image, the final reconstructed image is obtained.

[0336] Selectively, the reconstructed image may be used as a video output or as a reference for subsequent decoding.

[0337] In this embodiment, the calculation process for the optimal DIMD prediction block can be found in the description of the DIMD technology described above, and will not be detailed here to avoid duplication. Furthermore, the fusion weights of the optimal MIP prediction block and the optimal DIMD prediction block can be predetermined values, for example, the optimal MIP prediction block accounts for 5 / 9 and the optimal DIMD prediction block accounts for 4 / 9. Of course, in other alternative embodiments, the fusion weights of the optimal MIP prediction block and the optimal DIMD prediction block may be other values ​​and are not specifically limited in this application. Note that the second template region and the first template region may be the same or different and are not specifically limited in this application.

[0338] While preferred embodiments of this application have been described in detail above with reference to the attached drawings, this application is not limited to the detailed contents of the above embodiments. Within the scope of the technical idea of ​​this application, various simple modifications can be made to the technical proposal of this application, and any such simple modifications will fall within the scope of protection of this application. For example, each specific technical feature described in the above specific embodiments may be combined by any appropriate means, provided that they are not contradictory, and in order to avoid unnecessary redundancy, this application will not describe various possible combinations again. Furthermore, for example, between various different embodiments of this application, any combination should be considered disclosed in this application, as long as it does not contradict the idea of ​​this application. It should be understood that in the various method embodiments of this application, the magnitude of the sequence number of each process does not indicate the execution order. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0339] The method embodiments of this application have been described in detail above. Hereinafter, the apparatus embodiments of this application will be described in detail with reference to Figures 11 to 13.

[0340] Figure 11 is a block diagram showing a decoder 500 according to an embodiment of this application.

[0341] As shown in Figure 11, the decoder 500 may include an analysis unit 510, a conversion unit 520, and a reconstruction unit 530. The analysis unit 510 analyzes the bitstream of the current sequence and the current block First It is configured to obtain the conversion coefficient. The conversion unit 520 is configured to determine the first intra-prediction mode. The first intra-prediction mode includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for predicting the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of the optimal matrix-based intra-prediction (MIP) mode for predicting the current block; and one of the following: an intra-prediction mode derived from a DIMD mode or an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode used for reconstructed samples in a first template region adjacent to the current block. The conversion unit 520 is configured to perform a first conversion on a first conversion coefficient based on a conversion set corresponding to a first intra-prediction mode to obtain a second conversion coefficient for the current block. The conversion unit 520 is configured to perform a second conversion on a second conversion coefficient to obtain the residual block of the current block. The reconstruction unit 530 is configured to determine the reconstructed blocks of the current block based on the predicted blocks and residual blocks of the current block.

[0342] In some embodiments, the output vector of the optimal MIP mode is the vector before upsampling the output vector of the optimal MIP mode, or the output vector of the optimal MIP mode is the vector after upsampling the output vector of the optimal MIP mode.

[0343] In some embodiments, the conversion unit 520 is configured to determine a first intra-prediction mode based on a prediction mode for predicting the current block.

[0344] In some embodiments, the conversion unit 520 is configured such that, if the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode for predicting the current block, it determines an intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or determines an intra-prediction mode derived from the DIMD mode to be used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0345] In some embodiments, the conversion unit 520 is configured such that, specifically, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the TIMD mode, it determines the intra-prediction mode derived from the DIMD mode as the first intra-prediction mode to be used for predicting the current block, or determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0346] In some embodiments, the conversion unit 520 is configured such that, specifically, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for the reconstructed sample in the first template region, it determines the intra-prediction mode derived from a DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or determines the intra-prediction mode derived from a DIMD mode used for the reconstructed sample in the first template region as the first intra-prediction mode.

[0347] In some embodiments, the reconfiguration unit 530 is further configured to determine a second intra-prediction mode. The second intra-prediction mode includes a suboptimal MIP mode for predicting the current block, and one of the following intra-prediction modes used for reconstruction samples within the first template region: an intra-prediction mode derived from the DIMD mode and an intra-prediction mode derived from the TIMD mode. The reconstruction unit 530 is further configured to predict the current block based on the optimal MIP mode and a second intra-prediction mode, and to obtain a predicted block for the current block.

[0348] In some embodiments, the reconstruction unit 530 is configured to specifically predict the current block based on the optimal MIP mode and obtain a first predicted block, predict the current block based on the second intra-prediction mode and obtain a second predicted block, and perform a weighting process on the first and second predicted blocks based on the weights of the optimal MIP mode and the second intra-prediction mode to obtain a predicted block of the current block.

[0349] In some embodiments, the reconstruction unit 530 performs a weighting process on the first and second prediction blocks based on the optimal MIP mode weights and the second intra-prediction mode weights, and further, before obtaining the prediction block for the current block, If the prediction mode for predicting the current block includes an optimal MIP mode, a suboptimal MIP mode for predicting the current block, or an intra-prediction mode derived from the TIMD mode, then the weights of the optimal MIP mode and the second intra-prediction mode are determined based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the second intra-prediction mode. The system is configured such that, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the DIMD mode, which is used for reconstruction samples in the first template region, then the weights of both the optimal MIP mode and the second intra-prediction mode are determined to be preset values.

[0350] In some embodiments, the conversion unit 520 is specifically, Analyze the bitstream of the current sequence to obtain the first flag, The system is configured to confirm the second intra-prediction mode if the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode.

[0351] In some embodiments, the conversion unit 520 is specifically, If the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and the second intra-prediction mode, then the bitstream is parsed to obtain the second flag, The system is configured to confirm the second intra-prediction mode if the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode.

[0352] In some embodiments, the reconstruction unit 530 is further configured to determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. The strain costs corresponding to multiple MIP modes include strain costs obtained by using multiple MIP modes to predict samples in a second template region adjacent to the current block.

[0353] In some embodiments, the second template region and the first template region may be the same or different.

[0354] In some embodiments, the reconfiguration unit 530 is specifically, Based on the third flag and multiple MIP modes, predict the samples in the second template region and obtain the strain costs corresponding to the multiple MIP modes under each state of the third flag. The system is configured to determine the optimal MIP mode based on the distortion costs corresponding to multiple MIP modes under each state of the third flag. The third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP mode.

[0355] In some embodiments, the reconstruction unit 530, based on the strain costs corresponding to multiple MIP modes, further determines the optimal MIP mode before determining the optimal MIP mode. Get the MIP mode used for the adjacent block next to the current block, The configuration is set up to determine the MIP mode used for adjacent blocks as multiple MIP modes.

[0356] In some embodiments, the reconstruction unit 530, based on the strain costs corresponding to multiple MIP modes, further determines the optimal MIP mode before determining the optimal MIP mode. Reconstruction sample padding is performed on the reference region adjacent to the outside of the second template region to obtain the reference row and reference column of the second template region. Using the reference row and reference column as input, the samples in the second template region are predicted using each of the multiple MIP modes to obtain multiple prediction blocks corresponding to the multiple MIP modes. It is configured to determine the strain costs corresponding to multiple MIP modes based on multiple prediction blocks and reconstruction blocks in a second template region.

[0357] In some embodiments, the reconfiguration unit 530 is specifically, The input vector is obtained by downsampling the reference row and reference column. Using the input vector as input, samples within the second template region are predicted by traversing multiple MIP modes to obtain output vectors corresponding to multiple MIP modes. The system is configured to upsample output vectors corresponding to multiple MIP modes to obtain prediction blocks corresponding to multiple MIP modes.

[0358] In some embodiments, the reconstruction unit 530 is configured to determine the optimal MIP mode based on the differential transformation absolute sum (SATD) corresponding to multiple MIP modes in the second template region.

[0359] Figure 12 is a block diagram showing an encoder 600 according to an embodiment of this application.

[0360] As shown in Figure 12, the encoder 600 may include a residual unit 610, a conversion unit 620, and an encoding unit 630. The residual unit 610 is configured to retrieve the residual block of the current block in the current sequence. The conversion unit 620 is configured to perform a third conversion on the residual block of the current block to obtain the third conversion coefficient of the current block. The conversion unit 620 is configured to determine the first intra-prediction mode. The first intra-prediction mode includes an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode used for predicting the current block; an intra-prediction mode derived from a DIMD mode used for the output vector of the optimal, matrix-based intra-prediction (MIP) mode for predicting the current block; and one of the following: an intra-prediction mode derived from a DIMD mode or an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode used for reconstructed samples in a first template region adjacent to the current block. The transformation unit 620 is configured to perform a fourth transformation on the third transformation coefficients based on the transformation set corresponding to the first intra-prediction mode to obtain the fourth transformation coefficients for the current block. The encoding unit 630 is configured to encode the fourth conversion coefficient.

[0361] In some embodiments, the output vector of the optimal MIP mode is the vector before upsampling the output vector of the optimal MIP mode, or the output vector of the optimal MIP mode is the vector after upsampling the output vector of the optimal MIP mode.

[0362] In some embodiments, the conversion unit 620 is configured to determine a first intra-prediction mode based on a prediction mode for predicting the current block.

[0363] In some embodiments, the conversion unit 620 is configured such that, specifically, if the prediction mode for predicting the current block includes an optimal MIP mode and a suboptimal MIP mode for predicting the current block, it determines an intra-prediction mode derived from the DIMD mode to be used for the prediction block of the current block as the first intra-prediction mode, or determines an intra-prediction mode derived from the DIMD mode to be used for the output vector of the optimal MIP mode as the first intra-prediction mode.

[0364] In some embodiments, the conversion unit 620 is configured such that, specifically, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the TIMD mode, it determines the intra-prediction mode derived from the DIMD mode as the first intra-prediction mode to be used for predicting the current block, or determines the intra-prediction mode derived from the TIMD mode as the first intra-prediction mode.

[0365] In some embodiments, the conversion unit 620 is configured such that, specifically, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from a DIMD mode used for the reconstructed sample in the first template region, it determines the intra-prediction mode derived from a DIMD mode used for the prediction block of the current block as the first intra-prediction mode, or determines the intra-prediction mode derived from a DIMD mode used for the reconstructed sample in the first template region as the first intra-prediction mode.

[0366] In some embodiments, the residual unit 610 is configured to specifically determine a second intra-prediction mode, predict the current block based on the optimal MIP mode and the second intra-prediction mode to obtain a predicted block of the current block, and obtain a residual block of the current block based on the predicted block of the current block. The second intra-prediction mode includes a suboptimal MIP mode for predicting the current block, and one of the following intra-prediction modes used for reconstruction samples within the first template region: an intra-prediction mode derived from the DIMD mode and an intra-prediction mode derived from the TIMD mode.

[0367] In some embodiments, the residual unit 610 is specifically, Based on the optimal MIP mode, predict the current block and obtain the first predicted block. Based on the second intra prediction mode, predict the current block and obtain the second predicted block. The system is configured to perform a weighting process on the first and second prediction blocks based on the optimal MIP mode weight and the second intra prediction mode weight, in order to obtain the prediction block for the current block.

[0368] In some embodiments, the residual unit 610 performs a weighting process on the first and second prediction blocks based on the optimal MIP mode weight and the second intra-prediction mode weight, and further, before obtaining the prediction block for the current block, If the prediction mode for predicting the current block includes an optimal MIP mode, a suboptimal MIP mode for predicting the current block, or an intra-prediction mode derived from the TIMD mode, then the weights of the optimal MIP mode and the second intra-prediction mode are determined based on the strain cost corresponding to the optimal MIP mode and the strain cost corresponding to the second intra-prediction mode. The system is configured such that, if the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the DIMD mode, which is used for reconstruction samples in the first template region, then the weights of both the optimal MIP mode and the second intra-prediction mode are determined to be preset values.

[0369] In some embodiments, the residual unit 610 is configured to specifically acquire a first flag and determine a second intra-prediction mode if the first flag is used to indicate that it is permitted to predict image blocks in the current sequence using the optimal MIP mode and a second intra-prediction mode. The encoding unit 630 is configured to specifically encode a fourth conversion coefficient and the first flag.

[0370] In some embodiments, the residual unit 610 is specifically, If the first flag is used to indicate that it is permitted to predict the image block in the current sequence using the optimal MIP mode and the second intra-prediction mode, then predict the current block based on the optimal MIP mode and the second intra-prediction mode to obtain the first rate distortion cost. Predict the current block based on at least one intra-prediction mode to obtain at least one rate distortion cost, If the first rate distortion cost is less than or equal to the minimum of at least one rate distortion cost, the system is configured to determine the predicted block obtained by predicting the current block based on the optimal MIP mode and the second intra prediction mode, and to confirm that predicted block is the predicted block of the current block. The encoding unit 630 is specifically configured to encode the fourth conversion coefficient, the first flag, and the second flag. If the first rate distortion cost is less than or equal to the minimum of at least one rate distortion cost, the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode; if the first rate distortion cost is greater than the minimum of at least one rate distortion cost, the second flag is used to indicate that it is not permitted to predict the current block using the optimal MIP mode and the second intra-prediction mode.

[0371] In some embodiments, the residual unit 610 is further configured to determine the optimal MIP mode based on the strain costs corresponding to multiple MIP modes. The strain costs corresponding to multiple MIP modes include strain costs obtained by using multiple MIP modes to predict a sample in a second template region adjacent to the current block.

[0372] In some embodiments, the second template region and the first template region may be the same or different.

[0373] In some embodiments, the residual unit 610 is configured to specifically predict samples in a second template region based on a third flag and a plurality of MIP modes, obtain strain costs corresponding to the plurality of MIP modes under each state of the third flag, and determine the optimal MIP mode based on the strain costs corresponding to the plurality of MIP modes under each state of the third flag. The third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP mode.

[0374] In some embodiments, the residual unit 610, based on the strain costs corresponding to multiple MIP modes, further determines the optimal MIP mode before determining the optimal MIP mode. Get the MIP mode used for the adjacent block next to the current block, The configuration is set up to determine the MIP mode used for adjacent blocks as multiple MIP modes.

[0375] In some embodiments, the residual unit 610, based on the strain costs corresponding to multiple MIP modes, further determines the optimal MIP mode before determining the optimal MIP mode. Reconstruction sample padding is performed on the reference region adjacent to the outside of the second template region to obtain the reference row and reference column of the second template region. Using the reference row and reference column as input, the samples in the second template region are predicted using each of the multiple MIP modes to obtain multiple prediction blocks corresponding to the multiple MIP modes. It is configured to determine the strain costs corresponding to multiple MIP modes based on multiple prediction blocks and reconstruction blocks in a second template region.

[0376] In some embodiments, the residual unit 610 is specifically, The input vector is obtained by downsampling the reference row and reference column. Using the input vector as input, samples within the second template region are predicted by traversing multiple MIP modes to obtain output vectors corresponding to multiple MIP modes. The system is configured to upsample output vectors corresponding to multiple MIP modes to obtain prediction blocks corresponding to multiple MIP modes.

[0377] In some embodiments, the residual unit 610 is configured to determine the optimal MIP mode based on the difference transformation absolute sum (SATD) corresponding to multiple MIP modes in a second template region.

[0378] The apparatus embodiments can correspond to method embodiments, and similar descriptions can be found in the method embodiments. To avoid duplication, such descriptions are omitted here. Specifically, the decoder 500 shown in Figure 11 may correspond to an entity that performs method 300 in the embodiments of this application. Furthermore, the aforementioned and other operations and / or functions of each unit in the decoder 500 are used to realize the corresponding processes in each method, such as method 300. Similarly, the encoder 600 shown in Figure 12 may correspond to an entity that performs method 400 in the embodiments of this application. That is, the aforementioned and other operations and / or functions of each unit in the encoder 600 are used to realize the corresponding processes in each method, such as method 400.

[0379] Furthermore, each unit in the decoder 500 or encoder 600 according to the embodiment of this application may be integrated into one or several other units individually or in total, or some of these units may be further divided into several functionally smaller units. This allows similar operations to be achieved without affecting the realization of the technical effects of the embodiment of this application. The units are divided based on their logic functions. In practical applications, the function of one unit may be realized by several units, or the function of several units may be realized by one unit. In other embodiments of this application, the decoder 500 or encoder 600 may include other units, and in practical applications, these functions may be realized by the cooperation of the other units or by the cooperation of several units. According to another embodiment of this application, for example, a decoder 500 or encoder 600 according to an embodiment of this application is constructed by executing a computer program (including program code) capable of executing each step of the corresponding method in a general-purpose computer device such as a general-purpose computer equipped with processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), thereby realizing the encoding method or decoding method according to an embodiment of this application. The computer program can be recorded, for example, on a computer-readable storage medium, mounted on an electronic device via the computer-readable storage medium, and operated within it, thereby realizing the corresponding method in the embodiment of this application.

[0380] In other words, the above-mentioned unit may be implemented in hardware form, in software form by instructions, or in combination of hardware and software. Specifically, each step of the method embodiment in the embodiments of this application can be completed by an integrated logic circuit in hardware or by instructions in software form in a processor. The steps of the method disclosed in the embodiments of this application can be executed and completed directly by a hardware decoding processor, or by a combination of hardware and software in a decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium resides in memory. The processor reads information from memory and, together with the processor hardware, completes the steps of the method embodiment.

[0381] Figure 13 is a block diagram showing an electronic device 700 according to an embodiment of this application.

[0382] As shown in Figure 13, the electronic device 700 includes a processor 710 and a computer-readable storage medium 720. , transceiver 730 and It includes at least the following. Note that the processor 710 、 Computer-readable storage medium 720 , and transceiver 730 by bus or other means each otherIt may be connected. The computer-readable storage medium 720 is used to store a computer program 721 containing computer instructions. The processor 710 is used to execute the computer instructions stored in the computer-readable storage medium 720. The processor 710 is the computing core and control core of the electronic device 700 and is suitable for implementing one or more computer instructions, specifically, for implementing a corresponding process or corresponding function by loading and executing one or more computer instructions.

[0383] For example, the processor 710 may also be called a CPU. The processor 710 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0384] For example, the computer-readable storage medium 720 may be high-speed RAM memory, or non-volatile memory, such as at least one magnetic disk storage device. Selectively, the computer-readable storage medium 720 may be at least one computer-readable storage medium located away from the processor 710. Specifically, the computer-readable storage medium 720 includes, but is not limited to, volatile memory and / or non-volatile memory. Non-volatile memory may be ROM, programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or flash memory. Volatile memory may be RAM that functions as an external high-speed cache. As illustrative but not limited examples, various types of RAM are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DRRAM).

[0385] In one embodiment, the electronic device 700 may be an encoder or encoding frame according to the embodiment of this application. The computer-readable storage medium 720 stores a first computer instruction. The processor 710 loads and executes the first computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the encoding method according to the embodiment of this application. In other words, the first computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid redundancy, the explanation is omitted here.

[0386] In one embodiment, the electronic device 700 may be a decoder or decoding framework according to the embodiment of this application. The computer-readable storage medium 720 stores a second computer instruction. The processor 710 loads and executes the second computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the decoding method according to the embodiment of this application. In other words, the second computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid redundancy, the explanation is omitted here.

[0387] In another aspect of this application, a coding system is provided in an embodiment of this application. The coding system includes the encoder and decoder described above.

[0388] In another aspect of this application, embodiments of this application provide a computer-readable storage medium (Memory). The computer-readable storage medium is a storage device in the electronic device 700 and is used to store programs and data. For example, the computer-readable storage medium may be a computer-readable storage medium 720. To make it clear, the computer-readable storage medium 720 herein may include an internal storage medium in the electronic device 700, and of course may include an extended storage medium supported by the electronic device 700. The computer-readable storage medium provides a storage space in which the operating system of the electronic device 700 is stored. Furthermore, the storage space stores one or more computer instructions suitable for being loaded and executed by the processor 710, and these computer instructions may be one or more computer programs 721 (including program code).

[0389] According to another aspect of this application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions, which are stored on a computer-readable storage medium, and for example, the computer instructions may be computer program 721. In this case, electronic equipment 700 may be a computer, and the processor 710 reads a computer instruction from the computer-readable storage medium 720, and the processor 710 executes the computer instruction, thereby causing the computer to execute an encoding method or decoding method according to the various selectable methods described above.

[0390] In other words, when implemented by software, all or part of the above embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes described in the embodiments of this application are executed, or the functions described in the embodiments of this application are realized. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored on a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic cable, digital subscriber line (DSL), etc.) or wirelessly (e.g., infrared, radio, microwave, etc.).

[0391] In conjunction with the exemplary units and process steps described in the embodiments disclosed herein, it will be apparent to those skilled in the art that the present application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software will depend on the specific application of the invention and design limitations. Those skilled in the art may implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0392] Finally, the above is merely a specific embodiment of the present application, and the scope of protection of this application is not limited thereto. Any modification or substitution that a person skilled in the art could easily conceive within the scope of the art disclosed in this application should be included within the scope of protection of this application. Accordingly, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. A decoding method, The bitstream is analyzed to obtain the first transformation coefficient of the current block, If the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the DIMD mode used for reconstructed samples in a first template region adjacent to the current block, then the intra-prediction mode derived from the DIMD mode used for reconstructed samples in the first template region is determined to be the first intra-prediction mode. Based on the transformation set corresponding to the first intra prediction mode, a first transformation is performed on the first transformation coefficient to obtain the second transformation coefficient of the current block, Performing a second transformation on the second transformation coefficient to obtain the residual block of the current block, Based on the predicted block of the current block and the residual block of the current block, the reconstructed block of the current block is determined. including, A decoding method characterized by the following features.

2. The aforementioned method, Predicting samples in a second template region adjacent to the current block based on a third flag and a plurality of MIP modes, and obtaining strain costs corresponding to the plurality of MIP modes under each state of the third flag, wherein the third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP modes, and the second template region and the first template region may be the same or different, and obtaining the results. The optimal MIP mode is determined based on the strain cost corresponding to the plurality of MIP modes under each state of the third flag, Further including, The decoding method according to feature 1.

3. An encoding method, wherein the method is applied to an encoder, and the method is Obtain the residual blocks of the current block, Perform a third transformation on the residual block of the current block to obtain the third transformation coefficient of the current block, If the prediction mode for predicting the current block includes an optimal MIP mode and an intra-prediction mode derived from the DIMD mode used for reconstructed samples in a first template region adjacent to the current block, then the intra-prediction mode derived from the DIMD mode used for reconstructed samples in the first template region is determined to be the first intra-prediction mode. Based on the transformation set corresponding to the first intra prediction mode, a fourth transformation is performed on the third transformation coefficient to obtain the fourth transformation coefficient of the current block. Encoding the fourth conversion coefficient, including, An encoding method characterized by the following features.

4. The aforementioned method, Predicting samples in a second template region adjacent to the current block based on a third flag and a plurality of MIP modes, and obtaining strain costs corresponding to the plurality of MIP modes under each state of the third flag, wherein the third flag is used to indicate whether or not to transpose the input and output vectors corresponding to the MIP modes, and the second template region and the first template region may be the same or different, and obtaining the results. The optimal MIP mode is determined based on the strain cost corresponding to the plurality of MIP modes under each state of the third flag, Further including, The encoding method according to feature 3.

5. A method for transmitting a bitstream, The method includes generating a bitstream by performing the encoding method described in claim 3 or 4, and transmitting the bitstream. A method for transmitting a bitstream characterized by the following.

Images