Decoding method, encoding method, decoder, and encoder

Abstract

In an embodiment of the present application, a decoding method, an encoding method, a decoder and an encoder are provided. The decoding method includes: decoding a bitstream to obtain a first transform coefficient of a current block; performing a first transform on the first transform coefficient to obtain a residual block of the current block; performing a first transform on the first transform coefficient to obtain a residual block of the current block; performing a prediction on the current block according to a first prediction mode and a second prediction mode corresponding to a geometric partition mode to obtain a predicted block of the current block; and obtaining a reconstructed block of the current block according to the predicted block of the current block and the residual block of the current block. In the technical solution of the present application, the first transform is performed on the first transform coefficient of the current block, and prediction is performed on the current block according to the first prediction mode and the second prediction mode corresponding to the geometric partition mode, thereby improving the decompression efficiency of the current block.

Description

[Technical Field]

[0001] Embodiments of the present application relate to the technical field of image and video coding, and more specifically to decoding methods, encoding methods, decoders, and encoders. [Background technology]

[0002] Digital video compression technology is primarily a technique for compressing and transmitting / storing large amounts of digital video data. With the dramatic increase in internet video and the ever-growing demand for higher video clarity, while existing digital video compression standards can handle video decompression, better digital video decompression technology is needed to improve compression efficiency. [Overview of the Initiative]

[0003] Embodiments of the present invention provide a decoding method, an encoding method, a decoder, and an encoder, which can thereby improve compression efficiency.

[0004] In a first aspect, the present invention provides a decoding method, which includes decoding a bitstream to obtain a first transformation coefficient of the current block; performing a first transformation on the first transformation coefficient to obtain a residual block of the current block; making a prediction on the current block based on a first and second prediction mode corresponding to a geometric partitioning mode to obtain a predicted block of the current block; and obtaining a reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0005] In a second aspect, the present invention provides an encoding method, which includes: making a prediction for a current block based on a first prediction mode and a second prediction mode corresponding to a geometric division mode, obtaining a predicted block of the current block; obtaining a residual block of the current block based on the predicted block of the current block; performing a third transformation on the residual block of the current block, obtaining a third transformation coefficient of the current block; and encoding the third transformation coefficient.

[0006] In a third aspect, the present invention provides a decoding method. This decoding method includes decoding a bitstream to obtain a first transformation coefficient of the current block; performing a first transformation on the first transformation coefficient to obtain a residual block of the current block; making a prediction on the current block based on a first intra-prediction mode and a second intra-prediction mode derived in a prediction mode derivation mode to obtain a predicted block of the current block; and obtaining a reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0007] In a fourth aspect, the present invention provides an encoding method. This encoding method includes: making a prediction for the current block based on a first intra-prediction mode and a second intra-prediction mode derived in a prediction mode derivation mode, and obtaining a predicted block of the current block; obtaining a residual block of the current block based on the predicted block of the current block; performing a third transformation on the residual block of the current block, and obtaining a third transformation coefficient of the current block; and encoding the third transformation coefficient.

[0008] In a fifth aspect, the present invention provides a decoder. This decoder includes a decoding unit for decoding a bitstream and obtaining a first transformation coefficient of the current block; a transformation unit for performing a first transformation on the first transformation coefficient and obtaining a residual block of the current block; a prediction unit for making predictions on the current block based on a first prediction mode and a second prediction mode corresponding to geometric partitioning modes and obtaining a predicted block of the current block; and a reconstruction unit for obtaining a reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0009] In a sixth aspect, the present invention provides an encoder, which includes a prediction unit for making predictions about a current block based on a first prediction mode and a second prediction mode corresponding to a geometric division mode and obtaining a predicted block of the current block; a residual unit for obtaining a residual block of the current block based on the predicted block of the current block; a transformation unit for performing a third transformation on the residual block of the current block and obtaining a third transformation coefficient of the current block; and an encoding unit for encoding the third transformation coefficient.

[0010] In a seventh aspect, the present invention provides a decoder. This decoder includes a decoding unit for decoding a bitstream and obtaining a first transformation coefficient of the current block; a transformation unit for performing a first transformation on the first transformation coefficient and obtaining a residual block of the current block; a prediction unit for performing a prediction on the current block based on a first intra-prediction mode and a second intra-prediction mode derived in a prediction mode derivation mode and obtaining a predicted block of the current block; and a reconstruction unit for obtaining a reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0011] In an eighth aspect, the present invention provides an encoder, which includes a prediction unit for making predictions on a current block based on a first intra-prediction mode and a second intra-prediction mode derived in a prediction mode derivation mode, and for obtaining a predicted block of the current block; a residual unit for obtaining a residual block of the current block based on the predicted block of the current block; a transformation unit for performing a third transformation on the residual block of the current block, and for obtaining a third transformation coefficient of the current block; and an encoding unit for encoding the third transformation coefficient.

[0012] In a ninth aspect of this application, a decoder is provided, which includes a processor and a computer-readable storage medium. The processor is configured to execute computer instructions. Computer instructions are stored in the computer-readable storage medium, and when the computer instructions are read and executed by the processor, the decoding method of the first aspect, the third aspect, or each of their embodiments is realized.

[0013] In one embodiment, there is one or more processors and one or more memories.

[0014] In one embodiment, the computer-readable storage medium may be integrated with the processor, or it may be installed separately from the processor.

[0015] This application is the first 10 In one embodiment, an encoder is provided. This encoder includes a processor and a computer-readable storage medium. The processor is configured to execute computer instructions. Computer instructions are stored in the computer-readable storage medium, and when the computer instructions are read and executed by the processor, the encoding method of the second embodiment, the fourth embodiment, or each of its embodiments is realized.

[0016] In one embodiment, there is one or more of these processors and one or more of the memories.

[0017] In one embodiment, this computer-readable storage medium may be integrated with the processor, or this computer-readable storage medium may be installed separately from the processor.

[0018] This application 11 provides a computer-readable storage medium in an aspect. Computer instructions are stored in this computer-readable storage medium. When these computer instructions are read and executed by a processor of a computer device, the computer device executes the decoding method according to the first aspect or the third aspect or the encoding method according to the second aspect or the fourth aspect.

[0019] This application 12 provides a bitstream in an aspect. This bitstream is the bitstream according to the first aspect or the third aspect or the bitstream according to the second aspect or the fourth aspect.

[0020] Based on the above technical solutions, this application performs a first transformation on the first transformation coefficient of the current block and performs prediction on the current block based on the first prediction mode and the second prediction mode corresponding to the geometric division mode, thereby improving the thawing efficiency of the current block.

Brief Description of Drawings

[0021] [Figure 1] It is a block diagram showing an encoding framework according to an embodiment of this application. [Figure 2] It is a schematic diagram showing the specific directions of 65 types of angle prediction modes according to an embodiment of this application. [Figure 3] It is an example of a reference sample of a wide-angle prediction mode according to an embodiment of this application. [Figure 4] It is a schematic diagram of a MIP mode according to an embodiment of this application. [Figure 5] It is a schematic diagram showing the prediction mode derivation based on DIMD according to an embodiment of the present application. [Figure 6] It is a schematic diagram showing the prediction block derivation based on DIMD according to an embodiment of the present application. [Figure 7] It is a schematic diagram of a template used in TIMD according to an embodiment of the present application. [Figure 8] It is an example of a weight diagram corresponding to 64 weight derivation modes in GPM for a square block according to an embodiment of the present application. [Figure 9] It is an example of a dividing line of the weight derivation mode according to an embodiment of the present application. [Figure 10] It is an example of a weight diagram corresponding to 56 weight derivation modes in AWP for a square block according to an embodiment of the present application. [Figure 11] It is a schematic diagram of GPM or AWP according to an embodiment of the present application. [Figure 12] It is an example of a DCT2-type base image according to an embodiment of the present application. [Figure 13] It is an example of LFNST according to an embodiment of the present application. [Figure 14] It is an example of a conversion matrix set of LFNST according to an embodiment of the present application. [Figure 15] It is a block diagram showing a decoding framework according to an embodiment of the present application. [Figure 16] It is a flowchart showing a decoding method according to an embodiment of the present application. [Figure 17] It is a flowchart showing an encoding method according to an embodiment of the present application. [Figure 18] It is another flowchart showing a decoding method according to an embodiment of the present application. [Figure 19] It is another flowchart showing an encoding method according to an embodiment of the present application. [Figure 20] It is a block diagram showing a decoder according to an embodiment of the present application. [Figure 21]A block diagram of an encoder according to an embodiment of the present application. [Figure 22] This is another block diagram showing a decoder according to an embodiment of the present application. [Figure 23] This is another block diagram showing an encoder according to an embodiment of the present application. [Figure 24] A branch office of an electronic device according to an embodiment of the present application. [Modes for carrying out the invention]

[0022] The technical proposal in the embodiment of this application will be described below with reference to the drawings.

[0023] The solutions according to the embodiments of this application can be applied to the technical field of digital video coding, which includes, but is not limited to, the fields of image coding, video coding, hardware video coding, dedicated circuit video coding, and real-time video coding. Furthermore, the solutions according to the embodiments of this application can be combined with audio video coding standards (AVS), second-generation AVS standards (AVS2), or third-generation AVS standards (AVS3). For example, they can be combined with, but are not limited to, the H.264 / audio video coding (AVC) standard, the H.265 / high-efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. In addition, the solutions according to the embodiments of this application can perform lossy compression or lossless compression on images. The lossless compression in question may be visually lossless compression or mathematically lossless compression.

[0024] A block-based mixed coding framework is used in video coding standards. Each image (frame) in video is divided into a square largest coding unit (LCU) or coding tree unit (CTU) of the same size (e.g., 128x128, 64x64, etc.). Each largest coding unit or coding tree unit can also be divided into rectangular coding units (CU) based on rules. Coding units can be further divided into prediction units (PU), transform units (TU), etc. The mixed coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. Because there is a strong correlation between adjacent samples in a single image of a video, video coding techniques utilize intra-prediction methods to eliminate spatial redundancy between adjacent samples. In intra-prediction, sample information within the current segmented block is predicted by referring only to information from the same single frame of image. Because there is a strong similarity between adjacent images in a video, video coding techniques can improve coding efficiency by using inter-prediction methods to eliminate temporal redundancy between adjacent images. In inter-prediction, image information from different frames is referred to, and motion estimation is used to find the motion vector information that best matches the current segmented block. Through transformation, the predicted block is transformed into the frequency domain, and energy redistribution is performed.By combining transformation and quantization, information that is difficult for the human eye to perceive can be removed, and this is used to eliminate visual redundancy. Entropy coding can remove character redundancy based on the current context model and the probabilistic information of the binary bitstream.

[0025] In the digital video encoding process, the encoder can first read a grayscale or color image from the original video sequence, and then encode the grayscale or color image. Here, the grayscale image may contain samples of the lumen component, and the color image may contain samples of the chroma component. Selectively, the color image may further contain samples of the lumen component. The color format of the original video sequence may be a luminance-chroma (YCbCr, YUV) format or a red-green-blue (RGB) format, etc. Specifically, after reading a grayscale or color image, the encoder divides it into blocks, generates a predicted block of the current block by performing intra-prediction or inter-prediction on the current block, obtains a residual block by subtracting the predicted block from the original block of the current block, transforms and quantizes the residual block to obtain a quantization coefficient matrix, and outputs the quantization coefficient matrix as an entropy code to a bitstream. In the digital video decoding process, the decoding side generates a predicted block of the current block by performing intra-prediction or inter-prediction on the current block. Furthermore, the decoding side decodes the bitstream to obtain the quantization coefficient matrix, inversely quantizes and inversely transforms the quantization coefficient matrix to obtain the residual block, and adds the predicted block and the residual block to obtain the reconstructed block. The reconstructed block forms the reconstructed image. The decoding side loop-filters the reconstructed image based on the image or block to obtain the decoded image.

[0026] The current block can be the current coding unit (CU) or the current prediction unit (PU), among others.

[0027] Furthermore, the encoding side also requires processing similar to that on the decoding side in order to obtain the decoded image. The decoded image can be used as a reference image for interpretation of subsequent images. Mode information or parameter information such as block partitioning information, prediction, transformation, quantization, entropy coding, and loop filtering, which are determined on the encoding side, are output to the bitstream as needed. The decoding side analyzes and processes the existing information to determine the same block partitioning information, prediction, transformation, quantization, entropy coding, and loop filtering mode information or parameter information as on the encoding side. This ensures that the decoded image obtained on the encoding side is the same as the decoded image obtained on the decoding side. The decoded image obtained on the encoding side is usually also called the reconstructed image. During prediction, the current block may be divided into prediction units, and during transformation, the current block may be divided into transformation units, and the division of the prediction units and transformation units may be the same or different. Naturally, the above is the basic flow of video coding in a block-based mixed coding framework. As technology advances, some modules or steps in the flow of the framework may be optimized. This application applies to the basic flow of a video codec in a mixed coding framework based on the said block.

[0028] To facilitate understanding, we will first briefly explain the encoding framework related to this application.

[0029] Figure 1 is a block diagram showing an encoding framework 100 according to an embodiment of the present application.

[0030] As shown in Figure 1, the encoding framework 100 may include an intra-prediction unit 180, an inter-prediction unit 170, a residual unit 110, a transform and quantization unit 120, an entropy encoding unit 130, an inverse transform and inverse quantization unit 140, and a loop filtering unit 150. Optionally, the encoding framework 100 may further include a decoded image buffer unit 160. The encoding framework 100 is also called a mixed framework encoding mode.

[0031] The intra-prediction unit 180 or inter-prediction unit 170 can make predictions for blocks awaiting coding and output predicted blocks. The residual unit 110 can calculate residual blocks, i.e., the difference between predicted blocks and blocks awaiting coding, based on the predicted blocks and blocks awaiting coding. The transformation and quantization unit 120 is used to perform operations such as transformation and quantization on residual blocks, thereby removing information that is difficult for the human eye to perceive and eliminating visual redundancy. Selectively, residual blocks before transformation and quantization by the transformation and quantization unit 120 may be called temporal residual blocks, and temporal residual blocks after transformation and quantization by the transformation and quantization unit 120 may be called frequency residual blocks or frequency-domain residual blocks. The entropy encoding unit 130 can receive the quantized transform coefficient output by the transformation and quantization unit 120 and output a bitstream based on the quantized transform coefficient. For example, the entropy encoding unit 130 can eliminate character redundancy based on a target context model and probabilistic information of the binary bitstream. For example, the entropy encoding unit 130 can be used in context-based adaptive binary arithmetic coding (CABAC). The entropy encoding unit 130 may also be called a header information encoding unit. Optionally, in this application, the block awaiting coding may also be called an original block or a target block. The prediction block may also be called a prediction image block or an image prediction block, and may also be called a prediction signal or prediction information.A reconstructed block may also be called a reconstructed image block or image reconstruction block, and may also be called a reconstructed signal or reconstructed information. Furthermore, on the encoding side, the block awaiting coding may also be called an encoding block or an encoded image block. On the decoding side, the block awaiting coding may also be called a decoding block or a decoded image block. The block awaiting coding may be a CTU or a CU.

[0032] In the encoding framework 100, the difference between the predicted block and the block awaiting coding is calculated to obtain the residual block. Processes such as transformation and quantization are performed on the residual block, and the residual block is transmitted to the decoding side. Accordingly, the decoding side receives the bitstream, decodes it, obtains the residual block through steps such as inverse transformation and inverse quantization, and obtains the reconstructed block by adding the residual block to the predicted block obtained by the decoding side.

[0033] Furthermore, the inverse transform and inverse quantization unit 140, the loop filtering unit 150, and the decoding image buffer unit 160 within the encoding framework 100 can be used to form a decoder. The intra-prediction unit 180 or inter-prediction unit 170 can predict blocks awaiting coding based on existing reconstruction blocks, thereby ensuring that the encoding and decoding sides utilize the reference frame in the same manner. In other words, the encoder can replicate the decoder's processing loop, thereby generating the same predictions as the decoding side. Specifically, the quantized transformation coefficients are inversely transformed and inversely quantized by the inverse transform and inverse quantization unit 140 to replicate the approximate residual block on the decoding side. After the prediction block is added to this approximate residual block, the effects of block-based processing and blocking artifacts due to quantization can be smoothly filtered through the loop filtering unit 150. The blocks output from the loop filtering unit 150 can be stored in the decoding image buffer unit 160 for use in predicting subsequent images.

[0034] Figure 1 is merely one example of this application and should not be interpreted as a limitation of this application.

[0035] For example, the loop filtering unit 150 within the encoding framework 100 may include a deblocking filter (DBF) and a sample adaptive offset (SAO). The role of the DBF is to remove deblocking artifacts, and the role of the SAO is to remove ringing effects. In other embodiments of the present invention, a neural-network-based loop filtering algorithm can be used in the encoding framework 100 to improve the video compression efficiency. Alternatively, the encoding framework 100 may be a deep learning neural network-based video coding hybrid framework. In one embodiment, the results after sample filtering can be calculated using a convolutional neural network-based model based on the deblocking filter and the SAO. The network structure in the luminance component and the network structure in the saturation component of the loop filtering unit 150 may be the same or different. Given that the luminance component contains more visual information, the luminance component can be used to guide the filtering of the saturation component in order to improve the reconstruction quality of the saturation component.

[0036] Next, we will explain the relationship between intra-prediction and inter-prediction.

[0037] Interpretation uses image information from different frames and motion estimation to search for the motion vector information that best matches the block awaiting coding, thereby eliminating temporal redundancy. The images used for interpretation may be P-frames and / or B-frames, where P-frames refer to forward prediction frames and B-frames refer to bidirectional prediction frames.

[0038] In intra-prediction, the intra-prediction refers only to information from the same image to predict sample information within blocks awaiting coding, which is used to eliminate spatial redundancy. The frame used for intra-prediction may be an I-frame. For example, following a coding order from left to right and top to bottom, the top-left block, the upper block, and the left-side block can be used as reference information to predict blocks awaiting coding. The blocks awaiting coding are also used as reference information for the next block. In this way, the entire image can be predicted. If the input digital video has a color format such as YUV4:2:0 format, each of the four pixels in each image frame of the digital video consists of four Y components and two UV components. The encoding framework can encode the Y components (i.e., luminance blocks) and UV components (i.e., chromaticity blocks), respectively. Similarly, the decoding side can decode according to the format.

[0039] In the intra-prediction process, the intra-prediction uses angle prediction mode and non-angle prediction mode to make predictions on blocks awaiting coding and acquire predicted blocks. Based on rate distortion information calculated from the predicted blocks and blocks awaiting coding, the optimal prediction mode for the blocks awaiting coding is selected, and this prediction mode can be transmitted to the decoding side via a bitstream. The decoding side can analyze and acquire the prediction mode, make predictions to acquire predicted blocks of the target decoding blocks, and acquire reconstructed blocks by superimposing the time-domain residual blocks acquired through bitstream transmission.

[0040] Through the development of successive digital video coding standards, non-angle prediction modes have remained relatively stable, consisting of average mode and planar mode. Angle prediction modes have increased with the evolution of digital video coding standards. Taking the H-series of international digital video coding standards as an example, the H.264 / AVC standard has only 8 angle prediction modes and 1 non-angle prediction mode. H.265 / HEVC expands this to 33 angle prediction modes and 2 non-angle prediction modes. In H.266 / VVC, intra-prediction modes are further expanded to include 67 conventional prediction modes and non-conventional prediction modes, namely matrix-weighted intra-frame prediction (MIP) modes, for luminance blocks. The 67 conventional prediction modes include planar mode, DC mode, and 65 angle prediction modes. Planar mode is typically used to process blocks with progressively changing textures, DC mode is typically used to process flat areas as its name suggests, and angle prediction mode is typically used to process blocks where the angle texture is relatively obvious.

[0041] In this application, the current block used for intra prediction may be a square block or a rectangular block.

[0042] Furthermore, since all intra-prediction blocks are square, the probability of using each angle prediction mode is equal. Currently, when the length and width of a block are not equal, for horizontal blocks (width greater than height), the probability of using the upper reference sample is greater than the probability of using the left reference sample, and for vertical blocks (height greater than width), the probability of using the upper reference sample is less than the probability of using the left reference sample. Based on this, the present invention incorporates a wide-angle prediction mode. When predicting a rectangular block, the conventional angle prediction mode is changed to the wide-angle prediction mode, and when predicting a rectangular block using the wide-angle prediction mode, the predicted angle range of the current block is greater than the predicted angle range when predicting a rectangular block using the conventional angle prediction mode. Selectively, when using the wide-angle prediction mode, the index of the conventional angle prediction mode can still be used to transmit the signal, and accordingly, after receiving the signal, the decoding side can change from the conventional angle prediction mode to the wide-angle prediction mode, thereby not changing the total number of intra-prediction modes or the intra-mode encoding method, and the intra-mode encoding method does not change.

[0043] Figure 2 is a schematic diagram showing the specific directions of 65 types of angle prediction modes according to the embodiment of the present application.

[0044] As shown in Figure 2, index 0 represents the planar mode, index 1 represents the DC mode, and indices -14 to 80 each represent different angle prediction modes. Specifically, indices 2 to 66 represent the conventional angle prediction mode, and indices -1 to -14 and indices 67 to 80 represent the wide-angle prediction mode. In other words, the conventional intra-prediction mode represented by indices 2 to 66 can be used to predict square blocks, and the wide-angle prediction modes represented by indices -1 to -14 and 67 to 80 can be used to predict rectangular blocks.

[0045] It should be understood that the prediction mode indicated by the index x according to the present application can also be called prediction mode x. For example, the intra prediction mode indicated by index 2 may sometimes be called intra prediction mode 2.

[0046] FIG. 3 is an example of a reference sample of a wide-angle prediction mode according to an embodiment of the present application.

[0047] As shown in FIG. 3, in the wide-angle prediction mode, for a CU of size W×H, the number of reference samples above it is 2W + 1, and the number of reference samples on its left side is 2H + 1. Specifically, as shown in (a) of FIG. 3, when W > H (for example, a CU of 8×4), near intra prediction mode 2 (greater than intra prediction mode 2), there are cases where the lower right corner point of the CU cannot be indexed to the reference sample, and near intra prediction mode 66 (greater than intra prediction mode 66), there are still points that can be indexed to the reference sample. Therefore, some of the quasi-horizontal angle modes near intra prediction mode 2 (greater than intra prediction mode 2) need to be replaced with some of the quasi-vertical angle modes near intra prediction mode 66 (greater than intra prediction mode 66), thereby expanding the prediction angle range. Similarly, as shown in (b) of FIG. 3, when W < H (for example, a CU of 4×8), near intra prediction mode 66 (less than intra prediction mode 66), there may be cases where some points cannot be indexed to the reference sample, and near intra prediction mode 2 (less than intra prediction mode 2), there are still points that can be indexed to the reference sample. Therefore, some of the quasi-vertical angle modes near intra prediction mode 66 (less than intra prediction mode 66) need to be replaced with some of the quasi-horizontal angle modes smaller than those near intra prediction mode 2 (less than intra prediction mode 2), thereby expanding the prediction angle range.

[0048] In some cases, the intra-prediction mode to be executed can be determined or selected based on the size of the current block. For example, an intra-prediction can be performed on the current block by determining or selecting a wide-angle prediction mode based on the size of the current block. For example, if the current block is a rectangular block (different width and height), an intra-prediction can be performed on the current block using the wide-angle prediction mode. The ratio of the width to height of the current block can be used to determine the angle prediction mode that is replaced in the wide-angle prediction mode and the angle prediction mode after the replacement. For example, when predicting the current block, any intra-prediction mode for angles that do not exceed the diagonals of the current block (from the bottom left corner to the top right corner of the current block) can be selected as the replacement angle prediction mode.

[0049] The following describes other intra-prediction modes related to this application.

[0050] (1) Matrix-based Intra Prediction (MIP) mode

[0051] The MIP mode is also called the Matrix weighted Intra Prediction mode. The process related to the MIP mode can be divided into three main steps: a downsampling process, a matrix multiplication process, and an upsampling process. Specifically, first, in the downsampling process, spatially adjacent reconstructed samples are downsampled. Next, the obtained downsampled sample sequence is used as the input vector of the matrix multiplication process, that is, the output vector of the downsampling process is used as the input vector of the matrix multiplication process, and the preset matrix in the input vector of the matrix multiplication process is multiplied, and a bias vector is added to the result to output the calculated sample vector. Finally, the output vector of the matrix multiplication process is used as the input vector of the upsampling process, and the final prediction block is obtained by upsampling.

[0052] Figure 4 is a schematic diagram of the MIP mode according to an embodiment of the present application.

[0053] As shown in Figure 4, in the MIP mode, in the downsampling process, the reconstructed samples adjacent above the current coding unit are averaged to obtain the downsampled upper adjacent reconstructed sample vector bdry top and the reconstructed samples adjacent to the left of the current coding unit are averaged to obtain the downsampled left adjacent reconstructed sample vector bdry left . After obtaining bdry top and bdry left , they are used as the input vector bdry red of the matrix multiplication process. Specifically, the sample vector can be obtained based on the vector bdry red of the top row of bdry top red , bdry left , A k· bdry red +b k . A kb is a pre-configured matrix, k is a pre-set bias vector, and k is the index of the MIP mode. After obtaining the sample vector, the sample vector is upsampled by linear interpolation to obtain a predicted sample block with a number of samples that matches the actual number of samples in the coding unit.

[0054] In other words, to predict a block with width W and height H, MIP requires H reconstructed samples from the left column of the current block and W reconstructed samples from the row above the current block as input. MIP generates the predicted block based on three steps: averaging of reference samples, matrix-vector multiplication, and interpolation. The core of MIP is matrix-vector multiplication, which can be considered the process of generating the predicted block using input samples (reference samples) in a matrix-vector multiplication manner. Various matrices are provided for MIP, and differences in prediction methods can be reflected in differences in matrices; if the input samples are the same, different matrices will yield different results. Furthermore, the processes of averaging and interpolation of reference samples are designed to consider the trade-off between performance and complexity. For blocks with large sizes, averaging of reference samples can achieve an effect that approximates downsampling, allowing the input to be adapted to a relatively small matrix. Interpolation provides an upsampling effect. Thus, it is no longer necessary to provide a separate MIP matrix for each block size; instead, only one or a few matrices of specific sizes need to be provided. As the need for compression performance increases and hardware performance improves, more complex MIPs may appear in next-generation standards.

[0055] In MIP mode, the MIP mode can be simplified from a neural network; for example, the matrix used in the MIP mode can be obtained based on training. Therefore, the MIP mode possesses relatively strong generalization ability and predictive effects that cannot be achieved by conventional prediction modes. The MIP mode is a model obtained by performing multiple simplifications of hardware and software complexity on an intra-predictive model based on a neural network. Based on a large number of training samples, multiple prediction modes represent multiple models and parameters, better covering the texture of natural sequences.

[0056] MIP mode is somewhat similar to planar mode, but clearly, MIP mode is more complex and flexible than planar mode.

[0057] The number of MIP modes can vary depending on the coding unit's block size. For example, a 4x4 coding unit has 16 prediction modes. An 8x8 coding unit, or a coding unit with a width equal to 4 or a height equal to 4, has 8 prediction modes. For other sizes of coding units, it has 6 prediction modes. MIP modes also have a transpose function. For prediction modes that match the current size, MIP modes allow the encoder to attempt a transpose calculation. Therefore, MIP modes require a flag indicating whether or not MIP modes are used for the current coding unit. Furthermore, if MIP modes are used for the current coding unit, a transpose flag must also be transmitted to the decoder.

[0058] (2) Decoder side Intra Mode Derivation (DIMD) mode

[0059] The core of DIMD mode is that the decoder derives the intra-predictive mode using the same method as the encoder. This avoids transmitting the intra-predictive mode index of the current coding unit in the bitstream, thereby saving bit overhead.

[0060] The specific process for DIMD mode can be divided into the following two main steps:

[0061] Step 1: Derive the prediction mode.

[0062] Figure 5 is a schematic diagram showing the derivation of a prediction mode based on DIMD according to an embodiment of the present application.

[0063] As shown in Figure 5(a), DIMD derives prediction modes using samples within a template in the reconstruction region (reconstruction samples to the left and above the current block). For example, the template can include three adjacent rows of reconstruction samples above the current block, three adjacent columns of reconstruction samples to the left, and the corresponding adjacent reconstruction sample to the upper left. Based on this, multiple gradient values ​​can be determined within the template according to a window (e.g., the window shown in Figure 5(b) or Figure 5(c)). Each gradient value is used to obtain one intra prediction mode (ipm) that fits its gradient direction. Based on this, the encoder can derive prediction modes by selecting the prediction mode corresponding to the largest gradient value and the prediction mode corresponding to the second largest gradient value among the multiple gradient values. For example, as shown in Figure 5(b), for a 4x4 block, all samples for which gradient values ​​need to be determined are analyzed to obtain the corresponding gradient histogram. For example, as shown in Figure 5(c), for blocks of other sizes, all samples for which gradient values ​​need to be determined are analyzed to obtain the corresponding gradient histogram. Finally, the prediction modes corresponding to the largest gradient and the second largest gradient in the gradient histogram are defined as the derived prediction modes.

[0064] Of course, the gradient histogram in this application is merely an example for determining the derived prediction mode, and when implemented in practice, it can be carried out in various simple forms, and this application is not particularly limited thereto. Furthermore, this application is not limited to the method of statistically calculating the gradient histogram; for example, the gradient histogram can be statistically calculated using the Sobel operator or other methods.

[0065] Step 2: Derive the predicted block.

[0066] Figure 6 is a schematic diagram showing the derivation of a predicted block based on DIMD according to an embodiment of the present application.

[0067] As shown in Figure 6, the encoder can weight the predicted values ​​corresponding to three intra-prediction modes (planar mode and two intra-prediction modes derived based on DIMD). The encoder and decoder use the same prediction block derivation method to obtain the predicted block for the current block. Assuming that the prediction mode corresponding to the largest gradient value is prediction mode 1 and the prediction mode corresponding to the second largest gradient value is prediction mode 2, the encoder determines the following two conditions. 1. The gradient value for prediction mode 2 is not 0. 2. Neither Prediction Mode 1 nor Prediction Mode 2 is a Planar Mode or a DC Prediction Mode.

[0068] If the above two conditions are not met simultaneously, the predicted sample value of the current block is calculated using only prediction mode 1, i.e., the normal prediction process is applied to prediction mode 1. If not, i.e., if the above two conditions are met simultaneously, the predicted block of the current block is derived using the weighted averaging method. The specific method is as follows: Planar mode accounts for 1 / 3 of the weight, and the remaining 2 / 3 is the combined weight of prediction mode 1 and prediction mode 2. For example, the gradient amplitude value of prediction mode 1 is divided by the sum of the gradient amplitude values ​​of prediction mode 1 and prediction mode 2, and the result is used as the weight of prediction mode 1. The gradient amplitude value of prediction mode 2 is divided by the sum of the gradient amplitude values ​​of prediction mode 1 and prediction mode 2, and the result is used as the weight of prediction mode 2. Finally, a weighted averaging is performed on the predicted blocks obtained based on the above three prediction modes, i.e., predicted block 1, predicted block 2, and predicted block 3 obtained based on planar mode, prediction mode 1, and prediction mode 2, respectively, to obtain the predicted block of the current coding unit. The decoder also obtains its predicted block in the same steps.

[0069] In other words, the specific weights for step 2 above are calculated as follows: Weight(PLANAR) = 1 / 3; Weight(mode1) = 2 / 3 * (amp1 / (amp1+amp2)); Weight(mode2) = 1 - Weight(PLANAR) - Weight(mode1); mode1 and mode2 represent prediction mode 1 and prediction mode 2, respectively, and amp1 and amp2 represent the gradient amplitude values ​​for prediction mode 1 and prediction mode 2, respectively. In DIMD mode, a flag must be transmitted to the decoder. This flag is used to indicate whether DIMD mode is being used for the current coder unit.

[0070] Of course, the weighted averaging method described above is merely one example of this application and should not be understood as a limitation of this application.

[0071] In summary, DIMD uses gradient analysis of the reconstructed sample to select an intra-prediction mode, and depending on the analysis results, it is possible to weight two intra-prediction modes and a planar mode. The advantage of DIMD is that, if a DIMD mode is selected for a block, there is no need to specifically indicate which intra-prediction mode is being used in the bitstream, as this is derived by the decoder itself through the process described above, thus saving some overhead.

[0072] (3) Template-based Intra Mode Derivation (TIMD) Prediction Mode

[0073] The technical principle of TIMD mode is relatively similar to that of DIMD mode described above, and in both cases, the codec performs the same operation to derive the prediction mode, saving the overhead of transmitting the mode index. TIMD mode can be understood in two main parts. First, cost information for each prediction mode is calculated based on a template, and the prediction mode corresponding to the smallest cost and the prediction mode corresponding to the second smallest cost are selected. The prediction mode corresponding to the smallest cost is denoted as prediction mode 1, and the prediction mode corresponding to the second smallest cost is denoted as prediction mode 2. If the ratio of the second smallest cost (costMode2) to the smallest cost (costMode1) satisfies a pre-set condition such as costMode2 < 2*costMode1, then the prediction block corresponding to prediction mode 1 and the prediction block corresponding to prediction mode 2 are weighted and merged according to the weights corresponding to prediction mode 1 and prediction mode 2 to obtain the final prediction block.

[0074] For example, the weights corresponding to prediction mode 1 and prediction mode 2 are determined based on the following method. weight1 = costMode2 / (costMode1+ costMode2); weight2 = 1 - weight1; weight1 is the weight of the prediction block corresponding to prediction mode 1, and weight2 is the weight of the prediction block corresponding to prediction mode 2. If the ratio of the second smallest cost costMode2 to the smallest cost costMode1 does not satisfy a predetermined condition, weight fusion between prediction blocks does not occur, and the prediction block corresponding to prediction mode 1 becomes a TIMD prediction block.

[0075] Furthermore, when performing intraprediction on the current block using TIMD mode, if the reconstruction sample template for the current block does not contain any available adjacent reconstruction samples, TIMD mode will select planar mode and perform intraprediction on the current block, i.e., will not perform weighted fusion. Similar to DIMD mode, TIMD mode requires the transmission of a flag to the decoder. This flag is used to indicate whether TIMD mode is being used for the current coder unit.

[0076] Figure 7 is a schematic diagram of a template used in TIMD according to the embodiment of the present application.

[0077] As shown in Figure 7, if the current block is a coding unit with width equal to M and height equal to N, the codec can select a reference template for the current block from a coding unit with width equal to 2(M+L1)+1 and height equal to 2(N+L2)+1, and calculate the template for the current block. In this case, if the template for the current block does not contain any available adjacent reconstruction samples, the TIMD mode selects planar mode and performs intra-prediction for the current block. For example, available adjacent reconstruction samples may be samples adjacent to the left and above the current CU in Figure 7, i.e., there are no available reconstruction samples in the shaded area. In other words, if there are no available reconstruction samples in the shaded area, the TIMD mode selects planar mode and performs intra-prediction for the current block.

[0078] Except in the case of boundaries, when coding the current block, theoretically, reconstructed values ​​can be obtained for the left and above the current block, i.e., the template of the current block contains available adjacent reconstructed samples. In a specific embodiment, the decoder predicts the template using a certain intra-prediction mode and compares the predicted value to the reconstructed value to obtain the cost of that intra-prediction mode in the template. Examples include the Sum of Absolute Differences (SAD), the Sum of Absolute Transformed Difference (SATD), and the Sum of Squared Error (SSE). Since the template and the current block are adjacent to each other, the reconstructed samples in the template and the samples in the current block are correlated. Therefore, the behavior of the prediction mode on the template can be used to estimate the behavior of this prediction mode on the current block. In TIMD, the template is predicted using several candidate intra-prediction modes to obtain the costs of the candidate intra-prediction modes on the template, and the predicted value of the one or two intra-prediction modes with the lowest cost is taken as the intra-prediction value of the current block. If the difference between the two costs corresponding to the two intra-prediction modes on the template is not large, the compression performance can be improved by performing a weighted average on the predicted values ​​of the two intra-prediction modes. Selectively, the weights of the predicted values ​​of the two prediction modes are related to the above costs, for example, the weights are inversely proportional to the costs.

[0079] In summary, TIMD allows for the selection of an intra-prediction mode by leveraging the predictive effect of the intra-prediction mode on the template, and weighting two intra-prediction modes according to their cost on the template. The advantage of TIMD is that, once a TIMD mode is selected for a block, there is no need to specifically indicate which intra-prediction mode is being used in the bitstream, as this is derived by the decoder itself through the process described above, thus saving some overhead.

[0080] Through a brief introduction to the above intra-prediction modes, the following can be observed: The technical principles of DIMD mode and TIMD mode are similar. Both utilize the fact that the decoder performs the same operation as the encoder to estimate the prediction mode of the current coding unit. In such prediction modes, if the complexity is acceptable, the transmission of the prediction mode index can be omitted, saving overhead and improving compression efficiency. However, due to the limitations of the available information and the fact that they do not significantly improve prediction quality in themselves, DIMD and TIMD modes are more effective in large areas where texture characteristics are consistent. When the texture changes slightly or the template area cannot cover it, the prediction effect of such prediction modes is inferior.

[0081] Furthermore, both DIMD and TIMD modes merge or weight prediction blocks obtained based on multiple conventional prediction modes. Merging prediction blocks can produce effects that cannot be achieved with a single prediction mode. In DIMD mode, the planar mode can be incorporated as an additional weighted prediction mode to enhance the spatial relevance between adjacent reconstructed samples and predicted samples, thereby improving the prediction effect of intra-prediction. However, because the prediction principle of the planar mode is relatively simple, using the planar mode as an additional weighted prediction mode for prediction blocks where there is a clear difference between the upper right corner and the lower left corner may have a counterproductive effect.

[0082] (4) Geometric partitioning mode (GPM) and angular weighted prediction (AWP).

[0083] In video coding standards, conventional unidirectional prediction searches for only one reference block of the same size as the current block, while conventional bidirectional prediction uses two reference blocks of the same size as the current block, and the sample value of each sample in the prediction block is the average of the samples at the corresponding positions in the two reference blocks, meaning that all samples in each reference block each account for 50%. Furthermore, with bidirectional weighted prediction, the proportions of the two reference blocks can differ; for example, all samples in the first reference block might account for 75%, and all samples in the second reference block might account for 25%, while the proportions of all samples in the same reference block remain the same. In addition, several optimization methods, such as decoder-side motion vector refinement (DMVR) and bidirectional optical flow (BIO or BDOF), can introduce some variation into the reference or prediction samples.

[0084] In both GPM and AWP, two reference blocks of the same size as the current block are used, but at some sample locations, 100% of the sample values ​​from the corresponding location in the first reference block are used, at some sample locations, 100% of the sample values ​​from the corresponding location in the second reference block are used, and in boundary regions (or regions called transition regions), a certain proportion of the sample values ​​from the corresponding locations in the two reference blocks are used. The weights of the boundary regions also change gradually. The specific method of assigning these weights is determined by the weight derivation mode of GPM or AWP. The weight of each sample location is determined based on the weight derivation mode of GPM or AWP.

[0085] Of course, in some cases, for example when the block size is very small, in some GPM or AWP modes, it may not be possible to guarantee that 100% of the sample values ​​of the corresponding positions in the first reference block are used at some sample positions, and 100% of the sample values ​​of the corresponding positions in the second reference block are used at some sample pixel positions. In this case, the GPM or AWP can be understood as using two reference blocks of different sizes from the current block, that is, using the required portion of each as a reference block, or in other words, using the portion with a weight of 0 as a reference block and removing the portion with a weight of 0, and the present application does not limit the specific implementation of this.

[0086] Figure 8 shows an example of a weight diagram corresponding to 64 different weight derivation modes in the GPM for a square block according to an embodiment of the present application.

[0087] As shown in Figure 8, GPM has weight diagrams corresponding to 64 different weight derivation modes for a square block. Here, for each weight derivation mode, the black area indicates that the weight value of the corresponding position in the first reference block is 0%, the white area indicates that the weight value of the corresponding position in the first reference block is 100%, and the gray area indicates, depending on the shade of color, that the weight value of the corresponding position in the first reference block is a weight value greater than 0% but less than 100%. The weight value of the corresponding position in the second reference block is the value obtained by subtracting the weight value of the corresponding position in the first reference block from 100%.

[0088] Figure 9 shows an example of a dividing line for a weight derivation mode according to an embodiment of the present application.

[0089] As shown in Figure 9, the division line for the weight derivation mode may be a line consisting of points where the weights of two prediction modes corresponding to the GPM are the same. In other words, in the weight matrix of the GPM, the division line is a line consisting of points where the weights of two prediction modes corresponding to the GPM are the same. In other words, the division line may be a line consisting of points where the weights of two prediction modes corresponding to the GPM are the same within the region where the weights change in the weight matrix of the GPM. In other words, the division line is a line consisting of points where the weight is the midpoint, or the division line is a line consisting of points corresponding to the midpoint weight, and a point with the midpoint weight may or may not be in the midpoint of the entire sample. For example, if the weights are from 0 to 8, the midpoint weight may be 4.

[0090] Figure 10 shows an example of a weight diagram corresponding to 56 different weight derivation modes in AWP for a square block according to an embodiment of the present application.

[0091] As shown in Figure 10, weight diagrams corresponding to 56 different weight derivation modes in AWP for square blocks are shown. Here, for each weight derivation mode, the black area indicates that the weight value of the corresponding position in the first reference block is 0%, the white area indicates that the weight value of the corresponding position in the first reference block is 100%, and the gray area indicates that the weight value of the corresponding position in the first reference block is a weight value greater than 0% but less than 100%, depending on the shade of color. The weight value of the corresponding position in the second reference block is the value obtained by subtracting the weight value of the corresponding position in the first reference block from 100%.

[0092] Note that the weight derivation methods for GPM and AWP can differ. For example, in GPM, the angle and offset amount are determined for each weight derivation mode, and then a corresponding weight diagram is calculated for each weight derivation mode. In AWP, a one-dimensional weight line is first determined based on each weight derivation mode, and then the one-dimensional weight line is tiled across the entire image using a method similar to intra-angle prediction to obtain a weight diagram corresponding to each weight derivation mode. Of course, in other alternative embodiments, the weight diagram corresponding to each weight derivation mode can also be called a weight matrix.

[0093] Next, we will explain the weight derivation method using GPM as an example.

[0094] The encoder can determine the corresponding partition line based on each weight derivation mode, and then determine the corresponding weight matrix based on the partition line. For example, the encoder can determine the angle index variable angleIdx and the distance index variable distanceIdx corresponding to the weight derivation mode using Table 1, and determine the weight derivation mode merge_gpm_partition_idx. The angle index variable angleIdx and the distance index variable distanceIdx can be understood as variables for determining the partition line, that is, variables for determining the angle and offset amount of the partition line, respectively. After determining the partition line corresponding to each weight derivation mode, the encoder can determine the weight matrix corresponding to each weight derivation mode based on the partition line corresponding to each weight derivation mode.

[0095] [Table 1]

[0096] As shown in Table 1, there are 64 types of weight derivation modes (for example, the 64 modes shown in Figure 8), and the index values ​​(merge_gpm_partition_idx) for these modes range from 0 to 63. Each of the 64 weight derivation modes can correspond to one angle index variable angleIdx and one distance index variable distanceIdx. That is, each weight derivation mode can correspond to one partition line. Of course, one angle index variable angleIdx or one distance index variable distanceIdx can correspond to the index of one or more weight derivation modes, and Table 1 is merely an example of the present invention and should not be understood as a limitation on the present invention.

[0097] Since GPM can be used for three components (e.g., Y, Cb, Cr), the process of generating a GPM prediction sample matrix for one component can be made a subprocess, namely a weighted sample prediction process for geometric partitioning mode. This process can be invoked for three components, differing only in the invocation parameters, and this application will explain using the luminance component as an example. Exemplarily, the weighted sample prediction process for GPM can be used to derive the prediction matrix predSamplesL[ xL ][ yL ] for the current luminance block, where xL = 0..cbWidth-1 and yL = 0..cbHeight-1, where nCbW is cbWidth and nCbH is cbHeight.

[0098] The input to the weighted prediction process for GPM is the current block width nCbW, the current block height nCbH, two (nCbW)x(nCbH) predicted sample matrices predSamplesLA and predSamplesLB, the angle index variable angleIdx for the "split" of the GPM, the distance index variable distanceIdx for the GPM, and the component index variable cIdx, where, for example, if cIdx is 0, the component index variable represents the luminance component. The output to the weighted prediction process for GPM is the GPM predicted sample matrix pbSamples[x][y] of (nCbW)x(nCbH), where x=0..nCbW-1 and y=0..nCbH-1.

[0099] The predicted sample matrix pbSamples[x][y] can be derived in the following way: For example, the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor can be derived in the following way: nW = ( cIdx = = 0 ) ? nCbW : nCbW * SubWidthC. nH = ( cIdx = = 0 ) ? nCbH : nCbH * SubHeightC. shift1 = Max( 5, 17 - BitDepth ), where BitDepth is the bit depth of the coding. offset1 = 1 << ( shift1 - 1 ). DimensionX = angleIdx. displacementY = (angleIdx + 8) % 32. partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1. shiftHor = ( angleIdx % 16 = = 8 | | ( angleIdx % 16 != 0 && nH >= nW ) ) ? 0 : 1. Next, the variables offsetX and offsetY can be derived in the following way: If the value of shiftHor is 0, offsetX = ( -nW ) >> 1; offsetY = ( ( -nH ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nH ) >> 3 : -( ( distanceIdx * nH ) >> 3 ) ); If not (i.e., the value of shiftHor is 1), offsetX = ( ( -nW ) >> 1 ) + ( angleIdx < 16 ? ( distanceIdx * nW ) >> 3 : -( ( distanceIdx * nW ) >> 3 ); offsetY = ( -nH ) >> 1. Subsequently, the predicted sample matrix pbSamples[x][y] can be derived using the following method (x = 0..nCbW - 1, y = 0..nCbH - 1), Variables xL and yL are derived in the following way: xL = ( cIdx = = 0 ) ? x : x * SubWidthC; yL = ( cIdx = = 0 ) ? y : y * SubHeightC; weightIdx = ( ( ( xL + offsetX ) << 1 ) + 1 ) * disLut[ displacementX ] + ( ( ( yL + offsetY ) << 1 ) + 1 ) * disLut[ displacementY ]; The disLut[displacementX] can be determined from Table 2: [Table 2] weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx; wValue = Clip3( 0, 8, ( weightIdxL + 4 ) >> 3 ); pbSamples[ x ][ y ] = Clip3( 0, ( 1 << BitDepth ) - 1, ( predSamplesLA[ x ][ y ] * wValue + predSamplesLB[ x ][ y ] * ( 8 - wValue ) + offset1 ) >> shift1 ).

[0100] Here, pbSamples[x][y] represents the predicted sample at point (x,y). wValue represents the weights of the predicted values ​​predSamplesLA[x][y] of the prediction matrix for one prediction mode at point (x,y), and (8-wValue) represents the weights of the predicted values ​​predSamplesLB[x][y] of the prediction matrix for another prediction mode at point (x,y).

[0101] Furthermore, for one weight derivation mode, it is possible to use it to derive one weight value wValue for each point, and then calculate one GPM prediction value pbSamples[x][y]. In this method, the weight wValue does not need to be in matrix form, but it can be understood that if the wValue for each position is stored in a matrix, it becomes a weight matrix. The principle of calculating a weight for each point and weighting it to obtain a GPM prediction value is the same as the principle of calculating all the weights and weighting them uniformly to obtain a GPM prediction sample matrix. The reason why "weight matrix" is used in many descriptions in this specification is to make the description easier to understand and to obtain a more intuitive diagram using the weight matrix, but in reality it can also be described using weights for each position. For example, the weight matrix derivation mode can also be called a weight derivation mode, but this application does not specifically limit it.

[0102] Furthermore, the partitioning of CU, PU, ​​and TU all belong to partitioning methods based on rectangles. However, GPM and AWP achieve the effect of non-rectangular partitioning of predictions without partitioning. GPM and AWP use a weight mask of two reference blocks, i.e., the weight diagram or weight matrix described above. This mask determines the weights of the two reference blocks when generating the prediction block, or it can be easily understood that some positions of the prediction block are obtained based on the first reference block and other positions are obtained based on the second reference block, and the blending area is obtained by weighting the corresponding positions of the two reference blocks, thereby making the transition smoother. In GPM and AWP, since the current block is not partitioned into two CUs or PUs, the current block is treated as a single unit in post-prediction residual transformations, quantization, inverse transformations, and inverse quantization.

[0103] In GPM, two inter-prediction blocks can be combined using a weight matrix. This application extends this to combining any two prediction blocks. For example, two inter-prediction blocks, two intra-prediction blocks, or one inter-prediction block and one intra-prediction block can be combined. Furthermore, in screen content coding, one or two prediction blocks from intra-block copy (IBC) mode or palette mode can be used. For convenience of explanation, this application collectively refers to intra-mode, inter-mode, IBC mode, and palette mode as prediction modes. A prediction mode can be understood as a mode on which the codec can generate information for one prediction block of the current block. Exemplaryly, in intra-prediction, the prediction mode may be an intra-prediction mode such as DC mode, planar mode, or various types of intra-angle prediction modes. Of course, some or more auxiliary information can also be superimposed, such as an optimization method for intra-reference samples or an optimization method after generating a rudimentary prediction block (e.g., filtering). For example, in interpretation, the prediction mode may be merge mode, merge with motion vector difference (MMVD) mode, or advanced motion vector prediction (AMVP). For example, the prediction mode may be unidirectional, bidirectional, or multi-hypothesis prediction. Furthermore, in interpretation mode, if unidirectional prediction is used and one piece of motion information can be determined, the prediction block can be determined based on the motion information. In interpretation mode, if bidirectional prediction is used and two pieces of motion information can be determined, the prediction block can be determined based on the motion information.

[0104] Figure 11 is a schematic diagram of a GPM or AWP according to an embodiment of the present application.

[0105] As shown in Figure 11, the information that needs to be determined in GPM can be represented as one weight derivation mode and two prediction modes. The weight derivation mode is used to determine the weight matrix or weights, and each of the two prediction modes determines one prediction block or prediction value. The weight derivation mode is also called the partition mode or weight matrix derivation mode. The two prediction modes may be the same or different prediction modes, and include, but are not limited to, the intra-prediction mode, inter-prediction mode, IBC mode, and palette mode.

[0106] The following explains the process of converting residual blocks.

[0107] When encoding, a prediction is first made to the current block, using spatial or temporal correlations to obtain an image that is the same as or similar to the current block. For one block, the predicted block and the current block may be exactly the same, but it is difficult to guarantee that this will be the case for all blocks in a video. In particular, in natural video or video shot with a camera, the image texture is complex and there are factors such as noise in the image, so the predicted block and the current block are usually similar but different. Also, due to irregular motion, twisting deformation, occlusion, and changes in brightness in the video, it is difficult to perfectly predict the current block. Therefore, in a hybrid coding framework, the residual image is obtained by subtracting the predicted image from the original image of the current block, or in other words, the residual block is obtained by subtracting the predicted block from the current block. Since the residual block is usually much simpler than the original image, the compression efficiency can be greatly improved through prediction. Rather than encoding the residual block directly, a transformation is usually performed first. The transformation is to convert the residual image from the spatial domain to the frequency domain and remove the correlation of the residual image. After the residual image is converted to the frequency domain, the energy is often concentrated in the low-frequency region, so the non-zero coefficients after conversion are often concentrated in the upper left corner. Next, quantization is used for further compression. Also, since the human eye has difficulty perceiving high frequencies, a larger quantization step size can be used in the high-frequency region.

[0108] Image transformation technology transforms an original image so that it can be represented by an orthogonal function or orthogonal matrix. This transformation is two-dimensional, linear, and reversible. Generally, the original image is called a spatial domain image, the transformed image is called a transformed domain image (also called a frequency domain image), and the transformed domain image can be inversely transformed back into a spatial domain image. After image transformation, on the one hand, the characteristics of the image itself can be reflected more effectively, and on the other hand, energy can be concentrated on a small amount of data, which is advantageous for image storage, transmission, and processing.

[0109] In the field of image-video coding, an encoder can obtain residual blocks and then transform those residual blocks. Transformation methods include, but are not limited to, the Discrete Cosine Transform (DCT) and the Discrete Sine Transform (DST). Because the DCT has strong energy concentration characteristics, after the original image undergoes a DCT transformation, non-zero coefficients may only exist in certain regions (e.g., the upper-left corner region). Of course, in video coding, images are processed by dividing them into blocks, so transformations are also performed based on these blocks. DCT types usable in video coding include, but are not limited to, DCT2 and DCT8, and DST types usable in video coding include, but are not limited to, DST7. Here, DCT2 is a transformation commonly used in video compression standards, and VVC can use DCT8 and DST7. While transformations are very useful for normal video compression, it's not always necessary to transform every block; sometimes, not transforming yields better compression results. Therefore, encoders may sometimes allow users to choose whether or not to use transformations on a given block.

[0110] When an encoder transforms the current block in the current image, it is possible to transform the residual block of the current block using a basis function or a basis image. A basis image is an image representation of a basis function.

[0111] Figure 12 shows an example of a DCT2 type base image according to an embodiment of the present application.

[0112] As shown in Figure 12, the DCT2 type basis image may be a basis image consisting of 8 × 8 small blocks created based on a basis function, where each small block consists of 8 × 8 elements (subblocks). In a specific embodiment, an 8 × 8 size block can be transformed using a basis image consisting of 8 × 8 small blocks, and the result is an 8 × 8 transformation coefficient matrix.

[0113] As mentioned above, VVC can perform basic transformations on residual blocks using the DCT2 type, as well as using the DCT8 and DST7 types, meaning that the multiple transform selection (MTS) technique in VVC can be used. The transformation type corresponding to the basis function used for the basic transformation is also called the transformation kernel type used for the basic transformation. When performing a basic transformation, the encoder can improve compression performance by selecting the optimal transformation kernel type based on different residual distribution characteristics. The basic transformation is also called the core transform. In MTS, the transformation kernel type can be selected through several syntax elements. Below, we show MTS for selecting the transformation kernel type through syntax elements in conjunction with Table 3.

[0114] [Table 3]

[0115] As shown in Table 3, if the value of MTS_CU_flag is 0, the transformation kernel type for all horizontal and vertical base transformations is DCT2. If the value of MTS_CU_flag is 1, and the value of MTS_Hor_flag is 0, and the value of MTS_Ver_flag is 0, then the horizontal transformation kernel type is DST7, and the vertical transformation kernel type is DST7.

[0116] The VVC standard also allows for rewriting, or in other words, simplifying, the syntax of MTS. Specifically, VVC can use a single syntax element, mts_idx, to determine the transformation kernel type of the underlying transformation.

[0117] [Table 4]

[0118] As shown in Table 4, trTypeHor indicates the conversion kernel type for horizontal transformation, and trTypeVer indicates the conversion kernel type for vertical transformation. A value of 0 for both trTypeHor and trTypeVer indicates a DCT2 type transformation, a value of 1 indicates a DST7 type transformation, and a value of 2 indicates a DCT8 type transformation.

[0119] Since there is a certain correlation between the residual distribution and the intra-prediction mode, this correlation can also be utilized in the underlying transformation. One method is to group the MTS transformation kernel types based on the intra-prediction mode. An example of one grouping is shown in the table below.

[0120] [Table 5]

[0121] As shown in Table 5, if the intra-prediction mode index is 0 or 1, the MTS selects the transformation kernel typeset with index 0. In VVC, the mode with index 0 is Planar, and the mode with index 1 is DC. Both DC and Planar produce flat prediction values. If the intra-prediction mode index is between 2 and 12, the MTS selects the transformation kernel typeset with index 1. According to the intra-prediction mode diagram, angles 2 through 12 all point downwards to the left.

[0122] Each transformation kernel type set may have one choice of transformation kernel type for horizontal and vertical transformations, or it may have multiple choices for horizontal and vertical transformation kernel types. In other words, after selecting a transformation kernel type set based on the intra-prediction mode, it is possible to further subdivide it, for example, by flags or block size information, but this will not be explained here. The emphasis is that in the base transformation, the transformation kernel type set can be selected based on the intra-prediction mode. This method of selecting the transformation kernel type set for the base transformation based on the intra-prediction mode may also allow for more detailed grouping of base transformations in the future, and this application does not specifically limit this.

[0123] Furthermore, in this application, the transformation kernel type relating to the main transformation is also referred to as a transformation matrix, transformation type, or transformation kernel, and the transformation kernel type set relating to the main transformation is also referred to as a transformation matrix set, transformation type set, or transformation kernel set, and this application does not specifically limit these terms. That is, the selection of a transformation kernel type or transformation kernel type set relating to this application is also referred to as the selection of a transformation matrix or transformation matrix set, the selection of a transformation type or transformation type set, and the selection of a transformation kernel or transformation kernel set. The transformation kernel type or transformation type may include DCT2, DCT8, DST7, etc., and may also include DCT5, DST4, DST1, or Identity Transform (IDTR), etc.

[0124] Furthermore, a corresponding size conversion kernel type can be used for blocks of different sizes, but this will not be described in this application.

[0125] Since all images are two-dimensional, and the computational complexity and memory overhead required for direct two-dimensional conversion cannot be handled by hardware limitations, it should be noted that the aforementioned DCT2, DCT8, and DST7 conversions are all divided into horizontal and vertical one-dimensional conversions, i.e., performed in two steps. For example, the horizontal conversion can be performed first, followed by the vertical conversion, or vice versa. The above conversion methods are relatively effective for horizontal and vertical textures, but less effective for diagonal textures. Since horizontal and vertical textures are the most common, the above conversion methods are very useful for improving compression efficiency. However, with technological advancements, processing only the residuals of horizontal and vertical textures is no longer sufficient to meet the needs of compression efficiency.

[0126] Based on this, the present invention introduces the concept of a secondary transformation, that is, the encoder can perform a secondary transformation based on the primary transformation, thereby improving compression efficiency.

[0127] Exemplary, the primary transform can be used to process horizontal and vertical textures, also called the basic transform, and includes, but is not limited to, the DCT2, DCT8, and DST7 transforms mentioned above. The secondary transform is used to process diagonal textures. The secondary transform includes, but is not limited to, the low-frequency non-separable transform (LFNST). On the encoding side, the secondary transform is performed after the primary transform and before quantization. On the decoding side, the secondary transform is performed after inverse quantization and before the inverse primary transform.

[0128] Figure 13 shows an example of an LFNST according to an embodiment of the present application.

[0129] As shown in Figure 13, on the encoding side, a quadratic transformation is performed on the low-frequency coefficients in the upper left corner after the basic transformation using LFNST. The main transformation concentrates the energy in the upper left corner by removing correlations to the image. Then, a quadratic transformation removes correlations again to the low-frequency coefficients after the main transformation. On the encoding side, if 16 coefficients are input to a 4x4 LFNST, 8 coefficients are output. If 64 coefficients are input to an 8x8 LFNST, 16 coefficients are output. On the decoding side, if 8 coefficients are input to a 4x4 inverse LFNST, 16 coefficients are output. If 16 coefficients are input to an 8x8 inverse LFNST, 64 coefficients are output.

[0130] When the encoder performs a quadratic transformation on the current block in the current image, it can use a transformation matrix from a selected set of transformation matrices to transform the residual block of the current block. For example, if the quadratic transformation is LFNST, the transformation matrix can refer to a matrix for transforming a texture in a certain diagonal direction, and the set of transformation matrices can contain matrices for transforming several similar textures in diagonal directions.

[0131] Figure 14 shows an example of a transformation matrix set for LFNST according to the present invention.

[0132] As shown in Figures 14(a) to (d), LFNST can have four sets of transformation matrices, and transformation matrices within the same set have similar diagonal textures. For example, the set of transformation matrices shown in Figure 14(a) may be a set of transformation matrices with index 0, the set of transformation matrices shown in Figure 14(b) may be a set of transformation matrices with index 1, the set of transformation matrices shown in Figure 14(c) may be a set of transformation matrices with index 2, and the set of transformation matrices shown in Figure 14(d) may be a set of transformation matrices with index 3.

[0133] To ensure understanding, in this application, the transformation matrix relating to a quadratic transformation is also referred to as a transformation kernel, transformation kernel type, or basis function, or any similar or identical term; and the set of transformation matrices relating to a quadratic transformation is also referred to as a transformation kernel set, transformation kernel type set, or basis function set, or any similar or identical term; and is not particularly limited in this application. That is, the selection of a transformation matrix or set of transformation matrices relating to this application is also referred to as the selection of a transformation kernel type or set of transformation kernel types, the selection of a transformation type or set of transformation types, and the selection of a transformation kernel or set of transformation kernels.

[0134] The following describes a correlation scheme for applying LFNST to intra-encoded blocks.

[0135] In intra prediction, a prediction is made for the current block using reconstructed samples around the current block as a reference. Currently, since video is generally encoded from left to right and top to bottom, the reference samples available for the current block are typically on the left and top sides. In angular prediction, the reference samples in the current block are arranged according to a given angle to obtain the predicted value, which means that the predicted block has an obvious directional texture, and the residual after the current block undergoes angular prediction exhibits a statistically obvious angular characteristic. Therefore, the transformation matrix selected for LFNST can be associated with the intra prediction mode, i.e., after the intra prediction mode is determined, LFNST can use a set of transformation matrices in which the texture direction fits the angular features of the intra prediction mode.

[0136] For illustrative purposes, we assume there are a total of four sets of transformation matrices for LFNST, with two transformation matrices per set. Table 6 shows the correspondence between the intra-prediction modes and the sets of transformation matrices.

[0137] [Table 6]

[0138] As shown in Table 6, intra-prediction modes 0 to 81 can be associated with indices in four sets of transformation matrices.

[0139] It should be noted that the cross-component prediction modes used for saturation intra-prediction are 81-83, while these modes are not available for luminance intra-prediction. By transposing the LFNST transformation matrix, a single set of transformation matrices can handle more angles. For example, both intra-prediction modes 13-23 and intra-prediction modes 45-55 correspond to transformation matrix set 2, but intra-prediction modes 13-23 are clearly closer to horizontal, and intra-prediction modes 45-55 are clearly closer to vertical, and therefore need to be adapted by transposing after the transformation or inverse transformation corresponding to intra-prediction modes 45-55.

[0140] In a specific embodiment, since there are four sets of transformation matrices for LFNST, the encoding side can determine which set of transformation matrices to use for LFNST based on the intra-prediction mode currently used for the block, and further determine which transformation matrices to use within the determined set. In other words, by utilizing the correlation between the intra-prediction mode and the LFNST transformation matrix sets, the transmission of selecting LFNST transformation matrices in the bitstream can be reduced. Whether to use LFNST for the current block, and if so, whether to use the first or second transformation matrix from the transformation matrix set, can be determined by the bitstream and several conditions.

[0141] Of course, considering that there are 67 normal intra-prediction modes and LFNST only has 4 sets of transformation matrices, multiple near-angle prediction modes can only be represented by one set of LFNST transformation matrices. This is a result of considering the trade-off between performance and complexity, as each transformation matrix requires memory space to store the coefficients of the transformation matrix. As the demand for compression efficiency increases and hardware capabilities improve, LFNST can also be designed to be more complex. For example, by using larger transformation matrices, using more sets of transformation matrices, and using more transformation matrices in each set of transformation matrices. Exemplarily, Table 7 shows another correspondence between intra-prediction modes and sets of transformation matrices.

[0142] [Table 7]

[0143] As shown in Table 7, 35 transformation matrix sets are used, with 3 transformation matrices used in each transformation matrix set. The correspondence between the transformation matrix sets and intra-prediction modes can be realized as follows: Intra-prediction modes 0 to 34 correspond to transformation matrix sets 0 to 34 in the forward direction, meaning that the larger the prediction mode number, the larger the index of the transformation matrix set; intra-prediction modes 35 to 67 correspond to transformation matrix sets 2 to 33 in the reverse direction due to transposition, meaning that the larger the prediction mode number, the smaller the index of the transformation matrix set; the remaining prediction modes can be uniformly corresponded to the transformation matrix set with index 2. In other words, if transposition is not considered, one intra-prediction mode corresponds to one transformation matrix set. With this design, a more appropriate LFNST transformation matrix can be obtained for the residuals corresponding to each intra-prediction mode, and the compression performance is also improved.

[0144] Of course, theoretically, a one-to-one ratio can be achieved even in wide-angle mode, but the cost-effectiveness of such a design is relatively low, and this application will not provide a detailed explanation of this.

[0145] Furthermore, in the case of LFNST, in order to adapt MIP to the transformation matrix set, in this application, the transformation matrix set adapted to planar modes can be made into the transformation matrix set adapted to MIP.

[0146] It should be noted that LFNST is merely one example of a quadratic transformation and should not be understood as a limitation of quadratic transformations. For example, LFNST is an inseparable quadratic transformation, and in other alternative embodiments, separable quadratic transformations can be used to improve the compression efficiency of diagonal texture residuals.

[0147] Figure 15 is a block diagram showing a decoding framework 200 according to an embodiment of the present application.

[0148] As shown in Figure 15, the decoding framework 200 may include an entropy decoding unit 210, an inverse transform / inverse quantization unit 220, a residual unit 230, an intra prediction unit 240, an inter prediction unit 250, a loop filtering unit 260, and a decoding image buffer unit 270.

[0149] The entropy decoding unit 210 receives and decodes the bitstream to obtain a prediction block and a frequency-domain residual block. The frequency-domain residual block can be subjected to steps such as inverse transformation and inverse quantization through the inverse transformation / inverse quantization unit 220 to obtain a time-domain residual block. The residual unit 230 can obtain a reconstructed block by adding the prediction block obtained by the intra-prediction unit 240 or inter-prediction unit 250 to the time-domain residual block obtained after inverse transformation and inverse quantization through the inverse transformation / inverse quantization unit 220.

[0150] Figure 16 is a flowchart illustrating a decoding method 310 according to an embodiment of the present application. To make it clear, the decoding method 310 can be performed by a decoder. For example, the decoding method 310 is applied to the decoding framework 200 shown in Figure 15. For the sake of clarity, a decoder will be described below as an example.

[0151] As shown in Figure 16, the decoding method 310 may include some or all of the following: S311: The decoder decodes the bitstream and obtains the first transformation coefficient of the current block. S312: The decoder performs the first transformation on the first transformation coefficient and obtains the residual block of the current block. S313: The decoder makes a prediction for the current block based on the first and second prediction modes corresponding to the geometric division mode, and obtains the predicted block for the current block. S314: The decoder retrieves the reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0152] In this embodiment, the decoder can perform a first transformation on the first transformation coefficient of the current block, and at the same time make predictions on the current block based on a first prediction mode and a second prediction mode corresponding to the geometric division mode, which can improve the decompression efficiency of the current block.

[0153] Of course, in other alternative embodiments, a quadratic transformation may be introduced in addition to the first transformation, and the method for matching the geometric partitioning mode with the first transformation may also be applied to the method for matching the geometric partitioning mode with the quadratic transformation. For example, the quadratic transformation may be the LFNST described above, i.e., the method for matching the geometric partitioning mode with the first transformation may also be applied to the method for matching the geometric partitioning mode with the LFNST. As another example, the LFNST is an inseparable quadratic transformation, and in other alternative embodiments, the geometric partitioning mode may also be applied to a separable quadratic transformation, i.e., the method for matching the geometric partitioning mode with the first transformation may also be applied to the method for matching the geometric partitioning mode with the inseparable quadratic transformation.

[0154] The geometric partitioning mode may be used for intra-prediction or inter-prediction. In other words, the prediction modes corresponding to the geometric partitioning mode (i.e., the first prediction mode and the second prediction mode) may both be intra-prediction modes, or both be inter-prediction modes, or may include both intra-prediction modes and inter-prediction modes simultaneously. In other words, in the geometric partitioning mode, two arbitrary prediction blocks can be combined using a weight matrix. For example, two inter-prediction blocks, or two intra-prediction blocks, or one inter-prediction block and one intra-prediction block can be combined. Furthermore, in screen content coding, one or two prediction blocks from intra-block copy (IBC) mode or palette mode can be used. For convenience of explanation, in this application, intra-mode, inter-mode, IBC mode, and palette mode are collectively referred to as prediction modes. A prediction mode can be understood as a basis on which the codec can generate information for one prediction block of the current block. Illustratively, in intra-prediction, the prediction mode may be an intra-prediction mode such as DC mode, planar mode, or various types of intra-angle prediction modes. Of course, some or more auxiliary information may be superimposed, such as an optimization method for intra-reference samples or an optimization method after generating rudimentary prediction blocks (e.g., filtering). Illustratively, in inter-prediction, the prediction mode may be a merge mode, a Merge with Motion Vector Difference (MMVD) mode, or an Advanced Motion Vector Prediction (AMVP) mode. Illustratively, the prediction mode may be unidirectional prediction, bidirectional prediction, or multi-hypothesis prediction. Furthermore, in inter-prediction modes, if unidirectional prediction is used and one piece of motion information can be determined, then prediction blocks can be determined based on the motion information.In interpretation mode, bidirectional prediction is used, and if two pieces of motion information can be determined, the prediction block can be determined based on the motion information.

[0155] Of course, the scheme provided in this application is also applicable to any prediction mode that performs weighted predictions based on multiple prediction modes, such as AWP, and this application does not specifically limit it to such modes.

[0156] In some embodiments, S312 is Decode the bitstream and obtain the first and second flags, If the first flag indicates that the use of geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the use of the first transformation is permitted to make transformations for blocks in the current sequence, then the first transformation is performed on the first transformation coefficients to obtain the residual blocks, It can include...

[0157] For example, the current sequence is the image sequence that includes the current block.

[0158] For example, the first flag is used to control whether to use geometric partitioning mode for the current sequence.

[0159] Exemplary, if the value of the first flag is the first value, the first flag indicates that the use of geometric partitioning mode to make predictions for blocks in the current sequence is permitted. If the value of the first flag is the second value, the first flag indicates that the use of geometric partitioning mode to make predictions for blocks in the current sequence is not permitted. In one embodiment, the first value is 0 and the second value is 1, and in another embodiment, the first value is 1 and the second value is 0. Of course, the first value or the second value may be any other value.

[0160] For example, the second flag is used to control whether to use the first transformation for the current sequence.

[0161] For example, if the value of the second flag is the third value, the second flag indicates that the first transformation is permitted to be used to perform a transformation on the block in the current sequence. If the value of the second flag is the fourth value, the second flag indicates that the first transformation is not permitted to be used to perform a transformation on the block in the current sequence. In one embodiment, the third value is 0 and the fourth value is 1, and in another embodiment, the third value is 1 and the fourth value is 0. Of course, the third value or the fourth value may be any other value.

[0162] For example, if we denote the first flag as sps_gpm_enabled_flag and the second flag as sps_mts_enabled_flag, then if both sps_gpm_enabled_flag and sps_mts_enabled_flag have a value of 1, we perform the first transformation on the first transformation coefficient to obtain the residual block.

[0163] For example, if the first flag indicates that the use of geometric partitioning mode is not permitted to make predictions for blocks in the current sequence, and / or the second flag indicates that the use of the first transformation is not permitted to make transformations for blocks in the current sequence, then the first transformation can be omitted from the first transformation coefficients, or in other words, another type of transformation can be directly applied to the first transformation coefficients to obtain the residual values ​​of the current blocks.

[0164] Of course, in other alternative embodiments, the first flag and / or the second flag may be replaced with flags at the picture, slice, largest coding unit (LCU), coding tree unit (CTU), coding unit (CU), prediction unit (PU), or transform unit (TU) levels. Alternatively, additional flags at the picture, slice, LCU, CTU, CU, PU, ​​or TU levels may be added based on the first and second flags to indicate whether to use the geometric partitioning mode or the first transform. The embodiments of this application are not limited thereto.

[0165] In some embodiments, S312 is If the first flag indicates that the use of geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the use of the first transformation is permitted to make transformations for blocks in the current sequence, then decode the bitstream and obtain the third flag. If the third flag indicates that both the geometric partitioning mode and the first transformation are permitted for the block in the current sequence, then the first transformation is performed on the first transformation coefficients to obtain the residual block, It can include...

[0166] For example, the third flag is used to control whether both the geometric partitioning mode and the first transformation can be used.

[0167] For example, if the value of the third flag is the fifth value, the third flag indicates that both the geometric partitioning mode and the first transformation are permitted for the blocks in the current sequence. If the value of the third flag is the sixth value, the third flag indicates that both the geometric partitioning mode and the first transformation are not permitted for the blocks in the current sequence. In one embodiment, the fifth value is 0 and the sixth value is 1, and in another embodiment, the fifth value is 1 and the sixth value is 0. Of course, the fifth value or the sixth value may be any other value.

[0168] For example, if we denote the first flag as sps_gpm_enabled_flag, the second flag as sps_mts_enabled_flag, and the third flag as sps_gpm_mts_enabled_flag, then if both sps_gpm_enabled_flag and sps_mts_enabled_flag have a value of 1, we determine whether sps_gpm_mts_enabled_flag is 1. If sps_gpm_mts_enabled_flag is 1, we perform the first transformation on the first transformation coefficient to obtain the residual block.

[0169] Of course, in other alternative embodiments, the third flag may be replaced with a flag at the picture, slice, largest coding unit (LCU), coding tree unit (CTU), coding unit (CU), prediction unit (PU), or transform unit (TU) level. Alternatively, based on the third flag, additional flag information at the picture, slice, LCU, CTU, CU, PU, ​​or TU level may be added indicating whether to use the geometric partitioning mode or the first transform. The embodiments of this application are not limited thereto.

[0170] In some embodiments, S312 is If a first flag indicates that a geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and a second flag indicates that a first transformation is permitted to make transformations for blocks in the current sequence, the decoder may include performing a first transformation on a first transformation coefficient to obtain a residual block when the height and / or width of the current block is greater than or equal to a first threshold.

[0171] For example, if we denote the first flag as sps_gpm_enabled_flag, the second flag as sps_mts_enabled_flag, and the third flag as sps_gpm_mts_enabled_flag, then if both sps_gpm_enabled_flag and sps_mts_enabled_flag have a value of 1, the decoder determines the height and / or width of the current block, and if the height and / or width of the current block is greater than or equal to the first threshold, it performs the first transformation on the first transformation coefficient to obtain the residual block.

[0172] For example, the first threshold can be 4, 8, 16, 32, 64, or any other value.

[0173] In some embodiments, method 310 may further include, prior to S312, having the decoder determine the conversion kernel typeset to be used for the first conversion.

[0174] In geometric partitioning mode, two prediction modes (i.e., a first prediction mode and a second prediction mode) are combined to predict the current block. Predicted blocks obtained by predicting the current block using different prediction modes may have different texture characteristics. Therefore, when geometric partitioning mode is selected for the current block, the first prediction mode may result in one texture characteristic for the predicted block of the current block, and the second prediction mode may result in another texture characteristic for the predicted block of the current block. In other words, after predicting the current block, from a statistical standpoint, the residual block of the current block will also show two texture characteristics. That is, the residual block of the current block does not necessarily conform to the rules that can be reflected in a given prediction mode. Therefore, for geometric partitioning mode, the decoder needs to determine a transformation kernel typeset that fits the characteristics before performing the first transformation on the first transformation coefficients. However, since the transformation kernel typeset used for the first transformation is usually defined based on a single intra-prediction mode, the relevant scheme for determining the transformation kernel typeset used for the first transformation needs to be further improved for geometric partitioning mode, and each embodiment is described below illustratively.

[0175] In some embodiments, the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0176] For example, when the decoder checks the predicted mode of the current block, if a geometric partitioning mode is used for the current block, the decoder classifies the geometric partitioning mode and the planar mode (or DC mode) into one type and determines the conversion kernel type set to be used for the first conversion based on the planar mode (or DC mode). Alternatively, when the decoder checks the predicted mode of the current block, if a geometric partitioning mode is used for the current block, the encoder may return the planar mode (or DC mode) as the predicted mode of the current block, and based on this, the decoder may determine the conversion kernel type set that conforms to the planar mode (or DC mode) as the conversion kernel type set to be used for the first conversion. Alternatively, when the decoder checks the predicted mode of the current block, if a geometric partitioning mode is used for the current block, the decoder may consider that the conversion kernel type set used for the first conversion of the current block may be a conversion kernel type set that conforms to the planar mode (or DC mode).

[0177] In this embodiment, since both the planar mode (or DC mode) and the geometric partitioning mode can represent various texture characteristics, by determining a transformation kernel typeset that matches the planar mode or DC mode as the transformation kernel typeset used for the first transformation, it is not only possible to decode the current block based on the geometric partitioning mode and the first transformation, but it is also guaranteed that the texture characteristics of the transformation kernel typeset used for the first transformation will be as similar as possible to the texture characteristics of the residual block of the current block, thereby improving decompression efficiency.

[0178] For example, if the prediction modes corresponding to the geometrically divided modes (i.e., the first and second prediction modes) are both intra-prediction modes, or if the geometrically divided modes are used for intra-prediction, the decoder determines that the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the planar mode or DC mode.

[0179] For example, when the decoder checks the prediction mode of the current block, if a geometric division mode is used for the current block and both of the two prediction modes corresponding to the geometric division mode are intra-prediction modes, the decoder classifies the geometric division mode and the planar mode (or DC mode) into one type and determines the conversion kernel typeset to be used for the first conversion based on the planar mode (or DC mode). Alternatively, when the decoder checks the prediction mode of the current block, if a geometric division mode is used for the current block and both of the two prediction modes corresponding to the geometric division mode are intra-prediction modes, the encoder can return the planar mode (or DC mode) as the prediction mode of the current block, and based on this, the decoder can determine the conversion kernel typeset that conforms to the planar mode (or DC mode) as the conversion kernel typeset to be used for the first conversion. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block and both of the two prediction modes corresponding to the geometric partitioning mode are intra-prediction modes, the decoder may consider that the conversion kernel typeset used for the first conversion of the current block may be a conversion kernel typeset that conforms to planar mode (or DC mode).

[0180] For example, if the prediction modes corresponding to the geometrically divided modes (i.e., the first and second prediction modes) are both interpredictive modes, or if the geometrically divided modes are used for interprediction, the decoder determines that the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the planar mode or DC mode.

[0181] For example, when the decoder checks the prediction mode of the current block, if a geometric division mode is used for the current block and both of the two prediction modes corresponding to the geometric division mode are inter-prediction modes, the decoder classifies the geometric division mode and the planar mode (or DC mode) into one type and determines the conversion kernel typeset to be used for the first conversion based on the planar mode (or DC mode). Alternatively, when the decoder checks the prediction mode of the current block, if a geometric division mode is used for the current block and both of the two prediction modes corresponding to the geometric division mode are inter-prediction modes, the encoder can return the planar mode (or DC mode) as the prediction mode of the current block, and based on this, the decoder can determine the conversion kernel typeset that conforms to the planar mode (or DC mode) as the conversion kernel typeset to be used for the first conversion. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric division mode is used for the current block and both of the two prediction modes corresponding to the geometric division mode are inter-prediction modes, the decoder can consider that the conversion kernel typeset used for the first conversion of the current block may be a conversion kernel typeset that conforms to the planar mode (or DC mode).

[0182] In some embodiments, the decoder first determines a dividing line consisting of points where the weights of the first prediction mode and the weights of the second prediction mode are the same, then determines the angular index of the dividing line, and then, based on the angular index, determines the transformation kernel typeset to be used for the first transformation.

[0183] For example, if the prediction modes corresponding to the geometric partitioning modes (i.e., the first and second prediction modes) are both intra-prediction modes, or if the geometric partitioning modes are used for intra-prediction, the decoder first determines a partitioning line consisting of points where the weights of the first prediction mode and the weights of the second prediction mode are the same, then determines the angular index of the partitioning line, and then, based on the angular index, determines the transformation kernel typeset to be used for the first transformation. For example, the decoder determines the transformation kernel typeset to be used for the first transformation as the transformation kernel typeset that fits the prediction mode corresponding to the angular index (e.g., intra-prediction mode).

[0184] For example, if the prediction modes corresponding to the GPM geometric partitioning modes (i.e., the first and second prediction modes) are both interprediction modes, or if the geometric partitioning modes are used for interprediction, the decoder first determines a partition line consisting of points where the weights of the first prediction mode and the weights of the second prediction mode are the same, then determines the angular index of the partition line, and then, based on the angular index, determines the transformation kernel typeset to be used for the first transformation. For example, the decoder determines the transformation kernel typeset that fits the prediction mode corresponding to the angular index (e.g., the intraprediction mode) as the transformation kernel typeset to be used for the first transformation.

[0185] For example, when predicting a current block using a geometric partitioning mode, the decoder can classify the geometric partitioning mode and the intra-prediction mode corresponding to the partition line into one type based on the partition line of the geometric partitioning mode when selecting the transformation kernel typeset to be used for the first transformation. That is, it can determine the transformation kernel typeset to be used for the first transformation based on the angle index. Specifically, it determines the angle index (angleIdx) of the partition line based on the weight derivation mode used for the geometric partitioning mode, and then determines the transformation kernel typeset that fits the intra-prediction mode corresponding to the angle index as the transformation kernel typeset to be used for the first transformation. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the encoder can return the intra-prediction mode corresponding to the partition line as the prediction mode of the current block. Based on this, the decoder can determine the transformation kernel typeset that fits the intra-prediction mode corresponding to the partition line as the transformation kernel typeset to be used for the first transformation. Alternatively, when the decoder checks the prediction mode for the current block, if a geometric partitioning mode is used for the current block, the decoder may assume that the transformation kernel typeset used for the first transformation of the current block may be a transformation kernel typeset that conforms to the intra-prediction mode corresponding to the partition line.

[0186] In this embodiment, the division lines or angle index can not only represent the characteristics of the geometric division mode, but also to some extent the texture characteristics of the residual block of the current block. By associating the angle index with the intra prediction mode, it is possible to decode the current block based on the geometric division mode and the first transformation. Furthermore, it is ensured that the texture characteristics of the transformation kernel typeset used in the first transformation are as similar as possible to the texture characteristics of the residual block of the current block, thereby improving decompression efficiency.

[0187] To ensure clarity, the division lines described above may be for the weight derivation mode, the weight matrix, or the weight matrix derivation mode. Their specific details can be found in the explanations and related information in Figure 8 or 9; they are not explained here to avoid repetition.

[0188] Exemplary, a decoder may determine a set of transformation kernel types to be used for a first transformation based on a first mapping relationship and an angular index. The first mapping relationship includes a correspondence between at least one index and at least one intra-prediction mode, where this at least one index includes an angular index. Of course, the first mapping relationship may be implemented in the form of a table, an array, or other form, and is not specifically limited thereto.

[0189] For example, the first mapping relationship can be realized as shown in Table 8 below.

[0190] [Table 8]

[0191] As shown in Table 8, the first mapping relationship can include 32 indices and an intra-prediction mode corresponding to each index.

[0192] The indices in Table 8 may include angleIdx shown in Table 1, and of course, may also include indices other than angleIdx shown in Table 1. For example, some indices correspond to 0. This is because these indices are not used in geometric division modes, that is, they are not included in Table 1. Furthermore, if some indices change in a certain version, for example, if more indices are used in the future, or if the intra-angle prediction mode is changed, for example, if there are more intra-angle prediction modes in the future, the correspondence table shown in Table 8 may also be changed accordingly, and is not particularly limited in this application.

[0193] In this embodiment, the decoder may determine the transformation kernel type set used for the first transformation based solely on the angle index of the dividing line. In other alternative embodiments, the decoder may also determine the transformation kernel type set used for the first transformation based solely on the distance index of the dividing line. For example, the decoder may determine the transformation kernel type set used for the first transformation to be one that conforms to the prediction mode (e.g., intra-prediction mode) corresponding to the distance index of the dividing line. Alternatively, the decoder may determine the transformation kernel type set used for the first transformation based on the dividing line (i.e., the angle index of the dividing line and the distance index of the dividing line). For example, the decoder may determine the transformation kernel type set used for the first transformation to be one that conforms to the prediction mode (e.g., intra-prediction mode) corresponding to the dividing line (i.e., the angle index of the dividing line and the distance index of the dividing line). This invention is not particularly limited thereto.

[0194] In some embodiments, the decoder first determines the weight derivation mode to be used for the geometric partitioning mode, and then determines the transformation kernel typeset that fits the intra-prediction mode corresponding to the weight derivation mode as the transformation kernel typeset to be used for the first transformation.

[0195] For example, if the prediction modes corresponding to the geometric partitioning modes (i.e., the first and second prediction modes) are both intra-prediction modes, or if the geometric partitioning modes are used for intra-prediction, the decoder first determines the weight derivation mode used for the geometric partitioning modes, and then determines the transformation kernel typeset that fits the intra-prediction modes corresponding to the weight derivation modes as the transformation kernel typeset used for the first transformation.

[0196] For example, if the prediction modes corresponding to the geometric partitioning modes (i.e., the first and second prediction modes) are both interprediction modes, or if the geometric partitioning modes are used for interprediction, the decoder first determines the weight derivation mode used for the geometric partitioning modes, and then determines the transformation kernel typeset that fits the intraprediction mode corresponding to the weight derivation modes as the transformation kernel typeset used for the first transformation.

[0197] For example, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the decoder classifies the geometric partitioning mode and the intra-prediction mode corresponding to the weight derivation mode into one type, and determines the transformation kernel typeset to be used for the first transformation based on the intra-prediction mode corresponding to the weight derivation mode. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the encoder can return the intra-prediction mode corresponding to the weight derivation mode as the prediction mode of the current block, and based on this, the decoder can determine the transformation kernel typeset that matches the intra-prediction mode corresponding to the weight derivation mode as the transformation kernel typeset to be used for the first transformation. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the decoder can consider that the transformation kernel typeset used for the first transformation of the current block may be a transformation kernel typeset that matches the intra-prediction mode corresponding to the weight derivation mode.

[0198] In this embodiment, the weight derivation mode can not only represent the characteristics of the geometric partitioning mode, but also to some extent the texture characteristics of the residual blocks of the current block. By associating the weight derivation mode with the intra prediction mode, it is possible to decode the current block based on the geometric partitioning mode and the first transformation. Furthermore, it is ensured that the texture characteristics of the transformation kernel typeset used in the first transformation are as similar as possible to the texture characteristics of the residual blocks of the current block, thereby improving decompression efficiency.

[0199] Exemplary, the decoder may determine, based on a second mapping relationship, a set of transformation kernel types that conforms to the intra-prediction modes corresponding to the weight derivation modes used for the geometric partitioning modes, as the set of transformation kernel types used for the first transformation. The second mapping relationship includes a correspondence between at least one weight derivation mode and at least one intra-prediction mode, where the at least one weight derivation mode includes a weight derivation mode used for the geometric partitioning modes.

[0200] Of course, the second mapping relationship may be implemented in the form of a table, or in other forms such as an array, and this application is not specifically limited to these forms.

[0201] The weight derivation mode described above can also be called the weight matrix or weight matrix derivation mode. For specific details, please refer to the explanation in Figure 8 and Table 1, and related information. To avoid repetition, no further explanation is provided here.

[0202] In some embodiments, the first prediction mode is the first intra-prediction mode, and the second prediction mode is the second intra-prediction mode. The decoder determines the third intra-prediction mode based on the first and second intra-prediction modes. The conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0203] For example, the decoder may determine a set of transformation kernel types that conforms to a third intra-prediction mode as the set of transformation kernel types used for the first transformation.

[0204] For example, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the decoder can determine a third intra-prediction mode based on the first and second intra-prediction modes, classify the geometric partitioning mode and the third intra-prediction mode into one type, and based on this, the decoder can determine the conversion kernel typeset to be used for the first conversion based on the third intra-prediction mode. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the encoder can return a third intra-prediction mode as the prediction mode of the current block, and based on this, the decoder can determine the conversion kernel typeset that matches the third intra-prediction mode as the conversion kernel typeset to be used for the first conversion. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, the decoder can consider the conversion kernel typeset used for the first conversion of the current block to be a conversion kernel typeset that matches the third intra-prediction mode.

[0205] Of course, in other alternative embodiments, the decoder does not necessarily have to explicitly determine a third intra-prediction mode first, and then determine the conversion kernel typeset to be used for the first conversion based on the third intra-prediction mode. Instead, the decoder directly uses the conversion kernel typeset that fits the third intra-prediction mode as the conversion kernel typeset to be used for the first conversion.

[0206] In some embodiments, the decoder determines the default prediction mode from among the first and second intra-prediction modes as the third intra-prediction mode. Alternatively, the decoder determines the intra-prediction mode from among the first and second intra-prediction modes that corresponds to the weight derivation mode used for the geometric partitioning mode as the third intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0207] For example, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, it can determine the conversion kernel typeset to be used for the first conversion based on the first intra-prediction mode and the second intra-prediction mode when selecting the conversion kernel typeset to be used for the first conversion. In one embodiment, the determination can be made using the first intra-prediction mode in any case, i.e., in any case, the conversion kernel typeset that conforms to the first intra-prediction mode is determined as the conversion kernel typeset to be used for the first conversion; or, the determination can be made using the second intra-prediction mode in any case, i.e., in any case, the conversion kernel typeset that conforms to the second intra-prediction mode is determined as the conversion kernel typeset to be used for the first conversion. In another embodiment, the conversion kernel typeset may be determined using a first intra-prediction mode, i.e., a conversion kernel typeset that conforms to the first intra-prediction mode may be determined as the conversion kernel typeset used for the first conversion; or, the conversion kernel typeset may be determined using a second intra-prediction mode, i.e., a conversion kernel typeset that conforms to the second intra-prediction mode may be determined as the conversion kernel typeset used for the first conversion; furthermore, the conversion kernel typeset may be determined using a planar mode or DC mode, i.e., a conversion kernel typeset that conforms to the planar mode or DC mode may be determined as the conversion kernel typeset used for the first conversion. Determining using a certain prediction mode means classifying the geometric partitioning mode and a certain prediction mode into one type, and based on this, the decoder can determine the conversion kernel typeset used for the first conversion based on a certain prediction mode. Alternatively, when the decoder checks the prediction mode of the current block, if a geometric partitioning mode is used for the current block, it can return a certain prediction mode, and based on this, the decoder can determine the conversion kernel typeset used for the first conversion based on a certain prediction mode.Alternatively, when the decoder checks the prediction mode for the current block, if a geometric partitioning mode is used for the current block, the decoder may assume that the transformation kernel typeset used for the first transformation of the current block may be a transformation kernel typeset that conforms to a certain prediction mode.

[0208] For example, if the decoder determines, among the first and second intra-prediction modes, the intra-prediction mode corresponding to the weight derivation mode used for the geometric partitioning mode as the third intra-prediction mode, then based on the third mapping relationship, the decoder can determine, as the third intra-prediction mode, the intra-prediction mode corresponding to the weight derivation mode used for the geometric partitioning mode.

[0209] The third mapping relationship includes a weight derivation mode corresponding to a first intra-prediction mode and a weight derivation mode corresponding to a second intra-prediction mode, wherein the weight derivation mode corresponding to the first intra-prediction mode includes a weight derivation mode used for geometric partitioning mode, or the weight derivation mode corresponding to the second intra-prediction mode includes a weight derivation mode used for geometric partitioning mode. In other words, based on the third mapping relationship, if the weight derivation mode corresponding to the first intra-prediction mode includes a weight derivation mode used for geometric partitioning mode, the first intra-prediction mode is determined as the third intra-prediction mode. If the weight derivation mode corresponding to the second intra-prediction mode includes a weight derivation mode used for geometric partitioning mode, the second intra-prediction mode is determined as the third intra-prediction mode. Alternatively, the third mapping relationship can be used to define a weight derivation mode corresponding to a first prediction mode and a weight derivation mode corresponding to a second prediction mode. In one specific embodiment, the third mapping relationship may include only the weight derivation mode corresponding to the first intra-prediction mode and the weight derivation mode corresponding to the second intra-prediction mode, while in another specific embodiment, the third mapping relationship may also include the weight derivation mode corresponding to intra-prediction modes other than the first intra-prediction mode and the second intra-prediction mode.

[0210] Of course, the third mapping relationship may be implemented in the form of a table, or in other forms such as an array, and this application is not specifically limited to these forms.

[0211] In some embodiments, the decoder can determine a third intra-prediction mode based on the weights of a first intra-prediction mode or a second intra-prediction mode used at the default position.

[0212] For example, the third intra-prediction mode is related to the weights of the first intra-prediction mode or the second intra-prediction mode used at the default position.

[0213] For example, the weights of the first intra-prediction mode used at the default position may be the weights used when predicting a point at the default position of the current block using the first intra-prediction mode. Similarly, the weights of the second intra-prediction mode used at the default position may be the weights used when predicting a point at the default position of the current block using the second intra-prediction mode. For example, in connection with the geometric partitioning modes described above, the weights of the first intra-prediction mode used at the default position may be the wValue calculated at the default position, and the weights of the second intra-prediction mode used at the default position may be the 8-wValue calculated at the default position.

[0214] In some embodiments, when the decoder determines a third intra-prediction mode based on the weights of a first intra-prediction mode and / or a second intra-prediction mode, it may determine the intra-prediction mode with the largest weight at the default position among the first and second intra-prediction modes as the third intra-prediction mode.

[0215] For example, if the weight of the first intra-prediction mode used at the default position is greater than the weight of the second intra-prediction mode used at the default position, the decoder can either determine the first intra-prediction mode as the third intra-prediction mode, or directly determine a conversion kernel typeset that conforms to the first intra-prediction mode as the conversion kernel typeset used for the first conversion. If the weight of the second intra-prediction mode used at the default position is greater than the weight of the first intra-prediction mode used at the default position, the decoder can either determine the second intra-prediction mode as the third intra-prediction mode, or directly determine a conversion kernel typeset that conforms to the second intra-prediction mode as the conversion kernel typeset used for the first conversion.

[0216] Of course, in other alternative embodiments, the third intra-prediction mode may be determined based solely on the weights of the first intra-prediction mode or the weights of the second intra-prediction mode. For example, if the weight of the first intra-prediction mode is greater than a certain threshold, the first intra-prediction mode can be determined as the third intra-prediction mode; otherwise, the second intra-prediction mode can be determined as the third intra-prediction mode. For example, if the weight of the second intra-prediction mode is greater than a certain threshold, the second intra-prediction mode can be determined as the third intra-prediction mode; otherwise, the first intra-prediction mode can be determined as the third intra-prediction mode.

[0217] Of course, in other alternative embodiments, the weights of the first intra-prediction mode may be the weights of the first intra-prediction mode statistically aggregated against the weight matrix used for the geometric partitioning mode, and similarly, the weights of the second intra-prediction mode may be the weights of the second intra-prediction mode statistically aggregated against the weight matrix used for the geometric partitioning mode. In other words, the decoder can determine the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the second intra-prediction mode by first calculating the weight value of each point and statistically aggregating the weight values ​​of all points for each of the first intra-prediction mode and / or the second intra-prediction mode. In one embodiment, if the weight of the first intra-prediction mode is greater than a threshold, the first intra-prediction mode can be determined as the third intra-prediction mode; otherwise, the second intra-prediction mode can be determined as the third intra-prediction mode. In another embodiment, the intra-prediction mode with the largest weight among the first intra-prediction mode and the second intra-prediction mode can be determined as the third intra-prediction mode.

[0218] In some embodiments, the default position is the center position.

[0219] Of course, in other alternative embodiments, the default position may be any other position, such as the top left, top right, bottom left, or bottom right.

[0220] In some embodiments, when the decoder determines a third intra-prediction mode based on the weights of a first intra-prediction mode or a second intra-prediction mode used at the default position, it may first determine a dividing line consisting of points where the weights of the first and second prediction modes are the same, and then determine the third intra-prediction mode based on the dividing line. For example, the decoder may determine the third intra-prediction mode based on the angle index and / or distance index of the dividing line.

[0221] For example, when the decoder determines the intra-prediction mode corresponding to the angle index of the dividing line from among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode, it can determine the intra-prediction mode corresponding to the angle index of the dividing line as the third intra-prediction mode based on a fourth mapping relationship.

[0222] The fourth mapping relationship includes an angle index corresponding to a first intra-prediction mode and an angle index corresponding to a second intra-prediction mode, wherein the angle index corresponding to the first intra-prediction mode includes an angle index used for the dividing line, or the angle index corresponding to the second intra-prediction mode includes an angle index used for the dividing line. In other words, based on the fourth mapping relationship, if the angle index corresponding to the first intra-prediction mode includes an angle index for the dividing line, the first intra-prediction mode is determined as the third intra-prediction mode. If the angle index corresponding to the second intra-prediction mode includes an angle index for the dividing line, the second intra-prediction mode is determined as the third intra-prediction mode. Alternatively, the fourth mapping relationship can be used to define the angle index corresponding to the first prediction mode and the angle index corresponding to the second prediction mode. In one specific embodiment, the fourth mapping relationship may include only the angular index corresponding to the first intra-prediction mode and the angular index corresponding to the second intra-prediction mode, while in another specific embodiment, the fourth mapping relationship may also include the angular index corresponding to intra-prediction modes other than the first intra-prediction mode and the second intra-prediction mode.

[0223] Of course, the fourth mapping relationship may be implemented in the form of a table, or in other forms such as an array, and this application is not specifically limited to these forms.

[0224] For example, when the decoder determines, as the third intra-prediction mode, the intra-prediction mode corresponding to the distance index used for the dividing line, from among the first and second intra-prediction modes, it can determine, as the third intra-prediction mode, based on a fifth mapping relationship.

[0225] The fifth mapping relationship includes a distance index corresponding to a first intra-prediction mode and a distance index corresponding to a second intra-prediction mode, wherein the distance index corresponding to the first intra-prediction mode includes a distance index used for dividing lines, or the distance index corresponding to the second intra-prediction mode includes a distance index used for dividing lines. In other words, based on the fifth mapping relationship, if the distance index corresponding to the first intra-prediction mode includes a distance index used for dividing lines, the first intra-prediction mode is determined as the third intra-prediction mode. If the distance index corresponding to the second intra-prediction mode includes a distance index used for dividing lines, the second intra-prediction mode is determined as the third intra-prediction mode. Alternatively, the fifth mapping relationship can be used to define the distance index corresponding to the first prediction mode and the distance index corresponding to the second prediction mode. In one specific embodiment, the fifth mapping relationship may include only distance indices corresponding to the first intra-prediction mode and distance indices corresponding to the second intra-prediction mode, while in another specific embodiment, the fifth mapping relationship may also include distance indices corresponding to intra-prediction modes other than the first intra-prediction mode and the second intra-prediction mode.

[0226] Of course, the fifth mapping relationship may be implemented in the form of a table, or in other forms such as an array, and this application is not specifically limited to these forms.

[0227] In some embodiments, when the decoder determines a third intra-prediction mode based on the type of a first intra-prediction mode and the type of a second intra-prediction mode, if the first and second intra-prediction modes include an angle prediction mode and a non-angle prediction mode, the decoder determines the angle prediction mode as the third intra-prediction mode.

[0228] For example, when a decoder determines a third intra-prediction mode based on the types of the first and second intra-prediction modes, the priority given to an angle prediction mode as the third intra-prediction mode is higher than the priority given to a non-angle prediction mode as the third intra-prediction mode. For instance, if the first intra-prediction mode is an angle prediction mode and the second intra-prediction mode is a non-angle prediction mode (e.g., a planar mode or a DC mode), the decoder determines the first intra-prediction mode (i.e., the angle prediction mode) as the third intra-prediction mode.

[0229] In some embodiments, when the decoder determines a third intra-prediction mode based on the prediction angle of a first intra-prediction mode and the prediction angle of a second intra-prediction mode, if the prediction angle of the first intra-prediction mode approaches the prediction angle of the second intra-prediction mode, the decoder may determine the first intra-prediction mode, the second intra-prediction mode, or an intra-prediction mode whose prediction angle is between the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode as the third intra-prediction mode. If the difference between the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode is large, the decoder may determine a planar mode or a DC mode as the third intra-prediction mode.

[0230] In some embodiments, if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode. The first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, the planar mode or DC mode is determined as the third intra-prediction mode.

[0231] In some embodiments, the decoder determines a set of transformation kernel types that conforms to the geometric partitioning mode as the set of transformation kernel types to be used for the first transformation.

[0232] For example, if both prediction modes corresponding to a geometric partitioning mode (i.e., the first and second prediction modes) are intra-prediction modes, or if a geometric partitioning mode is used for intra-prediction, the decoder determines the transformation kernel typeset that conforms to the geometric partitioning mode as the transformation kernel typeset used for the first transformation.

[0233] For example, if the prediction modes corresponding to the geometric partitioning mode (i.e., the first and second prediction modes) are both interprediction modes, or if the geometric partitioning mode is used for interprediction, the decoder determines the transformation kernel typeset that fits the geometric partitioning mode as the transformation kernel typeset used for the first transformation.

[0234] For example, a decoder can define a set of transformation kernel types suited to or dedicated to geometric partitioning modes.

[0235] In some embodiments, the first transformation is used to process the horizontal and vertical textures in the current block.

[0236] In some embodiments, S312 is Performing a second transformation on the first transformation coefficient and obtaining the second transformation coefficient, The first transformation is applied to the second transformation coefficient to obtain the residual block of the current block, It can include...

[0237] In some embodiments, the second transformation is used to process diagonal textures in the current block.

[0238] As described above, the decoding method according to the embodiment of the present application will be described in detail from the viewpoint of the decoder, and below, with reference to Figure 17, the encoding method according to the embodiment of the present application will be described from the viewpoint of the encoder.

[0239] Figure 17 is a flowchart illustrating an encoding method 410 according to an embodiment of the present application. It should be understood that the encoding method 410 may be performed by an encoder and, for example, is applied to the encoding framework 100 shown in Figure 1. For the sake of explanation, an encoder will be described as an example.

[0240] As shown in Figure 17, the encoding method 410 may include the following: S411: A prediction is made for the current block based on the first and second prediction modes corresponding to the geometric partitioning mode, and the predicted block for the current block is obtained. S412: Based on the predicted block of the current block, obtain the residual block of the current block. S413: Perform a third transformation on the residual block of the current block and obtain the third transformation coefficient of the current block. S414: Encode the third conversion coefficient.

[0241] In some embodiments, S414 may include encoding a first flag, a second flag, and a third conversion coefficient. The first flag indicates that the geometric partitioning mode is permitted to be used to make predictions for blocks in the current sequence, and the second flag indicates that the third transformation is permitted to be used to perform transformations on blocks in the current sequence.

[0242] In some embodiments, S414 may include encoding a first flag, a second flag, a third conversion coefficient, and a third flag. The third flag indicates that it is permitted to use both the geometric division mode and the third transformation for the block in the current sequence.

[0243] In some embodiments, S413 may include performing a third transformation on the residual block of the current block and obtaining third transformation coefficients when the height and / or width of the current block is greater than or equal to a first threshold.

[0244] In some embodiments, before S414, method 410 may further include determining a set of transformation kernel types used for the third transformation. In some embodiments, the set of transformation kernel types used for the third transformation is the same as the set of transformation kernel types adapted to the planar mode or the DC mode.

[0245] In some embodiments, determining the set of transformation kernel types used for the third transformation includes determining a dividing line consisting of the point that the weights of the first prediction mode and the second prediction mode are the same, determining the angle index of the dividing line, and determining the set of transformation kernel types used for the third transformation based on the angle index. and includes.

[0246] In some embodiments, determining the set of transformation kernel types used for the third transformation includes determining the weight derivation mode used for the geometric division mode, and determining, as the set of transformation kernel types used for the third transformation, the set of transformation kernel types adapted to the intra prediction mode corresponding to the weight derivation mode. and includes.

[0247] In some embodiments, the first prediction mode is the first intra-prediction mode, and the second prediction mode is the second intra-prediction mode. Determining the transformation kernel typeset to be used for the third transformation includes determining the third intra-prediction mode based on the first and second intra-prediction modes. The transformation kernel typeset to be used for the third transformation is the same as the transformation kernel typeset that conforms to the third intra-prediction mode.

[0248] In some embodiments, determining a third intra-prediction mode based on a first intra-prediction mode and a second intra-prediction mode is possible. To determine the default prediction mode from the first intra prediction mode and the second intra prediction mode as the third intra prediction mode, or To determine the intra-prediction mode corresponding to the weight derivation mode used in the geometric partitioning mode among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode, or, Determining a third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode, or Determining a third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode, or The third intra-prediction mode is determined based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode. Includes.

[0249] In some embodiments, determining a third intra-prediction mode based on the weights of a first intra-prediction mode and / or a second intra-prediction mode includes determining a third intra-prediction mode based on the weights of a first intra-prediction mode or a second intra-prediction mode used in the default position.

[0250] In some embodiments, determining a third intra-prediction mode based on the weights of a first intra-prediction mode or a second intra-prediction mode used at the default position includes determining the intra-prediction mode with the largest weight at the default position among the first and second intra-prediction modes as the third intra-prediction mode.

[0251] In some embodiments, the default position is the center position.

[0252] In some embodiments, determining a third intra-prediction mode based on the type of a first intra-prediction mode and the type of a second intra-prediction mode includes determining the angle prediction mode as the third intra-prediction mode if the first and second intra-prediction modes include an angle prediction mode and a non-angle prediction mode.

[0253] In some embodiments, determining a third intra-prediction mode based on the prediction angle of a first intra-prediction mode and the prediction angle of a second intra-prediction mode is possible. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, then the planar mode or DC mode is determined as the third intra-prediction mode. Includes.

[0254] In some embodiments, determining the transformation kernel typeset to be used for the third transformation includes determining a transformation kernel typeset that conforms to the geometric partitioning mode as the transformation kernel typeset to be used for the third transformation.

[0255] In some embodiments, a third transformation is used to process the horizontal and vertical textures in the current block.

[0256] In some embodiments, S414 may include performing a fourth transformation on a third transformation coefficient to obtain a fourth transformation coefficient, and encoding the fourth transformation coefficient.

[0257] In some embodiments, the fourth transformation is used to process diagonal textures in the current block.

[0258] It should be understood that the first transformation on the decoding side is the inverse transformation of the third transformation on the encoding side, and the second transformation on the decoding side is the inverse transformation of the fourth transformation on the encoding side. For example, the third transformation is the basic or principal transformation described above, and the fourth transformation is the quadratic transformation described above. Accordingly, the first transformation may be the inverse (or anti-transformation) of the basic or principal transformation, and the second transformation may be the inverse (or anti-transformation) of the quadratic transformation. For example, the first transformation may be the inverse (anti-)DCT2 type, inverse (anti-)DCT8 type, or inverse (anti-)DST7 type, and the second transformation may be the inverse (anti-)LFNST type. Accordingly, the third transformation may be the DCT2 type, DCT8 type, or DST7 type, and the fourth transformation may be the LFNST type.

[0259] The encoding method can be understood as the reverse process of the decoding method. Therefore, the specific method of encoding method 410 can be found by referring to the relevant content of decoding method 310, and for the sake of clarity, this specification will not describe it again.

[0260] Figure 18 is a flowchart illustrating a decoding method 320 according to an embodiment of the present application. It should be understood that the decoding method 320 may be performed by a decoder. For example, it is applied to the decoding framework 200 shown in Figure 15. For the sake of explanation, a decoder will be used as an example.

[0261] As shown in Figure 18, the decoding method 320 may include some or all of the following: S321: The decoder decodes the bitstream and obtains the first transformation coefficient of the current block. S322: The decoder performs the first transformation on the first transformation coefficient and obtains the residual block of the current block. S323: The decoder makes a prediction for the current block based on the first intra-prediction mode and the second intra-prediction mode derived in the prediction mode derivation mode, and obtains the predicted block for the current block. S324: The decoder retrieves the reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0262] In this embodiment, the decoder can perform a first transformation on the first transformation coefficient of the current block and predict the current block based on the first intra-prediction mode and the second intra-prediction mode derived in the prediction mode derivation mode, thereby improving the decompression efficiency of the current block.

[0263] Of course, in other alternative embodiments, in addition to the first conversion, a secondary conversion can also be introduced, and the method for adapting the prediction mode derivation mode to the first conversion can also be applied to the method for adapting the prediction mode derivation mode to the secondary conversion. For example, the secondary conversion may be the LFNST described above, that is, the method for adapting the prediction mode derivation mode to the first conversion can also be applied to the method for adapting the prediction mode derivation mode to the LFNST. As another example, the LFNST is an inseparable secondary conversion, and in other alternative embodiments, the prediction mode derivation mode can also be applied to a separable secondary conversion, that is, the method for adapting the prediction mode derivation mode to the first conversion can also be adapted to the inseparable secondary conversion.

[0264] In some embodiments, the prediction mode derivation mode can include the DIMD mode or the TIMD mode.

[0265] In some embodiments, S322 decodes the bitstream to obtain the first flag and the second flag, when the first flag indicates that it is permitted to use the prediction mode derivation mode to perform prediction on the block in the current sequence, and the second flag indicates that it is permitted to use the first conversion to perform conversion on the block in the current sequence, perform the first conversion on the first conversion coefficient to obtain the residual block of the current block, can include.

[0266] Exemplarily, the current sequence is an image sequence including the current block.

[0267] Exemplarily, the first flag is used to control whether to use the prediction mode derivation mode for the current sequence.

[0268] Exemplary, if the value of the first flag is the first value, the first flag indicates that the use of the predictive mode derivation mode to make predictions for blocks in the current sequence is permitted. If the value of the first flag is the second value, the first flag indicates that the use of the predictive mode derivation mode to make predictions for blocks in the current sequence is not permitted. In one embodiment, the first value is 0 and the second value is 1, and in another embodiment, the first value is 1 and the second value is 0. Of course, the first value or the second value may be any other value.

[0269] For example, the second flag is used to control whether to use the first transformation for the current sequence.

[0270] For example, if the value of the second flag is the third value, the second flag indicates that the first transformation is permitted to be used to perform a transformation on the block in the current sequence. If the value of the second flag is the fourth value, the second flag indicates that the first transformation is not permitted to be used to perform a transformation on the block in the current sequence. In one embodiment, the third value is 0 and the fourth value is 1, and in another embodiment, the third value is 1 and the fourth value is 0. Of course, the third value or the fourth value may be any other value.

[0271] For example, if the first flag is denoted as sps_timd / dimd_enabled_flag and the second flag as sps_mts_enabled_flag, and both sps_timd / dimd_enabled_flag and sps_mts_enabled_flag have a value of 1, then the first transformation is performed on the first transformation coefficient, and the residual block of the current block is obtained.

[0272] For example, if the first flag indicates that the use of the prediction mode derivation mode is not permitted to make predictions for blocks in the current sequence, and / or the second flag indicates that the use of the first transformation is not permitted to make transformations for blocks in the current sequence, then the first transformation can be omitted from the first transformation coefficients, or in other words, another type of transformation can be directly applied to the first transformation coefficients to obtain the residual values ​​of the current block.

[0273] Of course, in other alternative embodiments, the first flag and / or the second flag may be replaced with flags at the picture, slice, largest coding unit (LCU), coding tree unit (CTU), coding unit (CU), prediction unit (PU), or transform unit (TU) levels. Alternatively, additional flags at the picture, slice, LCU, CTU, CU, PU, ​​or TU levels may be added based on the first and second flags to indicate whether to use the prediction mode derivation mode or the first transform. The embodiments of this application are not limited thereto.

[0274] In some embodiments, S322 is If the first flag indicates that the prediction mode derivation mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the first transformation is permitted to make transformations for blocks in the current sequence, then decode the bitstream and obtain the third flag. If the third flag indicates that both the prediction mode derivation mode and the first transformation are permitted for the block in the current sequence, then the first transformation is performed on the first transformation coefficients to obtain the residual block of the current block, It can include...

[0275] For example, a third flag is used to control whether both the prediction mode derivation mode and the first transformation can be used.

[0276] For example, if the value of the third flag is the fifth value, the third flag indicates that both the predictive mode-derivative mode and the first transformation are permitted for the block in the current sequence. If the value of the third flag is the sixth value, the third flag indicates that both the predictive mode-derivative mode and the first transformation are not permitted for the block in the current sequence. In one embodiment, the fifth value is 0 and the sixth value is 1, and in another embodiment, the fifth value is 1 and the sixth value is 0. Of course, the fifth value or the sixth value may be any other value.

[0277] For example, if we denote the first flag as sps_timd / dimd_enabled_flag, the second flag as sps_mts_enabled_flag, and the third flag as sps_timd / dimd_lfnst_enabled_flag, then if both sps_timd / dimd_enabled_flag and sps_mts_enabled_flag are 1, we determine whether sps_timd / dimd_lfnst_enabled_flag is 1. If sps_timd / dimd_lfnst_enabled_flag is 1, we perform the first transformation on the first transformation coefficient and obtain the residual block of the current block.

[0278] Of course, in other alternative embodiments, the third flag may be replaced with a flag at the level of picture, slice, largest coding unit (LCU), coding tree unit (CTU), coding unit (CU), prediction unit (PU), or transform unit (TU). Alternatively, based on the third flag, additional flag information at the level of picture, slice, LCU, CTU, CU, PU, ​​or TU may be added to indicate whether to use the prediction mode derivation mode or the first transform. The embodiments of this application are not limited thereto.

[0279] In some embodiments, S322 is If a first flag indicates that the use of a prediction mode derivation mode is permitted to make predictions for blocks in the current sequence, and a second flag indicates that the use of a first transformation is permitted to make transformations for blocks in the current sequence, the decoder may include performing a first transformation on a first transformation coefficient to obtain the residual block of the current block when the height and / or width of the current block is greater than or equal to a first threshold.

[0280] For example, if the first flag is denoted as sps_timd / dimd_enabled_flag, the second flag as sps_mts_enabled_flag, and the third flag as sps_timd / dimd_lfnst_enabled_flag, then if both sps_timd / dimd_enabled_flag and sps_mts_enabled_flag have a value of 1, the decoder determines the height and / or width of the current block. If the height and / or width of the current block is greater than or equal to the first threshold, the decoder performs the first transformation on the first transformation coefficient and obtains the residual block of the current block.

[0281] For example, the first threshold can be 4, 8, 16, 32, 64, or any other value.

[0282] In some embodiments, prior to S322, method 320 may further include the decoder determining the conversion kernel typeset to be used for the first conversion.

[0283] In the prediction mode derivation mode, two prediction modes (i.e., the first intra prediction mode and the second intra prediction mode) are combined to predict the current block. Predicted blocks obtained by predicting the current block using different intra prediction modes may have different texture characteristics. Therefore, when the prediction mode derivation mode is selected for the current block, the first intra prediction mode may result in the predicted block of the current block exhibiting one texture characteristic, and the second intra prediction mode may result in the predicted block of the current block exhibiting another texture characteristic. In other words, after predicting the current block, from a statistical standpoint, the residual block of the current block will also exhibit two texture characteristics. That is, the residual block of the current block may not conform to the rules that can be reflected in a given prediction mode. Therefore, for the prediction mode derivation mode, the decoder needs to determine a set of transformation kernel types that fit the characteristics before performing the first transformation on the first transformation coefficient. However, since the transformation kernel typeset used for the first transformation is typically defined based on a single intra-prediction mode, the relevant scheme for determining the transformation kernel typeset used for the first transformation needs to be further improved for the prediction mode derivation mode, and each embodiment is described below illustratively.

[0284] In some embodiments, the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0285] For example, when the decoder checks the predicted mode of the current block, if the predicted mode derivation mode is used for the current block, the decoder classifies the predicted mode derivation mode and the planar mode (or DC mode) into one type and determines the conversion kernel type set to be used for the first conversion based on the planar mode (or DC mode). Alternatively, when the decoder checks the predicted mode of the current block, if the predicted mode derivation mode is used for the current block, the encoder may return the planar mode (or DC mode) as the predicted mode of the current block, and based on this, the decoder may determine the conversion kernel type set that conforms to the planar mode (or DC mode) as the conversion kernel type set to be used for the first conversion. Alternatively, when the decoder checks the predicted mode of the current block, if the predicted mode derivation mode is used for the current block, the decoder may consider that the conversion kernel type set used for the first conversion of the current block may be a conversion kernel type set that conforms to the planar mode (or DC mode).

[0286] In this embodiment, since both the planar mode (or DC mode) and the prediction mode derivation mode can represent various texture characteristics, by determining a transformation kernel typeset that matches the planar mode or DC mode as the transformation kernel typeset used for the first transformation, it is not only possible to decode the current block based on the prediction mode derivation mode and the first transformation, but it is also guaranteed that the texture characteristics of the transformation kernel typeset used for the first transformation will be as similar as possible to the texture characteristics of the residual block of the current block, thereby improving decompression efficiency.

[0287] In some embodiments, the decoder determines a third intra-prediction mode based on a first intra-prediction mode and a second intra-prediction mode. The conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0288] For example, the decoder may determine a set of transformation kernel types that conforms to a third intra-prediction mode as the set of transformation kernel types used for the first transformation.

[0289] For example, when the decoder checks the prediction mode of the current block, if the prediction mode derivation mode is used for the current block, the decoder can determine a third intra-prediction mode based on the first intra-prediction mode and the second intra-prediction mode, classify the prediction mode derivation mode and the third intra-prediction mode into one type, and based on this, the decoder can determine the conversion kernel type set to be used for the first conversion based on the third intra-prediction mode. Alternatively, when the decoder checks the prediction mode of the current block, if the prediction mode derivation mode is used for the current block, the encoder can return a third intra-prediction mode as the prediction mode of the current block, and based on this, the decoder can determine the conversion kernel type set that matches the third intra-prediction mode as the conversion kernel type set to be used for the first conversion. Alternatively, when the decoder checks the prediction mode of the current block, if the prediction mode derivation mode is used for the current block, the decoder can consider that the conversion kernel type set used for the first conversion of the current block may be a conversion kernel type set that matches the third intra-prediction mode.

[0290] Of course, in other alternative embodiments, the decoder does not necessarily have to explicitly determine a third intra-prediction mode first, and then determine the conversion kernel typeset to be used for the first conversion based on the third intra-prediction mode. Instead, the decoder directly uses the conversion kernel typeset that fits the third intra-prediction mode as the conversion kernel typeset to be used for the first conversion.

[0291] In some embodiments, the decoder determines the default prediction mode from among the first and second intra-prediction modes as the third intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode. Alternatively, the decoder determines the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0292] For example, when the decoder checks the prediction mode of the current block, if a prediction mode derivation mode is used for the current block, it can determine the conversion kernel type set to be used for the first conversion based on the first intra-prediction mode and the second intra-prediction mode when selecting the conversion kernel type set to be used for the first conversion. In one embodiment, the determination can be made using the first intra-prediction mode in any case, that is, in any case, the conversion kernel type set that conforms to the first intra-prediction mode is determined as the conversion kernel type set to be used for the first conversion; or, the determination can be made using the second intra-prediction mode in any case, that is, in any case, the conversion kernel type set that conforms to the second intra-prediction mode is determined as the conversion kernel type set to be used for the first conversion. In another embodiment, the determination may be made using a first intra-prediction mode, i.e., in some cases, a conversion kernel typeset that conforms to the first intra-prediction mode is determined as the conversion kernel typeset used for the first conversion; or, in some cases, the determination may be made using a second intra-prediction mode, i.e., in some cases, a conversion kernel typeset that conforms to the second intra-prediction mode is determined as the conversion kernel typeset used for the first conversion; furthermore, in some cases, the determination may be made using a planar mode or a DC mode, i.e., in some cases, a conversion kernel typeset that conforms to a planar mode or a DC mode is determined as the conversion kernel typeset used for the first conversion. Determining using a certain prediction mode means classifying the prediction mode derivation mode and a certain prediction mode into one type, and based on this, the decoder can determine the conversion kernel typeset used for the first conversion based on a certain prediction mode. Alternatively, when the decoder checks the prediction mode of the current block, if a prediction mode derivation mode is used for the current block, it can return a certain prediction mode, and based on this, the decoder can determine the conversion kernel typeset used for the first conversion based on a certain prediction mode.Alternatively, when the decoder checks the prediction mode of the current block, if a prediction mode derivation mode is used for the current block, the decoder may assume that the transformation kernel typeset used for the first transformation of the current block may be a transformation kernel typeset that conforms to a certain prediction mode.

[0293] In some embodiments, when the decoder determines a third intra-prediction mode based on the weights of a first intra-prediction mode and / or a second intra-prediction mode, it may determine the intra-prediction mode with the largest weight among the first and second intra-prediction modes as the third intra-prediction mode.

[0294] For example, when a decoder determines a third intra-prediction mode based on the weights of a first intra-prediction mode and / or a second intra-prediction mode, the priority of an intra-prediction mode with a larger weight is higher than the priority of an intra-prediction mode with a smaller weight. For instance, if the weight of the first intra-prediction mode is greater than the weight of the second intra-prediction mode, the decoder determines the first intra-prediction mode as the third intra-prediction mode; if the weight of the second intra-prediction mode is greater than the weight of the first intra-prediction mode, the decoder determines the second intra-prediction mode as the third intra-prediction mode.

[0295] In some embodiments, when the decoder determines a third intra-prediction mode based on the type of a first intra-prediction mode and the type of a second intra-prediction mode, if the first and second intra-prediction modes include an angle prediction mode and a non-angle prediction mode, the decoder determines the angle prediction mode as the third intra-prediction mode.

[0296] For example, when a decoder determines a third intra-prediction mode based on the types of the first and second intra-prediction modes, the priority given to an angle prediction mode as the third intra-prediction mode is higher than the priority given to a non-angle prediction mode as the third intra-prediction mode. For instance, if the first intra-prediction mode is an angle prediction mode and the second intra-prediction mode is a non-angle prediction mode (e.g., a planar mode or a DC mode), the decoder determines the first intra-prediction mode (i.e., the angle prediction mode) as the third intra-prediction mode.

[0297] In some embodiments, if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode. The first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, the planar mode or DC mode is determined as the third intra-prediction mode.

[0298] For example, when the decoder determines a third intra-prediction mode based on the predicted angles of the first intra-prediction mode and the second intra-prediction mode, if the predicted angles of the first and second intra-prediction modes are relatively close, the decoder can determine the first intra-prediction mode, the second intra-prediction mode, or an intra-prediction mode whose predicted angle is between the predicted angles of the first and second intra-prediction modes as the third intra-prediction mode. For example, if the difference between the predicted angles of the first and second intra-prediction modes is large, the decoder can determine the planar mode or DC mode as the third intra-prediction mode.

[0299] In some embodiments, the decoder determines a set of transformation kernel types that conforms to the predictive mode derivation mode as the set of transformation kernel types used for the first transformation.

[0300] For example, a decoder can define a set of transformation kernel types that are adapted to or dedicated to the prediction mode and the derivation mode.

[0301] In some embodiments, the first transformation is used to process the horizontal and vertical textures in the current block.

[0302] In some embodiments, S322 is Performing a second transformation on the first transformation coefficient and obtaining the second transformation coefficient, The first transformation is applied to the second transformation coefficient to obtain the residual block of the current block, It can include...

[0303] In some embodiments, the second transformation is used to process diagonal textures in the current block.

[0304] The decoding method according to the embodiment of this application has been described in detail above from the perspective of the decoder angle. The encoding method according to the embodiment of this application will now be described below from the perspective of the encoder angle, with reference to Figure 19.

[0305] Figure 19 is a flowchart illustrating an encoding method 420 according to an embodiment of the present application. It should be understood that the encoding method 420 may be performed by an encoder. For example, it is applied to the encoding framework 100 shown in Figure 1. For the sake of explanation, an encoder will be used as an example.

[0306] As shown in Figure 19, the encoding method 420 may include the following: S421: Prediction mode is derived. Based on the first intra-prediction mode and the second intra-prediction mode derived in the prediction mode, a prediction is made for the current block, and the predicted block for the current block is obtained. S422: Based on the predicted block of the current block, obtain the residual block of the current block. S423: Perform a third transformation on the residual block of the current block and obtain the third transformation coefficient of the current block. S424: Encode the third conversion coefficient.

[0307] In some embodiments, the prediction mode derivation mode may include a DIMD mode or a TIMD mode.

[0308] In some embodiments, S424 may include encoding a first flag, a second flag, and a third conversion coefficient. The first flag indicates that the prediction mode derivation mode is permitted to be used to make predictions for blocks in the current sequence, and the second flag indicates that the third transformation mode is permitted to be used to make transformations for blocks in the current sequence.

[0309] In some embodiments, S424 may include encoding a first flag, a second flag, a third conversion coefficient, and a third flag. The third flag indicates that both predictive mode, derived mode, and the third transformation are permitted for the block in the current sequence.

[0310] In some embodiments, S423 may include performing a third transformation on the residual block of the current block and obtaining a third transformation coefficient when the height and / or width of the current block is greater than or equal to a first threshold.

[0311] In some embodiments, prior to S423, method 420 may further include determining the set of transformation kernel types to be used for the third transformation.

[0312] In some embodiments, the conversion kernel typeset used for the third conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0313] In some embodiments, determining the transformation kernel typeset used for the third transformation includes determining the third intra-prediction mode based on the first intra-prediction mode and the second intra-prediction mode. The transformation kernel typeset used for the third transformation is the same as the transformation kernel typeset that fits the third intra-prediction mode.

[0314] In some embodiments, determining a third intra-prediction mode based on a first intra-prediction mode and a second intra-prediction mode is possible. To determine the default prediction mode from the first intra prediction mode and the second intra prediction mode as the third intra prediction mode, or Determining a third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode, or Determining a third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode, or The third intra-prediction mode is determined based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode. Includes.

[0315] In some embodiments, determining a third intra-prediction mode based on the weights of a first intra-prediction mode and / or a second intra-prediction mode is: This includes determining the intra-prediction mode with the largest weight among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode.

[0316] In some embodiments, determining a third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode is possible. The method includes determining the angle prediction mode as the third intra-prediction mode if the first intra-prediction mode and the second intra-prediction mode include an angle prediction mode and a non-angle prediction mode.

[0317] In some embodiments, determining a third intra-prediction mode based on the prediction angle of a first intra-prediction mode and the prediction angle of a second intra-prediction mode is possible. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, then the planar mode or DC mode is determined as the third intra-prediction mode. Includes.

[0318] In some embodiments, determining the transformation kernel typeset to be used for the third transformation includes determining a transformation kernel typeset that conforms to the predictive mode derivation mode as the transformation kernel typeset to be used for the third transformation.

[0319] In some embodiments, a third transformation is used to process the horizontal and vertical textures in the current block.

[0320] In some embodiments, S424 is, Perform a fourth transformation on the third transformation coefficient to obtain the fourth transformation coefficient, Encoding the fourth conversion coefficient, It can include...

[0321] In some embodiments, the fourth transformation is used to process diagonal textures in the current block.

[0322] It should be understood that the first transformation on the decoding side is the inverse transformation of the third transformation on the encoding side, and the second transformation on the decoding side is the inverse transformation of the fourth transformation on the encoding side. For example, the third transformation is the basic or principal transformation described above, and the fourth transformation is the quadratic transformation described above. Accordingly, the first transformation may be the inverse (or anti-transformation) of the basic or principal transformation, and the second transformation may be the inverse (or anti-transformation) of the quadratic transformation. For example, the first transformation may be the inverse (anti-)DCT2 type, inverse (anti-)DCT8 type, or inverse (anti-)DST7 type, and the second transformation may be the inverse (anti-)LFNST type. Accordingly, the third transformation may be the DCT2 type, DCT8 type, or DST7 type, and the fourth transformation may be the LFNST type.

[0323] Furthermore, the encoding method can be understood as the reverse process of the decoding method. Therefore, the specific scheme of the encoding method 420 can be found in the relevant section of the decoding method 320, and for the sake of clarity, it will not be described in detail in this application.

[0324] While preferred embodiments of the present application have been described in detail above with reference to the attached drawings, the present application is not limited to the detailed contents of the above embodiments. Within the scope of the technical idea of ​​the present application, various simple modifications can be made to the technical proposal of the present application, and any such simple modifications will fall within the scope of protection of the present application. For example, each specific technical feature described in the above specific embodiments may be combined by any appropriate means, provided that they are not contradictory, and in order to avoid unnecessary redundancy, the present application will not describe various possible combinations again. Furthermore, for example, between various different embodiments of the present application, any combination should be considered disclosed in the present application, as long as it does not contradict the idea of ​​the present application. It should be understood that in the various method embodiments of the present application, the magnitude of the sequence number of each process does not indicate the execution order. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0325] The above description details the method embodiment of the present application. Hereinafter, the apparatus embodiment of the present application will be described in detail with reference to Figures 20 to 24.

[0326] Figure 20 is a block diagram showing a decoder 510 according to an embodiment of the present application.

[0327] As shown in Figure 20, the decoder 510 may include a decoding unit 511, a conversion unit 512, a prediction unit 513, and a reconstruction unit 514. The decoding unit 511 is configured to decode a bitstream and obtain a first conversion coefficient of the current block. The conversion unit 512 is configured to perform a first conversion on the first conversion coefficient and obtain the residual block of the current block. The prediction unit 513 is configured to make a prediction on the current block based on a first prediction mode and a second prediction mode corresponding to the geometric partitioning mode and obtain the predicted block of the current block. The reconstruction unit 514 is configured to obtain the reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0328] In some embodiments, the transformation unit 512 is configured to specifically perform the following: decode the bitstream and obtain a first flag and a second flag; if the first flag indicates that a geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the first transformation is permitted to make transformations for blocks in the current sequence, then perform the first transformation on the first transformation coefficients and obtain the residual block.

[0329] In some embodiments, the transformation unit 512 is configured to specifically perform the following: if a first flag indicates that the geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and a second flag indicates that the first transformation is permitted to make transformations for blocks in the current sequence, then decode the bitstream and obtain a third flag; if the third flag indicates that both the geometric partitioning mode and the first transformation are permitted to make predictions for blocks in the current sequence, then perform the first transformation on the first transformation coefficients and obtain the residual block.

[0330] In some embodiments, the transformation unit 512 is configured to specifically perform the following: If a first flag indicates that a geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and a second flag indicates that a first transformation is permitted to make transformations for blocks in the current sequence, then if the height and / or width of the current block is greater than or equal to a first threshold, the first transformation is performed on the first transformation coefficients to obtain the residual block.

[0331] In some embodiments, the conversion unit 512 is configured to determine the conversion kernel type set to be used for the first conversion before it is configured to perform the first conversion on the first conversion coefficients and obtain the residual block of the current block.

[0332] In some embodiments, the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0333] In some embodiments, the transformation unit 512 is configured to specifically perform the following: determine a dividing line consisting of points where the weights of the first prediction mode and the weights of the second prediction mode are the same; determine the angular index of the dividing line; and determine the transformation kernel typeset to be used for the first transformation based on the angular index.

[0334] In some embodiments, the transformation unit 512 is configured to specifically perform the following: determine the weight derivation mode to be used for the geometric partitioning mode; and determine the transformation kernel typeset that fits the intra-prediction mode corresponding to the weight derivation mode as the transformation kernel typeset to be used for the first transformation.

[0335] In some embodiments, the first prediction mode is a first intra-prediction mode, and the second prediction mode is a second intra-prediction mode. Specifically, the conversion unit 512 is configured to determine a third intra-prediction mode based on the first and second intra-prediction modes. The conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0336] In some embodiments, the conversion unit 512 is configured to specifically perform the following: determine the default prediction mode from among the first and second intra-prediction modes as the third intra-prediction mode; or determine the intra-prediction mode from among the first and second intra-prediction modes that corresponds to the weight derivation mode used for the geometric partitioning mode as the third intra-prediction mode; or determine the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode; or determine the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode; or determine the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0337] In some embodiments, the conversion unit 512 is configured to determine a third intra-prediction mode based specifically on the weights of a first intra-prediction mode or a second intra-prediction mode used at the default position.

[0338] In some embodiments, the conversion unit 512 is configured to determine, specifically, as the third intra-prediction mode the intra-prediction mode that has the greatest weight at the default position among the first intra-prediction mode and the second intra-prediction mode.

[0339] In some embodiments, the default position is the center position.

[0340] In some embodiments, the conversion unit 512 is configured to determine the angle prediction mode as the third intra prediction mode if the first intra prediction mode and the second intra prediction mode include an angle prediction mode and a non-angle prediction mode.

[0341] In some embodiments, the conversion unit 512 is configured to specifically perform the following: if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode; if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, the planar mode or DC mode is determined as the third intra-prediction mode.

[0342] In some embodiments, the conversion unit 512 is configured to determine, specifically, a conversion kernel typeset that conforms to the geometric partitioning mode as the conversion kernel typeset to be used for the first conversion.

[0343] In some embodiments, the first transformation is used to process the horizontal and vertical textures in the current block.

[0344] In some embodiments, the conversion unit 512 is configured to specifically perform the following: perform a second conversion on a first conversion coefficient to obtain a second conversion coefficient; perform a first conversion on the second conversion coefficient to obtain a residual block of the current block.

[0345] In some embodiments, the second transformation is used to process diagonal textures in the current block.

[0346] Figure 21 is a block diagram showing an encoder 610 according to an embodiment of the present application.

[0347] As shown in Figure 21, the encoder 610 may include a prediction unit 611, a residual unit 612, a transformation unit 613, and an encoding unit 614. The prediction unit 611 is configured to make a prediction for the current block based on a first prediction mode and a second prediction mode corresponding to the geometric partitioning mode, and to obtain the predicted block of the current block. The residual unit 612 is configured to obtain the residual block of the current block based on the predicted block of the current block. The transformation unit 613 is configured to perform a third transformation on the residual block of the current block and to obtain the third transformation coefficient of the current block. The encoding unit 614 is configured to encode the third transformation coefficient.

[0348] In some embodiments, the encoding unit 614 is configured to encode, specifically, a first flag, a second flag, and a third transformation coefficient. The first flag indicates that a geometric partitioning mode is permitted to be used to make predictions for blocks in the current sequence, and the second flag indicates that a third transformation is permitted to be used to perform transformations for blocks in the current sequence.

[0349] In some embodiments, the encoding unit 614 is configured to encode, specifically, a first flag, a second flag, a third transformation coefficient, and a third flag. The third flag indicates that both the geometric partitioning mode and the third transformation are permitted for the block in the current sequence.

[0350] In some embodiments, the conversion unit 613 is configured to perform a third conversion on the residual block of the current block and obtain a third conversion coefficient when the height and / or width of the current block exceeds a first threshold.

[0351] In some embodiments, the conversion unit 613 is configured to further determine the conversion kernel typeset to be used for the third conversion before encoding the third conversion coefficients.

[0352] In some embodiments, the conversion kernel typeset used for the third conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0353] In some embodiments, the transformation unit 613 is configured to specifically perform the following: determine a dividing line consisting of points where the weights of the first prediction mode and the weights of the second prediction mode are the same; determine the angular index of the dividing line; and determine the transformation kernel typeset to be used for the third transformation based on the angular index.

[0354] In some embodiments, the transformation unit 613 is configured to specifically perform the following: determine the weight derivation mode to be used for the geometric partitioning mode; and determine the transformation kernel typeset that fits the intra-prediction mode corresponding to the weight derivation mode as the transformation kernel typeset to be used for the third transformation.

[0355] In some embodiments, the first prediction mode is the first intra-prediction mode, and the second prediction mode is the second intra-prediction mode. Specifically, the conversion unit 613 is configured to determine the third intra-prediction mode based on the first intra-prediction mode and the second intra-prediction mode. The conversion kernel typeset used for the third conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0356] In some embodiments, the conversion unit 613 is configured to specifically perform the following: determine the default prediction mode from among the first and second intra-prediction modes as the third intra-prediction mode; or determine the intra-prediction mode from among the first and second intra-prediction modes that corresponds to the weight derivation mode used for the geometric partitioning mode as the third intra-prediction mode; or determine the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode; or determine the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode; or determine the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0357] In some embodiments, the conversion unit 613 is configured to determine a third intra-prediction mode based specifically on the weights of a first intra-prediction mode or a second intra-prediction mode used at the default position.

[0358] In some embodiments, the conversion unit 613 is configured to determine, specifically, as the third intra-prediction mode the intra-prediction mode that has the greatest weight at the default position among the first intra-prediction mode and the second intra-prediction mode.

[0359] In some embodiments, the default position is the center position.

[0360] In some embodiments, the conversion unit 613 is configured to determine the angle prediction mode as the third intra prediction mode if the first intra prediction mode and the second intra prediction mode include an angle prediction mode and a non-angle prediction mode.

[0361] In some embodiments, determining a third intra-prediction mode based on the prediction angle of a first intra-prediction mode and the prediction angle of a second intra-prediction mode is possible. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode. If the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, then the planar mode or DC mode is determined as the third intra-prediction mode. Includes.

[0362] In some embodiments, the conversion unit 613 is configured to determine, specifically, a conversion kernel typeset that conforms to the geometric partitioning mode as the conversion kernel typeset to be used for the third conversion.

[0363] In some embodiments, a third transformation is used to process the horizontal and vertical textures in the current block.

[0364] In some embodiments, the encoding unit 614 is configured to specifically perform the following: perform a fourth transformation on a third transformation coefficient to obtain a fourth transformation coefficient; and encode the fourth transformation coefficient.

[0365] In some embodiments, the fourth transformation is used to process diagonal textures in the current block.

[0366] Figure 22 is a block diagram showing a decoder 520 according to an embodiment of the present application.

[0367] As shown in Figure 20, the decoder 520 may include a decoding unit 521, a conversion unit 522, a prediction unit 523, and a reconstruction unit 524. The decoding unit 521 is configured to decode a bitstream and obtain a first conversion coefficient of the current block. The conversion unit 522 is configured to perform a first conversion on the first conversion coefficient and obtain the residual block of the current block. The prediction unit 523 is configured to make a prediction on the current block based on a first intra-prediction mode and a second intra-prediction mode derived in the prediction mode derivation mode and obtain the predicted block of the current block. The reconstruction unit 524 is configured to obtain the reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block.

[0368] In some embodiments, the prediction mode derivation mode may include a DIMD mode or a TIMD mode.

[0369] In some embodiments, the conversion unit 522 is configured to specifically perform the following: decode the bitstream and obtain a first flag and a second flag; if the first flag indicates that the prediction mode derivation mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the first conversion is permitted to make conversions for blocks in the current sequence, then perform the first conversion on the first conversion coefficients and obtain the residual block of the current block.

[0370] In some embodiments, the conversion unit 522 is configured to specifically perform the following: if a first flag indicates that the prediction mode derivation mode is permitted to make predictions for blocks in the current sequence, and a second flag indicates that the first transformation is permitted to make transformations for blocks in the current sequence, then decode the bitstream and obtain a third flag; if the third flag indicates that both the prediction mode derivation mode and the first transformation are permitted to make predictions for blocks in the current sequence, then perform the first transformation on the first transformation coefficients and obtain the residual block of the current block.

[0371] In some embodiments, the conversion unit 522 is configured such that, specifically, if a first flag indicates that the prediction mode derivation mode is permitted to make a prediction for a block in the current sequence, and a second flag indicates that the first conversion is permitted to make a conversion for a block in the current sequence, then when the height and / or width of the current block is greater than or equal to a first threshold, the first conversion is performed on the first conversion coefficient to obtain the residual block of the current block.

[0372] In some embodiments, the conversion unit 522 is configured to perform a first conversion on a first conversion coefficient and obtain a residual block of the current block, and is further configured to determine the conversion kernel typeset to be used for the first conversion.

[0373] In some embodiments, the conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0374] In some embodiments, the conversion unit 522 is configured to determine a third intra-prediction mode based on a first intra-prediction mode and a second intra-prediction mode. The conversion kernel typeset used for the first conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0375] In some embodiments, the conversion unit 522 is configured to specifically perform the following: determine the default prediction mode among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode; or determine the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode; or determine the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode; or determine the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0376] In some embodiments, the conversion unit 522 is configured to determine, specifically, the intra-prediction mode with the greatest weight among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode.

[0377] In some embodiments, the conversion unit 522 is configured to determine the angle prediction mode as the third intra prediction mode if the first intra prediction mode and the second intra prediction mode include an angle prediction mode and a non-angle prediction mode.

[0378] In some embodiments, the conversion unit 522 is configured to specifically perform the following: if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode; if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, the planar mode or DC mode is determined as the third intra-prediction mode.

[0379] In some embodiments, the conversion unit 522 is configured to determine, specifically, a conversion kernel typeset that conforms to the prediction mode derivation mode as the conversion kernel typeset to be used for the first conversion.

[0380] In some embodiments, the first transformation is used to process the horizontal and vertical textures in the current block.

[0381] In some embodiments, the conversion unit 522 is configured to specifically perform the following: perform a second conversion on a first conversion coefficient to obtain a second conversion coefficient; perform a first conversion on the second conversion coefficient to obtain a residual block of the current block.

[0382] In some embodiments, the second transformation is used to process diagonal textures in the current block.

[0383] Figure 23 is a block diagram showing an encoder 620 according to an embodiment of the present application.

[0384] As shown in Figure 23, the encoder 620 may include a prediction unit 621, a residual unit 622, a transformation unit 623, and an encoding unit 624. The prediction unit 621 is configured to make a prediction for the current block based on a first intra-prediction mode and a second intra-prediction mode derived in the prediction mode derivation mode, and to obtain the predicted block of the current block. The residual unit 622 is configured to obtain the residual block of the current block based on the predicted block of the current block. The transformation unit 623 is configured to perform a third transformation on the residual block of the current block and to obtain the third transformation coefficient of the current block. The encoding unit 624 is configured to encode the third transformation coefficient.

[0385] In some embodiments, the prediction mode derivation mode may include a DIMD mode or a TIMD mode.

[0386] In some embodiments, the encoding unit 624 is configured to encode, specifically, a first flag, a second flag, and a third transformation coefficient. The first flag indicates that the prediction mode derivation mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the third transformation is permitted to make transformations for blocks in the current sequence.

[0387] In some embodiments, the encoding unit 624 is configured to encode, specifically, a first flag, a second flag, a third transformation coefficient, and a third flag. The third flag indicates that both the predictive mode derivation mode and the third transformation are permitted for the block in the current sequence.

[0388] In some embodiments, the conversion unit 623 is configured to perform a third conversion on the residual block of the current block and obtain a third conversion coefficient when the height and / or width of the current block exceeds a first threshold.

[0389] In some embodiments, the conversion unit 623 is configured to perform a third conversion on the residual block of the current block and to determine the conversion kernel type set to be used for the third conversion before obtaining the third conversion coefficients of the current block.

[0390] In some embodiments, the conversion kernel typeset used for the third conversion is the same as the conversion kernel typeset that conforms to planar mode or DC mode.

[0391] In some embodiments, the conversion unit 623 is configured to determine a third intra-prediction mode based specifically on a first intra-prediction mode and a second intra-prediction mode. The conversion kernel typeset used for the third conversion is the same as the conversion kernel typeset that conforms to the third intra-prediction mode.

[0392] In some embodiments, the conversion unit 623 is configured to specifically perform the following: determine the default prediction mode among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode; or determine the third intra-prediction mode based on the weights of the first intra-prediction mode and / or the weights of the second intra-prediction mode; or determine the third intra-prediction mode based on the type of the first intra-prediction mode and the type of the second intra-prediction mode; or determine the third intra-prediction mode based on the prediction angle of the first intra-prediction mode and the prediction angle of the second intra-prediction mode.

[0393] In some embodiments, the conversion unit 623 is configured to determine, specifically, the intra-prediction mode with the greatest weight among the first intra-prediction mode and the second intra-prediction mode as the third intra-prediction mode.

[0394] In some embodiments, the conversion unit 623 is configured to determine the angle prediction mode as the third intra prediction mode if the first intra prediction mode and the second intra prediction mode include an angle prediction mode and a non-angle prediction mode.

[0395] In some embodiments, the conversion unit 623 is configured to specifically perform the following: if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is less than or equal to a second threshold, the intra-prediction mode corresponding to the first predicted angle is determined as the third intra-prediction mode, and the first predicted angle is determined based on the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode; if the absolute value of the difference between the predicted angle of the first intra-prediction mode and the predicted angle of the second intra-prediction mode is greater than the second threshold, the planar mode or DC mode is determined as the third intra-prediction mode.

[0396] In some embodiments, the conversion unit 623 is configured to determine, specifically, a conversion kernel typeset that conforms to the prediction mode derivation mode as the conversion kernel typeset to be used for the third conversion.

[0397] In some embodiments, a third transformation is used to process the horizontal and vertical textures in the current block.

[0398] In some embodiments, the encoding unit 624 is configured to specifically perform the following: perform a fourth transformation on a third transformation coefficient to obtain a fourth transformation coefficient; and encode the fourth transformation coefficient.

[0399] In some embodiments, the fourth transformation is used to process diagonal textures in the current block.

[0400] Note that the apparatus embodiment and the method embodiment can correspond to each other, and for similar descriptions, refer to the method embodiment. To avoid duplication, such descriptions are omitted here. Specifically, the decoder 510 shown in Figure 20 may correspond to the entity that performs method 310 in the embodiment of the present application. Furthermore, the aforementioned and other operations and / or functions of each unit in the decoder 510 are each used to realize the corresponding process in each method, such as method 310. Similarly, the encoder 610 shown in Figure 21 may correspond to the entity that performs method 410 in the embodiment of the present application. That is, the aforementioned and other operations and / or functions of each unit in the encoder 610 are each used to realize the corresponding process in each method, such as method 410. The decoder 520 shown in Figure 22 may correspond to the main body that performs method 320 in the embodiment of the present application, and the aforementioned and other operations and / or functions of each unit in the decoder 520 are each used to realize the corresponding process in each method, such as method 320. Similarly, the encoder 620 shown in Figure 23 may correspond to a body that performs method 420 in the embodiments of the present invention, that is, each of the aforementioned and other operations and / or functions of the encoder 620 is used to implement the corresponding process in each of the methods, such as method 420.

[0401] Furthermore, each unit in the decoder 510, encoder 610, decoder 520, or encoder 620 according to the embodiments of the present application may be integrated into one or more other units, or some of these units may be further divided into several functionally smaller units. This allows similar operation to be achieved without affecting the realization of the technical effects of the embodiments of the present application. The units are divided based on logic functions. In practical applications, the function of one unit may be realized by multiple units, or the function of multiple units may be realized by one unit. In other embodiments of the present application, the decoder 510, encoder 610, decoder 520, or encoder 620 may include other units, and in practical applications, these functions may be realized by the cooperation of other units or by the cooperation of multiple units. According to another embodiment of the present application, for example, a decoder 510, encoder 610, decoder 520, or encoder 620 according to an embodiment of the present application can be constructed by executing a computer program (including program code) capable of executing each step of the corresponding method on a general-purpose computing device such as a general-purpose computer equipped with processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), thereby realizing the encoding method or decoding method according to an embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and mounted on an electronic device via the computer-readable storage medium, and by operating within it, the corresponding method in the embodiment of the present application can be realized.

[0402] In other words, the above-mentioned unit may be implemented in hardware form, in software form by instructions, or in combination of hardware and software. Specifically, each step of the method embodiment in the embodiments of the present application can be completed by hardware integrated logic circuits and / or software form instructions in a processor. The steps of the method disclosed in the embodiments of the present application can be executed and completed directly by a hardware decoding processor, or by a combination of hardware and software in a decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium resides in memory. The processor reads information from memory and uses the processor's hardware to complete the steps of the method embodiment.

[0403] Figure 24 is a block diagram showing an electronic device 700 according to an embodiment of the present application.

[0404] As shown in Figure 24, the electronic device 700 includes at least a processor 710 and a computer-readable storage medium 720. The processor 710 and the computer-readable storage medium 720 may be connected by a bus or other means. The computer-readable storage medium 720 is used to store a computer program 721 containing computer instructions. The processor 710 is used to execute the computer instructions stored in the computer-readable storage medium 720. The processor 710 is the computing core and control core of the electronic device 700 and is suitable for implementing one or more computer instructions, specifically, for implementing a corresponding process or corresponding function by loading and executing one or more computer instructions.

[0405] For example, the processor 710 may also be called a central processing unit (CPU). The processor 710 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0406] For example, the computer-readable storage medium 720 may be high-speed RAM memory, non-volatile memory, such as at least one magnetic disk storage device. Selectively, the computer-readable storage medium 720 may be at least one computer-readable storage medium located away from the processor 710. Specifically, the computer-readable storage medium 720 includes, but is not limited to, volatile memory and / or non-volatile memory. Non-volatile memory may be ROM, programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or flash memory. Volatile memory may be RAM that functions as an external high-speed cache. As illustrative but not limited examples, various types of RAM are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DRRAM).

[0407] In one embodiment, the electronic device 700 may be an encoder or encoding framework according to the embodiment of the present application. The computer-readable storage medium 720 stores a first computer instruction. The processor 710 loads and executes the first computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the encoding method according to the embodiment of the present application. In other words, the first computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid redundancy, the explanation is omitted here.

[0408] In one embodiment, the electronic device 700 may be a decoder or decoding framework according to the embodiment of the present application. The computer-readable storage medium 720 stores a second computer instruction. The processor 710 loads and executes the second computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the decoding method according to the embodiment of the present application. In other words, the second computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid redundancy, the explanation is omitted here.

[0409] In another embodiment of the present application, a coding system is provided, comprising the encoder and decoder described above.

[0410] In another aspect of the present application, an embodiment of the present application provides a computer-readable storage medium (Memory). The computer-readable storage medium is a storage device in the electronic device 700 and is used to store programs and data. For example, it may be a computer-readable storage medium 720. To make it clear, the computer-readable storage medium 720 herein may include an internal storage medium in the electronic device 700 and, of course, an extended storage medium supported by the electronic device 700. The computer-readable storage medium provides a storage space in which the operating system of the electronic device 700 is stored. Furthermore, the storage space stores one or more computer instructions suitable for being loaded and executed by the processor 710, which may be one or more computer programs 721 (including program code).

[0411] According to another aspect of the present application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions, which are stored in a computer-readable storage medium, and for example, the computer instructions may be computer program 721. In this case, electronic equipment 700 may be a computer, and the processor 710 reads a computer instruction from the computer-readable storage medium 720, and the processor 710 executes the computer instruction, thereby causing the computer to execute an encoding method or decoding method according to the various selectable methods described above.

[0412] In other words, when implemented by software, all or part of the above embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes described in the embodiments of this application are executed, or all or part of the functions described in the embodiments of this application are implemented. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored on a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic cable, digital subscriber line (DSL), etc.) or wirelessly (e.g., infrared, radio, microwave, etc.).

[0413] It will be apparent to those skilled in the art that, in conjunction with the exemplary units and process steps described in the embodiments disclosed herein, the present application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software will depend on the specific application of the technology and design limitations. Those skilled in the art can implement the described functions using different methods for each specific application, but these implementations should not be considered beyond the scope of the present application.

[0414] Finally, the above is merely a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Any modification or substitution that a person skilled in the art could easily conceive within the scope of the art disclosed herein should be included within the scope of protection of the present application. Accordingly, the scope of protection of the present application should be determined by the scope of protection of the claims.

Claims

1. A decoding method, Decode the bitstream and obtain the first transformation coefficient of the current block, Determining the weight derivation mode used for the geometric partitioning mode, The transformation kernel typeset that conforms to the intra-prediction mode corresponding to the weight derivation mode is determined as the transformation kernel typeset used for the first transformation, Perform the first transformation on the first transformation coefficient to obtain the residual block of the current block, Based on the first and second prediction modes corresponding to the geometric division mode, a prediction is made for the current block, and the predicted block of the current block is obtained. Based on the predicted block of the current block and the residual block of the current block, the reconstructed block of the current block is obtained. including, A decoding method characterized by the following features.

2. Performing the first transformation on the aforementioned first transformation coefficient and obtaining the residual block of the current block is, Decode the bitstream and obtain the first flag and the second flag, If the first flag indicates that the geometric partitioning mode is permitted to make predictions for blocks in the current sequence, and the second flag indicates that the first transformation is permitted to make transformations for blocks in the current sequence, then the first transformation is performed on the first transformation coefficients to obtain the residual block, including, The decoding method according to feature 1.

3. Performing the first transformation on the aforementioned first transformation coefficient and obtaining the residual block of the current block is, If the first flag indicates that the geometric division mode is permitted to be used to make predictions for the blocks in the current sequence, and the second flag indicates that the first transformation is permitted to be used to perform a transformation on the blocks in the current sequence, then if the height and / or width of the current block is greater than or equal to a first threshold, the first transformation is performed on the first transformation coefficients to obtain the residual block, The decoding method according to feature 2.

4. Performing the first transformation on the aforementioned first transformation coefficient and obtaining the residual block of the current block is, The process involves performing a second transformation on the first transformation coefficient to obtain the second transformation coefficient, Performing the first transformation on the second transformation coefficient and obtaining the residual block of the current block, including, The decoding method according to any one of claims 1 to 3.

5. An encoding method, Based on a first prediction mode and a second prediction mode corresponding to the geometric partitioning mode, a prediction is made for the current block, and the predicted block of the current block is obtained. Determining the weight derivation mode used for the aforementioned geometric partitioning mode, The transformation kernel typeset that conforms to the intra-prediction mode corresponding to the weight derivation mode is determined as the transformation kernel typeset used for the third transformation, Based on the predicted block of the current block, the residual block of the current block is obtained, The third transformation is performed on the residual block of the current block, and the third transformation coefficient of the current block is obtained. Encoding the third conversion coefficient, including, An encoding method characterized by the following features.

6. Encoding the aforementioned third conversion coefficient is This includes encoding the first flag, the second flag, and the third conversion coefficient, The first flag indicates that the geometric partitioning mode is permitted to be used to make predictions for blocks in the current sequence, and the second flag indicates that the third transformation is permitted to be used to perform transformations for blocks in the current sequence. The encoding method according to feature 5.

7. Performing a third transformation on the residual block of the current block as described above, and obtaining the third transformation coefficient of the current block, The process includes, when the height and / or width of the current block exceeds a first threshold, performing the third transformation on the residual block of the current block and obtaining the third transformation coefficient, The encoding method according to feature 6.

8. Encoding the aforementioned third conversion coefficient is The third conversion coefficient is subjected to a fourth conversion to obtain the fourth conversion coefficient, Encoding the fourth conversion coefficient, including, The encoding method according to feature 5.

9. It is a decoder, A decoding unit for decoding a bitstream and obtaining the first transformation coefficient of the current block, A transformation unit for determining the weight derivation mode to be used in the geometric partitioning mode, determining a transformation kernel typeset that fits the intra prediction mode corresponding to the weight derivation mode as the transformation kernel typeset to be used in the first transformation, performing the first transformation on the first transformation coefficients, and obtaining the residual block of the current block, A prediction unit for making predictions for the current block based on a first prediction mode and a second prediction mode corresponding to the geometric division mode, and for obtaining a predicted block of the current block, A reconstruction unit for obtaining the reconstructed block of the current block based on the predicted block of the current block and the residual block of the current block, including, A decoder characterized by the following features.

10. It is an encoder, A prediction unit for making predictions for the current block based on a first prediction mode and a second prediction mode corresponding to the geometric division mode, and for obtaining the predicted block of the current block, A residual unit for obtaining the residual block of the current block based on the predicted block of the current block, A transformation unit for determining the weight derivation mode used in the geometric partitioning mode, determining a transformation kernel typeset that fits the intra-prediction mode corresponding to the weight derivation mode as the transformation kernel typeset used in the third transformation, performing the third transformation on the residual block of the current block, and obtaining the third transformation coefficient of the current block, An encoding unit for encoding the third conversion coefficient, including, An encoder characterized by the following features.

11. A non-temporary computer-readable storage medium storing a computer program and a bitstream, When the computer program is executed by the processor, it causes the processor to execute the encoding method described in any one of claims 5 to 8 to generate the bitstream. A non-temporary computer-readable storage medium characterized by the following features.

Images