Encoding method, decoding method, code flow, encoder, decoder, and storage medium

Abstract

The present application describes an encoding method, a decoding method, a code stream, an encoder, a decoder, and a storage medium; the method comprises: determining a prediction mode of a current block; based on the prediction mode of the current block, determining at least one context model; based on the at least one context model, decoding the code stream to determine a string of binary symbols from the current block, wherein the string of binary symbols comprises at least one binary symbol; performing reverse binarization processing on the string of binary symbols to determine the value of the first syntax identification information;And when the first syntax identification information indicates that the current block uses a first transformation mode, determine a transformation kernel of the current block and on the basis of the transformation kernel, perform inverse transformation on the transformation coefficient of the current block to determine a residual block; thus, compression efficiency is improved.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] As demand for video display quality increases, high-resolution video, such as HD and UHD, has emerged. However, high-resolution video typically contains more information and therefore requires more bandwidth. To reduce bandwidth requirements, video coding standards involving video compression have been introduced.

[0003] Video coding standards include intra-frame prediction and inter-frame prediction. Intra-frame prediction predicts the current block based on reconstructed images around the current block, while inter-frame prediction predicts the current block based on images of the same object at different times. The statistical laws of the residuals of the two are different. As a result, although the Low Frequency Non-Separable Transform (LFNST) or Non-Separable Primary Transform (NSPT) can be used during the transformation process, the compression efficiency is low.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve compression efficiency.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Determine the prediction mode of the current block;

[0009] Determining at least one context model according to a prediction mode of a current block;

[0010] Decoding a code stream according to at least one context model to determine a binary symbol string of a current block; wherein the binary symbol string includes at least one binary symbol;

[0011] Performing debinarization processing on the binary symbol string to determine the value of the first grammar identification information;

[0012] When the first syntax identification information indicates that the current block uses the first transform mode, a transform kernel of the current block is determined, and transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block.

[0013] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0014] Determine the prediction mode of the current block;

[0015] Determining at least one context model according to a prediction mode of a current block;

[0016] Determining a value of the first syntax identification information, and performing binarization processing on the value of the first syntax identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol;

[0017] At least one binary symbol in the binary symbol string is respectively coded according to at least one context model, and the obtained coded bits are written into a bitstream.

[0018] In a third aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: a quantization coefficient of a current block, a value of first syntax identification information, a value of second syntax identification information, a value of third syntax identification information, and a value of fourth syntax identification information;

[0019] Among them, the first syntax identification information is used to indicate whether the current block uses the first transformation mode, the second syntax identification information is used to indicate whether the current block uses the multi-transform selection mode, the third syntax identification information is used to indicate whether the current block uses the inseparable basic transformation mode, and the fourth syntax identification information is used to indicate whether the current block uses the low-frequency inseparable transformation mode.

[0020] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a binarization unit, and an encoding unit, wherein:

[0021] A first determining unit configured to determine a prediction mode of a current block; and determine at least one context model according to the prediction mode of the current block;

[0022] a binarization unit configured to determine a value of the first syntax identification information and perform binarization processing on the value of the first syntax identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol;

[0023] The encoding unit is configured to respectively encode at least one binary symbol in the binary symbol string according to at least one context model, and write the obtained encoded bits into the code stream.

[0024] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor, wherein:

[0025] a first memory for storing a computer program capable of running on the first processor;

[0026] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0027] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second determination unit, a decoding unit, a debinarization unit, and an inverse transformation unit, wherein:

[0028] a second determining unit configured to determine a prediction mode of the current block; and determine at least one context model according to the prediction mode of the current block;

[0029] A decoding unit configured to decode the code stream according to at least one context model to determine a binary symbol string of a current block; wherein the binary symbol string includes at least one binary symbol;

[0030] a debinarization unit configured to perform debinarization processing on the binary symbol string to determine a value of the first grammar identification information;

[0031] The inverse transform unit is configured to determine a transform kernel of the current block when the first syntax identification information indicates that the current block uses the first transform mode, and inversely transform the transform coefficients of the current block according to the transform kernel to determine a residual block of the current block.

[0032] In a seventh aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor, wherein:

[0033] a second memory for storing a computer program capable of running on the second processor;

[0034] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0035] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method described in the first aspect or the method described in the second aspect.

[0036] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. At the encoding end, a prediction mode of a current block is determined; at least one context model is determined based on the prediction mode of the current block; a value of first syntax identification information is determined, and the value of the first syntax identification information is binarized to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; at least one binary symbol in the binary symbol string is encoded according to at least one context model, and the obtained coded bits are written into the code stream. At the decoding end, a prediction mode of the current block is determined; at least one context model is determined based on the prediction mode of the current block; the code stream is decoded according to at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; the binary symbol string is debinarized to determine a value of the first syntax identification information; when the first syntax identification information indicates that the current block uses a first transform mode, a transform kernel of the current block is determined, and the transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block. In this way, both the encoding and decoding ends first determine at least one context model based on the prediction mode of the current block, and then perform encoding and decoding on the binary symbol string corresponding to the first syntax identification information based on the at least one determined context model. In other words, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, so that the encoding and decoding method that conforms to the distribution law of the binary symbol string of LFNST / NSPT in the inter-frame prediction mode can be used, thereby improving not only compression efficiency but also encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a flow chart diagram of a hybrid coding framework;

[0038] FIG2 is a schematic diagram of the structure of a group of pictures GOP;

[0039] FIG3 is a schematic diagram of a block partition structure;

[0040] FIG4 is a schematic diagram of a block position relationship in the spatial domain / temporal domain of a current block;

[0041] FIG5 is a schematic diagram of motion information of a current block and a co-located block;

[0042] FIG6 is a schematic diagram of motion displacement using a co-located reference image;

[0043] FIG7 is a schematic diagram of a distribution using MVD characteristics;

[0044] FIG8 is a schematic diagram of an affine motion using 2 or 3 control points;

[0045] FIG9 is a schematic diagram of affine motion based on derivation of motion vectors of sub-blocks;

[0046] FIG10 is a schematic diagram of weights of various modes under a GPM mode;

[0047] FIG11 is a schematic diagram of motion information for bilateral matching;

[0048] FIG12 is a schematic diagram of template matching of a current block;

[0049] FIG13 is a schematic diagram of reference pixels of a current block;

[0050] FIG14 is a schematic diagram of multiple reference rows of a current block;

[0051] FIG15 is a schematic diagram 1 of multiple prediction modes corresponding to an intra-frame prediction;

[0052] FIG16 is a second schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0053] FIG17 is a third schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0054] FIG18 is a fourth schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0055] FIG19 is a schematic diagram of encoding of screen content;

[0056] FIG20 is a schematic diagram of a histogram of gradient and intra prediction mode;

[0057] FIG21 is a schematic diagram of weighted fusion of three intra-frame prediction modes;

[0058] FIG22 is a schematic diagram of reference pixels of chrominance components and luminance components;

[0059] FIG23 is a schematic diagram of a grouping of a multi-model CCLM;

[0060] FIG24 is a schematic diagram of a multi-model CCLM prediction;

[0061] FIG25 is a schematic diagram of a reference area for intra-frame CCCM;

[0062] FIG26 is a schematic diagram of brightness gradient calculation of a GLM;

[0063] FIG27 is a schematic diagram of an inter-frame CCCM prediction;

[0064] FIG28 is a schematic diagram of a DCT transformation;

[0065] FIG29 is a schematic diagram of a base image of a DCT transformation;

[0066] FIG30 is a schematic diagram of a process flow without LFNST transformation;

[0067] FIG31 is a schematic diagram of a process flow with LFNST transformation;

[0068] FIG32 is a detailed flowchart of a LFNST transformation;

[0069] FIG33 is a schematic diagram of a base image of multiple transformation kernel groups;

[0070] FIG34 is a schematic diagram of a base image of NSPT transformation;

[0071] FIG35 is a schematic diagram of a structure in which brightness and chromaticity are divided separately;

[0072] FIG36 is a schematic diagram of a network architecture of a video codec provided in an embodiment of the present application;

[0073] FIG37 is a schematic block diagram of a system composition of an encoder provided in an embodiment of the present application;

[0074] FIG38 is a schematic block diagram of a system composition of a decoder provided in an embodiment of the present application;

[0075] FIG39 is a flowchart diagram 1 of a decoding method provided in an embodiment of the present application;

[0076] FIG40 is a schematic diagram showing statistical results of lfnst_nspt_idx corresponding to an intra prediction mode provided by an embodiment of the present application;

[0077] FIG41 is a schematic diagram showing statistical results of lfnst_nspt_idx corresponding to an inter-frame prediction mode provided by an embodiment of the present application;

[0078] FIG42 is a schematic diagram of a single texture feature and a bidirectional texture feature provided in an embodiment of the present application;

[0079] FIG43 is a second flow chart of a decoding method provided in an embodiment of the present application;

[0080] FIG44 is a third flow chart of a decoding method provided in an embodiment of the present application;

[0081] FIG45 is a fourth flowchart of a decoding method provided in an embodiment of the present application;

[0082] FIG46 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0083] FIG47 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0084] FIG48 is a block diagram illustrating a method of writing a syntax element value into a code stream according to an embodiment of the present application;

[0085] FIG49 is a block diagram illustrating a method for decoding syntax element values ​​from a bitstream according to an embodiment of the present application;

[0086] FIG50 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0087] FIG51 is a schematic diagram of the hardware structure of an encoder provided in an embodiment of the present application;

[0088] FIG52 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0089] FIG53 is a schematic diagram of the hardware structure of a decoder provided in an embodiment of the present application;

[0090] Figure 54 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0093] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0094] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0095] In video images, a coding block (CB) is generally represented by a first color component, a second color component, and a third color component. These three color components are a luminance component, a blue chrominance component, and a red chrominance component. Specifically, the luminance component is typically represented by the symbol Y, the blue chrominance component is typically represented by the symbols Cb or U, and the red chrominance component is typically represented by the symbols Cr or V. Thus, video images can be represented in either the YCbCr or YUV format.

[0096] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0097] H.265 / High Efficiency Video Coding (HEVC);

[0098] H.266 / Versatile Video Coding (VVC);

[0099] VVC Test Model (VTM), a reference software testing platform for VVC;

[0100] A platform that improves compression performance after VVC (Enhanced Compression Model, ECM);

[0101] Joint Video Experts Team (JVET);

[0102] Coding Unit (CU);

[0103] Coding Tree Unit (CTU);

[0104] Largest Coding Unit (LCU);

[0105] Motion Vector (MV);

[0106] Prediction Unit (PU);

[0107] Transform Unit (TU);

[0108] Fusion technology (Merge);

[0109] Skip technology (Skip);

[0110] Quantization Parameter (QP);

[0111] Merge with Motion Vector Difference (MMVD)

[0112] Motion Vector Prediction (MVP);

[0113] Temporal Motion Vector Prediction (TMVP);

[0114] Subblock-based Temporal Motion Vector Prediction (SbTMVP);

[0115] Discrete Cosine Transform (DCT);

[0116] Discrete Sine Transform (DST);

[0117] Multiple Transform Selection (MTS);

[0118] Low Frequency Non-Separable Transform (LFNST);

[0119] Non-Separable Primary Transform (NSPT);

[0120] Context-based Adaptive Binary Arithmetic Coding (CABAC).

[0121] Currently, common video codec standards all adopt a block-based hybrid coding framework. Each image, sub-image, or frame in a video is divided into square maximum coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 256×256, 128×128, 64×64, etc.). Each LCU or CTU can be divided into rectangular CUs according to a set of rules. Coding units may also be divided into prediction units (PUs) and transform units (TUs). Specifically, as shown in Figure 1, the hybrid coding framework includes modules such as prediction, transform / quantization, entropy coding, inverse quantization / inverse transform, and in-loop filtering. The prediction module can include intra-frame prediction and inter-frame prediction, and inter-frame prediction can include motion estimation and motion compensation. Because adjacent pixels within a video image are highly correlated, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Furthermore, because adjacent images in a video image have strong similarities, inter-image prediction is used in video coding and decoding to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0122] The basic process of a video codec is as follows: On the encoder side, an image is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations as the decoder side to obtain a decoded image. The decoded image can serve as a reference image for inter-frame prediction of subsequent images. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoder are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing the bit stream and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The embodiment of the present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0123] In addition, in the embodiments of the present application, the current block (CB) can be the current coding unit, the current prediction unit, or the current transform unit. Due to the need for parallel processing, the image can be divided into slices, etc. Slices in the same image can be processed in parallel, that is, there is no data dependency between them. "Frame" is a commonly used term, and it can generally be understood that a frame is an image. The frame described in the embodiments of the present application can also be replaced by an image or a slice, etc.

[0124] It can be understood that in the prediction technology, intra-frame prediction and inter-frame prediction are described in detail below respectively.

[0125] (1) Inter-frame prediction.

[0126] Inter-frame prediction uses temporal correlation to eliminate redundancy. To minimize visual lag, typical video frame rates range from 30, 50, 60, or even 120 frames per second. In such videos, adjacent frames within the same scene are highly correlated. Inter-frame prediction leverages this correlation to predict the current encoding content, referencing the content of previously encoded and decoded frames. Inter-frame prediction can significantly improve encoding performance.

[0127] The most basic inter-frame prediction method is translational prediction. Translational prediction assumes that the content being predicted translates between the current image and the reference image. For example, if the content of the current block (coding unit or prediction unit) translates between the current image and the reference image, a motion vector can be used to find this content in the reference image and use it as the prediction block for the current block. Translational motion is a significant feature of video, and can be used to handle static backgrounds, moving objects, and camera movements.

[0128] Some content in natural videos doesn't simply translate; for example, the translation process involves subtle changes in shape and color. Bidirectional prediction uses two reference blocks from a reference image and performs a weighted average of these blocks to produce a predicted block that is as similar as possible to the current block. For example, in certain scenes, a weighted average of two reference blocks from before and after the current frame may produce a prediction that is more similar to the current block than a single reference block. This improves compression performance over unidirectional prediction.

[0129] The picture playback order (Picture Order Count, POC) can be used as an identifier of the image. In a video sequence, each image has a unique POC. In the embodiment of the present application, it is considered that the order of the POC and the playback order are the same. The P picture (P Frame) is an image that can only be predicted using the reference image whose POC is before the current image. The current reference image has only one reference picture list (Reference Picture List, RPL), denoted as RPL0. The reference picture list RPL0 contains all reference images whose POC is before the current image. The B picture (B Frame) is an image that can be predicted using the reference image whose POC is before the current image and the reference image whose POC is after the current image. The B picture has two reference picture lists, denoted as RPL0 and RPL1. One configuration method is that RPL0 contains all reference images whose POC is before the current image, and RPL1 contains all reference images whose POC is after the current image. For a current block, one can reference only the reference block of a certain image in RPL0 (also known as forward prediction); only the reference block of a certain image in RPL1 (also known as backward prediction); or simultaneously reference the reference block of a certain image in RPL0 and a reference block of a certain image in RPL1 (also known as bidirectional prediction). A simple method for simultaneously referencing two reference blocks is to average the pixels at each corresponding position in the two reference blocks to obtain the prediction block for the current block. Later, B-pictures no longer require that RPL0 only contain reference images with a Point of Occurrence (POC) before the current image, and RPL1 only contain reference images with a Point of Occurrence (POC) after the current image. Therefore, RPL0 can also contain reference images with a Point of Occurrence (POC) after the current image, and RPL1 can also contain reference images with a Point of Occurrence (POC) before the current image. The current block can then reference both reference images with a Point of Occurrence (POC) before the current image and reference images with a Point of Occurrence (POC) after the current image. This type of B-picture is also called a generalized B-picture.

[0130] Because the encoding and decoding order of the random access (RA) configuration is different from the POC order, B pictures can reference information before and after the current picture, significantly improving encoding performance. For example, a classic RA group of pictures (GOP) structure is shown in Figure 2. In Figure 2, arrows represent reference relationships. Since I pictures do not require reference pictures, after an I picture with a POC of 0 is decoded, a P picture with a POC of 4 will be decoded. When decoding a P picture with a POC of 4, the I picture with a POC of 0 can be referenced. After decoding a P picture with a POC of 4, a B picture with a POC of 2 is decoded. When decoding a B picture with a POC of 2, the I picture with a POC of 0 and the P picture with a POC of 4 can be referenced, and so on. Thus, according to Figure 2, when the POC order is {0 1 2 3 4 5 6 7 8}, the corresponding decoding order is {0 3 2 4 1 7 6 8 5}.

[0131] The encoding and decoding order of the Low Delay (LD) configuration is the same as the POC order. Therefore, the current image can only reference information before the current image. The Low Delay configuration is divided into Low Delay P and Low Delay B. Low Delay P is the traditional Low Delay configuration. Its typical structure is IPPP..., that is, an I picture is encoded and decoded first, and subsequent pictures are all P pictures. The typical structure of Low Delay B is IBBB..., which differs from Low Delay P in that each inter-frame picture is a B picture, that is, using two reference picture lists, the current block can simultaneously reference the reference block of a picture in RPL0 and the reference block of a picture in RPL1.

[0132] Generally speaking, the RA configuration has higher compression efficiency than the LD configuration, and the LDB configuration has higher compression efficiency than the LDP configuration. This is partly because bidirectional prediction can refer to backward information and partly because bidirectional prediction can reduce prediction errors through techniques such as weighted averaging.

[0133] A reference image list of the current image can have a maximum of several reference images, such as 2, 3 or 4. When encoding a current image, which reference images are in RPL0 and RPL1 is determined by a certain configuration or algorithm, which is not the focus of the embodiments of this application. However, the same reference image may appear in RPL0 and RPL1 at the same time. That is, the codec allows the current block to refer to two reference blocks of the same reference image at the same time. The codec usually uses the index value (index, idx) in the reference image list to correspond to the reference image. If the length of a reference image list is 4, the index has four values ​​0, 1, 2, and 3. For example, if the RPL0 of the current image has 4 reference images with POCs of 5, 4, 3, and 0. Then index 0 of RPL0 is the reference image with POC 5, index 1 of RPL0 is the reference image with POC 4, index 2 of RPL0 is the reference image with POC 3, and index 3 of RPL0 is the reference image with POC 0.

[0134] It can also be understood that inter-frame prediction uses motion information to represent "motion". Basic motion information includes reference picture information and motion vector (MV) information. In order for a block to use bidirectional prediction, it is naturally necessary to find two reference blocks, so two sets of reference image information and motion vector information are needed. Each of these sets can be understood as a unidirectional motion information, and combining these two sets together forms a bidirectional motion information. In specific implementation, unidirectional motion information and bidirectional motion information can use the same data structure, but the two sets of reference frame information and motion vector information of the bidirectional motion information are both valid, while one set of reference frame information and motion vector information of the unidirectional motion information is invalid. The valid can also be said to be "used", and the invalid can also be said to be "not used".

[0135] VVC supports two reference picture lists, denoted as RPL0 and RPL1. For the above-mentioned bidirectional motion information, VVC uses the reference picture index refIdxL0 corresponding to RPL0, the motion vector mvL0 corresponding to RPL0, the reference picture index refIdxL1 corresponding to RPL1, and the motion vector mvL1 corresponding to RPL1. The reference picture index corresponding to RPL0 and the reference picture index corresponding to RPL1 here can be understood as the above-mentioned reference picture information. VVC uses two identification information to indicate whether to use the motion information corresponding to RPL0 and whether to use the motion information corresponding to RPL1, respectively denoted as predFlagL0 and predFlagL1. It can also be understood that predFlagL0 and predFlagL1 indicate whether the above-mentioned unidirectional motion information is "valid". Therefore, although VVC does not explicitly mention the data structure of motion information, it uses the reference picture index, motion vector and "valid" flag corresponding to each reference picture list to represent the motion information. The VVC standard doesn't mention motion information, but instead uses motion vectors. The reference image index and the flag indicating whether to use the corresponding motion information can be considered as appendages of the motion vector. This document uses "motion information" for ease of description, but it should be understood that "motion vectors" can also be used. "Motion information" can also be called "motion parameters."

[0136] For a two-dimensional image, a motion vector can be represented by (x, y), with a horizontal component and a vertical component. Since videos are represented in pixels, there are distances between pixels. The motion of an object in adjacent images may not always correspond to a whole-pixel distance. For example, in a distant video, the distance between two pixels is 1 meter for the distant object, while the object moves 0.5 meters between two frames. This scenario cannot be well represented using a whole-pixel motion vector. Therefore, motion vectors can be expressed at the sub-pixel level, with precision of 1 / 2, 1 / 4, 1 / 8, or 1 / 16 pixels, to represent motion more precisely. Interpolation is then used to obtain the pixel value at the sub-pixel position in the reference image.

[0137] Both the unidirectional and bidirectional predictions in the aforementioned translation prediction are block-based, such as coding units (CUs) or prediction units (PUs). This means that prediction is performed using a pixel matrix as the unit. The most basic block is a rectangular block, such as a square or rectangle. Video codec standards such as HEVC and VVC allow the encoder to determine the size and division of CUs and PUs based on the video content. Regions with simple textures or motion tend to use larger blocks, while regions with complex textures or motion tend to use smaller blocks. The deeper the block division level, the more complex the blocks that more closely resemble the actual texture or motion, but the corresponding overhead for representing these divisions increases. Motion information may also need to be transmitted in the bitstream. Furthermore, generally, the finer the block division, the greater the motion information overhead.

[0138] The most primitive method of representing motion information directly writes the complete motion information. Later, experts discovered that motion vectors can be represented using motion vector prediction (MVP) plus motion vector difference (MVD), i.e., MV = MVP + MVD. The more accurate the MVP, the smaller the MVD, which in turn reduces the bitstream overhead.

[0139] It's also understandable that for merge mode, each inter-coded block requires motion information. To simplify matters, we assume that CU division equals PU division equals TU division, meaning that each coding unit has a prediction unit (PU) and a transform unit (TU) of the same size and position. In reality, with the increased flexibility of CU division, VVC tends to decouple PU and TU division from HEVC. Differences in any of the steps involved in prediction, transform, quantization, and entropy coding can lead to CU division. For example, if two regions have different motion information, the encoder may divide them into different CUs. Alternatively, if two regions have identical or similar motion information but significantly different residual characteristics, the encoder may also divide them into different CUs. The division method is determined based on overall compression efficiency, not solely on a single factor. Consequently, the same object or regions with identical or similar motion may be divided into different CUs.

[0140] Figure 3 is a schematic diagram of the HEVC block division structure. As shown in Figure 3, (a) is the original image. In (a), there is an iron rod moving in the direction indicated by the arrow, and the background area moves less. (b) is the block division of HEVC, and (c) is the block boundary with the same motion information in (b) removed. It can be seen that many adjacent blocks use the same motion information. In this case, if the motion information is encoded separately for each block, it will result in obvious waste. The complete motion information of VVC mentioned above includes the reference image index, MV and the flag of whether it is used for RPL0, and the reference image index, MV and the flag of whether it is used for RPL1. The basic principle of the merge mode is that the current block can inherit the motion information of the adjacent blocks, including the reference image information and motion vector information.

[0141] Merge mode can build a merge candidate list. If the current block uses merge mode, an index can be used to indicate which motion information to merge with the current block, eliminating the need to encode the complete motion information. When building the merge candidate list, you can include motion information of spatially adjacent blocks of the current block, motion information in the temporal domain, motion information of non-adjacent blocks in the spatial domain, motion information of non-adjacent blocks in the temporal domain, motion information based on historical information, and synthesized motion information.

[0142] The spatially adjacent blocks refer to blocks adjacent to the current block in the same image, and the spatially non-adjacent blocks refer to blocks non-adjacent to the current block in the same image. The temporal motion information and the motion information of temporally non-adjacent blocks refer to the motion information at a specified position in the collocated reference image. For example, as shown in FIG4 , the black-filled block is the current block, where positions 1, 2, 3, 4, and 5 are the positions of spatially adjacent blocks used for merging, and the corresponding positions of the other dot-filled blocks are the positions of spatially non-adjacent blocks used for merging. Position 6 is the position used for temporal motion information. If the corresponding position at the lower right corner of the current block is unavailable, the position corresponding to the center of the current block is used. The corresponding positions of the other grid-filled blocks are the positions used for motion information of temporally non-adjacent blocks. Temporal motion information is derived based on the motion information at the corresponding positions in the collocated reference image. The specific derivation method is described below. It should be noted that the background grid in FIG4 is only a schematic representation of pixel coordinates and does not represent a specific block division.

[0143] Historical motion information is independent of position. The codec maintains a first-in, first-out list of motion information. Each time a block is encoded or decoded, the codec updates the list with the motion information of that block, ensuring that there is no duplication with existing motion information in the list. Historical motion information is obtained from this list.

[0144] It can also be understood that for the derivation of temporal motion information (vector), temporal motion information prediction is used as a supplement to spatial motion information prediction. Generally speaking, the correlation between adjacent regions in the same image is stronger than the correlation between regions in different images. However, there are also cases where temporal motion information is more useful. For example, if the current block and the surrounding adjacent blocks in the current image belong to different objects and have completely different motions, the motion of the block belonging to the same object as the current block in a reference image can provide better motion information prediction for the current block.

[0145] The motion vector of the co-located block on the co-located reference image (here, the block that obtains temporal motion information is called the co-located block) is the vector between the co-located reference image col_pic and the reference image col_ref of the co-located block. Specifically, as shown in Figure 5, for the current block, the motion vector it needs is the vector between the current image curr_pic and the reference image curr_ref of the current block. Let the POC distance between col_pic and col_ref be td, and the POC distance between curr_pic and curr_ref be tb. Assuming that the motion on the co-located block is unchanged to the motion on the current block, the scaling ratio can be determined based on td and tb. If the motion vector of the co-located block is (col_mv_x, col_mv_y), the temporal motion vector prediction (tmvp_x, tmvp_y) can be derived as follows: tmvp_x = col_mv_x*tb / td, tmvp_y = col_mv_y*tb / td.

[0146] In VVC, the smallest unit for storing motion information in a co-located reference image is 4×4. That is, each 4×4 sub-block stores a set of motion information. It's understandable that, if hardware implementation costs are not considered, a co-located reference image can also store a set of motion information per pixel.

[0147] It's also understandable that VVC introduces sub-block-based temporal motion vector prediction, or SbTMVP. Conventionally, MVP and TMVP are applied to the entire block, meaning the entire block shares the same MVP. However, SbTMVP is based on sub-blocks, allowing it to generate a unique MVP for each sub-block. This is the fundamental difference between SbTMVP and TMVP.

[0148] On the other hand, TMVP uses the position of the lower right corner of the current block or the position of the current block center to locate the co-located block, while SbTMVP finds a motion offset based on the motion of the surrounding blocks to determine the position. In VVC, if the block at position A1 refers to the co-located reference image, then the motion offset is set to the motion vector of A1 using the co-located reference image. Otherwise, the motion offset is set to (0, 0). As shown in Figure 6, the position is found according to the motion offset, and then the MV corresponding to the position of each sub-block in the "co-located block" is scaled to obtain the MVP of each sub-block.

[0149] It can also be understood that for the fusion technology using motion vector difference, namely MMVD, the motion information in the merge candidate list directly selected by the Merge mode is used as the motion information of the current block. In actual videos, there may sometimes be some differences between the actual motion vector of the current block and the motion vector in the selected merge candidate list. MMVD is a special merge mode in VVC, which uses an efficient method to encode the MVD in this case. Ordinary Merge does not require encoding and decoding of MVD. Ordinary inter mode requires direct encoding and decoding of MVD. MMVD takes advantage of the fact that MVD is more distributed in a single horizontal direction or a single vertical direction, with more MVDs for smaller values ​​and fewer MVDs for larger values, as shown in Figure 7. In Figure 7, circles of different shapes can represent MVDs of different values.

[0150] MMVD can only represent MVDs of specific values ​​in specific directions, and it cannot represent any MVD. It uses mmvd_direction_idx to represent the direction of the MVD, which can also be understood as whether the x and y values ​​of the MVD are non-zero and their positive and negative signs. mmvd_distance_idx represents the absolute value of the non-zero x and y values ​​of the MVD, MmvdDistance.

[0151] Here, Table 1 shows a schematic relationship between mmvd_distance_idx[x0][y0] and MmvdDistance[x0][y0].

[0152] Table 1

[0153] Among them, ph_mmvd_fullpel_only_flag is an image header flag that can set two different combinations of MMVD.

[0154] Here, Table 2 shows a schematic relationship between mmvd_direction_idx[x0][y0] and MmvdSign[x0][y0].

[0155] Table 2

[0156] Among them, the MVD of MMVD is obtained as follows: MmvdOffset[x0][y0][0] = (MmvdDistance[x0][y0] << 2) * MmvdSign[x0][y0][0]; MmvdOffset[x0][y0][1] = (MmvdDistance[x0][y0] << 2) * MmvdSign[x0][y0][1].

[0157] It can also be understood that for affine Affine, the simplest and most commonly used translation motion was introduced above. In the real world, there are not only translations, but also many forms of motion such as reduction, enlargement, rotation, perspective (perspective: objects close to the lens appear large, objects far from the lens appear small), and many irregular forms of motion. Affine can be used to represent more complex motion than translation. As shown in Figure 8, Affine uses a linear model to calculate the motion vector of each sub-block or each pixel in the current block based on the motion vectors of 2 control points (4 parameters, a motion vector includes two parameters x and y) or 3 control points (6 parameters). Among them, (a) provides the case of 2 control points, such as v0 and v1; (b) provides the case of 3 control points, such as v0, v1, and v2.

[0158] For the 4-parameter affine model, the motion vector at the (x, y) position in the current block is derived as follows:

[0159] For the 6-parameter affine model, the motion vector at the (x, y) position in the current block is derived as follows:

[0160] Among them, (mv 0x ,mv 0y ) is the motion vector of the control point at the upper left corner of the current block, (mv 1x ,mv 1y ) is the motion vector of the control point in the upper right corner of the current block, (mv 2x ,mv 2y ) is the motion vector of the control point at the lower left corner of the current block.

[0161] To simplify hardware implementation complexity, Affine, used in VVC, divides the current block into 4×4 sub-blocks, calculates an MV for each sub-block, and performs motion compensation. Figure 9 is a schematic diagram of how Affine derives motion vectors based on sub-blocks. It is understandable that with the improvement of hardware processing capabilities, Affine can also perform pixel-based processing. That is, a motion vector is derived for each pixel, and motion compensation is performed on the pixel based on this motion vector. Here, Affine only needs a few control points to derive a separate motion vector for each sub-block or each pixel, allowing for more precise predictions compared to whole-block motion compensation. Furthermore, compared to dividing the CU into smaller units, Affine has much less overhead.

[0162] It can also be understood that for the Geometric Partitioning Mode (GPM), HEVC supports a maximum CTU of 64×64 and can recursively perform quadtree division. VVC supports a more flexible block division method than HEVC, supporting a maximum CTU of 128×128, including quadtree, ternary tree and binary tree division. These division methods. Although block division is becoming more and more flexible, whether it is CU, PU or TU, it can only be divided into rectangular blocks. It should be noted that VVC has weakened the division of PU and TU. The boundaries of texture or motion in natural videos are diverse. For example, when encountering an oblique object boundary, if you simply use rectangular blocks to approach the boundary, you will divide it into many small blocks, which will significantly increase the overhead. The GPM geometric division mode can better handle textures and boundaries in natural videos.

[0163] GPM uses two prediction blocks of the same size as the current block. Some pixel locations in the GPM prediction block use 100% of the pixel values ​​corresponding to the first prediction block, while others use 100% of the pixel values ​​corresponding to the second prediction block. In the boundary region, or transition region, pixel values ​​from both prediction blocks are used in a certain proportion. The weights in the boundary region also gradually transition. Of course, for scenarios such as screen content encoding, the transition region can be omitted. The specific distribution of these weights is determined by the GPM's "partitioning" mode. The weight of each pixel position is determined based on the GPM's partitioning mode. However, in certain situations, such as very small block sizes, some GPM modes may not guarantee that some pixel locations will use 100% of the pixel values ​​corresponding to the first prediction block, while others will use 100% of the pixel values ​​corresponding to the second prediction block. Alternatively, GPM can be considered to use two prediction blocks of different block sizes, taking the required portion from each and discarding any portions with zero weights.

[0164] Figure 10 shows the weights of the 64 GPM modes in VVC on a square block. As shown in Figure 10, black indicates a weight of 0% for the position corresponding to the first prediction block, white indicates a weight of 100%, and gray areas, depending on the depth of the color, indicate a weight greater than 0% and less than 100% for the position corresponding to the first prediction block. The weight of the position corresponding to the second reference block is 100% minus the weight of the position corresponding to the first reference block.

[0165] GPM can be considered a prediction mode or method, as it ultimately produces a prediction block. It can also be considered a "partitioning" mode, simulating a partitioning of the prediction block, similar to implementing PU partitioning without actually partitioning the prediction block. The first and second prediction blocks used in GPM can be prediction blocks generated by intra-frame prediction, inter-frame unidirectional prediction, or inter-frame bidirectional prediction.

[0166] Understandably, the bitrate of general consumer video is limited, so video compression typically seeks a compromise between bitrate overhead and distortion. Taking block partitioning as an example, for the same content, within a certain range, finer the partitioning, the higher the overhead and lower the distortion; coarser the partitioning, the lower the overhead and higher the distortion. Taking motion information encoding as an example, for the same content, within a certain range, more precise motion information leads to higher overhead and lower distortion; coarser motion information leads to lower overhead and higher distortion. Some decoder-side methods utilize decoder-side information for processing and calculation without incurring overhead, thereby improving motion information, enhancing prediction, and reducing distortion. This overhead-free approach means that the encoder does not require instructions based on the original image; processing is performed automatically based on available information. Two typical decoder-side methods in VVC are decoder-side motion vector refinement (DMVR) and bidirectional optical flow (BDOF), which are described in detail below.

[0167] In one possible implementation, for DMVR, one condition for starting DMVR in VVC is that the two reference images of the current block come from the front and back of the current image respectively, and the distances between the two reference images and the current image are equal. Another starting condition is that the current CU uses the whole-block merge mode (including skip), which does not include sub-block-based merges such as SbTMVP and affine merge, because the motion vector in the merge mode is prone to inaccuracy. There are some other conditions that are not described here. DMVR in VVC uses bilateral matching (BM), which is to calculate the matching cost for the reference blocks on both sides, such as the sum of absolute differences (SAD). DMVR searches for the matching cost of the MVs around the original MV. When moving, the MVs of the two reference images are moved in a mirrored manner, that is, the MVs are moved on the basis of their respective original MVs. diff , move the other side - MV diff , as shown in Figure 11. In Figure 11, the two reference images include reference image L0 (refPic in ListL0) and reference image L1 (refPic in ListL1). The search also supports sub-pixel search, so DMVR may find an MV with higher accuracy than the original MV. The search is performed according to certain rules. Generally, the integer pixel MV within a certain range is searched first to find the integer pixel MV with the lowest matching cost, and then the sub-pixel MV is searched based on the integer pixel MV. If an MV with a lower matching cost than the original MV is found, the MV with the lower matching cost is used for motion compensation prediction. The MV improved by DMVR can theoretically be used to store MVs and surrounding blocks. For example, when constructing a merge candidate list for the current block, if the MV of the surrounding blocks is improved by DMVR, using the improved MV to construct the merge candidate list can achieve better compression effect. However, due to hardware implementation considerations, VVC does not do this.

[0168] DMVR can be processed on a sub-block basis. In fact, in VVC, if the horizontal or vertical size of a block is larger than 16 pixels, it will be divided into sub-blocks of 16 pixels. This is partly due to the complexity of hardware implementation, because DMVR needs to be searched at the decoding end, and limiting the size of sub-blocks can reduce the cost of caching. On the other hand, dividing the processing into sub-blocks provides better flexibility. Each sub-block can independently improve the MV, which to a certain extent achieves the effect of improving the division accuracy, which also improves the compression efficiency.

[0169] In another possible implementation, BDOF is also a typical decoder-side method. As its name suggests, BDOF improves MV and prediction based on the principle of optical flow. Optical flow is the instantaneous velocity of pixels moving across the observation imaging plane for a moving object. Optical flow has some basic assumptions, such as constant brightness (i.e., the brightness of the same object does not change when it moves between different images), and temporal continuity or small motion. This means that changes in time do not cause drastic changes in the object's position.

[0170] A condition for starting BDOF in VVC is that the two reference images of the current block come from the previous and next images respectively, and the distance between the two reference images and the current image is equal. For each 4×4 sub-block, VVC will derive a motion vector deviation (v x ,v y ), which is calculated by minimizing the difference between the predicted values ​​in the two directions. This motion vector deviation is also used to adjust the predicted value in the corresponding sub-block. The derivation process is as follows:

[0171] First, calculate the horizontal and vertical gradients of the two prediction blocks and The details are as follows:

[0172] Among them, I (k) (i, j) is the predicted value of the coordinate (i, j) in the reference image list k, k = 0, 1, and shift1 is calculated according to the bit depth bitDepth of the brightness, shift1 = max(6, bitDepth-6).

[0173] Next, calculate S1, S2, S3, S5, and S6 as follows: S1 = ∑ (i,j)∈Ω Abs(ψ x (i,j)) (5) S2=∑ (i,j)∈Ω ψ x (i,j)·Sign(ψ y (i,j)) (6) S3=∑ (i,j)∈Ω θ(i,j)·Sign(ψ x (i,j)) (7) S5=∑ (i,j)∈Ω Abs(ψ y (i,j)) (8) S6=∑ (i,j)∈Ω θ(i,j)·Sign(ψ y (i,j)) (9)

[0174] in, θ(i,j)=(I (1) (i,j)>>n b )-(I (0) (i,j)>>n b ) (12)

[0175] Where Ω is a 6×6 window around the current 4×4 sub-block, n a is min(1,bitDepth-11), n b is min(4,bitDepth-8).

[0176] Again, the motion vector deviation (v x ,v y ) is calculated as follows:

[0177] in, th′ BIO =2max(5,BD-7), is rounded down, BD is bit depth bitDepth.

[0178] Again, based on the motion vector deviation and gradient, each prediction value within the 4×4 sub-block is adjusted as follows:

[0179] The final predicted value of BDOF is calculated as follows: pred BDOF (x,y)=(I (0) (x,y)+I (1) (x,y)+b(x,y)+o offset )>>shift (16)

[0180] Among them, offset and shift are calculated based on the bit depth of brightness. a 、n b Both scalar and shift are processed to reduce the bit width during the calculation process.

[0181] In this way, the motion vector deviation of BDOF can achieve high precision, making the prediction more accurate, and the sub-block-based processing also improves flexibility. These two aspects are similar to DMVR.

[0182] It's also understood that both DMVR and BDOF improve motion vectors. DMVR is based on block matching, while BDOF is based on optical flow. They can be used in combination. For example, this can be referred to as Multi-pass Decoder-side Motion Vector Refinement (MDMVR).

[0183] The first step is to improve the motion vector based on bidirectional matching of the entire block. The second step is to improve the motion vector based on bidirectional matching of sub-blocks. The block size for this step can be 16×16. The third step is to improve the motion vector based on bidirectional optical flow of sub-blocks. The sub-block size for this step can be 8×8. Further steps can be added to this system, such as a fourth step to improve the motion vector based on bidirectional optical flow of 4×4 sub-blocks, or a point-based bidirectional optical flow improvement.

[0184] It's also understandable that the Template Matching (TM) method was first used in inter-frame prediction. It leverages the correlation between adjacent pixels and uses areas surrounding the current block as templates. When the current block is encoded or decoded, its left and upper sides have already been encoded and decoded according to the coding order. Of course, existing hardware decoder implementations don't guarantee that the left and upper sides have already been decoded when decoding begins. This refers to inter-frame blocks. For example, in HEVC, when generating prediction blocks for inter-frame coded blocks, the surrounding reconstructed pixels are not required, allowing the prediction process for inter-frame blocks to proceed in parallel. However, intra-frame coded blocks must use reconstructed pixels on the left and upper sides as reference pixels. Theoretically, the left and upper sides are available, meaning that this can be achieved by adjusting the hardware design accordingly. Comparatively, the right and lower sides are unavailable under current coding standards such as VVC.

[0185] As shown in Figure 12, the rectangular areas to the left and above the current block are used as templates. The height of the left template portion is generally the same as the height of the current block, and the width of the upper template portion is generally the same as the width of the current block, but they can also be different. The best matching position of the template is found in the reference image L0 to determine the motion information, or motion vector, of the current block. This process can be roughly described as starting from a starting position in a reference image and searching within a certain range around it. Search rules, such as the search range and search step size, can be predefined. At each position, the degree of match between the template corresponding to that position and the templates surrounding the current block is calculated. The degree of match can be measured using distortion metrics such as the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), and the mean-square error (MSE). SATD generally uses the Hadamard transform. Smaller values ​​of SAD, SATD, and MSE indicate a higher degree of match. The cost is calculated using the predicted block of the template corresponding to the position and the reconstructed blocks of the template surrounding the current block. In addition to searching at whole-pixel locations, sub-pixel locations can also be searched, with the motion information of the current block determined based on the location with the highest degree of match. By leveraging the correlation between adjacent pixels, the appropriate motion information for the template may also be appropriate for the current block. Of course, template matching may not be applicable to all blocks, so methods can be used to determine whether template matching should be used for the current block, such as using a control switch to indicate whether template matching should be used for the current block. This template matching method is called decoder-side motion vector derivation (DMVD). Both the encoder and decoder can use the template to search to derive motion information or find better motion information based on the existing motion information. This method does not require the transmission of specific motion vectors or motion vector differences. Instead, both the encoder and decoder perform the same search rules to ensure consistent encoding and decoding. Template matching can improve compression performance, but it requires a "search" at the decoder, which introduces a certain degree of decoding complexity.

[0186] (2) Intra-frame prediction.

[0187] There is a strong spatial correlation between adjacent parts or adjacent pixels within the image. Intra-frame prediction is a prediction method that uses the spatial correlation between the coded and decoded pixels around the current block and the pixels within the current block. For example, as shown in Figure 13, the 4×4 white filling pixels are the current block, and the grid filling pixels in the left column and the upper row of the current block are the reference pixels of the current block. Intra-frame prediction uses these reference pixels to predict the current block. These reference pixels may all be available, that is, all have been coded and decoded. Some may also be unavailable. For example, if the current block is the leftmost of the entire frame, then the reference pixels on the left side of the current block are unavailable. Or when encoding and decoding the current block, the lower left part of the current block has not been encoded and decoded, then the reference pixels on the lower left are also unavailable. For the case where reference pixels are unavailable, available reference pixels or certain values ​​or certain methods can be used for filling, or no filling can be performed.

[0188] It should also be noted that the multiple reference line (MRL) intra prediction method can use more reference pixels to improve coding efficiency. As shown in Figure 14, this is a schematic diagram using four reference rows / columns.

[0189] There are multiple prediction modes for intra-frame prediction, as shown in Figure 15. Here are the nine modes for intra-frame prediction of 4×4 blocks in H.264. Among them, mode 0 (vertical mode) copies the pixels above the current block vertically to the current block as the prediction value, mode 1 (horizontal mode) copies the reference pixels on the left side horizontally to the current block as the prediction value, mode 2 (DC mode) uses the average of the eight points A to D and I to L as the prediction value for all points, and modes 3 to 8 copy the reference pixels to the corresponding positions of the current block at a certain angle. Because some positions of the current block cannot correspond exactly to the reference pixels, it may be necessary to use the weighted average of the reference pixels, or the sub-pixels of the interpolated reference pixels.

[0190] In addition, there are modes such as PLANE and PLANAR. With technological advancements and the expansion of block sizes, the number of angular prediction modes is also increasing. For example, HEVC uses intra-frame prediction modes including PLANAR, DC, and 33 angular modes, for a total of 35 prediction modes (see Figure 16 for details). VVC uses intra-frame modes including PLANAR, DC, and 65 angular modes, for a total of 67 prediction modes (see Figure 17 for details). Of course, in addition to the 67 modes mentioned above, VVC also provides wide-angle modes for rectangular blocks with a large difference between length and width. These modes, indicated by the dashed lines in the figure, represent the ranges -14 to -1 and 67 to 80 degrees. These modes replace some conventional modes (see Figure 18 for details).

[0191] It can also be understood that intra block copy (IBC) can significantly improve the compression efficiency of screen content coding (SCC), and therefore IBC has been used for screen content coding from HEVC to VVC. Screen content is different from camera-captured content. It is computer-generated, noise-free, and contains text, computer graphics, and has clear boundaries. There is a large amount of repeated content in the screen content, as shown in Figure 19.

[0192] In the embodiments of this application, IBC can be considered to apply the inter-frame prediction method to intra-frame prediction. Inter-frame prediction uses a reference block in a reference image, which is not the current image, to generate the prediction block for the current block. IBC, on the other hand, uses a reference block from the coded or reconstructed portion of the current image to generate the prediction block for the current block. IBC is also known as intra picture block compensation or current picture referencing (CPR).

[0193] IBC uses a block vector (BV) to represent the position difference between the current block and the reference block, similar to the MV used in inter-frame prediction. The encoder uses block matching within the search range to determine the best matching block for the current block and encodes the BV. There are various methods for encoding the BV, such as the merge mode, which is similar to inter-frame prediction and will not be discussed here.

[0194] IBC can be considered an intra-frame prediction method, or a different type of prediction method independent of intra-frame and inter-frame prediction. IBC is highly efficient for encoding screen content and can also improve compression efficiency in natural sequences captured by cameras.

[0195] Intra Template Matching Prediction (IntraTMP) in ECM10 can be considered a special type of IBC. IntraTMP also searches for a reference block from the coded portion of the current image to generate a prediction block for the current block. IntraTMP uses the reconstructed pixels within a certain range to the left and above the current block as the template for the current block. Each time a BV is searched, the template of the reference block of reconstructed pixels within a certain range to the left and above the corresponding reference block of the same size as the current block is calculated. The matching cost between the current block's template and the reference block's template is calculated, and the prediction block is generated based on the matching cost.

[0196] It can also be understood that for decoder-side intra mode derivation (DIMD), DIMD uses the reconstructed pixels on the left and top sides of the current block to derive the prediction mode, but it does not predict on the template, but analyzes the gradient of the reconstructed pixels.

[0197] As shown in Figure 20, DIMD analyzes the gradient of black points, such as horizontal gradient and vertical gradient, and adapts an intra-frame prediction mode according to its gradient. The analysis of all points that need to be checked can obtain a result similar to the bar graph below. That is, the statistics of the number of points matched by each intra-frame prediction mode. Of course, the so-called bar graph is just to help understanding, and it can be implemented in a variety of simple forms. The current DIMD selects the two highest intra-frame prediction modes in the histogram, plus the PLANAR mode, and the prediction values ​​of a total of three intra-frame prediction modes are weighted. The weights are related to the results of the analysis. For example, as shown in Figure 21, the three intra-frame prediction modes include M1 mode, M2 mode and PLANAR mode. The prediction values ​​obtained for these three intra-frame prediction modes are set to Pred1, Pred2, and Pred3 respectively, and the weight values ​​of these three intra-frame prediction modes are set to w1, w2, and w3 respectively. The specific calculation formulas are as follows:

[0198] The final prediction block can be shown as follows:

[0199] In summary, DIMD uses gradient analysis of reconstructed pixels to select intra prediction modes, and can weight two intra prediction modes plus planar based on the analysis results. The advantage of DIMD is that if DIMD mode is selected for the current block, the decoder does not need to indicate the specific intra prediction mode to be used. Instead, the decoder can derive the selected mode through the above process, which saves a certain amount of overhead.

[0200] It can also be understood that for cross-component prediction (Cross-Component Prediction, CCP), since different components of the same space have a strong correlation, video coding and decoding technology can use this correlation to improve compression efficiency. In some cases, the first component of the same space will be encoded and decoded first, and then the second and third components will be encoded and decoded, so that the second and third components can use some information of the first component. For example, in the YUV format, the pixels of U and V can be predicted using the reconstructed pixels of Y at the corresponding positions. If it is in YUV4:2:0 format, the pixel positions of U and V do not correspond one to one to the position of Y, and the corresponding reconstructed pixels of Y can be found for prediction through methods such as downsampling.

[0201] A typical example of CCP in VVC is Cross-Component Linear Model (CCLM). Other cross-component prediction technologies in ECM include Multi-Model CCLM (MM-CCLM), Convolutional Cross-Component Model (CCCM), Multi-Model CCCM (MM-CCCM), Gradient Linear Model (GLM), inter-CCCM, etc. There are other derivative technologies that are not listed one by one. They are all technologies that use some information of the first color component, such as the reconstruction value, to predict the second / third color component. Cross-component prediction technology can be used for blocks encoded within a frame as well as for blocks encoded between frames. For example, inter-CCCM is used for blocks encoded between frames.

[0202] (1) Cross-component linear model CCLM.

[0203] CCLM, as the name implies, is a technique that uses a linear model of the first color component and the second / third color component for prediction. Specifically, it is as follows: pred C (i,j)=α·rec L ′(i,j)+β (21)

[0204] Among them, pred C (i, j) represents the predicted value of the chrominance pixel at position (i, j), which represents rec L The reconstructed value of the downsampled luminance pixel at the ′(i,j) position. The model parameters (α and β) of CCLM are derived based on the adjacent downsampled luminance pixels and chrominance pixels of the current block. For example, as shown in Figure 22, a sampling diagram of the adjacent reference values ​​of the luminance component and the chrominance component of the current block is shown; wherein, in (a), the larger bold box is used to highlight the chrominance block 21, and the gray solid circle is used to indicate the adjacent reference value of the chrominance block 21; in (b), the larger bold box is used to highlight the luminance block 22, and the gray solid circle is used to indicate the adjacent reference value of the luminance block 22. Among them, the chrominance block 21 is N×N in size, and the luminance block 22 is 2N×2N in size. Here, the adjacent reference values ​​of the chrominance block 21 and the adjacent reference values ​​of the luminance block 22 are both used to derive the model parameters α and β.

[0205] The basic principle is to use the adjacent luminance and chrominance pixels of the current block to derive a linear model, then apply this linear model to the current block, and determine the predicted value of chrominance based on the reconstructed value of luminance and this linear model.

[0206] (2) Multi-model CCLM.

[0207] MM-CCLM is an extension of CCLM. CCLM uses only one linear model in the current block, while MM-CCLM, as the name implies, uses multiple linear models. Specifically, two linear models are used in ECM-10. To derive the two linear models, the adjacent luminance pixels on the left and above sides of the current block used to derive the linear models are divided into two groups, with the grouping threshold being the median of these luminance pixel values. When predicting the current block, the same threshold is used to divide the luminance pixels of the current block into two groups, and prediction is performed using their respective models. Figure 23 is a schematic diagram of MM-CCLM grouping, and Figure 24 is a schematic diagram of MM-CCLM prediction.

[0208] (3) Convolutional Cross-Component Model CCCM.

[0209] Similar to CCLM, CCCM also derives a cross-component model based on the reconstructed portions to the left and above the current block. However, CCCM uses a larger reconstruction area, allowing it to derive a nonlinear model. Another difference is that CCLM predicts a chrominance value from a downsampled luma value, while CCCM uses the reconstructed luma value at the corresponding chrominance pixel, as well as the surrounding pixels above, below, and to the left and right of the pixel.

[0210] Specifically, CCCM uses a 7-tap convolution filter that includes five spatially adjacent pixels. C (center) is the luma pixel corresponding to the current chroma pixel, N (north) is the luma pixel above C, S (south) is the luma pixel below C, W (west) is the luma pixel to the left of C, and E is the luma pixel to the right of C. There is also a nonlinear term P, P = (C × C + midVal) >> bitDepth. Here, midVal means the median, and bitDepth is the bit width. For the 10-bit case, P = (C × C + 512) >> 10. There is also an offset term B, which is set to the median, which is 512 for the 10-bit case. The predicted value of the chroma pixel is predChromaVal = C0C + C1N + C2S + C3E + C4W + C5P + C6B.

[0211] In order to determine C0, C1, C2, C3, C4, C5, and C6, CCCM will apply the same model to predict the chrominance in the reference area as shown in Figure 25 and compare it with the reconstructed chrominance value to solve C0, C1, C2, C3, C4, C5, and C6 that minimizes the mean square error.

[0212] (4) Multi-model convolutional cross-component model MM-CCCM.

[0213] Similar to MM-CCLM, multiple models are applied to the current block. ECM-10 uses two models. The derivation method also uses a threshold to separate the reconstructed luma pixels and the luma pixels of the current block into two groups. CCCM models are derived for each group, and the predictions are combined into a single prediction block.

[0214] (5) Gradient linear model GLM.

[0215] GLM is a method for predicting chromaticity based on the gradient of brightness. There are two GLM modes, one is 2-parameter GLM and the other is 3-parameter GLM.

[0216] The formula for a 2-parameter GLM is pred C (i,j)=α·grad L (i,j)+β;

[0217] The formula for the 3-parameter GLM is pred C (i,j)=α0·grad L (i,j)+α1·rec L ′(i,j)+β.

[0218] Here, the 2-parameter GLM replaces the reconstructed luminance pixel values ​​from the downsampled CCLM with luminance gradients. The 3-parameter GLM adds a luminance gradient to the CCLM. There are four ways to calculate the luminance gradient, as shown in Figure 26. The filter squares correspond to luminance pixels, and the circles correspond to chrominance pixels. This is a set of filters designed for the YUV 4:2:0 format.

[0219] (6) CCCM between frames.

[0220] Intra-frame prediction and inter-frame prediction are different. Intra-frame prediction uses reconstructed pixels around the current block to predict the interior of the current block, while inter-frame prediction uses images of the same object at different times. The most basic principle of inter-frame prediction is translational motion: using a motion vector to find a reference block and use the reference block as the prediction block. Since the same object is found at different times, the reference image contains the same number of components as the current image. For YUV sequences, all three YUV components can be predicted from the reference block. Cross-component models such as CCLM and CCCM follow a similar approach to intra-frame prediction, using surrounding reconstructed pixels to derive the model. However, inter-CCCM uses the original inter-frame prediction block to derive the cross-component model (filter coefficients), as shown in Figure 27. resY, resCb, and resCr represent the luma residual, blue chroma residual, and red chroma residual, respectively. Adding the corresponding predicted values ​​yields the reconstructed luma value Y, blue chroma reconstructed value Cb, and red chroma reconstructed value Cr. In other words, after luma reconstruction, the derived model (filter coefficients) is applied to the luma reconstruction value to obtain a second chroma prediction value. The original prediction value and the second prediction value obtained by inter-CCCM are weighted to form the new prediction value. Inter-CCCM transforms the reconstruction value containing the luma residual into a chroma prediction through the model, thereby improving compression efficiency.

[0221] Furthermore, the following introduces the transformation technology.

[0222] During encoding, commonly used hybrid coding frameworks first perform a prediction. This prediction leverages spatial or temporal correlation to produce an image identical or similar to the current block. While it's possible for the predicted block to be identical to the current block for a given block, it's difficult to guarantee this for all blocks in a video, especially in natural video or video captured by a camera. Irregular motion, distortion, occlusion, and brightness changes in video are difficult to fully predict. Therefore, hybrid coding frameworks subtract the predicted image from the original image of the current block to produce a residual image, or, in other words, subtract the predicted block from the current block to produce a residual block. This residual block is typically much simpler than the original image, so prediction can significantly improve compression efficiency. The residual block isn't encoded directly; instead, it's usually first transformed. This transform converts the residual image from the spatial domain to the frequency domain to remove correlation. After the residual image is transformed to the frequency domain, since most of the energy is concentrated in the low-frequency region, the non-zero coefficients are concentrated in the upper left corner. Quantization is then used to further compress the image. Furthermore, since the human eye is less sensitive to high frequencies, a larger quantization step size can be used in high-frequency regions.

[0223] Figure 28 is a schematic diagram of a DCT transform. As shown in Figure 28, after the DCT transform of the original image, only the upper left corner has non-zero coefficients. Of course, this example performs a DCT transform on the entire image. In video codecs, images are processed by dividing them into blocks, so the transform is also performed on a block-by-block basis.

[0224] Transforms are very useful in typical video compression, but not all blocks require transforms. In some cases, transforms can even yield worse compression results than non-transforms. Therefore, in some standards, such as VVC, the encoder can choose whether to use transforms for the current block. DCT-II is the most commonly used transform in video compression standards, and its base image is shown in Figure 29.

[0225] In addition, VVC can also use DCT8 type (DCT-VIII) and DST7 type (DST-VII). The basic formulas of these transforms are shown in Table 3, which shows the basic transform formulas of DCT2, DCT8 and DST7 for N-point input.

[0226] Table 3

[0227] Because images are all two-dimensional, the computational complexity and memory overhead of performing a direct two-dimensional transform were prohibitive for the hardware available at the time. Therefore, the DCT2, DCT8, and DST7 transforms used in the standards were split into two steps: horizontal and vertical one-dimensional transforms. For example, the horizontal transform was performed first, followed by the vertical transform, or the vertical transform was performed first, followed by the horizontal transform.

[0228] (1) Multi-transformation selection MTS.

[0229] VVC supports transform kernels such as DCT2, DCT8, and DST7. For a block, the encoder selects the appropriate transform kernel and transmits the index to the bitstream. The decoder then uses the index to determine the transform kernel for the inverse transform. Different transform kernels can be selected for the horizontal and vertical directions, such as using DCT8 horizontally and DST7 vertically. This technique is generally referred to as MTS.

[0230] (2) Low-frequency non-separable transform LFNST.

[0231] The above transformation method is effective for horizontal and vertical textures, but less so for diagonal textures. Indeed, horizontal and vertical textures are the most common, making the above transformation method very useful for improving compression efficiency. As the demand for compression efficiency continues to increase, more efficient processing of diagonal textures could further improve compression efficiency.

[0232] In order to more effectively process the residual of oblique texture, LFNST transform is used in VVC. The above transforms such as DCT2, DCT8, and DST7 are called primary transforms. At the encoding end of VVC, LFNST is used after DCT2 transform and before quantization. At the decoding end of VVC, LFNST is used after inverse quantization and before inverse DCT2 transform. Because it is transformed on the basis of DCT2 (basic transform), LFNST is a secondary transform. Figure 30 is a schematic diagram of the encoding and decoding process without LFNST (secondary transform), and Figure 31 is a schematic diagram of the encoding and decoding process with LFNST (secondary transform). Of course, the encoding end can directly inverse quantize the saved quantization coefficients instead of entropy decoding, because entropy coding is lossless.

[0233] Figure 32 is a detailed schematic diagram of the encoding and decoding process with LFNST (secondary transform). As shown in Figure 32, at the encoding end, the first step is the forward basic transform, and then the LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the basic transform. For example, there are 16 input coefficients for 4×4 LFNST and 64 input coefficients for 8×8 LFNST; the coefficients after the LFNST transform are then quantized and the quantized coefficients are written into the bitstream. At the decoding end, the transform coefficients can be obtained by decoding the bitstream and inverse quantization. Then, in the inverse LFNST transform, there are 8 input coefficients for 4×4 inverse LFNST and 16 input coefficients for 8×8 inverse LFNST. Finally, the inverse basic transform can obtain the residual block.

[0234] That is, at the encoder, LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the base transform. The base transform decorrelates the image, concentrating the energy in the upper left corner. The secondary transform further decorrelates the low-frequency coefficients of the base transform. The result is intuitively shown in Figure 32. At the encoder, 16 coefficients are input to a 4×4 LFNST, and the output is 8 coefficients; 64 coefficients are input to an 8×8 LFNST, and the output is 16 coefficients. At the decoder, 8 coefficients are input to a 4×4 inverse LFNST, and the output is 16 coefficients; 16 coefficients are input to an 8×8 inverse LFNST, and the output is 64 coefficients.

[0235] Figure 33 shows some base images for the LFNST algorithm in VVC. Figure 33 shows some obvious diagonal textures. In addition to kernels optimized for certain diagonal textures, the LFNST algorithm also has kernels optimized for flat, gradient textures, such as kernel group 0 in the LFNST algorithm in VVC.

[0236] LFNST is only applied to intra-coded blocks. Angular prediction tiles the reference pixels at a specified angle onto the current block as the prediction value. This means the predicted block will have a distinct directional texture, and the residual of the current block after angular prediction will also statistically exhibit significant angular characteristics. Therefore, the transform kernel selected by LFNST can be tied to the intra-prediction mode. That is, once the intra-prediction mode is determined, LFNST can only use the set of transform kernels corresponding to that intra-prediction mode.

[0237] Specifically, the LFNST in VVC has a total of 4 groups of transform kernels, and each group can select 2 transform kernels. Table 4 shows the correspondence between intra prediction modes and transform kernel groups. Note that the cross-component prediction modes used for chroma intra prediction are 81 to 83, and there are no such modes for luma intra prediction. The transform kernel of LFNST can be transposed to process more angles with one transform kernel group. For example, modes 13 to 23 and 45 to 55 both correspond to transform kernel group 2, but 13 to 23 is obviously close to the horizontal mode and 45 to 55 is obviously close to the vertical mode.

[0238] Table 4

[0239] VVC's LFNST uses four sets of transform kernels, with the intra-prediction mode specifying which set to use. This leverages the correlation between the intra-prediction mode and the LFNST transform kernel, reducing the transmission of the selected LFNST transform kernel in the bitstream. Whether the current block uses LFNST, and if so, whether to use the first or second set within a set, is determined by the bitstream and certain conditions.

[0240] In the subsequent evolution of ECM technology, LFNST was further expanded. LFNST has more transform kernel groups, 35 in ECM. The correspondence between the transform kernel group index (LFNST set index) and the intra prediction mode (Intra pred.mode) is shown in Table 5. Each transform kernel group is more efficient for textures at the corresponding angle. Here, each transform kernel group can select three transform kernels.

[0241] Table 5

[0242] (3) Non-separable basis transformation NSPT.

[0243] LFNST is a horizontally and vertically inseparable transform. Because it involves a secondary transform, DCT2 can be called the base transform. This approach of performing DCT2 before LFNST is a compromise between performance and complexity. While directly performing the inseparable base transform is more efficient, it also incurs higher complexity, such as increased computational effort and storage space required for the transform kernel.

[0244] In ECM10, some small blocks can use NSPT, while large blocks still use DCT2+LFNST. The sizes of small blocks are 4×4, 4×8, 8×4, 8×8, 4×16, 16×4, 8×16, 16×8, 8×32, and 32×8. In ECM10, NSPT also matches the transform kernel group according to the intra prediction mode. The matching method can refer to the LFNST method. Each transform kernel group has 3 transform kernels to choose from. For example, an 8×8 base image of an NSPT in ECM10 is shown in Figure 34. Figure 34 corresponds to inter-frame angle prediction mode 7. It can be seen that it handles the texture of the corresponding angle better. It should be noted that NSPT is only applied to intra-coded blocks.

[0245] Furthermore, for the partitioning technology, it can be divided into single tree partitioning and dual tree partitioning. In the Single tree, luminance and chrominance are divided together, and a CU contains a luminance block and a chrominance block at the same position. In the Dual tree, luminance and chrominance are divided separately, and a CU has only luminance blocks or only chrominance blocks. HEVC only has Single tree. Dual tree is introduced in VVC, but the P and B slices in VVC can still only use Single tree. The I slice in VVC can use Dual tree. As shown in Figure 35, luminance and chrominance are divided separately, and it can be seen that luminance has a finer division than chrominance, because luminance has more details than chrominance, and generally luminance requires better quality than chrominance.

[0246] In summary, LFNST and NSPT can be applied to intra-coded blocks, or to inter-coded blocks. Although for the transform, the residual block and transform coefficients are provided here, and only the transform and inverse transform are required. But in fact, the current block may use LFNST / NSPT or may not use LFNST / NSPT, and there are multiple transform cores to choose from in a transform core group of LFNST / NSPT. In other words, the encoder can choose to use or not use LFNST / NSPT, and specifically which transform core of LFNST / NSPT to use. If LFNST / NSPT is not used, other transform methods can be selected, such as selecting horizontal and vertical basic transforms from DCT2, DCT8, and DST7.

[0247] Furthermore, the residuals generated by intra-frame prediction and inter-frame prediction are statistically significantly different. Intra-frame prediction predicts the current block based on the reconstructed image surrounding the current block. If the original image of the current block undergoes changes that the surrounding reconstructed images do not, intra-frame prediction will struggle to produce a good prediction, resulting in significant residuals. Inter-frame prediction, on the other hand, uses images of the same object at different times to predict the current block. If the object in the current block does not undergo significant changes or occlusion between the reference and current images, inter-frame prediction can produce a good prediction, typically resulting in fewer residuals. Of course, inter-frame prediction can also produce poor predictions, such as when the object undergoes significant changes or occlusions. Fundamentally, this means that good content cannot be found in other coded images, resulting in significant residuals. Simply put, because of the fundamental differences between intra-frame and inter-frame prediction, the statistical patterns of their residuals differ. This is why, while LFNST / NSPT can be used for both intra-frame and inter-frame prediction, there are still differences.

[0248] An embodiment of the present application provides an encoding method, which determines a prediction mode of a current block; determines at least one context model based on the prediction mode of the current block; determines a value of first syntax identification information, and binarizes the value of the first syntax identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; encodes at least one binary symbol in the binary symbol string according to at least one context model, and writes the obtained coded bits into a bitstream. An embodiment of the present application also provides a decoding method, which determines a prediction mode of a current block; determines at least one context model based on the prediction mode of the current block; decodes the bitstream according to at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; debinarizes the binary symbol string to determine a value of the first syntax identification information; when the first syntax identification information indicates that the current block uses a first transform mode, determines a transform kernel of the current block, and de-transforms the transform coefficients of the current block according to the transform kernel to determine a residual block of the current block.

[0249] In this way, both the encoding and decoding ends first determine at least one context model based on the prediction mode of the current block, and then perform encoding and decoding on the binary symbol string corresponding to the first syntax identification information based on the at least one determined context model. In other words, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, so that the binary symbol string of LFNST / NSPT in the inter-frame prediction mode can be encoded and decoded using an encoding and decoding method that conforms to its distribution law. This not only improves compression efficiency, that is, video encoding and decoding efficiency, but also improves encoding and decoding performance.

[0250] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0251] FIG36 is a schematic diagram of a network architecture for video encoding and decoding provided in an embodiment of the present application. As shown in FIG36 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein electronic devices 13 to 1N can perform video interaction via the communication network 01. During implementation, the electronic devices can be various types of devices with video encoding and decoding capabilities. For example, the electronic devices can include mobile phones, tablet computers, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensor devices, servers, etc., and the embodiments of the present application do not limit this.

[0252] In an embodiment of the present application, a network architecture of a video encoding and decoding system including a decoding method and an encoding method is provided. The decoder or encoder in the embodiment of the present application can be the aforementioned electronic device. In other words, the electronic device in the embodiment of the present application has video encoding and decoding capabilities, and generally includes a video encoder (i.e., encoder) and a video decoder (i.e., decoder).

[0253] FIG37 is a block diagram of the system composition of an encoder provided in an embodiment of the present application. As shown in FIG37 , the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a series of pictures or a single still picture, and the output of the encoder 100 may be a bitstream (also referred to as a "codestream") used to represent a compressed version of the input video.

[0254] Among them, the segmentation unit 101 segments the picture in the input video into one or more Coding Tree Units (CTUs). The segmentation unit 101 divides the picture into multiple tiles (or tiles), and can further divide a tile into one or more bricks. Here, a tile or a brick may include one or more complete and / or partial CTUs. In addition, the segmentation unit 101 can form one or more slices, where a slice can include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture. The segmentation unit 101 can also form one or more sub-pictures, where a sub-picture can include one or more slices, tiles or bricks.

[0255] During the encoding process of encoder 100, segmentation unit 101 transmits the CTU to prediction unit 102. Generally, prediction unit 102 may be composed of block segmentation unit 103, motion estimation (ME) unit 104, motion compensation (MC) unit 105, and intra prediction unit 106. Specifically, block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (CUs). Prediction unit 102 may use ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks for the CU. Intra-frame prediction unit 106 may use various intra-frame prediction modes, including MIP mode, to obtain intra-frame prediction blocks for the CU. In an example, a rate-distortion optimized motion estimation method may be used by ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks, and a rate-distortion optimized mode determination method may be used by intra-frame prediction unit 106 to obtain intra-frame prediction blocks.

[0256] The prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU in the output of the segmentation unit 101 and the prediction block of the CU, i.e., the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels). The inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstructed residual. The second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference. After all CUs in the picture or sub-picture are reconstructed, the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping and chroma scaling (LMCS) filter, and a neural network-based filter. Alternatively, when the filtering unit 113 determines that a CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.

[0257] The output of the filtering unit 113 is a decoded picture or sub-picture, which is cached to the DPB unit 114. The DPB unit 114 outputs the decoded picture or sub-picture based on the timing and control information. Here, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, the entropy coding unit 115 converts the parameters required for decoding the picture from the encoder 100 (such as control parameters and supplementary information, etc.) into binary form and writes this binary form into the code stream according to the syntax structure of each data unit. That is, the encoder 100 finally outputs the code stream.

[0258] Furthermore, the encoder 100 can be a device having a first processor and a first memory for recording a computer program. When the first processor reads and executes the computer program, the encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, the encoder 100 can be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in FIG37 .

[0259] Figure 38 is a block diagram of the system components of a decoder provided in an embodiment of the present application. As shown in Figure 38, the decoder 200 may include: a parsing unit 201, a prediction unit 202, an inverse quantization unit 205, an inverse transform unit 206, an adder 207, a filtering unit 208, and a decoded image buffer unit 209. Here, the input of the decoder 200 is a bitstream representing a compressed version of a video or a still image, and the output of the decoder 200 may be a decoded video consisting of a series of images or a decoded still image.

[0260] The input codestream to decoder 200 may be the codestream generated by encoder 100. Parsing unit 201 parses the input codestream and obtains syntax element values ​​from the input codestream. Parsing unit 201 converts the binary representation of the syntax elements into digital values ​​and sends the digital values ​​to units within decoder 200 to obtain one or more decoded pictures. Parsing unit 201 may also parse one or more syntax elements from the input codestream to display decoded pictures.

[0261] During the decoding process of the decoder 200 , the parsing unit 201 sends the values ​​of the syntax elements and one or more variables set or determined according to the values ​​of the syntax elements and used to obtain one or more decoded pictures to the units in the decoder 200 .

[0262] The prediction unit 202 determines a prediction block for the current decoding block (e.g., CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra-frame prediction unit 204. Specifically, when the inter-frame decoding mode is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the motion compensation unit 203 to obtain an inter-frame prediction block; when the intra-frame prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the intra-frame prediction unit 204 to obtain an intra-frame prediction block.

[0263] The inverse quantization unit 205 has the same functionality as the inverse quantization unit 110 in the encoder 100. The inverse quantization unit 205 performs a scaling operation on the quantized coefficients (i.e., levels) from the parsing unit 201 to obtain reconstructed coefficients. The inverse transform unit 206 has the same functionality as the inverse transform unit 111 in the encoder 100. The inverse transform unit 206 performs one or more transform operations (i.e., the inverse of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual. The adder 207 adds its inputs (the prediction block from the prediction unit 202 and the reconstructed residual from the inverse transform unit 206) to obtain a reconstructed block for the currently decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in intra-frame prediction mode.

[0264] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chroma scaling filter, and a neural network-based filter. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of the filtering unit 208 is a decoded picture or sub-picture, which is cached to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture based on timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0265] Furthermore, the decoder 200 can be a second memory having a second processor and a computer program. When the first processor reads and runs the computer program, the decoder 200 reads the input code stream and generates the corresponding decoded video. In addition, the decoder 200 can also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have connections and data exchange functions similar to those of the corresponding units in Figure 38.

[0266] It should also be noted that when the embodiment of the present application is applied to the encoder 100, the "current block" specifically refers to the current block to be encoded in the video image (which can also be simply referred to as the "encoding block"); when the embodiment of the present application is applied to the decoder 200, the "current block" specifically refers to the current block to be decoded in the video image (which can also be simply referred to as the "decoding block").

[0267] In one embodiment of the present application, referring to FIG39 , a flowchart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG39 , the method may include:

[0268] S3901: Determine the prediction mode of the current block.

[0269] It should be noted that in the embodiments of the present application, the method is applied to a decoder. Specifically, based on the structure of the decoder 200 shown in FIG38 , the decoding method of the embodiments of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, the optimization scheme proposed is mainly for the NSPT and LFNST transforms in the inter-frame prediction mode to improve compression efficiency.

[0270] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0271] S3902: Determine at least one context model according to the prediction mode of the current block.

[0272] It should be noted that in the embodiment of the present application, it is determined that at least one context model corresponding to the inter prediction mode is different from at least one context model corresponding to the intra prediction mode. In other words, the inter prediction mode and the intra prediction mode do not share a common context model.

[0273] S3903: Decode the code stream according to at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol.

[0274] S3904: Debinarize the binary symbol string to determine the value of the first syntax identification information.

[0275] It should also be noted that in the embodiments of the present application, the number of context models is related to the number of binary symbols. If the binary symbol string of the current block includes n binary symbols, then n context models need to be determined based on the prediction mode of the current block. Each binary symbol uses a context model during decoding. Here, n is a positive integer.

[0276] In a possible implementation, for the intra prediction mode and the inter prediction mode, the binary symbol string may use the same debinarization method but different context models.

[0277] In some embodiments, decoding the code stream according to at least one context model to determine the binary symbol string of the current block may include: when the prediction mode of the current block is an intra-frame prediction mode, decoding the code stream according to at least one first context model to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is an inter-frame prediction mode, decoding the code stream according to at least one second context model to determine the second binary symbol string of the current block.

[0278] In the embodiment of the present application, at least one first context model is different from at least one second context model, that is, the inter-frame prediction mode and the intra-frame prediction mode do not share a common context model.

[0279] In some embodiments, performing debinarization processing on the binary symbol string to determine the value of the first syntax identification information may include: when the prediction mode of the current block is the intra-frame prediction mode and / or the inter-frame prediction mode, performing debinarization processing on the binary symbol string based on the first method to determine the value of the first syntax identification information.

[0280] In the embodiment of the present application, both the inter-frame prediction mode and the intra-frame prediction mode can be decoded using the first method. The first method can be a fixed-length decoding method, also known as a fixed-length (FL) decoding method, that is, using the same number of binary symbols to represent different values ​​of the first syntax identification information; or the first method can be a truncated unary code method, that is, using different numbers of binary symbols to represent different values ​​of the first syntax identification information; or the first method can even be other methods, such as a unary code method, a K-order exponential Golomb decoding method, etc., which are not specifically limited here.

[0281] In an embodiment of the present application, the first syntax identification information can be represented by lfnst_nspt_idx. For example, the value of lfnst_nspt_idx can be 0, 1, 2, or 3. If the first method is a fixed-length decoding method, the binarization table shown in Table 6 can be used. In Table 6, the values ​​of the first syntax identification information all use two binary symbols, and these two binary symbols can both be decoded using a context model. Among them, the first binary symbol (binIdx is equal to 0) uses one context model, and the second binary symbol (binIdx is equal to 1) uses one context model.

[0282] Table 6

[0283] If the first mode is the truncated unary code mode, the binarization table shown in Table 7 can be used. In Table 7, different binary symbol string lengths can be used for the values of the first syntax identification information. Among them, the 3 binary symbols in Table 7 can all be decoded by the context model. Among them, the first (binIdx equals 0) binary symbol uses one context model, the second (binIdx equals 1) binary symbol uses one context model, and the third (binIdx equals 2) binary symbol uses one context model.

[0284] Table 7

[0285] It should be noted that in the embodiments of the present application, the decoding rule of the unary code is that for the binary symbol "x" >= 0 to be decoded, the decoded binary symbol string can be composed of x "1"s plus one "0". For example, if x = 5, then after decoding according to the unary code mode, the binary symbol string is "111110".

[0286] It should also be noted that in the embodiments of the present application, the truncated unary code belongs to a variant of the unary code and is used in the case where the maximum value Max of the syntax element to be decoded is known. Assume that the symbol to be decoded is x. If 0 < x < Max, then the binary symbol string corresponding to x adopts the unary code mode; if x = Max, then the binary symbol string corresponding to x is all composed of 1s and the length is Max. For example, assume Max = 6. If x = 6, then after decoding according to the truncated unary code mode, the binary symbol string is "111111"; if x = 3, then after decoding according to the truncated unary code mode, the binary symbol string is "1110".

[0287] It can be understood that in the embodiments of the present application, both NSPT and LFNST are transforms for processing textures of various angles. There may be multiple transform kernels here, and one transform kernel may be optimized for a specific angle texture. In addition to the angle textures, NSPT and LFNST also include transform kernels for processing gradient textures. In fact, these transform kernels can also be said to be trained Karhunen-Loeve Transforms (KLTs). That is to say, both NSPT and LFNST can have multiple transform kernels, and each transform kernel is designed for a specific texture. Among them, specific textures include angle textures, gradient textures, etc. In addition, gradient textures can be further extended such as horizontal gradient textures, vertical gradient textures, oblique gradient textures, etc. Further, the technical solution of the present application is not limited to inseparable transforms such as NSPT and LFNST, and the technical solution is also applicable to separable transforms optimized for specific textures.

[0288] It should also be noted that in the embodiments of this application, both NSPT and LFNST include multiple transform kernel groups, each of which includes three selectable transform kernels. Including the case where LFNST / NSPT is not used, this means that a block has four possible choices for LFNST / NSPT. In this embodiment, the syntax element lfnst_idx can be used to indicate whether LFNST is used and which LFNST transform kernel is used. If lfnst_idx is 0, LFNST is not used for the current block; if lfnst_idx is 1, the first LFNST transform kernel is used for the current block; and if lfnst_idx is 2, the second LFNST transform kernel is used for the current block. It should be noted that the transform kernel group can be determined by the intra-frame prediction mode or texture feature index, in which case no syntax element indication is required. Furthermore, for a transform kernel group, some technologies may only select two transform kernels, while others may select three. Furthermore, for small blocks of a certain size, NSPT may be used, or three transform kernels may be used. In the embodiment of the present application, lfnst_nspt_idx can be used to indicate whether LFNST / NSPT is used and which LFNST / NSPT transform core is used. This is because in one possible implementation, it indicates the transform core index of NSPT for smaller blocks and the transform core index of LFNST for larger blocks. In other words, the value of lfnst_nspt_idx can be 0, 1, 2, or 3. Of course, the embodiment of the present application can continue to use the syntax element lfnst_idx, or can also use two syntax elements, lfnst_idx and nspt_idx, where lfnst_idx is used to indicate whether LFNST is used and which LFNST transform core is used, and nspt_idx is used to indicate whether NSPT is used and which NSPT transform core is used; the two are essentially the same and are not specifically limited here.

[0289] For example, for the value of lfnst_nspt_idx in intra prediction mode, its binarization method can be as shown in Table 6 above. It can be seen that the binarization method of lfnst_nspt_idx in intra prediction mode uses two binary symbols, and these two binary symbols are both decoded by context models. The first binary symbol (binIdx is equal to 0) uses one context model, and the second binary symbol (binIdx is equal to 1) uses another context model.

[0290] It should be noted that the binarization method and context model configuration set in this way are related to its statistical laws. As shown in Figure 40, a schematic diagram of the statistical results of lfnst_nspt_idx corresponding to the intra prediction mode in some technologies is shown. Specifically, Figure 40 is a statistical result of the value of intra lfnst_nspt_idx in the test sequence MarketPlace under all intra QP 32. According to Figure 40, it can be seen that under this condition, for the value of lfnst_nspt_idx, the possibilities of the four choices are similar, and the possibilities of 1 and 2 are even more than 0. It should be noted that this is only an example, and the statistical results corresponding to different test sequences and different QPs are different.

[0291] Furthermore, using the same binarization method, the lfnst_nspt_idx statistics for inter-frame prediction modes across several technologies are shown in Figure 41. Specifically, Figure 41 shows the inter-frame lfnst_nspt_idx statistics for the test sequence BasketballDrive at a random access QP of 32. Figure 41 shows that under this condition, the likelihood of the four possible lfnst_nspt_idx values ​​varies considerably. 0 is significantly higher than 1, 2, and 3, and 1 is significantly higher than 2 and 3. Of course, it should be noted that this is just an example; the statistical results will vary for different test sequences and different QPs.

[0292] Based on the theoretical analysis and statistical data of the probability distribution of inter-frame lfnst_nspt_idx, the embodiment of the present application proposes a binarization method for inter-frame lfnst_nspt_idx, as shown in Table 7 above. According to Table 7, it can be seen that the binarization method of inter-frame lfnst_nspt_idx uses different binary symbol string lengths, which is a truncated unary code method. Here, the three binary symbols can all be decoded using context models. The first (binIdx equals 0) binary symbol uses a context model, the second (binIdx equals 1) binary symbol uses a context model, and the third (binIdx equals 2) binary symbol uses a context model.

[0293] That is, in another possible implementation, for the intra-frame prediction mode and the inter-frame prediction mode, the binary symbol string may use different debinarization methods and different context models.

[0294] In some embodiments, decoding the code stream according to at least one context model to determine the binary symbol string of the current block may include: when the prediction mode of the current block is an intra-frame prediction mode, decoding the code stream according to at least one first context model and a first method to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is an inter-frame prediction mode, decoding the code stream according to at least one second context model and a second method to determine the second binary symbol string of the current block.

[0295] In some embodiments, debinarizing the binary symbol string to determine the value of the first syntax identification information may include: when the prediction mode of the current block is the intra-frame prediction mode, debinarizing the binary symbol string based on the first method to determine the value of the first syntax identification information; or, when the prediction mode of the current block is the inter-frame prediction mode, debinarizing the binary symbol string based on the second method to determine the value of the first syntax identification information.

[0296] In an embodiment of the present application, at least one first context model is different from at least one second context model, i.e., the inter-frame prediction mode and the intra-frame prediction mode do not share a common context model. In addition, the first mode is different from the second mode, i.e., the inter-frame prediction mode and the intra-frame prediction mode use different binarization methods.

[0297] In a specific embodiment, the first method may be a fixed-length decoding method, and the second method may be a truncated unary code method. Thus, when the prediction mode of the current block is an intra-frame prediction mode, the bitstream is decoded according to at least one first context model and the fixed-length decoding method to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is an inter-frame prediction mode, the bitstream is decoded according to at least one second context model and the truncated unary code method to determine the second binary symbol string of the current block. In this implementation, for different values ​​of the first syntax identifier lfnst_nspt_idx, the length of the first binary symbol string remains unchanged, while the length of the second binary symbol string varies.

[0298] In short, for the transform kernel index (i.e., the value of lfnst_nspt_idx) of the intra prediction mode and the inter prediction mode, different decoding methods can be used here, and different context models can also be used. Specifically, the context model used by lfnst_nspt_idx in the inter prediction mode is different from the context model used by lfnst_nspt_idx in the intra prediction mode, or in other words, lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode do not share the same context model. Among them, one possible implementation method is that lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode use the same binarization method, but different context models. Another possible implementation method is that lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode use different binarization methods and different context models.

[0299] It should also be noted that in this embodiment of the present application, LFNST / NSPT can share a syntax element lfnst_nspt_idx, specifically indicating the transform core index of the NSPT for smaller blocks and the transform core index of the LFNST for larger blocks. Alternatively, LFNST / NSPT may each use different syntax elements, for example, LFNST uses the syntax element lfnst_idx and NSPT uses the syntax element nspt_idx. However, the above method still applies to the binarization of lfnst_idx and nspt_idx.

[0300] S3905: When the first syntax identification information indicates that the current block uses the first transform mode, determine a transform kernel of the current block, and perform inverse transform on the transform coefficients of the current block according to the transform kernel to determine a residual block of the current block.

[0301] It should be noted that, in the embodiment of the present application, determining the transformation core of the current block may include: determining the transformation core group of the current block; and determining the transformation core of the current block according to the values ​​of the transformation core group and the first syntax identification information.

[0302] Here, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group. In this way, the transform core of the current block can be determined based on the transform core group and the value of the first syntax identification information.

[0303] It should also be noted that, in the embodiment of the present application, when intra-frame prediction is performed on the current block, the transform core group of the current block can be determined based on the correspondence between the intra-frame prediction mode and the transform core group. When inter-frame prediction is performed on the current block, the transform core group of the current block can be determined based on the correspondence between the texture feature index (i.e., the virtual intra-frame prediction mode) and the transform core group. The texture feature index of the current block can be derived using the prediction block or reference block of the current block or the reconstructed area around the current block.

[0304] In some embodiments, determining the transform core group for the current block may include: when the prediction mode of the current block is an intra-frame prediction mode, determining that the transform core group includes M candidate transform cores; or when the prediction mode of the current block is an inter-frame prediction mode, determining that the transform core group includes N candidate transform cores, where M and N are both positive integers, and M is greater than or equal to N.

[0305] That is, in embodiments of the present application, LFNST / NSPT may have more or fewer options, and the number of transform cores available for LFNST / NSPT for inter-frame prediction may be different from the number of transform cores available for LFNST / NSPT for intra-frame prediction. For example, because intra-frame prediction has more residuals and inter-frame prediction has fewer residuals, a transform core group for intra-frame prediction has three transform cores, while a transform core group for inter-frame prediction has two transform cores. In this case, the value of lfnst_nspt_idx for inter-frame prediction can be 0, 1, or 2.

[0306] For example, Table 8 provides another binarization method of lfnst_nspt_idx for inter-frame prediction, as shown below.

[0307] Table 8

[0308] As can be seen from Table 8, the binarization method of lfnst_nspt_idx for inter-frame prediction can use different binary symbol string lengths, which is a truncated unary code method. These two binary symbols can both be decoded using context models. The first binary symbol (binIdx equals 0) uses one context model, and the second binary symbol (binIdx equals 1) uses another context model.

[0309] It should also be noted that the context model used by lfnst_nspt_idx for inter prediction is different from the context model used by lfnst_nspt_idx for intra prediction, or in other words, lfnst_nspt_idx for inter prediction and lfnst_nspt_idx for intra prediction do not share the same context model. In addition, LFNST / NSPT may also use different syntax elements, and the above method still applies.

[0310] In some embodiments, the method further includes: when M is greater than N, setting the N candidate transformation cores to a subset of the M candidate transformation cores; or, when M is equal to N, setting the N candidate transformation cores to be at least partially different from the M candidate transformation cores.

[0311] It should be noted that, in this embodiment of the present application, the transform cores available for inter-frame LFNST / NSPT can be the same as the transform cores available for intra-frame LFNST / NSPT. If the number of transform cores available for inter-frame LFNST / NSPT is less than the number of transform cores available for intra-frame LFNST / NSPT, then the transform cores available for inter-frame LFNST / NSPT can be a subset of the transform cores available for intra-frame LFNST / NSPT. For example, inter-frame prediction can use two transform cores, and intra-frame prediction can use three transform cores. The transform cores available for inter-frame prediction can be the first two, the last two, or the first or third of the transform cores available for intra-frame prediction, without specific limitation here.

[0312] It should also be noted that in the embodiments of the present application, the transform kernels available for inter-frame LFNST / NSPT can be different from the transform kernels available for intra-frame LFNST / NSPT. That is, transform kernels are designed separately for the residual features of inter-frame prediction and intra-frame prediction, which can achieve higher compression efficiency. For example, three transform kernels can be used for inter-frame prediction, and three transform kernels can also be used for intra-frame prediction, but the three transform kernels for inter-frame prediction and intra-frame prediction are partially or completely different.

[0313] That is, in the embodiment of the present application, the transform core used for inter prediction and the transform core used for intra prediction can be the same, or can be partially or completely different. In addition, in the embodiment of the present application, the available transform core refers to the transform core available in a transform core group.

[0314] In some embodiments, when determining the transform kernel group of the current block, the method may further include: determining at least one texture feature of the current block; performing transform kernel training according to the at least one texture feature to determine the transform kernel group of the current block.

[0315] It should be noted that in related art, intra-frame prediction can correspond to a transform kernel group based on the intra-frame prediction mode. In the embodiment of the present application, inter-frame prediction can use the prediction block to derive a virtual intra-frame prediction mode (texture feature index) corresponding to a transform kernel group. Inter-frame prediction can also derive texture features different from those of intra-frame prediction, and correspondingly, such texture features are used to train the transform kernel.

[0316] For example, the current LFNST and NSPT training are relatively single texture features, such as texture in a specific direction, as shown in (a) and (b) in Figure 42. It should be noted that this is just a schematic diagram, but for inter-frame prediction, residuals appear more often at the edges of objects. For example, if residuals with two-directional textures as shown in (c) in Figure 42 often appear between frames, then such LFNST / NSPT transformation kernels can also be trained. When analyzing the texture features of the current block (such as the process of deriving the virtual intra-frame prediction mode mentioned above), if it is found that the prediction block of the current block or the surrounding reconstructed area has textures in two directions, then the transformation kernel with such texture features can be matched.

[0317] The decoding order of the syntax elements will be described in detail below in combination with the second identification information, the third syntax identification information and the fourth syntax identification information.

[0318] In one possible implementation, for the inter-frame prediction mode, referring to FIG. 43 , the method may include:

[0319] S4301: Decode the code stream and determine the value of the second syntax identification information of the current block.

[0320] S4302: When the second syntax identification information indicates that the current block does not use the multi-transform selection mode and the current block meets the use condition of the first transform mode, decode the code stream and determine the value of the first syntax identification information of the current block.

[0321] S4303: When the first syntax identification information indicates that the current block uses the first transform mode, determine a transform kernel of the current block, and perform inverse transform on the transform coefficients of the current block according to the transform kernel to determine a residual block of the current block.

[0322] It should be noted that, in the embodiment of the present application, both the first grammar identification information and the second grammar identification information can be decoded using a context model. Specifically, for step S4302, decoding the code stream and determining the value of the first grammar identification information of the current block may include: decoding the code stream according to at least one context model to determine the binary symbol string of the first grammar identification information; and performing debinarization processing on the binary symbol string of the first grammar identification information to determine the value of the first grammar identification information. For step S4301, decoding the code stream and determining the value of the second grammar identification information of the current block may include: decoding the code stream according to at least one context model to determine the binary symbol string of the second grammar identification information; and performing debinarization processing on the binary symbol string of the second grammar identification information to determine the value of the second grammar identification information. Here, each binary symbol in the binary symbol string uses a context model.

[0323] It should also be noted that in this embodiment of the present application, the first syntax identification information is represented by lfnst_nspt_idx, and the second syntax identification information is represented by mts_idx. lfnst_nspt_idx is used to indicate whether the current block uses the first transform mode (i.e., LFNST / NSPT), and mts_idx is used to indicate whether the current block uses the multiple transform selection mode (i.e., MTS). In other words, when decoding the bitstream, the syntax element lfnst_nspt_idx of LFNST / NSPT can be after the syntax element mts_idx of MTS.

[0324] In some embodiments, if the value of the first syntax identification information is a first value, it can be determined that the current block does not use LFNST / NSPT; if the value of the first syntax identification information is a second value, it can be determined that the current block uses LFNST / NSPT and the corresponding transform kernel used. The first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, 3, etc.

[0325] In some embodiments, if the value of the second syntax identification information is a first value, it can be determined that the current block does not use the multi-transform selection mode; if the value of the second syntax identification information is a second value, it can be determined that the current block uses the multi-transform selection mode and the corresponding transform kernel to be used. The first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, 3, etc.

[0326] It should also be noted that in this embodiment of the present application, if the second syntax identification information indicates that the current block uses the multi-transform selection mode, then there is no need to decode lfnst_nspt_idx. In this case, multiple transform kernels of the current block can be determined, and the transform coefficients of the current block can be inversely transformed according to the multiple transform kernels to determine the residual block of the current block. For example, mts_idx is used to indicate which (basic) transform kernel is used in the horizontal / vertical direction of the current block.

[0327] In another possible implementation, with respect to the intra prediction mode, referring to FIG. 44 , the method may include:

[0328] S4401: Decode the code stream and determine the value of the first syntax identification information of the current block.

[0329] S4402: When the first syntax identification information indicates that the current block does not use the first transform mode and the current block meets the use condition of the multiple transform selection mode, decode the code stream and determine the value of the second syntax identification information of the current block.

[0330] S4403: When the second syntax identification information indicates that the current block uses a multi-transform selection mode, determine multiple transform kernels of the current block, and perform inverse transform on the transform coefficients of the current block according to the multiple transform kernels to determine a residual block of the current block.

[0331] It should be noted that, in this embodiment of the present application, with respect to step S4401, decoding the bitstream and determining the value of the first syntax identification information of the current block may include: decoding the bitstream according to at least one context model to determine a binary symbol string of the first syntax identification information; and performing debinarization processing on the binary symbol string of the first syntax identification information to determine the value of the first syntax identification information. With respect to step S4402, decoding the bitstream and determining the value of the second syntax identification information of the current block may include: decoding the bitstream according to at least one context model to determine a binary symbol string of the second syntax identification information; and performing debinarization processing on the binary symbol string of the second syntax identification information to determine the value of the second syntax identification information. Here, each binary symbol in the binary symbol string uses a context model.

[0332] It should also be noted that in this embodiment of the present application, the first syntax identification information is represented by lfnst_nspt_idx, and the second syntax identification information is represented by mts_idx. lfnst_nspt_idx is used to indicate whether the current block uses the first transform mode (i.e., LFNST / NSPT), and mts_idx is used to indicate whether the current block uses the multiple transform selection mode (i.e., MTS). In other words, when decoding the bitstream, the syntax element lfnst_nspt_idx of LFNST / NSPT can precede the syntax element mts_idx of MTS.

[0333] It should also be noted that in an embodiment of the present application, if the first syntax identification information indicates that the current block uses the first transform mode, then there is no need to decode mts_idx. At this time, the transform kernel of LFNST / NSPT can be determined, and the transform coefficients of the current block can be inversely transformed according to the transform kernel to determine the residual block of the current block.

[0334] That is to say, in an embodiment of the present application, the decoding order of LFNST / NSPT and MTS for inter-frame prediction and intra-frame prediction is different. Specifically, when decoding the code stream, the syntax element lfnst_nspt_idx of LFNST / NSPT precedes the syntax element mts_idx of MTS. That is, lfnst_nspt_idx is parsed first. If the value of lfnst_nspt_idx is 0 and the current situation meets the application conditions of MTS, then continue to parse mts_idx. Here, mts_idx indicates which (basic) transform kernel is used in the horizontal and vertical directions of the current block respectively. In addition, in intra-frame prediction, the parsing of lfnst_nspt_idx can also be before mts_idx. In other words, whether mts_idx is parsed depends on the value of lfnst_nspt_idx.

[0335] In this embodiment of the present application, for inter-frame prediction of the current block, mts_idx can be parsed before lfnst_nspt_idx. In other words, whether lfnst_nspt_idx is parsed depends on the value of mts_idx. That is, mts_idx is parsed first. If mts_idx is 0 and the current situation meets the application conditions of LFNST / NSPT, then lfnst_nspt_idx is parsed.

[0336] It should be noted that the inter-frame or inter-frame prediction in the embodiments of the present application can be understood as an inter-frame coded block or coding unit or transformation unit, and the intra-frame or intra-frame prediction can be understood as an intra-frame coded block or coding unit or transformation unit.

[0337] It should also be noted that in the embodiment of the present application, two syntax identification information (third syntax identification information and fourth syntax identification information) can also be used to replace lfnst_nspt_idx. Among them, the third syntax identification information is represented by nspt_idx and can be used to indicate whether the current block uses NSPT and which NSPT transformation core is used. The fourth syntax identification information is represented by lfnst_idx and can be used to indicate whether the current block uses LFNST and which LFNST transformation core is used.

[0338] In some embodiments, the method further includes: decoding a code stream based on at least one third context model to determine a third binary symbol string of the current block; performing debinarization on the third binary symbol string to determine a value of third syntax identification information; and when the third syntax identification information indicates that the current block uses an inseparable basic transform mode, determining a transform kernel of the current block, and performing a deverse transform on the transform coefficients of the current block based on the transform kernel to determine a residual block of the current block.

[0339] In some embodiments, the method further includes: decoding a code stream based on at least one fourth context model to determine a fourth binary symbol string of the current block; performing debinarization on the fourth binary symbol string to determine a value of fourth syntax identification information; and when the fourth syntax identification information indicates that the current block uses a low-frequency inseparable transform mode, determining the transform kernel of the current block, and performing a deverse transform on the transform coefficients of the current block based on the transform kernel to determine a residual block of the current block.

[0340] That is, in the embodiments of the present application, LFNST / NSPT can each use different syntax elements, but the above method still applies. Specifically, intra prediction and inter prediction use different decoding methods for LFNST syntax elements and NSPT syntax elements, intra prediction and inter prediction use different context models for LFNST syntax elements and NSPT syntax elements, intra prediction and inter prediction can use different numbers of transform kernels for LFNST and NSPT, intra prediction and inter prediction can use different transform kernels for LFNST and NSPT, and intra prediction and inter prediction LFNST / NSPT and MTS are decoded in different orders.

[0341] It should also be noted that in the embodiments of the present application, the "inverse transformation" of the transformation coefficients by the decoding end may also be referred to as "transformation" in the standard text. The "transformation" and "inverse transformation" in this article correspond to two opposite processes. For example, if the "transformation" converts the numerical value of the spatial domain to the coefficient of the frequency domain, then the "inverse transformation" converts the coefficient of the frequency domain to the numerical value of the spatial domain. "Inverse" is relative to "positive", and they are essentially both transformations. It should be noted that if the standard only stipulates decoding, then the "transformation" in the standard text is the decoding part, specifically referring to the "inverse transformation" in this article.

[0342] It can also be understood that in the embodiment of the present application, in addition to applying LFNST and NSPT to intra-frame prediction / inter-frame prediction blocks, they can also be applied to intra-frame block copy (IBC) blocks, specifically by replacing inter-frame prediction with IBC.

[0343] In another possible implementation, lfnst_nspt_idx may be parsed only when the last non-zero coefficient position of the current block meets certain conditions. Referring to FIG45 , the method may include:

[0344] S4501: Decode the code stream and determine the quantization coefficient of the current block.

[0345] S4502: Dequantize the quantized coefficients of the current block to determine the transform coefficients of the current block.

[0346] S4503: When the position of the last non-zero coefficient in the transformation coefficients of the current block meets a preset condition, decode the code stream and determine the value of the first syntax identification information of the current block.

[0347] S4504: When the first syntax identification information indicates that the current block uses the first transform mode, determine a transform kernel of the current block, and perform inverse transform on the transform coefficients of the current block according to the transform kernel to determine a residual block of the current block.

[0348] It should be noted that, in the embodiment of the present application, for step S4503, decoding the code stream and determining the value of the first syntax identification information of the current block may include: decoding the code stream according to at least one context model to determine the binary symbol string of the first syntax identification information; and performing debinarization processing on the binary symbol string of the first syntax identification information to determine the value of the first syntax identification information.

[0349] It should also be noted that in this embodiment of the present application, when the first syntax identification information indicates that the current block does not use the first transform mode, a transform kernel of a second transform mode is determined for the current block, and the transform coefficients of the current block are inversely transformed based on the transform kernel of the second transform mode to determine the residual block of the current block. For example, the first transform mode may be LFNST / NSPT, and the second transform mode may be MTS, but this is not specifically limited.

[0350] In related technologies, lfnst_nspt_idx is parsed only when certain conditions are met. One of these conditions is that the last non-zero coefficient position is greater than or equal to 1. Quantized coefficients are scanned using a hierarchical diagonal scan. A coefficient block can be divided into 4×4 sub-blocks, which are scanned diagonally between each other, and within each 4×4 sub-block. The position (0, 0) at the upper left corner of the current block is 0 in the scan order, and the last non-zero coefficient position is greater than or equal to 1, which means that the current block has coefficients other than (0, 0). For DCT2 transform, (0, 0) is the DC coefficient. In other words, if the current block has only a non-zero coefficient at position (0, 0) or no non-zero coefficients, the syntax element lfnst_nspt_idx does not need to be parsed and defaults to 0. Because LFNST / NSPT is unlikely to be used in this case, this restriction significantly reduces unnecessary lfnst_nspt_idx overhead. Furthermore, related technologies require that the last non-zero coefficient position of all color components must be greater than or equal to 1. For example, in a single-tree block in YUV format, the last non-zero coefficient position of all three color components must be greater than or equal to 1.

[0351] Thus, for inter-frame predicted blocks, the residuals are inherently smaller than those for intra-frame predicted blocks. Furthermore, given that the quality requirements for chrominance components are generally much lower than those for luminance components, inter-frame predicted blocks often have certain luminance coefficients but no chrominance coefficients. Since this LFNST / NSPT condition is designed for intra-frame prediction, and LFNST / NSPT can currently also be used for inter-frame prediction, but their coefficients and residual patterns are different, the present invention can modify this restriction.

[0352] In one possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0353] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0354] It should be noted that, in the embodiment of the present application, the first threshold value may also be referred to as the minimum threshold value, wherein the minimum threshold value may be set to 1, or a larger value such as 2, 3, etc., which is not specifically limited here.

[0355] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the first threshold. In other words, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is greater than the first threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is greater than the first threshold.

[0356] For example, assuming the minimum threshold is 1, the relevant techniques are still used for intra-predicted blocks. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is greater than or equal to 1. For inter-predicted blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is greater than or equal to 1. For single-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required.

[0357] It's also understandable that in related art, lfnst_nspt_idx is only parsed if certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot exceed a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the current block size parameters, namely the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input is the 4x4 DCT2 transformed coefficients, and the output is 8 coefficients. This means that for a 4×4 block, the LFNST output can only have a maximum of 8 coefficients. Therefore, for the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), it is definitely not an LFNST output. This allows the decoder to know that the current block is definitely not using LFNST and therefore does not need to parse its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx. Furthermore, related art requires that the last non-zero coefficient position of any color component cannot exceed its maximum threshold. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three color components Y, U, and V cannot be greater than its maximum threshold.

[0358] In this way, considering that this condition of LFNST / NSPT is designed for intra-frame prediction, and LFNST / NSPT can also be used for inter-frame prediction, but their coefficients and residual rules are different, this restriction condition can be modified.

[0359] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0360] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0361] It should be noted that in the embodiments of the present application, the second threshold may also be referred to as a maximum threshold. In one specific embodiment, the second threshold may be determined based on the size parameters of the current block. For example, the second threshold may be determined based on the length and width of the current block. In another specific embodiment, the second threshold may be set to the difference between the maximum number of coefficients in the transform coefficients of the current block and one. For example, if the LFNST output of the current block may have a maximum of 8 coefficients, the second threshold may be set to 7.

[0362] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the second threshold. That is, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is less than the second threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is less than the second threshold.

[0363] For example, for intra-frame predicted blocks, the relevant technology is still used. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is not greater than its maximum threshold. For inter-frame predicted blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all its components is not greater than the maximum threshold. For single-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required.

[0364] That is, in the embodiment of the present application, lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is greater than or equal to the minimum threshold, and the U and V components are no longer required. Alternatively, lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the U and V components are no longer required.

[0365] Furthermore, in some embodiments, the method further includes: determining a prediction block of the current block; and determining a reconstructed block of the current block according to the prediction block of the current block and the residual block of the current block.

[0366] It should be noted that, in the embodiment of the present application, if the prediction mode of the current block is the intra-frame prediction mode, then intra-frame prediction is performed on the current block to determine the prediction block of the current block; then, an addition operation is performed on the prediction block of the current block and the residual block of the current block to determine the reconstructed block of the current block. It should also be noted that, in the embodiment of the present application, if the prediction mode of the current block is the inter-frame prediction mode, then inter-frame prediction is performed on the current block to determine the prediction block of the current block; then, an addition operation is performed on the prediction block of the current block and the residual block of the current block to determine the reconstructed block of the current block.

[0367] This embodiment provides a decoding method, which determines a prediction mode for a current block; determines at least one context model based on the prediction mode of the current block; decodes a bitstream based on the at least one context model to determine a binary symbol string for the current block; wherein the binary symbol string includes at least one binary symbol; debinarizes the binary symbol string to determine a value of first syntax identification information; and when the first syntax identification information indicates that the current block uses a first transform mode, determines a transform kernel for the current block, and inversely transforms the transform coefficients of the current block based on the transform kernel to determine a residual block for the current block. Thus, at least one context model is first determined based on the prediction mode of the current block, and then the binary symbol string corresponding to the first syntax identification information is decoded based on the determined at least one context model. In this way, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, so that a decoding method that conforms to the distribution law of the binary symbol string of LFNST / NSPT in the inter-frame prediction mode can be used, thereby improving not only compression efficiency but also encoding and decoding performance.

[0368] In one embodiment of the present application, referring to FIG46 , a schematic flow chart of an encoding method provided by an embodiment of the present application is shown. As shown in FIG46 , the method may include:

[0369] S4601: Determine the prediction mode of the current block.

[0370] It should be noted that in the embodiments of the present application, the method is applied to an encoder. Specifically, based on the structure of the encoder 100 shown in FIG37 , the encoding method of the embodiments of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, the optimization scheme proposed is mainly for the NSPT and LFNST transforms in the inter-frame prediction mode to improve compression efficiency.

[0371] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0372] S4602: Determine at least one context model based on the prediction mode of the current block.

[0373] It should be noted that in the embodiment of the present application, it is determined that at least one context model corresponding to the inter prediction mode is different from at least one context model corresponding to the intra prediction mode. In other words, the inter prediction mode and the intra prediction mode do not share a common context model.

[0374] S4603: Determine a value of the first syntax identification information, and perform binarization processing on the value of the first syntax identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol.

[0375] S4604: Encode at least one binary symbol in the binary symbol string according to the at least one context model, and write the obtained encoded bits into the code stream.

[0376] It should also be noted that in the embodiments of the present application, the number of context models is related to the number of binary symbols. If the binary symbol string of the current block includes n binary symbols, then n context models need to be determined based on the prediction mode of the current block. Each binary symbol uses a context model during encoding. Here, n is a positive integer.

[0377] It should also be noted that, in an embodiment of the present application, determining the value of the first grammar identification information may include: if the current block does not use the first transformation mode, determining the value of the first grammar identification information to be the first value; and / or, if the current block uses the first transformation mode, determining the value of the first grammar identification information to be the second value.

[0378] It should be noted that in the embodiment of the present application, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. Specifically, the first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, 3, etc.

[0379] It can be understood that in the embodiments of the present application, NSPT and LFNST are both transformations that process textures at various angles. There may be multiple transformation kernels, and one transformation kernel may be specifically optimized for a certain specific angle texture. In addition to angle textures, NSPT and LFNST also include transformation kernels that process gradient textures. In fact, these transformation kernels can also be said to be trained KL transforms (Karhunen-Loeve Transform, KLT). That is to say, both NSPT and LFNST can have multiple transformation kernels, each of which is designed for a specific texture. Among them, specific textures include angle textures, gradient textures, etc. In addition, gradient textures can be further extended to include horizontal gradient textures, vertical gradient textures, oblique gradient textures, etc. Furthermore, this technical solution is not limited to being used only for inseparable transformations such as NSPT and LFNST, and this technical solution can also be applied to separable transformations optimized for specific textures.

[0380] It should also be noted that in this embodiment of the present application, both NSPT and LFNST include multiple transform kernel groups, each of which includes three selectable transform kernels. Including the case where LFNST / NSPT is not used, this means that a block has four possible choices for LFNST / NSPT. In this embodiment of the present application, the syntax element lfnst_idx can be used to indicate whether LFNST is used and which LFNST transform kernel is used. If lfnst_idx is 0, LFNST is not used for the current block; if lfnst_idx is 1, the first LFNST transform kernel is used for the current block; and if lfnst_idx is 2, the second LFNST transform kernel is used for the current block. It should be noted that the transform kernel group can be determined by the intra-frame prediction mode or texture feature index, in which case no syntax element indication is required. Furthermore, for a transform kernel group, some technologies may only select two transform kernels, while others may select three. Furthermore, for small blocks of a certain size, NSPT may be used, or three transform kernels may be used. In this embodiment of the present application, lfnst_nspt_idx can be used to indicate whether to use LFNST / NSPT and which LFNST / NSPT transform core to use. In one possible implementation, it indicates the transform core index of the NSPT for smaller blocks and the transform core index of the LFNST for larger blocks. That is, the value of lfnst_nspt_idx can be 0, 1, 2, or 3.

[0381] In an embodiment of the present application, the first syntax identification information can be represented by lfnst_nspt_idx, or the syntax element lfnst_idx can continue to be used, or two syntax elements lfnst_idx and nspt_idx can be used, where lfnst_idx is used to indicate whether LFNST is used and which LFNST transformation core is used, and nspt_idx is used to indicate whether NSPT is used and which NSPT transformation core is used; these are essentially the same and are not specifically limited here.

[0382] In a possible implementation, for the intra prediction mode and the inter prediction mode, the binary symbol string corresponding to the first syntax identification information may use the same binarization method but different context models.

[0383] In some embodiments, binarizing the value of the first syntax identification information to determine the binary symbol string of the current block may include: when the prediction mode of the current block is an intra-frame prediction mode or an inter-frame prediction mode, binarizing the value of the first syntax identification information based on the first method to determine the binary symbol string of the current block.

[0384] In some embodiments, encoding at least one binary symbol in a binary symbol string is performed separately according to at least one context model, which may include: when the prediction mode of the current block is an intra-frame prediction mode, encoding at least one binary symbol in a binary symbol string is performed separately according to at least one first context model; or, when the prediction mode of the current block is an inter-frame prediction mode, encoding at least one binary symbol in a binary symbol string is performed separately according to at least one second context model.

[0385] It should be noted that, in the embodiment of the present application, at least one first context model is different from at least one second context model, that is, the inter-frame prediction mode and the intra-frame prediction mode do not share the same context model.

[0386] It should also be noted that, in the embodiment of the present application, both the inter-frame prediction mode and the intra-frame prediction mode can be encoded using the first method. The first method can be a fixed-length encoding method, also known as a fixed-length (FL) encoding method, in which the same number of binary symbols are used to represent different values ​​of the first syntax identification information; or the first method can be a truncated unary code method, in which different numbers of binary symbols are used to represent different values ​​of the first syntax identification information; or the first method can even be other methods, such as a unary code method, a K-order exponential Golomb coding method, etc., which are not specifically limited here.

[0387] Exemplarily, the value of lfnst_nspt_idx can be 0, 1, 2, or 3. If the first method is a fixed-length coding method, the binarization table shown in Table 6 can be used. In Table 6, the values ​​of the first syntax identification information all use two binary symbols, and these two binary symbols can both be encoded using context models. Among them, the first (binIdx is equal to 0) binary symbol uses one context model, and the second (binIdx is equal to 1) binary symbol uses one context model. Alternatively, if the first method is a truncated unary code method, the binarization table shown in Table 7 can be used. In Table 7, the values ​​of the first syntax identification information can use different binary symbol string lengths. Among them, the three binary symbols in Table 7 can all be encoded using context models. Among them, the first (binIdx is equal to 0) binary symbol uses one context model, the second (binIdx is equal to 1) binary symbol uses one context model, and the third (binIdx is equal to 2) binary symbol uses one context model.

[0388] In another possible implementation, for the intra-frame prediction mode and the inter-frame prediction mode, the binary symbol string corresponding to the first syntax identification information may use different binarization methods and different context models.

[0389] In some embodiments, binarizing the value of the first syntax identification information to determine the binary symbol string of the current block may include: when the prediction mode of the current block is the intra-frame prediction mode, binarizing the value of the first syntax identification information based on the first method to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is the inter-frame prediction mode, binarizing the value of the first syntax identification information based on the second method to determine the second binary symbol string of the current block.

[0390] In some embodiments, encoding at least one binary symbol in a binary symbol string is performed separately according to at least one context model, which may include: when the prediction mode of the current block is an intra-frame prediction mode, encoding at least one binary symbol in a first binary symbol string is performed separately according to at least one first context model and a first method; or, when the prediction mode of the current block is an inter-frame prediction mode, encoding at least one binary symbol in a second binary symbol string is performed separately according to at least one second context model and a second method.

[0391] It should be noted that in the embodiment of the present application, at least one first context model is different from at least one second context model, that is, the inter-frame prediction mode and the intra-frame prediction mode do not share a context model. In addition, the first method is different from the second method, that is, the inter-frame prediction mode and the intra-frame prediction mode use different binarization methods.

[0392] In a specific embodiment, the first method is a fixed-length encoding method, and the second method is a truncated unary code method.

[0393] Thus, when the prediction mode of the current block is intra-frame prediction mode, the first binary symbol string is encoded according to at least one first context model and a fixed-length coding method; or, when the prediction mode of the current block is inter-frame prediction mode, the second binary symbol string is encoded according to at least one second context model and a truncated unary coding method. In this implementation, for different values ​​of the first syntax identification information lfnst_nspt_idx, the length of the first binary symbol string remains unchanged, while the length of the second binary symbol string varies.

[0394] In short, for the transform kernel index (i.e., the value of lfnst_nspt_idx) of the intra prediction mode and the inter prediction mode, different encoding methods can be used here, and different context models can also be used. Specifically, the context model used by lfnst_nspt_idx in the inter prediction mode is different from the context model used by lfnst_nspt_idx in the intra prediction mode, or in other words, lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode do not share the same context model. Among them, one possible implementation method is that lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode use the same binarization method, but different context models. Another possible implementation method is that lfnst_nspt_idx in the inter prediction mode and lfnst_nspt_idx in the intra prediction mode use different binarization methods and different context models.

[0395] It should also be noted that in this embodiment of the present application, LFNST / NSPT can share a syntax element lfnst_nspt_idx, specifically indicating the transform core index of the NSPT for smaller blocks and the transform core index of the LFNST for larger blocks. Alternatively, LFNST / NSPT may each use different syntax elements, for example, LFNST uses the syntax element lfnst_idx and NSPT uses the syntax element nspt_idx. However, the above method still applies to the binarization of lfnst_idx and nspt_idx.

[0396] Furthermore, if the current block uses LFNST / NSPT, then in some embodiments, referring to FIG. 47 , the method further includes:

[0397] S4701: When the current block uses the first transform mode, determine a transform kernel of the current block.

[0398] S4702: Determine a residual block of the current block, and transform the residual block of the current block according to a transformation kernel to determine a transformation coefficient of the current block.

[0399] S4703: Quantize the transformation coefficients of the current block to determine the quantization coefficients of the current block.

[0400] S4704: Encode the quantized coefficients of the current block and write the obtained coded bits into the bitstream.

[0401] It should be noted that in an embodiment of the present application, when LFNST / NSPT is used for the current block, the transform kernel and residual block of the current block are first determined, and then the residual block of the current block is transformed according to the transform kernel to determine the transform coefficient of the current block. Finally, the transform coefficient of the current block is encoded, and the obtained coded bits are written into the bitstream.

[0402] It should also be noted that, in the embodiment of the present application, determining the transformation core of the current block may include: determining the transformation core group of the current block; and determining the transformation core of the current block according to the values ​​of the transformation core group and the first syntax identification information.

[0403] Here, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group. In this way, the transform core of the current block can be determined based on the transform core group and the value of the first syntax identification information.

[0404] It should also be noted that, in the embodiment of the present application, when intra-frame prediction is performed on the current block, the transform core group of the current block can be determined based on the correspondence between the intra-frame prediction mode and the transform core group. When inter-frame prediction is performed on the current block, the transform core group of the current block can be determined based on the correspondence between the texture feature index (i.e., the virtual intra-frame prediction mode) and the transform core group. The texture feature index of the current block can be derived using the prediction block or reference block of the current block or the reconstructed area around the current block.

[0405] In some embodiments, determining the transform core group for the current block may include: when the prediction mode of the current block is an intra-frame prediction mode, determining that the transform core group includes M candidate transform cores; or when the prediction mode of the current block is an inter-frame prediction mode, determining that the transform core group includes N candidate transform cores, where M and N are both positive integers, and M is greater than or equal to N.

[0406] That is, in embodiments of the present application, LFNST / NSPT may have more or fewer options, and the number of transform cores available for LFNST / NSPT for inter-frame prediction may be different from the number of transform cores available for LFNST / NSPT for intra-frame prediction. For example, because intra-frame prediction has more residuals and inter-frame prediction has fewer residuals, a transform core group for intra-frame prediction has three transform cores, while a transform core group for inter-frame prediction has two transform cores. In this case, the value of lfnst_nspt_idx for inter-frame prediction can be 0, 1, or 2.

[0407] For example, Table 8 provides another binarization method for inter-frame prediction lfnst_nspt_idx. As can be seen from Table 8, the binarization method for inter-frame prediction lfnst_nspt_idx can use different binary symbol string lengths, which is a truncated unary code method. These two binary symbols can both be encoded using context models. The first binary symbol (binIdx equals 0) uses one context model, and the second binary symbol (binIdx equals 1) uses another context model.

[0408] It should also be noted that the context model used by lfnst_nspt_idx for inter prediction is different from the context model used by lfnst_nspt_idx for intra prediction, or in other words, lfnst_nspt_idx for inter prediction and lfnst_nspt_idx for intra prediction do not share the same context model. In addition, LFNST / NSPT may also use different syntax elements, and the above method still applies.

[0409] In some embodiments, the method further includes: when M is greater than N, setting the N candidate transformation cores to a subset of the M candidate transformation cores; or, when M is equal to N, setting the N candidate transformation cores to be at least partially different from the M candidate transformation cores.

[0410] It should be noted that, in this embodiment of the present application, the transform cores available for inter-frame LFNST / NSPT can be the same as the transform cores available for intra-frame LFNST / NSPT. If the number of transform cores available for inter-frame LFNST / NSPT is less than the number of transform cores available for intra-frame LFNST / NSPT, then the transform cores available for inter-frame LFNST / NSPT can be a subset of the transform cores available for intra-frame LFNST / NSPT. For example, inter-frame prediction can use two transform cores, and intra-frame prediction can use three transform cores. The transform cores available for inter-frame prediction can be the first two, the last two, or the first or third of the transform cores available for intra-frame prediction, without specific limitation here.

[0411] It should also be noted that in the embodiments of the present application, the transform kernels available for inter-frame LFNST / NSPT can be different from the transform kernels available for intra-frame LFNST / NSPT. That is, transform kernels are designed separately for the residual features of inter-frame prediction and intra-frame prediction, which can achieve higher compression efficiency. For example, three transform kernels can be used for inter-frame prediction, and three transform kernels can also be used for intra-frame prediction, but the three transform kernels for inter-frame prediction and intra-frame prediction are partially or completely different.

[0412] That is, in the embodiment of the present application, the transform core used for inter prediction and the transform core used for intra prediction can be the same, or can be partially or completely different. In addition, in the embodiment of the present application, the available transform core refers to the transform core available in a transform core group.

[0413] In some embodiments, when determining the transform kernel group of the current block, the method may further include: determining at least one texture feature of the current block; performing transform kernel training according to the at least one texture feature to determine the transform kernel group of the current block.

[0414] It should be noted that in related art, intra-frame prediction can correspond to a transform kernel group based on the intra-frame prediction mode. In the embodiment of the present application, inter-frame prediction can use the prediction block to derive a virtual intra-frame prediction mode (texture feature index) corresponding to a transform kernel group. Inter-frame prediction can also derive texture features different from those of intra-frame prediction, and correspondingly, such texture features are used to train the transform kernel.

[0415] In some embodiments, the method further includes: determining a value of second syntax identification information of the current block; encoding the value of the second syntax identification information, and writing the obtained encoding bits into the bitstream.

[0416] Here, the second syntax identification information may be represented by mts_idx, and is used to indicate whether the current block uses the multi-transform selection mode, and which (basic) transform kernel is used in the horizontal / vertical directions of the current block.

[0417] It should also be noted that if the current block does not use the MTS mode, the value of the second syntax identification information is determined to be a first value; if the current block uses the MTS mode, the value of the second syntax identification information is determined to be a second value. The first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, or 3. The magnitude of the second value is related to the transform kernel used for the current block.

[0418] In some embodiments, the method further includes: when the current block does not use the multi-transform selection mode and the current block meets the use condition of the first transform mode, executing the step of determining the value of the first syntax identification information.

[0419] It should be noted that if the current block does not use LFNST / NSPT, the value of the first syntax identification information is determined to be a first value; if the current block uses LFNST / NSPT, the value of the first syntax identification information is determined to be a second value. The first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, or 3. The size of the second value depends on the transform kernel used for the current block.

[0420] It should also be noted that, in this embodiment of the present application, both the first syntax identification information and the second syntax identification information can be encoded using a context model. Furthermore, the first syntax identification information is represented by lfnst_nspt_idx, and the second syntax identification information is represented by mts_idx. lfnst_nspt_idx indicates whether the current block uses the first transform mode (i.e., LFNST / NSPT), and mts_idx indicates whether the current block uses the multi-transform selection mode (i.e., MTS). That is, when writing into the bitstream, the syntax element lfnst_nspt_idx of LFNST / NSPT can follow the syntax element mts_idx of MTS.

[0421] It should also be noted that, in this embodiment of the present application, if the second syntax identification information indicates that the current block uses a multi-transform selection mode, then encoding lfnst_nspt_idx is no longer necessary. In this case, multiple transform kernels for the current block can be determined, and the residual block of the current block can be transformed according to the multiple transform kernels to determine the transform coefficients of the current block. For example, mts_idx can be used to indicate which (basic) transform kernel is used in the horizontal / vertical direction of the current block, respectively.

[0422] In some embodiments, the method further includes: when the current block does not use the first transform mode and the current block meets the use condition of the multiple transform selection mode, executing the step of determining the value of the second syntax identification information of the current block.

[0423] It should be noted that, in the embodiment of the present application, the first syntax identification information and the second syntax identification information can also be encoded using a context model. The first syntax identification information is represented by lfnst_nspt_idx, and the second syntax identification information is represented by mts_idx. lfnst_nspt_idx is used to indicate whether the current block uses the first transform mode (i.e., LFNST / NSPT), and mts_idx is used to indicate whether the current block uses the multi-transform selection mode (i.e., MTS). That is, when writing the bitstream, the syntax element lfnst_nspt_idx of LFNST / NSPT can precede the syntax element mts_idx of MTS.

[0424] It should also be noted that in an embodiment of the present application, if the current block uses the first transform mode, then there is no need to encode mts_idx. At this time, the transform kernel of LFNST / NSPT can be determined, and the residual block of the current block can be transformed according to the transform kernel to determine the transform coefficient of the current block.

[0425] That is to say, in an embodiment of the present application, the encoding order of LFNST / NSPT and MTS for inter-frame prediction and intra-frame prediction is different. Specifically, during the encoding process, the syntax element lfnst_nspt_idx of LFNST / NSPT precedes the syntax element mts_idx of MTS. That is, lfnst_nspt_idx is first parsed at the decoding end. If the value of lfnst_nspt_idx is 0 and the current situation meets the application conditions of MTS, then mts_idx is continued to be parsed. Here, mts_idx indicates which (basic) transform kernel is used in the horizontal and vertical directions of the current block respectively. In addition, in intra-frame prediction, the encoding of lfnst_nspt_idx can also precede mts_idx. In other words, whether mts_idx is encoded depends on the value of lfnst_nspt_idx.

[0426] In this embodiment of the present application, for inter-frame prediction of the current block, mts_idx can be encoded before lfnst_nspt_idx. In other words, whether lfnst_nspt_idx is encoded depends on the value of mts_idx. That is, the decoder first parses mts_idx. If mts_idx is 0 and the current situation meets the application conditions of LFNST / NSPT, then parse lfnst_nspt_idx.

[0427] It should be noted that the inter-frame or inter-frame prediction in the embodiments of the present application can be understood as an inter-frame coded block or coding unit or transformation unit, and the intra-frame or intra-frame prediction can be understood as an intra-frame coded block or coding unit or transformation unit.

[0428] It should also be noted that in the embodiment of the present application, two syntax identification information (third syntax identification information and fourth syntax identification information) can also be used to replace lfnst_nspt_idx. Among them, the third syntax identification information is represented by nspt_idx and can be used to indicate whether the current block uses NSPT and which NSPT transformation core is used. The fourth syntax identification information is represented by lfnst_idx and can be used to indicate whether the current block uses LFNST and which LFNST transformation core is used.

[0429] In some embodiments, the method further includes: determining a value of third syntax identification information of the current block; wherein the third syntax identification information is used to indicate whether the current block uses an inseparable basic transform mode; encoding the value of the third syntax identification information, and writing the obtained coded bits into the bitstream.

[0430] In an embodiment of the present application, if the current block does not use the inseparable basic transform mode, then the value of the third syntax identification information is determined to be the first value; if the current block uses the inseparable basic transform mode, then the value of the third syntax identification information is determined to be the second value.

[0431] In an embodiment of the present application, when the current block uses the non-separable basic transform mode, a transform kernel of the current block is determined, and an NSPT transform is performed on the residual block of the current block according to the transform kernel to determine a transform coefficient of the current block.

[0432] In some embodiments, the method further includes: determining a value of fourth grammatical identification information of the current block; wherein the fourth grammatical identification information is used to indicate whether the current block uses a low-frequency inseparable transform mode; encoding the value of the fourth grammatical identification information, and writing the obtained coded bits into the bitstream.

[0433] In an embodiment of the present application, if the current block does not use the low-frequency inseparable transform mode, then the value of the fourth grammar identification information is determined to be the first value; if the current block uses the low-frequency inseparable transform mode, then the value of the fourth grammar identification information is determined to be the second value.

[0434] In an embodiment of the present application, when the current block uses the low-frequency inseparable transform mode, a transform kernel of the current block is determined, and a residual block of the current block is subjected to LFNST transform according to the transform kernel to determine a transform coefficient of the current block.

[0435] It should also be noted that, in the embodiment of the present application, the first value can be set to 0, and the second value can be set to non-zero, such as 1, 2, 3, etc. Here, the size of the second value is related to the type of transform kernel used for the current block.

[0436] That is, in the embodiments of the present application, LFNST / NSPT can each use different syntax elements, but the above method still applies. Specifically, intra-frame prediction and inter-frame prediction use different encoding methods for LFNST syntax elements and NSPT syntax elements, intra-frame prediction and inter-frame prediction use different context models for LFNST syntax elements and NSPT syntax elements, intra-frame prediction and inter-frame prediction can use different numbers of transform kernels for LFNST and NSPT, intra-frame prediction and inter-frame prediction can use different transform kernels, intra-frame prediction and inter-frame prediction LFNST and NSPT can use different transform kernels, and intra-frame prediction and inter-frame prediction LFNST / NSPT and MTS have different encoding orders.

[0437] It can also be understood that in the embodiment of the present application, in addition to applying LFNST and NSPT to intra-frame prediction / inter-frame prediction blocks, they can also be applied to intra-frame block copy (IBC) blocks, specifically by replacing inter-frame prediction with IBC.

[0438] In another possible implementation, lfnst_nspt_idx may be encoded into the bitstream only when the last non-zero coefficient position of the current block meets certain conditions. In some embodiments, the method further includes: when the last non-zero coefficient position among the transform coefficients of the current block meets a preset condition, determining a value of first syntax identification information, binarizing the value of the first syntax identification information, and determining a binary symbol string for the current block.

[0439] In related art, lfnst_nspt_idx is only encoded if certain conditions are met. One of these conditions is that the last non-zero coefficient position is greater than or equal to 1. Quantized coefficients are scanned using a hierarchical diagonal scan. A coefficient block can be divided into 4×4 sub-blocks, and the 4×4 sub-blocks are scanned diagonally, as are the sub-blocks themselves. The position (0, 0) at the top left corner of the current block is 0 in the scan order. The last non-zero coefficient position is greater than or equal to 1, indicating that the current block has coefficients other than (0, 0). For the DCT2 transform, (0, 0) is the DC coefficient. In other words, if the current block has only a non-zero coefficient at position (0, 0) or no non-zero coefficients, the lfnst_nspt_idx syntax element does not need to be encoded and defaults to 0. Because the likelihood of LFNST / NSPT being used in this situation is minimal, this restriction significantly reduces unnecessary lfnst_nspt_idx overhead. Moreover, the relevant technology requires that the last non-zero coefficient position of all color components must be greater than or equal to 1. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three color components Y, U, and V must be greater than or equal to 1.

[0440] Thus, for inter-frame predicted blocks, the residuals are inherently smaller than those for intra-frame predicted blocks. Furthermore, given that the quality requirements for chrominance components are generally much lower than those for luminance components, inter-frame predicted blocks often have certain luminance coefficients but no chrominance coefficients. Since this LFNST / NSPT condition is designed for intra-frame prediction, and LFNST / NSPT can currently also be used for inter-frame prediction, but their coefficients and residual patterns are different, the present invention can modify this restriction.

[0441] In one possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0442] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0443] It should be noted that, in the embodiment of the present application, the first threshold value may also be referred to as the minimum threshold value, wherein the minimum threshold value may be set to 1, or a larger value such as 2, 3, etc., which is not specifically limited here.

[0444] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the first threshold. In other words, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is greater than the first threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is greater than the first threshold.

[0445] For example, assuming the minimum threshold is equal to 1, the relevant technology is still used for intra-frame predicted blocks. That is, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all color components is greater than or equal to 1; for inter-frame predicted blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all its color components is greater than or equal to 1; for single-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required.

[0446] It's also understandable that in related art, lfnst_nspt_idx is only encoded if certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot exceed a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the current block size parameters, namely the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input is the 4x4 DCT2 transformed coefficients, and the output is 8 coefficients. This means that for a 4×4 block, the LFNST output can only have a maximum of 8 coefficients. Therefore, for the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), it is definitely not an LFNST output. This allows the decoder to know that the current block is definitely not using LFNST and therefore does not need to encode its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx. Furthermore, related art requires that the last non-zero coefficient position of any color component cannot exceed its maximum threshold. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three color components Y, U, and V cannot be greater than its maximum threshold.

[0447] In this way, considering that this condition of LFNST / NSPT is designed for intra-frame prediction, and LFNST / NSPT can also be used for inter-frame prediction, but their coefficients and residual rules are different, this restriction condition can be modified.

[0448] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0449] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0450] It should be noted that in the embodiments of the present application, the second threshold may also be referred to as a maximum threshold. In one specific embodiment, the second threshold may be determined based on the size parameters of the current block. For example, the second threshold may be determined based on the length and width of the current block. In another specific embodiment, the second threshold may be set to the difference between the maximum number of coefficients in the transform coefficients of the current block and one. For example, if the LFNST output of the current block may have a maximum of 8 coefficients, the second threshold may be set to 7.

[0451] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the second threshold. That is, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is less than the second threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is less than the second threshold.

[0452] For example, for intra-frame predicted blocks, the relevant technology is still used. That is, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all color components is not greater than its maximum threshold; for inter-frame predicted blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all its components is not greater than the maximum threshold; for single-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required.

[0453] That is, in the embodiment of the present application, lfnst_nspt_idx can be encoded only when the last non-zero coefficient position of the Y component is greater than or equal to the minimum threshold, and the U and V components are no longer required. Alternatively, lfnst_nspt_idx can be encoded only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the U and V components are no longer required.

[0454] Furthermore, in some embodiments, the method further includes: determining a prediction block of the current block; and determining a residual block of the current block according to the original block of the current block and the prediction block of the current block.

[0455] It should be noted that, in the embodiment of the present application, if the prediction mode of the current block is the intra-frame prediction mode, then intra-frame prediction is performed on the current block to determine the prediction block of the current block; then a subtraction operation is performed on the original block of the current block and the prediction block of the current block to determine the residual block of the current block. It should also be noted that, in the embodiment of the present application, if the prediction mode of the current block is the inter-frame prediction mode, then inter-frame prediction is performed on the current block to determine the prediction block of the current block; then a subtraction operation is performed on the original block of the current block and the prediction block of the current block to determine the residual block of the current block.

[0456] In a possible implementation, transforming the residual block of the current block according to the transform kernel to determine the transform coefficient of the current block may include performing an inseparable basic transform on the residual block of the current block according to the transform kernel to determine the transform coefficient of the current block.

[0457] In another possible implementation, transforming the residual block of the current block according to the transform kernel to determine the transform coefficient of the current block may include: performing a discrete cosine transform on the residual block of the current block to determine the transform block of the current block; and performing a low-frequency inseparable transform on the transform block of the current block according to the transform kernel to determine the transform coefficient of the current block.

[0458] That is to say, in an embodiment of the present application, if lfnst_nspt_idx indicates that the current block uses the first transform mode, then when the residual block of the current block is transformed according to the transform kernel, it can include: if the size parameter of the current block meets the first condition, then the residual block of the current block is subjected to an inseparable basic transform according to the transform kernel to determine the transform coefficient of the current block; if the size parameter of the current block meets the second condition, then the residual block of the current block is subjected to a discrete cosine transform to determine the transform block of the current block; and the transform block of the current block is subjected to a low-frequency inseparable transform according to the transform kernel to determine the transform coefficient of the current block.

[0459] Here, the size parameter of the current block satisfies the first condition, which may include: the size parameter of the current block is small, for example, the size parameter of the current block is less than a certain threshold. That is, for blocks of smaller sizes, the NSPT transform kernel is used here, that is, the residual block of the current block is subjected to NSPT transform according to the transform kernel to determine the transform coefficient of the current block. The size parameter of the current block satisfies the second condition, which may include: the size parameter of the current block is large, for example, the size parameter of the current block is greater than a certain threshold. That is, for blocks of larger sizes, the LFNST transform kernel is used here, that is, the residual block of the current block is first subjected to the basic transform of DCT2, and then the transform block of the current block is subjected to LFNST transform according to the transform kernel to determine the transform coefficient of the current block.

[0460] It should also be noted that in the embodiments of this application, the "transformation" of the residual block by the encoder can also be called a "forward transform," specifically referring to the transformation from the spatial domain to the frequency domain to remove residual correlation. It should be noted that if the standard only specifies decoding, then the "transformation" in the standard text refers to the decoding part, specifically referring to the "inverse transform" in this article.

[0461] In another embodiment of the present application, the embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: a quantization coefficient of the current block, a value of the first syntax identification information, a value of the second syntax identification information, a value of the third syntax identification information, and a value of the fourth syntax identification information.

[0462] In an embodiment of the present application, the first syntax identification information is used to indicate whether the current block uses the first transform mode, the second syntax identification information is used to indicate whether the current block uses the multi-transform selection mode, the third syntax identification information is used to indicate whether the current block uses the inseparable basic transform mode, and the fourth syntax identification information is used to indicate whether the current block uses the low-frequency inseparable transform mode.

[0463] This embodiment provides an encoding method, comprising: determining a prediction mode for a current block; determining at least one context model based on the prediction mode of the current block; decoding a bitstream based on the at least one context model to determine a binary symbol string for the current block; wherein the binary symbol string includes at least one binary symbol; performing debinarization on the binary symbol string to determine a value of first syntax identification information; and, when the first syntax identification information indicates that the current block uses a first transform mode, determining a transform kernel for the current block, and performing an inverse transform on the transform coefficients of the current block based on the transform kernel to determine a residual block for the current block. Thus, at least one context model is first determined based on the prediction mode of the current block, and then the binary symbol string corresponding to the first syntax identification information is encoded based on the determined at least one context model. In this way, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, so that a coding method that conforms to the distribution law of the LFNST / NSPT binary symbol string in the inter-frame prediction mode can be used, thereby improving not only compression efficiency and coding efficiency but also coding performance.

[0464] In another embodiment of the present application, based on the encoding and decoding methods described in the preceding embodiments, and considering that in related art, a LFNST / NSPT transform core group has three selectable transform cores, coupled with the case where LFNST / NSPT is not used, a block has four possible choices for LFNST / NSPT. In this embodiment of the present application, the syntax element lfnst_idx can be used to indicate whether and which LFNST transform core is used. An lfnst_idx of 0 indicates that LFNST is not used for the current block; an lfnst_idx of 1 indicates that the first LFNST transform core is used for the current block; and an lfnst_idx of 2 indicates that the second LFNST transform core is used for the current block. It should be noted that the transform core group is determined by the intra-frame prediction mode and does not require syntax element indication. For example, in some technologies, only two transform cores can be selected, while in other technologies, three transform cores can be used. Furthermore, when NSPT is used for small blocks of a certain size, three transform cores are also used. Here, lfnst_nspt_idx can be used to indicate whether to use the LFNST / NSPT transform kernel and which LFNST / NSPT transform kernel to use. In one implementation, it indicates the index of the NSPT transform kernel for smaller blocks and the index of the LFNST transform kernel for larger blocks. In other words, the value of lfnst_nspt_idx can be 0, 1, 2, or 3. Alternatively, you can continue to use the lfnst_idx syntax element, or use both the lfnst_idx and nspt_idx syntax elements. These elements are essentially equivalent and are not specifically defined here.

[0465] Figure 48 shows a block diagram of a syntax element value written into a code stream, and Figure 49 shows a block diagram of a syntax element value decoded from a code stream. During encoding, the syntax element value is binarized into a binary symbol string, and the binary symbol string is converted into a bit stream through CABAC. During decoding, when the decoder wants to parse a syntax element, it decodes the binary symbol string from the read code stream through CABAC, and the binary symbol string is debinarized to obtain the syntax element value. When CABAC (context-based adaptive binary arithmetic coding) is encoded, if a binary symbol uses a context model, the probability of the corresponding context model is used for encoding, otherwise, equal probability is used for encoding, and the same is true when decoding. It should be noted that the syntax element value here is the value of the syntax identification information in the aforementioned embodiment.

[0466] In related art, the original lfnst_nspt_idx encoding method is optimized for intra-frame prediction, so this can be directly referred to as the intra-frame lfnst_nspt_idx binarization method, as shown in Table 6. As can be seen, the intra-frame lfnst_nspt_idx binarization method uses two binary symbols, both of which are context-coded. The first binary symbol (binIdx = 0) uses one context model, and the second binary symbol (binIdx = 1) uses another context model.

[0467] The binarization method and context model configuration are related to their statistical behavior. Figure 40 shows the statistical results of intra-frame lfnst_nspt_idx values ​​in the MarketPlace test sequence at all intra QP 32. Figure 40 shows that under this condition, the four possible choices for lfnst_nspt_idx are similar, with 1 and 2 being even more likely than 0. Note that this is just an example; the statistical results may vary for different test sequences and different QPs.

[0468] Furthermore, using the same binarization method, Figure 41 shows the inter-frame lfnst_nspt_idx statistics for the test sequence BasketballDrive at a random access QP of 32. This shows that under this condition, there is a certain degree of difference in the likelihood of the four possible values ​​for lfnst_nspt_idx. 0 is significantly higher than 1, 2, and 3, and 1 is higher than 2 and 3. Of course, it should be noted that this is just an example; different test sequences and different QPs will produce different statistical results.

[0469] Based on the theoretical analysis and statistical data of the probability distribution of inter-frame lfnst_nspt_idx, the embodiment of the present application proposes a binarization method for inter-frame lfnst_nspt_idx, as shown in Table 7 above. It can be seen that the binarization method of inter-frame lfnst_nspt_idx uses different binary symbol string lengths and is a truncated unary code. These three binary symbols can all be context-mode encoded. The first (binIdx equals 0) binary symbol uses a context model, the second (binIdx equals 1) binary symbol uses a context model, and the third (binIdx equals 2) binary symbol uses a context model.

[0470] That is, the context model used by inter-frame lfnst_nspt_idx is different from the context model used by intra-frame lfnst_nspt_idx, or the context model used by inter-frame lfnst_nspt_idx and intra-frame lfnst_nspt_idx is not shared. One possible implementation is to use the same binarization method but different context models for inter-frame lfnst_nspt_idx and intra-frame lfnst_nspt_idx. Another possible implementation is to use different binarization methods and different context models for inter-frame lfnst_nspt_idx and intra-frame lfnst_nspt_idx.

[0471] It should also be noted that LFNST / NSPT may use different syntax elements, but the principles of the above method remain unchanged.

[0472] In an embodiment of the present application, LFNST / NSPT may have more or fewer options, and the number of transform cores available for LFNST / NSPT between frames may be different from the number of transform cores available for LFNST / NSPT within a frame. For example, since there are more residuals within a frame and fewer residuals between frames, a transform core group within a frame has 3 transform cores, while a transform core group between frames has 2 transform cores. Then the encoding method of lfnst_nspt_idx between frames is as shown in Table 8 above. From this, it can be seen that the binarization method of lfnst_nspt_idx between frames uses different binary symbol string lengths, which is a truncated unary code. These two binary symbols can both be context mode encoded. The first (binIdx equals 0) binary symbol uses one context model, and the second (binIdx equals 1) binary symbol uses one context model.

[0473] Here, the context model used by inter-frame lfnst_nspt_idx is different from the context model used by intra-frame lfnst_nspt_idx, or in other words, inter-frame lfnst_nspt_idx and intra-frame lfnst_nspt_idx do not share the same context model. In addition, LFNST / NSPT may use different syntax elements, and the principles of the above method remain unchanged.

[0474] Furthermore, the transform cores available for inter-frame LFNST / NSPT can be the same as the transform cores available for intra-frame LFNST / NSPT. If the number of transform cores available for inter-frame LFNST / NSPT is less than the number of transform cores available for intra-frame LFNST / NSPT, the transform cores available for inter-frame LFNST / NSPT can be a subset of the transform cores available for intra-frame LFNST / NSPT. For example, if two transform cores are available for inter-frame and three transform cores are available for intra-frame, the transform cores available for inter-frame are the first two, the last two, or the first and third transform cores available for intra-frame.

[0475] Furthermore, the transform kernels available for inter-frame LFNST / NSPT can be different from those available for intra-frame LFNST / NSPT. Specifically, separate transform kernels can be designed for inter-frame and intra-frame residual features, achieving higher compression efficiency. For example, three transform kernels can be used for inter-frame and three transform kernels can be used for intra-frame, but some or all of the three transform kernels differ between inter-frame and intra-frame.

[0476] Optionally, the available transform kernels refer to the available transform kernels in a transform kernel group. In related art, each intra-frame corresponds to a transform kernel group based on the intra-frame prediction mode. In this solution, a virtual intra-frame prediction mode (texture feature index) derived from an inter-frame prediction block corresponds to a transform kernel group. Texture features different from those derived from intra-frames can also be derived for inter-frames, and transform kernels are trained accordingly using these texture features.

[0477] For example, the current LFNST and NSPT training are relatively single texture features, such as texture in a specific direction, as shown in (a) and (b) in Figure 42. It should be noted that this is just a schematic diagram, but for inter-frame, residuals appear more often at the edges of objects. For example, if there are residuals with two-directional textures as shown in (c) in Figure 42 between frames, then such LFNST / NSPT transformation kernels can also be trained. When analyzing the texture features of the current block (such as the process of deriving the virtual intra-frame prediction mode mentioned above), if it is found that the prediction block of the current block or the surrounding reconstructed area has textures in two directions, then the transformation kernel with such texture features can be matched.

[0478] In an embodiment of the present application, when parsing the bitstream, the LFNST syntax element lfnst_nspt_idx precedes the MTS syntax element mts_idx. That is, lfnst_nspt_idx is parsed first. If lfnst_nspt_idx is 0 and the current situation meets the application conditions of MTS, mts_idx is parsed. Here, mts_idx is used to indicate which (basic) transform kernel is used in the horizontal and vertical directions of the current block. Similarly, during intra-frame coding, lfnst_nspt_idx can also be parsed before mts_idx, and whether mts_idx is parsed depends on the value of lfnst_nspt_idx.

[0479] In this embodiment of the present application, during inter-frame coding, mts_idx is parsed before lfnst_nspt_idx, and whether lfnst_nspt_idx is parsed depends on the value of mts_idx. In other words, mts_idx is parsed first. If mts_idx is 0 and the current situation meets the application conditions of LFNST / NSPT, lfnst_nspt_idx is parsed.

[0480] It should be noted that the inter-frame described in the embodiments of the present application can be understood as an inter-frame coded block or coding unit, a transformation unit, and the intra-frame can be understood as an intra-frame coded block or coding unit, a transformation unit.

[0481] It should also be noted that this technical solution can also be applied to the IBC mode, by replacing the interframe in the above text with IBC.

[0482] In the embodiment of the present application, lfnst_nspt_idx is parsed only when certain conditions are met. One of the conditions is that the position of the last non-zero coefficient is greater than or equal to 1. The scanning of the quantized coefficients is performed according to the hierarchical diagonal scan. A coefficient block is divided into 4×4 sub-blocks. The 4×4 sub-blocks are scanned according to the diagonal scan, and the 4×4 sub-blocks are also scanned according to the diagonal scan. The position (0, 0) in the upper left corner of the current block is 0 in the scan order. The position of the last non-zero coefficient is greater than or equal to 1, which means that the current block has coefficients other than (0, 0). For DCT2 transform, (0, 0) is the DC coefficient. That is to say, if the coefficients of the current block only have non-zero coefficients at position (0, 0) or no non-zero coefficients, then the syntax element lfnst_nspt_idx does not need to be parsed and defaults to 0. Because the possibility of LFNST / NSPT being used in this case is very small, this restriction can significantly reduce the unnecessary overhead of lfnst_nspt_idx. Furthermore, the relevant technology requires that the last non-zero coefficient position of all components is greater than or equal to 1. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three components Y, U, and V must be greater than or equal to 1.

[0483] Inter-coded blocks inherently have fewer residuals than intra-coded blocks. Furthermore, given that chroma quality requirements are generally much lower than luma, inter-coded blocks often have certain luma coefficients but no chroma coefficients. While the LFNST / NSPT constraint was designed for intra-coded blocks, inter-coded blocks can now also use LFNST / NSPT. However, their coefficients and residual patterns differ, so this constraint can be modified.

[0484] One possible implementation is to continue using the relevant techniques for intra-coded blocks. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all components is greater than or equal to 1. For inter-coded blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1. The UV component is no longer required.

[0485] Another possible implementation is: for a block partitioned by a dual tree, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all its components is greater than or equal to 1. For a block partitioned by a single tree, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1. The UV component is no longer required.

[0486] Furthermore, in the related art, the judgment condition is set to be that the last non-zero coefficient position is greater than or equal to 1, that is, the minimum possible threshold is 1, which is called the minimum threshold here. The minimum threshold can also be set to a larger value, such as 2, 3, etc. For example, lfnst_nspt_idx is only parsed when the last non-zero coefficient position is greater than or equal to 2.

[0487] In related art, lfnst_nspt_idx is parsed only when certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot be greater than a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the block size, i.e., the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input to the LFNST is the 4×4 coefficients after the DCT2 transform, and the output is 8 coefficients. This means that for a 4x4 block, the LFNST output can only have a maximum of 8 coefficients. For the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), then it is definitely not an LFNST output. The decoder then knows that the current block is definitely not using LFNST and does not need to parse its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx.

[0488] Furthermore, related technologies require that the last non-zero coefficient position of any component cannot exceed its maximum threshold. For example, in a block partitioned by a single tree in YUV format, the last non-zero coefficient position of each of the Y, U, and V components cannot exceed their maximum thresholds. This constraint is designed for intra-frame coding, and while inter-frame coding can now also use LFNST / NSPT, their coefficients and residual patterns differ, so this constraint can be modified.

[0489] One possible implementation is to continue using the relevant techniques for intra-coded blocks. Specifically, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of any component is no greater than its maximum threshold. For inter-coded blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is no greater than its maximum threshold. The UV component is no longer required.

[0490] Another possible implementation is to parse lfnst_nspt_idx for blocks partitioned by dual trees only when the last non-zero coefficient position of all its components is not greater than the maximum threshold. For blocks partitioned by single trees, parse lfnst_nspt_idx only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold. UV components are no longer required.

[0491] In the embodiments of the present application, the specific implementation of the aforementioned embodiments is described in detail through the aforementioned embodiments. It can be seen that according to the technical solutions of the aforementioned embodiments, on the one hand, different encoding methods are used for the transform kernel index of LFNST / NSPT in intra-frame and inter-frame, different context models are used for the index of LFNST / NSPT in intra-frame and inter-frame, the number of transform kernels that can be used for LFNST / NSPT in intra-frame and inter-frame is different, the transform kernels that can be used for LFNST / NSPT in intra-frame and inter-frame are different, and the encoding order of LFNST / NSPT and MTS in inter-frame and intra-frame is different; on the other hand, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the UV component is no longer required; and lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the UV component is no longer required; thereby, an encoding method that conforms to the distribution law of the transform kernel index of LFNST / NSPT in inter-frame is used, thereby improving compression efficiency and thereby improving encoding and decoding performance.

[0492] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, see Figure 50, which shows a schematic diagram of the composition structure of an encoder provided by an embodiment of the present application. As shown in Figure 50, the encoder 500 may include a first determining unit 5001, a binarization unit 5002, and an encoding unit 5003, wherein:

[0493] A first determining unit 5001 is configured to determine a prediction mode of a current block; and determine at least one context model according to the prediction mode of the current block;

[0494] A binarization unit 5002 is configured to determine a value of the first syntax identification information and perform binarization processing on the value of the first syntax identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol;

[0495] The encoding unit 5003 is configured to perform encoding processing on at least one binary symbol in the binary symbol string according to at least one context model, and write the obtained encoded bits into the bit stream.

[0496] In some embodiments, the prediction mode of the current block includes: an inter-frame prediction mode and / or an intra-frame prediction mode; accordingly, the first determination unit 5001 is also configured to determine that at least one context model corresponding to the inter-frame prediction mode is different from at least one context model corresponding to the intra-frame prediction mode.

[0497] In some embodiments, the binarization unit 5002 is further configured to binarize the value of the first syntax identification information based on a first method to determine the binary symbol string of the current block when the prediction mode of the current block is an intra-frame prediction mode and / or an inter-frame prediction mode.

[0498] In some embodiments, the binarization unit 5002 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, binarize the value of the first syntax identification information based on a first method to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is an inter-frame prediction mode, binarize the value of the first syntax identification information based on a second method to determine the second binary symbol string of the current block; wherein the first method is different from the second method.

[0499] In some embodiments, the first method is a fixed-length encoding method, and the second method is a truncated unary code method.

[0500] In some embodiments, the first determination unit 5001 is further configured to determine that the value of the first syntax identification information is the first value if the current block does not use the first transformation mode; if the current block uses the first transformation mode, determine that the value of the first syntax identification information is the second value.

[0501] In some embodiments, referring to FIG. 50 , the encoder 500 may further include a transform unit 5004 , wherein:

[0502] The first determining unit 5001 is further configured to determine a transform kernel of the current block when the current block uses the first transform mode;

[0503] a transform unit 5004 configured to determine a residual block of a current block, and transform the residual block of the current block according to a transform kernel to determine a transform coefficient of the current block;

[0504] The encoding unit 5003 is further configured to perform encoding processing on the transform coefficients of the current block and write the obtained encoding bits into the bitstream.

[0505] In some embodiments, the first determination unit 5001 is further configured to determine the transform core group of the current block; and determine the transform core of the current block based on the transform core group and the value of the first syntax identification information; wherein, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group.

[0506] In some embodiments, the first determination unit 5001 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, determine that the transform core group includes M candidate transform cores; or, when the prediction mode of the current block is an inter-frame prediction mode, determine that the transform core group includes N candidate transform cores; wherein M and N are both positive integers, and M is greater than or equal to N.

[0507] In some embodiments, the first determination unit 5001 is further configured to, when M is greater than N, set the N candidate transformation cores to a subset of the M candidate transformation cores; or, when M is equal to N, set the N candidate transformation cores to be at least partially different from the M candidate transformation cores.

[0508] In some embodiments, the first determining unit 5001 is further configured to determine at least one texture feature of the current block; and perform transform kernel training according to the at least one texture feature to determine the transform kernel group of the current block.

[0509] In some embodiments, the first determination unit 5001 is further configured to determine a value of second syntax identification information of the current block; wherein the second syntax identification information is used to indicate whether the current block uses a multi-transform selection mode; the encoding unit 5003 is further configured to encode the value of the second syntax identification information and write the obtained coded bits into the bitstream.

[0510] In some embodiments, the first determining unit 5001 is further configured to, when the current block does not use the multi-transform selection mode and the current block meets the use condition of the first transform mode, perform the step of determining the value of the first syntax identification information.

[0511] In some embodiments, the first determining unit 5001 is further configured to, when the current block does not use the first transform mode and the current block meets the use conditions of the multiple transform selection mode, execute the step of determining the value of the second syntax identification information of the current block.

[0512] In some embodiments, the first determination unit 5001 is further configured to determine a value of third syntax identification information of the current block; wherein the third syntax identification information is used to indicate whether the current block uses an inseparable basic transform mode; the encoding unit 5003 is further configured to encode the value of the third syntax identification information and write the obtained coded bits into the bitstream.

[0513] In some embodiments, the first determination unit 5001 is further configured to determine the value of the fourth grammar identification information of the current block; wherein the fourth grammar identification information is used to indicate whether the current block uses the low-frequency inseparable transform mode; the encoding unit 5003 is further configured to encode the value of the fourth grammar identification information and write the obtained coded bits into the bitstream.

[0514] In some embodiments, referring to FIG. 50 , the encoder 500 may further include a quantization unit 5005 configured to quantize the transform coefficients of the current block and determine the quantization coefficients of the current block; and an encoding unit 5003 configured to encode the quantization coefficients of the current block and write the obtained encoding bits into the bitstream.

[0515] In some embodiments, the binarization unit 5002 is further configured to determine the value of the first syntax identification information and binarize the value of the first syntax identification information to determine the binary symbol string of the current block when the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition.

[0516] In some embodiments, the first determination unit 5001 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0517] In some embodiments, the first determination unit 5001 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0518] In some embodiments, the first determination unit 5001 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0519] In some embodiments, the first determination unit 5001 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0520] In some embodiments, the first determining unit 5001 is further configured to determine the second threshold according to a size parameter of the current block.

[0521] In some embodiments, the first determining unit 5001 is further configured to set the second threshold to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

[0522] In some embodiments, the first determining unit 5001 is further configured to determine a prediction block of the current block; and determine a residual block of the current block according to the original block of the current block and the prediction block of the current block.

[0523] In some embodiments, the transform unit 5004 is further configured to perform a non-separable basic transform on the residual block of the current block according to the transform kernel to determine a transform coefficient of the current block.

[0524] In some embodiments, the transformation unit 5004 is further configured to perform discrete cosine transform on the residual block of the current block to determine the transformation block of the current block; and perform low-frequency inseparable transform on the transformation block of the current block according to the transformation kernel to determine the transformation coefficient of the current block.

[0525] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0526] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0527] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 500. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0528] Based on the composition of the encoder 500 and the computer-readable storage medium, refer to Figure 51, which shows a specific hardware structure diagram of the encoder 500 provided in an embodiment of the present application. As shown in Figure 51, the encoder 500 may include: a first communication interface 5101, a first memory 5102 and a first processor 5103; each component is coupled together through a first bus system 5104. It can be understood that the first bus system 5104 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 5104 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as the first bus system 5104 in Figure 51. Among them,

[0529] The first communication interface 5101 is used to receive and send signals when sending and receiving information with other external network elements;

[0530] A first memory 5102 is used to store computer programs that can be run on the first processor 5103;

[0531] The first processor 5103 is configured to, when running the computer program, execute:

[0532] Determine a prediction mode for a current block; determine at least one context model based on the prediction mode of the current block; determine a value of first syntax identification information, and binarize the value of the first syntax identification information to determine a binary symbol string for the current block; wherein the binary symbol string includes at least one binary symbol; encode at least one binary symbol in the binary symbol string according to at least one context model, and write the obtained encoded bits into a bitstream.

[0533] It is understood that the first memory 5102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 5102 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0534] The first processor 5103 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 5103. The above-mentioned first processor 5103 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented as a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 5102 , and the first processor 5103 reads the information in the first memory 5102 and completes the steps of the above method in combination with its hardware.

[0535] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0536] Optionally, as another embodiment, the first processor 5103 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0537] This embodiment provides an encoder that first determines at least one context model based on a prediction mode of a current block, and then encodes a binary symbol string corresponding to first syntax identification information based on the at least one determined context model. In other words, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, enabling the use of an encoding method that conforms to the distribution pattern of the binary symbol string of LFNST / NSPT in the inter-frame prediction mode, thereby improving not only compression efficiency but also encoding and decoding performance.

[0538] Based on the same inventive concept as the above embodiment, referring to FIG52 , a schematic diagram of the structure of a decoder provided by an embodiment of the present application is shown. As shown in FIG52 , the decoder 520 may include a second determination unit 5201, a decoding unit 5202, a debinarization unit 5203, and an inverse transformation unit 5204, wherein:

[0539] The second determining unit 5201 is configured to determine a prediction mode of the current block; and determine at least one context model according to the prediction mode of the current block;

[0540] The decoding unit 5202 is configured to decode the code stream according to at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol;

[0541] a debinarization unit 5203 configured to perform debinarization processing on the binary symbol string to determine a value of the first syntax identification information;

[0542] The inverse transform unit 5204 is configured to determine the transform kernel of the current block when the first syntax identification information indicates that the current block uses the first transform mode, and inversely transform the transform coefficients of the current block according to the transform kernel to determine the residual block of the current block.

[0543] In some embodiments, the prediction mode of the current block includes: an inter-frame prediction mode and / or an intra-frame prediction mode; accordingly, the second determination unit 5201 is also configured to determine that at least one context model corresponding to the inter-frame prediction mode is different from at least one context model corresponding to the intra-frame prediction mode.

[0544] In some embodiments, the debinarization unit 5203 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode and / or an inter-frame prediction mode, perform debinarization on the binary symbol string based on the first method to determine the value of the first syntax identification information.

[0545] In some embodiments, the debinarization unit 5203 is further configured to, when the prediction mode of the current block is the intra-frame prediction mode, perform debinarization on the binary symbol string based on a first method to determine the value of the first syntax identification information; or, when the prediction mode of the current block is the inter-frame prediction mode, perform debinarization on the binary symbol string based on a second method to determine the value of the first syntax identification information; wherein the first method is different from the second method.

[0546] In some embodiments, the second determination unit 5201 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, decode the code stream according to at least one first context model and a fixed-length decoding method to determine the first binary symbol string of the current block; or, when the prediction mode of the current block is an inter-frame prediction mode, decode the code stream according to at least one second context model and a truncated unary code method to determine the second binary symbol string of the current block; wherein the at least one first context model is different from the at least one second context model.

[0547] In some embodiments, the second determination unit 5201 is further configured to determine the transform core group of the current block; and determine the transform core of the current block based on the transform core group and the value of the first syntax identification information; wherein, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group.

[0548] In some embodiments, the second determination unit 5201 is further configured to determine that the transform core group includes M candidate transform cores when the prediction mode of the current block is an intra-frame prediction mode; or, when the prediction mode of the current block is an inter-frame prediction mode, determine that the transform core group includes N candidate transform cores; wherein M and N are both positive integers, and M is greater than or equal to N.

[0549] In some embodiments, the second determination unit 5201 is further configured to set the N candidate transformation cores to a subset of the M candidate transformation cores when M is greater than N; or, when M is equal to N, set the N candidate transformation cores to be at least partially different from the M candidate transformation cores.

[0550] In some embodiments, the second determining unit 5201 is further configured to determine at least one texture feature of the current block; perform transform kernel training according to the at least one texture feature to determine the transform kernel group of the current block.

[0551] In some embodiments, the decoding unit 5202 is further configured to decode the code stream and determine the value of the second syntax identification information of the current block; the debinarization unit 5203 is further configured to decode the code stream according to at least one context model to determine the binary symbol string of the current block when the second syntax identification information indicates that the current block does not use the multi-transform selection mode and the current block meets the use conditions of the first transform mode; and debinarize the binary symbol string to determine the value of the first syntax identification information.

[0552] In some embodiments, the decoding unit 5202 is further configured to decode the code stream and determine the value of the second syntax identification information of the current block when the first syntax identification information indicates that the current block does not use the first transform mode and the current block meets the use conditions of the multi-transform selection mode; the inverse transform unit 5204 is further configured to determine multiple transform kernels of the current block when the second syntax identification information indicates that the current block uses the multi-transform selection mode, and perform inverse transform on the transform coefficients of the current block according to the multiple transform kernels to determine the residual block of the current block.

[0553] In some embodiments, the second determining unit 5201 is further configured to decode the code stream according to at least one third context model to determine a third binary symbol string of the current block; the debinarization unit 5203 is further configured to perform debinarization processing on the third binary symbol string to determine a value of the third syntax identification information;

[0554] The inverse transform unit 5204 is further configured to determine the transform kernel of the current block when the third syntax identification information indicates that the current block uses the inseparable basic transform mode, and inversely transform the transform coefficients of the current block according to the transform kernel to determine the residual block of the current block.

[0555] In some embodiments, the second determining unit 5201 is further configured to decode the code stream according to at least one fourth context model to determine a fourth binary symbol string of the current block; the debinarization unit 5203 is further configured to perform debinarization processing on the fourth binary symbol string to determine a value of the fourth syntax identification information;

[0556] The inverse transform unit 5204 is further configured to determine the transform kernel of the current block when the fourth syntax identification information indicates that the current block uses the low-frequency inseparable transform mode, and inversely transform the transform coefficients of the current block according to the transform kernel to determine the residual block of the current block.

[0557] In some embodiments, referring to FIG. 52 , the decoder 500 further includes an inverse quantization unit 5005 , wherein:

[0558] The decoding unit 5202 is further configured to decode the code stream and determine the quantization coefficient of the current block;

[0559] The inverse quantization unit 5005 is configured to perform inverse quantization on the quantization coefficients of the current block to determine the transformation coefficients of the current block.

[0560] In some embodiments, the debinarization unit 5203 is further configured to, when the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, execute the steps of decoding the code stream according to the at least one context model to determine the binary symbol string of the current block; and debinarize the binary symbol string to determine the value of the first syntax identification information.

[0561] In some embodiments, the second determination unit 5201 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0562] In some embodiments, the second determination unit 5201 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of the color component in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0563] In some embodiments, the second determination unit 5201 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0564] In some embodiments, the second determination unit 5201 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the brightness component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0565] In some embodiments, the second determining unit 5201 is further configured to determine a second threshold value according to a size parameter of the current block.

[0566] In some embodiments, the second determining unit 5201 is further configured to set the second threshold to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

[0567] In some embodiments, the second determining unit 5201 is further configured to determine a prediction block of the current block; and determine a reconstructed block of the current block according to the prediction block of the current block and the residual block of the current block.

[0568] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0569] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 520 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0570] Based on the composition of the decoder 520 and the computer-readable storage medium, refer to Figure 53, which shows a specific hardware structure diagram of the decoder 520 provided in an embodiment of the present application. As shown in Figure 53, the decoder 520 may include: a second communication interface 5301, a second memory 5302 and a second processor 5303; each component is coupled together through a second bus system 5304. It can be understood that the second bus system 5304 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 5304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 5304 in Figure 53. Among them,

[0571] The second communication interface 5301 is used to receive and send signals when sending and receiving information with other external network elements;

[0572] The second memory 5302 is used to store computer programs that can be run on the second processor 5303;

[0573] The second processor 5303 is configured to, when running the computer program, execute:

[0574] Determine a prediction mode for a current block; determine at least one context model based on the prediction mode of the current block; decode a code stream based on the at least one context model to determine a binary symbol string for the current block; wherein the binary symbol string includes at least one binary symbol; perform debinarization on the binary symbol string to determine a value of first syntax identification information; when the first syntax identification information indicates that the current block uses a first transform mode, determine a transform kernel for the current block, and perform deconverted transform on transform coefficients of the current block based on the transform kernel to determine a residual block for the current block.

[0575] Optionally, as another embodiment, the second processor 5303 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0576] It can be understood that the hardware functions of the second memory 5302 are similar to those of the first memory 5102, and the hardware functions of the second processor 5303 are similar to those of the first processor 5103; they will not be described in detail here.

[0577] This embodiment provides a decoder that first determines at least one context model based on the prediction mode of the current block, and then decodes the binary symbol string corresponding to the first syntax identification information based on the at least one determined context model. In other words, different context models can be selected for intra-frame prediction mode and inter-frame prediction mode, so that the binary symbol string of LFNST / NSPT in the inter-frame prediction mode can be decoded using a method that conforms to its distribution pattern, thereby improving not only compression efficiency but also encoding and decoding performance.

[0578] In yet another embodiment of the present application, referring to FIG54 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG54 , the coding and decoding system 540 may include an encoder 5401 and a decoder 5402 .

[0579] In the embodiment of the present application, the encoder 5401 may be the encoder described in any one of the aforementioned embodiments, and the decoder 5402 may be the decoder described in any one of the aforementioned embodiments.

[0580] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0581] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0582] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0583] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0584] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0585] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0586] In an embodiment of the present application, at the encoding end, a prediction mode of the current block is determined; at least one context model is determined based on the prediction mode of the current block; a value of the first syntax identification information is determined, and the value of the first syntax identification information is binarized to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; at least one binary symbol in the binary symbol string is encoded separately according to at least one context model, and the obtained coded bits are written into the bitstream. At the decoding end, a prediction mode of the current block is determined; at least one context model is determined based on the prediction mode of the current block; the bitstream is decoded according to at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; the binary symbol string is debinarized to determine a value of the first syntax identification information; when the first syntax identification information indicates that the current block uses the first transform mode, a transform kernel of the current block is determined, and the transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block. In this way, both the encoding and decoding ends first determine at least one context model based on the prediction mode of the current block, and then perform encoding and decoding on the binary symbol string corresponding to the first syntax identification information based on the at least one determined context model. In other words, different context models can be selected for intra-frame prediction mode or inter-frame prediction mode, so that the encoding and decoding method that conforms to the distribution law of the binary symbol string of LFNST / NSPT in the inter-frame prediction mode can be used, thereby improving not only compression efficiency but also encoding and decoding performance.

Claims

1. A decoding method, applied to a decoder, the method comprising: Determine the prediction mode of the current block; Determining at least one context model according to a prediction mode of the current block; Decoding the bitstream according to the at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; Debinarizing the binary symbol string to determine a value of the first grammar identification information; When the first syntax identification information indicates that the current block uses a first transform mode, a transform kernel of the current block is determined, and transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block.

2. The method according to claim 1, wherein: The prediction mode of the current block includes: an inter-frame prediction mode and / or an intra-frame prediction mode, and the method further includes: It is determined that at least one context model corresponding to the inter-frame prediction mode is different from at least one context model corresponding to the intra-frame prediction mode.

3. The method according to claim 2, wherein: The performing debinarization processing on the binary symbol string to determine the value of the first grammar identification information includes: When the prediction mode of the current block is the intra-frame prediction mode and / or the inter-frame prediction mode, the binary symbol string is debinarized based on a first method to determine a value of the first syntax identification information.

4. The method according to claim 2, wherein: The performing debinarization processing on the binary symbol string to determine the value of the first grammar identification information includes: When the prediction mode of the current block is the intra prediction mode, performing debinarization processing on the binary symbol string based on a first method to determine a value of the first syntax identification information; or When the prediction mode of the current block is the inter-frame prediction mode, performing debinarization processing on the binary symbol string based on the second method to determine the value of the first syntax identification information; The first method is different from the second method.

5. The method according to claim 1, wherein: The step of decoding the bitstream according to the at least one context model to determine the binary symbol string of the current block includes: When the prediction mode of the current block is an intra-frame prediction mode, decoding a bitstream according to at least one first context model and a fixed-length decoding method to determine a first binary symbol string of the current block; or When the prediction mode of the current block is the inter-frame prediction mode, determining a second binary symbol string of the current block according to at least one second context model and a decoded bitstream in a truncated unary code mode; Wherein, the at least one first context model is different from the at least one second context model.

6. The method according to claim 1, wherein: The determining of the transformation kernel of the current block comprises: Determining a transformation core group of the current block; The transform core of the current block is determined according to the values ​​of the transform core group and the first syntax identification information; wherein, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group.

7. The method according to claim 6, wherein: The determining of the transformation core group of the current block includes: When the prediction mode of the current block is an intra prediction mode, determining that the transform core group includes M candidate transform cores; or, When the prediction mode of the current block is the inter-frame prediction mode, it is determined that the transform core group includes N candidate transform cores; wherein M and N are both positive integers, and M is greater than or equal to N.

8. The method according to claim 7, wherein: The method further comprises: When M is greater than N, setting the N candidate transformation cores to a subset of the M candidate transformation cores; or, When M is equal to N, the N candidate transformation cores are set to be at least partially different from the M candidate transformation cores.

9. The method according to claim 6, wherein: The step of determining a transformation core group of the current block includes: determining at least one texture feature of the current block; A transformation kernel training is performed according to the at least one texture feature to determine a transformation kernel group of the current block.

10. The method according to any one of claims 1 to 9, wherein: The method further comprises: Decoding the code stream to determine a value of second syntax identification information of the current block; When the second syntax identification information indicates that the current block does not use the multiple transform selection mode and the current block satisfies the use of the first transform mode When the condition is used, the steps of decoding the code stream according to the at least one context model to determine the binary symbol string of the current block; and debinarizing the binary symbol string to determine the value of the first syntax identification information.

11. The method according to any one of claims 1 to 9, wherein: The method further comprises: When the first syntax identification information indicates that the current block does not use the first transform mode and the current block meets the use condition of the multiple transform selection mode, decoding the bitstream to determine the value of the second syntax identification information of the current block; When the second syntax identification information indicates that the current block uses a multi-transform selection mode, multiple transform kernels of the current block are determined, and transform coefficients of the current block are inversely transformed according to the multiple transform kernels to determine a residual block of the current block.

12. The method according to any one of claims 1 to 9, wherein: The method further comprises: Decoding the bitstream according to at least one third context model to determine a third binary symbol string of the current block; Performing a debinarization process on the third binary symbol string to determine a value of the third grammar identification information; When the third syntax identification information indicates that the current block uses a non-separable basic transform mode, a transform kernel of the current block is determined, and transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block.

13. The method according to any one of claims 1 to 9, wherein: The method further comprises: Decoding a bitstream according to at least one fourth context model to determine a fourth binary symbol string of the current block; Performing a debinarization process on the fourth binary symbol string to determine a value of fourth grammar identification information; When the fourth syntax identification information indicates that the current block uses a low-frequency inseparable transform mode, a transform kernel of the current block is determined, and transform coefficients of the current block are inversely transformed according to the transform kernel to determine a residual block of the current block.

14. The method according to any one of claims 1 to 9, wherein: The method further comprises: Decoding a bitstream to determine a quantization coefficient of the current block; De-quantize the quantized coefficients of the current block to determine the transform coefficients of the current block.

15. The method according to claim 14, wherein: The method further comprises: When the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, the steps of decoding the code stream according to the at least one context model to determine the binary symbol string of the current block; and debinarizing the binary symbol string to determine the value of the first syntax identification information.

16. The method according to claim 15, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to a first threshold.

17. The method according to claim 15, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to a first threshold.

18. The method according to claim 15, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to the second threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to a second threshold.

19. The method according to claim 15, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to a second threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to a second threshold.

20. The method according to claim 18 or 19, wherein: The method further comprises: The second threshold is determined according to a size parameter of the current block.

21. The method according to claim 18 or 19, wherein: The method further comprises: The second threshold is set to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

22. The method according to any one of claims 1 to 21, wherein: The method further comprises: Determining a prediction block of the current block; A reconstructed block of the current block is determined according to the prediction block of the current block and the residual block of the current block.

23. A coding method, applied to an encoder, the method comprising: Determine the prediction mode of the current block; Determining at least one context model according to a prediction mode of the current block; Determine a value of first grammar identification information, and perform binarization processing on the value of the first grammar identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; At least one binary symbol in the binary symbol string is respectively encoded according to the at least one context model, and the obtained encoded bits are written into a bit stream.

24. The method according to claim 23, wherein: The prediction mode of the current block includes: an inter-frame prediction mode and / or an intra-frame prediction mode, and the method further includes: It is determined that at least one context model corresponding to the inter-frame prediction mode is different from at least one context model corresponding to the intra-frame prediction mode.

25. The method according to claim 24, wherein: The binarizing the value of the first syntax identification information to determine the binary symbol string of the current block includes: When the prediction mode of the current block is the intra-frame prediction mode and / or the inter-frame prediction mode, the value of the first syntax identification information is binarized based on a first method to determine a binary symbol string of the current block.

26. The method according to claim 24, wherein: The binarizing the value of the first syntax identification information to determine the binary symbol string of the current block includes: When the prediction mode of the current block is the intra prediction mode, binarizing the value of the first syntax identification information based on a first method to determine a first binary symbol string of the current block; or When the prediction mode of the current block is the inter-frame prediction mode, binarizing the value of the first syntax identification information based on a second method to determine a second binary symbol string of the current block; The first method is different from the second method.

27. The method according to claim 26, wherein: The first method is a fixed-length encoding method, and the second method is a truncated unary code method.

28. The method of claim 23, wherein: The determining a value of the first grammar identification information includes: If the current block does not use the first transform mode, determining that the value of the first syntax identification information is a first value; If the current block uses the first transform mode, it is determined that the value of the first syntax identification information is a second value.

29. The method of claim 23, wherein: The method further comprises: When the current block uses a first transform mode, determining a transform kernel of the current block; Determine a residual block of the current block, and transform the residual block of the current block according to the transform kernel to determine a transform coefficient of the current block; The transform coefficients of the current block are coded, and the obtained coded bits are written into a bit stream.

30. The method of claim 29, wherein: The determining of the transformation kernel of the current block comprises: Determining a transformation core group of the current block; The transform core of the current block is determined according to the values ​​of the transform core group and the first syntax identification information; wherein, when the current block uses the first transform mode, the value of the first syntax identification information is also used to indicate the number of the transform core of the current block in the transform core group.

31. The method according to claim 30, wherein: The determining of the transformation core group of the current block includes: When the prediction mode of the current block is an intra prediction mode, determining that the transform core group includes M candidate transform cores; or, When the prediction mode of the current block is the inter-frame prediction mode, it is determined that the transform core group includes N candidate transform cores; wherein M and N are both positive integers, and M is greater than or equal to N.

32. The method according to claim 31, wherein: The method further comprises: When M is greater than N, setting the N candidate transformation cores to a subset of the M candidate transformation cores; or, When M is equal to N, the N candidate transformation cores are set to be at least partially different from the M candidate transformation cores.

33. The method of claim 30, wherein: The step of determining a transformation core group of the current block includes: determining at least one texture feature of the current block; A transformation kernel training is performed according to the at least one texture feature to determine a transformation kernel group of the current block.

34. The method according to any one of claims 23 to 33, wherein: The method further comprises: Determine a value of second syntax identification information of the current block; wherein the second syntax identification information is used to indicate whether the current block uses a multi-transform selection mode; The value of the second syntax identification information is encoded, and the obtained encoded bits are written into a bit stream.

35. The method of claim 34, wherein: The method further comprises: When the current block does not use the multiple transform selection mode and the current block meets the use condition of the first transform mode, a step of determining a value of the first syntax identification information is performed.

36. The method according to any one of claims 23 to 33, wherein: The method further comprises: When the current block does not use the first transform mode and the current block meets the use condition of the multiple transform selection mode, a step of determining a value of the second syntax identification information of the current block is performed.

37. The method according to any one of claims 23 to 33, wherein: The method further comprises: Determine a value of third syntax identification information of the current block; wherein the third syntax identification information is used to indicate whether the current block uses a non-separable basic transform mode; The value of the third syntax identification information is encoded, and the obtained encoded bits are written into a bit stream.

38. The method according to any one of claims 23 to 33, wherein: The method further comprises: Determine a value of fourth syntax identification information of the current block; wherein the fourth syntax identification information is used to indicate whether the current block uses a low-frequency inseparable transform mode; The value of the fourth syntax identification information is encoded, and the obtained encoded bits are written into a bit stream.

39. The method of claim 29, wherein: The encoding process is performed on the transform coefficient of the current block, and the obtained encoding bits are written into a bit stream, including: quantizing the transform coefficients of the current block to determine the quantization coefficients of the current block; The quantized coefficients of the current block are coded, and the obtained coded bits are written into a bit stream.

40. The method of claim 29, wherein: The method further comprises: When the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, the step of determining a value of the first syntax identification information and binarizing the value of the first syntax identification information to determine the binary symbol string of the current block is executed.

41. The method of claim 40, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to a first threshold.

42. The method of claim 40, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to a first threshold.

43. The method of claim 40, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to the second threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to a second threshold.

44. The method of claim 40, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to a second threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to a second threshold.

45. The method according to claim 43 or 44, wherein: The method further comprises: The second threshold is determined according to a size parameter of the current block.

46. ​​The method of claim 43 or 44, wherein: The method further comprises: The second threshold is set to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

47. The method of claim 29, wherein: The determining of the residual block of the current block comprises: Determining a prediction block of the current block; A residual block of the current block is determined according to an original block of the current block and a predicted block of the current block.

48. The method of claim 29, wherein: The transforming the residual block of the current block according to the transform kernel to determine the transform coefficient of the current block includes: Performing an inseparable basic transform on the residual block of the current block according to the transform kernel to determine a transform coefficient of the current block.

49. The method of claim 29, wherein: The transforming the residual block of the current block according to the transform kernel to determine the transform coefficient of the current block includes: Performing discrete cosine transform on the residual block of the current block to determine a transform block of the current block; A low-frequency inseparable transform is performed on a transform block of the current block according to the transform kernel to determine a transform coefficient of the current block.

50. A code stream, wherein The code stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: The quantization coefficient of the current block, the value of the first syntax identification information, the value of the second syntax identification information, the value of the third syntax identification information, and the value of the fourth syntax identification information; Among them, the first grammar identification information is used to indicate whether the current block uses the first transformation mode, the second grammar identification information is used to indicate whether the current block uses the multi-transform selection mode, the third grammar identification information is used to indicate whether the current block uses the inseparable basic transformation mode, and the fourth grammar identification information is used to indicate whether the current block uses the low-frequency inseparable transformation mode.

51. An encoder, comprising a first determining unit, a binarization unit and an encoding unit, wherein: The first determining unit is configured to determine a prediction mode of a current block; and determining at least one context model according to the prediction mode of the current block; The binarization unit is configured to determine a value of the first grammar identification information, and perform binarization processing on the value of the first grammar identification information to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; The encoding unit is configured to respectively encode at least one binary symbol in the binary symbol string according to the at least one context model, and write the obtained encoded bits into a bit stream.

52. An encoder, comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 23 to 49 when running the computer program.

53. A decoder, comprising a second determination unit, a decoding unit, a debinarization unit and an inverse transformation unit, wherein: The second determining unit is configured to determine a prediction mode of the current block; and determining at least one context model according to the prediction mode of the current block; The decoding unit is configured to decode the code stream according to the at least one context model to determine a binary symbol string of the current block; wherein the binary symbol string includes at least one binary symbol; The debinarization unit is configured to perform debinarization processing on the binary symbol string to determine a value of the first grammar identification information; The inverse transformation unit is configured to determine a transformation kernel of the current block when the first syntax identification information indicates that the current block uses a first transformation mode, and inversely transform the transformation coefficients of the current block according to the transformation kernel to determine a residual block of the current block.

54. A decoder, comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 22 when running the computer program.

55. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 22 when executed by a first processor, or implements the method according to any one of claims 23 to 49 when executed by a second processor.