Encoding method, decoding method, code stream, encoder, decoder, and storage medium

Abstract

The present application discloses an encoding method, a decoding method, a code stream, an encoder, a decoder, and a storage medium. The method comprises: determining mode parameters of a chroma component of a current block; when the mode parameters satisfy a first condition, determining a first transform kernel of the chroma component of the current block; and determining a transform coefficient of the chroma component of the current block, and performing inverse transform on the transform coefficient of the chroma component of the current block on the basis of the first transform kernel, to determine a residual block of the chroma component of the current block. Therefore, the compression efficiency can be improved.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] As demand for video display quality increases, high-resolution video, such as HD and UHD, has emerged. However, high-resolution video typically contains more information and therefore requires more bandwidth. To reduce bandwidth requirements, video coding standards involving video compression have been introduced.

[0003] and video coding standards for video compression.

[0004] In video coding standards, the current block partitioning techniques include single-tree partitioning and dual-tree partitioning. If the current block uses single-tree partitioning, the luma block can use transform modes such as Multiple Transform Selection (MTS), Low Frequency Non-Separable Transform (LFNST), or Non-Separable Primary Transform (NSPT), but the chroma block uses the default Discrete Cosine Transform (DCT), which is not conducive to improving compression efficiency.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve compression efficiency.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0009] determining mode parameters for the chroma components of the current block;

[0010] When the mode parameter satisfies a first condition, determining a first transform kernel of the chrominance component of the current block;

[0011] A transform coefficient of a chroma component of a current block is determined, and an inverse transform is performed on the transform coefficient of the chroma component of the current block according to a first transform kernel to determine a residual block of the chroma component of the current block.

[0012] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] determining mode parameters for the chroma components of the current block;

[0014] When the mode parameter satisfies a first condition, determining a first transform kernel of the chrominance component of the current block;

[0015] Determine a residual block of a chrominance component of a current block, and transform the residual block of the chrominance component of the current block according to a first transform kernel to determine a transform coefficient of the chrominance component of the current block;

[0016] The transform coefficients of the chrominance components of the current block are coded, and the resulting coded bits are written into the bitstream.

[0017] In a third aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: a quantization coefficient of a chrominance component of a current block and a value of a first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block is allowed to use a first transform mode and the corresponding transform kernel index used.

[0018] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a transforming unit, and an encoding unit, wherein:

[0019] a first determining unit configured to determine a mode parameter of a chroma component of a current block; and determine a first transform kernel of the chroma component of the current block when the mode parameter satisfies a first condition;

[0020] a transform unit configured to determine a residual block of a chroma component of a current block, and transform the residual block of the chroma component of the current block according to a first transform kernel to determine a transform coefficient of the chroma component of the current block;

[0021] The encoding unit is configured to perform encoding processing on the transform coefficients of the chrominance components of the current block and write the obtained encoding bits into the bit stream.

[0022] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor, wherein:

[0023] a first memory for storing a computer program capable of running on the first processor;

[0024] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0025] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second determination unit and an inverse transformation unit, wherein:

[0026] a second determining unit configured to determine a mode parameter of the chroma component of the current block; and determine a first transform kernel of the chroma component of the current block when the mode parameter satisfies a first condition;

[0027] The inverse transform unit is configured to determine transform coefficients of the chrominance components of the current block, and perform inverse transform on the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine a residual block of the chrominance components of the current block.

[0028] In a seventh aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor, wherein:

[0029] a second memory for storing a computer program capable of running on the second processor;

[0030] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0031] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method described in the first aspect or the method described in the second aspect.

[0032] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. At the encoding end, the mode parameters of the chroma components of the current block are determined; when the mode parameters meet a first condition, the first transform kernel of the chroma components of the current block is determined; the residual block of the chroma components of the current block is determined, and the residual block of the chroma components of the current block is transformed according to the first transform kernel to determine the transform coefficients of the chroma components of the current block; the transform coefficients of the chroma components of the current block are encoded, and the obtained encoded bits are written into the code stream. At the decoding end, the mode parameters of the chroma components of the current block are determined; when the mode parameters meet the first condition, the first transform kernel of the chroma components of the current block is determined; the transform coefficients of the chroma components of the current block are determined, and the transform coefficients of the chroma components of the current block are inversely transformed according to the first transform kernel to determine the residual block of the chroma components of the current block. In this way, both the encoding and decoding ends first determine the mode parameters for the current block. When the mode parameters meet the first condition, the first transform kernel for the chroma components of the current block is determined, and the residual block of the chroma components of the current block is transformed based on the first transform kernel. In other words, the determination of the first transform kernel is related to the mode parameters, enabling the application of MTS / LFNST / NSPT transforms to the chroma components of a single tree partition, thereby improving not only compression efficiency but also encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a flow chart diagram of a hybrid coding framework;

[0034] FIG2 is a schematic diagram of the structure of a group of pictures GOP;

[0035] FIG3 is a schematic diagram of a block partition structure;

[0036] FIG4 is a schematic diagram of a block position relationship in the spatial domain / temporal domain of a current block;

[0037] FIG5 is a schematic diagram of motion information of a current block and a co-located block;

[0038] FIG6 is a schematic diagram of motion displacement using a co-located reference image;

[0039] FIG7 is a schematic diagram of a distribution using MVD characteristics;

[0040] FIG8 is a schematic diagram of an affine motion using 2 or 3 control points;

[0041] FIG9 is a schematic diagram of affine motion based on derivation of motion vectors of sub-blocks;

[0042] FIG10 is a schematic diagram of weights of various modes under a GPM mode;

[0043] FIG11 is a schematic diagram of motion information for bilateral matching;

[0044] FIG12 is a schematic diagram of template matching of a current block;

[0045] FIG13 is a schematic diagram of reference pixels of a current block;

[0046] FIG14 is a schematic diagram of multiple reference rows of a current block;

[0047] FIG15 is a schematic diagram 1 of multiple prediction modes corresponding to an intra-frame prediction;

[0048] FIG16 is a second schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0049] FIG17 is a third schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0050] FIG18 is a fourth schematic diagram of multiple prediction modes corresponding to an intra-frame prediction;

[0051] FIG19 is a schematic diagram of encoding of screen content;

[0052] FIG20 is a schematic diagram of a histogram of gradient and intra prediction mode;

[0053] FIG21 is a schematic diagram of weighted fusion of three intra-frame prediction modes;

[0054] FIG22 is a schematic diagram of reference pixels of chrominance components and luminance components;

[0055] FIG23 is a schematic diagram of a grouping of a multi-model CCLM;

[0056] FIG24 is a schematic diagram of a multi-model CCLM prediction;

[0057] FIG25 is a schematic diagram of a reference area for intra-frame CCCM;

[0058] FIG26 is a schematic diagram of brightness gradient calculation of a GLM;

[0059] FIG27 is a schematic diagram of an inter-frame CCCM prediction;

[0060] FIG28 is a schematic diagram of a DCT transformation;

[0061] FIG29 is a schematic diagram of a base image of a DCT transformation;

[0062] FIG30 is a schematic diagram of a process flow without LFNST transformation;

[0063] FIG31 is a schematic diagram of a process flow with LFNST transformation;

[0064] FIG32 is a detailed flowchart of a LFNST transformation;

[0065] FIG33 is a schematic diagram of a base image of multiple transformation kernel groups;

[0066] FIG34 is a schematic diagram of a base image of NSPT transformation;

[0067] FIG35 is a schematic diagram of a structure in which brightness and chromaticity are divided separately;

[0068] FIG36 is a schematic diagram of a network architecture of a video codec provided in an embodiment of the present application;

[0069] FIG37 is a schematic block diagram of a system composition of an encoder provided in an embodiment of the present application;

[0070] FIG38 is a schematic block diagram of a system composition of a decoder provided in an embodiment of the present application;

[0071] FIG39 is a flowchart diagram 1 of a decoding method provided in an embodiment of the present application;

[0072] FIG40 is a second flow chart of a decoding method provided in an embodiment of the present application;

[0073] FIG41 is a third flow chart of a decoding method provided in an embodiment of the present application;

[0074] FIG42 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0075] FIG43 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0076] FIG44 is a third flow chart of an encoding method provided in an embodiment of the present application;

[0077] FIG45 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0078] FIG46 is a schematic diagram of the hardware structure of an encoder provided in an embodiment of the present application;

[0079] FIG47 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0080] FIG48 is a schematic diagram of the hardware structure of a decoder provided in an embodiment of the present application;

[0081] Figure 49 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0084] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0085] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0086] In video images, a coding block (CB) is generally represented by a first color component, a second color component, and a third color component. These three color components are a luminance component, a blue chrominance component, and a red chrominance component. Specifically, the luminance component is typically represented by the symbol Y, the blue chrominance component is typically represented by the symbols Cb or U, and the red chrominance component is typically represented by the symbols Cr or V. Thus, video images can be represented in either the YCbCr or YUV format.

[0087] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0088] H.265 / High Efficiency Video Coding (HEVC);

[0089] H.266 / Versatile Video Coding (VVC);

[0090] VVC Test Model (VTM), a reference software testing platform for VVC;

[0091] A platform that improves compression performance after VVC (Enhanced Compression Model, ECM);

[0092] Joint Video Experts Team (JVET);

[0093] Coding Unit (CU);

[0094] Coding Tree Unit (CTU);

[0095] Largest Coding Unit (LCU);

[0096] Motion Vector (MV);

[0097] Prediction Unit (PU);

[0098] Transform Unit (TU);

[0099] Fusion technology (Merge);

[0100] Skip technology (Skip);

[0101] Quantization Parameter (QP);

[0102] Merge with Motion Vector Difference (MMVD)

[0103] Motion Vector Prediction (MVP);

[0104] Temporal Motion Vector Prediction (TMVP);

[0105] Subblock-based Temporal Motion Vector Prediction (SbTMVP);

[0106] Discrete Cosine Transform (DCT);

[0107] Discrete Sine Transform (DST);

[0108] Multiple Transform Selection (MTS);

[0109] Low Frequency Non-Separable Transform (LFNST);

[0110] Non-Separable Primary Transform (NSPT);

[0111] Context-based Adaptive Binary Arithmetic Coding (CABAC).

[0112] Currently, common video codec standards all adopt a block-based hybrid coding framework. Each image, sub-image, or frame in a video is divided into square maximum coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 256×256, 128×128, 64×64, etc.). Each LCU or CTU can be divided into rectangular CUs according to a set of rules. Coding units may also be divided into prediction units (PUs) and transform units (TUs). Specifically, as shown in Figure 1, the hybrid coding framework includes modules such as prediction, transform / quantization, entropy coding, inverse quantization / inverse transform, and in-loop filtering. The prediction module can include intra-frame prediction and inter-frame prediction, and inter-frame prediction can include motion estimation and motion compensation. Because adjacent pixels within a video image are highly correlated, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Furthermore, because adjacent images in a video image have strong similarities, inter-image prediction is used in video coding and decoding to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0113] The basic process of a video codec is as follows: On the encoder side, an image is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations as the decoder side to obtain a decoded image. The decoded image can serve as a reference image for inter-frame prediction of subsequent images. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoder are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing the bit stream and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The embodiment of the present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0114] In addition, in the embodiments of the present application, the current block (CB) can be the current coding unit, the current prediction unit, or the current transform unit. Due to the need for parallel processing, the image can be divided into slices, etc. Slices in the same image can be processed in parallel, that is, there is no data dependency between them. "Frame" is a commonly used term, and it can generally be understood that a frame is an image. The frame described in the embodiments of the present application can also be replaced by an image or a slice, etc.

[0115] It can be understood that in the prediction technology, intra-frame prediction and inter-frame prediction are described in detail below respectively.

[0116] (1) Inter-frame prediction.

[0117] Inter-frame prediction uses temporal correlation to eliminate redundancy. To minimize visual lag, typical video frame rates range from 30, 50, 60, or even 120 frames per second. In such videos, adjacent frames within the same scene are highly correlated. Inter-frame prediction leverages this correlation to predict the current encoding content, referencing the content of previously encoded and decoded frames. Inter-frame prediction can significantly improve encoding performance.

[0118] The most basic inter-frame prediction method is translational prediction. Translational prediction assumes that the content being predicted translates between the current image and the reference image. For example, if the content of the current block (coding unit or prediction unit) translates between the current image and the reference image, a motion vector can be used to find this content in the reference image and use it as the prediction block for the current block. Translational motion is a significant feature of video, and can be used to handle static backgrounds, moving objects, and camera movements.

[0119] Some content in natural videos doesn't simply translate; for example, the translation process involves subtle changes in shape and color. Bidirectional prediction uses two reference blocks from a reference image and performs a weighted average of these blocks to produce a predicted block that is as similar as possible to the current block. For example, in certain scenes, a weighted average of two reference blocks from before and after the current frame may produce a prediction that is more similar to the current block than a single reference block. This improves compression performance over unidirectional prediction.

[0120] The picture playback order (Picture Order Count, POC) can be used as an identifier of the image. In a video sequence, each image has a unique POC. In the embodiment of the present application, it is considered that the order of the POC and the playback order are the same. The P picture (P Frame) is an image that can only be predicted using the reference image whose POC is before the current image. The current reference image has only one reference picture list (Reference Picture List, RPL), denoted as RPL0. The reference picture list RPL0 contains all reference images whose POC is before the current image. The B picture (B Frame) is an image that can be predicted using the reference image whose POC is before the current image and the reference image whose POC is after the current image. The B picture has two reference picture lists, denoted as RPL0 and RPL1. One configuration method is that RPL0 contains all reference images whose POC is before the current image, and RPL1 contains all reference images whose POC is after the current image. For a current block, one can reference only the reference block of a certain image in RPL0 (also known as forward prediction); only the reference block of a certain image in RPL1 (also known as backward prediction); or simultaneously reference the reference block of a certain image in RPL0 and a reference block of a certain image in RPL1 (also known as bidirectional prediction). A simple method for simultaneously referencing two reference blocks is to average the pixels at each corresponding position in the two reference blocks to obtain the prediction block for the current block. Later, B-pictures no longer require that RPL0 only contain reference images with a Point of Occurrence (POC) before the current image, and RPL1 only contain reference images with a Point of Occurrence (POC) after the current image. Therefore, RPL0 can also contain reference images with a Point of Occurrence (POC) after the current image, and RPL1 can also contain reference images with a Point of Occurrence (POC) before the current image. The current block can then reference both reference images with a Point of Occurrence (POC) before the current image and reference images with a Point of Occurrence (POC) after the current image. This type of B-picture is also called a generalized B-picture.

[0121] Because the encoding and decoding order of the random access (RA) configuration is different from the POC order, B pictures can reference information before and after the current picture, significantly improving encoding performance. For example, a classic RA group of pictures (GOP) structure is shown in Figure 2. In Figure 2, arrows represent reference relationships. Since I pictures do not require reference pictures, after an I picture with a POC of 0 is decoded, a P picture with a POC of 4 will be decoded. When decoding a P picture with a POC of 4, the I picture with a POC of 0 can be referenced. After decoding a P picture with a POC of 4, a B picture with a POC of 2 is decoded. When decoding a B picture with a POC of 2, the I picture with a POC of 0 and the P picture with a POC of 4 can be referenced, and so on. Thus, according to Figure 2, when the POC order is {0 1 2 3 4 5 6 7 8}, the corresponding decoding order is {0 3 2 4 1 7 6 8 5}.

[0122] The encoding and decoding order of the Low Delay (LD) configuration is the same as the POC order. Therefore, the current image can only reference information before the current image. The Low Delay configuration is divided into Low Delay P and Low Delay B. Low Delay P is the traditional Low Delay configuration. Its typical structure is IPPP..., that is, an I picture is encoded and decoded first, and subsequent pictures are all P pictures. The typical structure of Low Delay B is IBBB..., which differs from Low Delay P in that each inter-frame picture is a B picture, that is, using two reference picture lists, the current block can simultaneously reference the reference block of a picture in RPL0 and the reference block of a picture in RPL1.

[0123] Generally speaking, the RA configuration has higher compression efficiency than the LD configuration, and the LDB configuration has higher compression efficiency than the LDP configuration. This is partly because bidirectional prediction can refer to backward information and partly because bidirectional prediction can reduce prediction errors through techniques such as weighted averaging.

[0124] A reference image list of the current image can have a maximum of several reference images, such as 2, 3 or 4. When encoding a current image, which reference images are in RPL0 and RPL1 is determined by a certain configuration or algorithm, which is not the focus of the embodiments of this application. However, the same reference image may appear in RPL0 and RPL1 at the same time. That is, the codec allows the current block to refer to two reference blocks of the same reference image at the same time. The codec usually uses the index value (index, idx) in the reference image list to correspond to the reference image. If the length of a reference image list is 4, the index has four values ​​0, 1, 2, and 3. For example, if the RPL0 of the current image has 4 reference images with POCs of 5, 4, 3, and 0. Then index 0 of RPL0 is the reference image with POC 5, index 1 of RPL0 is the reference image with POC 4, index 2 of RPL0 is the reference image with POC 3, and index 3 of RPL0 is the reference image with POC 0.

[0125] It can also be understood that inter-frame prediction uses motion information to represent "motion". Basic motion information includes reference picture information and motion vector (MV) information. In order for a block to use bidirectional prediction, it is naturally necessary to find two reference blocks, so two sets of reference image information and motion vector information are needed. Each of these sets can be understood as a unidirectional motion information, and combining these two sets together forms a bidirectional motion information. In specific implementation, unidirectional motion information and bidirectional motion information can use the same data structure, but the two sets of reference frame information and motion vector information of the bidirectional motion information are both valid, while one set of reference frame information and motion vector information of the unidirectional motion information is invalid. The valid can also be said to be "used", and the invalid can also be said to be "not used".

[0126] VVC supports two reference picture lists, denoted as RPL0 and RPL1. For the above-mentioned bidirectional motion information, VVC uses the reference picture index refIdxL0 corresponding to RPL0, the motion vector mvL0 corresponding to RPL0, the reference picture index refIdxL1 corresponding to RPL1, and the motion vector mvL1 corresponding to RPL1. The reference picture index corresponding to RPL0 and the reference picture index corresponding to RPL1 here can be understood as the above-mentioned reference picture information. VVC uses two identification information to indicate whether to use the motion information corresponding to RPL0 and whether to use the motion information corresponding to RPL1, respectively denoted as predFlagL0 and predFlagL1. It can also be understood that predFlagL0 and predFlagL1 indicate whether the above-mentioned unidirectional motion information is "valid". Therefore, although VVC does not explicitly mention the data structure of motion information, it uses the reference picture index, motion vector and "valid" flag corresponding to each reference picture list to represent the motion information. The VVC standard doesn't mention motion information, but instead uses motion vectors. The reference image index and the flag indicating whether to use the corresponding motion information can be considered as appendages of the motion vector. This document uses "motion information" for ease of description, but it should be understood that "motion vectors" can also be used. "Motion information" can also be called "motion parameters."

[0127] For a two-dimensional image, a motion vector can be represented by (x, y), with a horizontal component and a vertical component. Since videos are represented in pixels, there are distances between pixels. The motion of an object in adjacent images may not always correspond to a whole-pixel distance. For example, in a distant video, the distance between two pixels is 1 meter for the distant object, while the object moves 0.5 meters between two frames. This scenario cannot be well represented using a whole-pixel motion vector. Therefore, motion vectors can be expressed at the sub-pixel level, with precision of 1 / 2, 1 / 4, 1 / 8, or 1 / 16 pixels, to represent motion more precisely. Interpolation is then used to obtain the pixel value at the sub-pixel position in the reference image.

[0128] Both the unidirectional and bidirectional predictions in the aforementioned translation prediction are block-based, such as coding units (CUs) or prediction units (PUs). This means that prediction is performed using a pixel matrix as the unit. The most basic block is a rectangular block, such as a square or rectangle. Video codec standards such as HEVC and VVC allow the encoder to determine the size and division of CUs and PUs based on the video content. Regions with simple textures or motion tend to use larger blocks, while regions with complex textures or motion tend to use smaller blocks. The deeper the block division level, the more complex the blocks that more closely resemble the actual texture or motion, but the corresponding overhead for representing these divisions increases. Motion information may also need to be transmitted in the bitstream. Furthermore, generally, the finer the block division, the greater the motion information overhead.

[0129] The most primitive method of representing motion information directly writes the complete motion information. Later, experts discovered that motion vectors can be represented using motion vector prediction (MVP) plus motion vector difference (MVD), i.e., MV = MVP + MVD. The more accurate the MVP, the smaller the MVD, which in turn reduces the bitstream overhead.

[0130] It's also understandable that for merge mode, each inter-coded block requires motion information. To simplify matters, we assume that CU division equals PU division equals TU division, meaning that each coding unit has a prediction unit (PU) and a transform unit (TU) of the same size and position. In reality, with the increased flexibility of CU division, VVC tends to decouple PU and TU division from HEVC. Differences in any of the steps involved in prediction, transform, quantization, and entropy coding can lead to CU division. For example, if two regions have different motion information, the encoder may divide them into different CUs. Alternatively, if two regions have identical or similar motion information but significantly different residual characteristics, the encoder may also divide them into different CUs. The division method is determined based on overall compression efficiency, not solely on a single factor. Consequently, the same object or regions with identical or similar motion may be divided into different CUs.

[0131] Figure 3 is a schematic diagram of the HEVC block division structure. As shown in Figure 3, (a) is the original image. In (a), there is an iron rod moving in the direction indicated by the arrow, and the background area moves less. (b) is the block division of HEVC, and (c) is the block boundary with the same motion information in (b) removed. It can be seen that many adjacent blocks use the same motion information. In this case, if the motion information is encoded separately for each block, it will result in obvious waste. The complete motion information of VVC mentioned above includes the reference image index, MV and the flag of whether it is used for RPL0, and the reference image index, MV and the flag of whether it is used for RPL1. The basic principle of the merge mode is that the current block can inherit the motion information of the adjacent blocks, including the reference image information and motion vector information.

[0132] Merge mode can build a merge candidate list. If the current block uses merge mode, an index can be used to indicate which motion information to merge with the current block, eliminating the need to encode the complete motion information. When building the merge candidate list, you can include motion information of spatially adjacent blocks of the current block, motion information in the temporal domain, motion information of non-adjacent blocks in the spatial domain, motion information of non-adjacent blocks in the temporal domain, motion information based on historical information, and synthesized motion information.

[0133] The spatially adjacent blocks refer to blocks adjacent to the current block in the same image, and the spatially non-adjacent blocks refer to blocks non-adjacent to the current block in the same image. The temporal motion information and the motion information of temporally non-adjacent blocks refer to the motion information at a specified position in the collocated reference image. For example, as shown in FIG4 , the black-filled block is the current block, where positions 1, 2, 3, 4, and 5 are the positions of spatially adjacent blocks used for merging, and the corresponding positions of the other dot-filled blocks are the positions of spatially non-adjacent blocks used for merging. Position 6 is the position used for temporal motion information. If the corresponding position at the lower right corner of the current block is unavailable, the position corresponding to the center of the current block is used. The corresponding positions of the other grid-filled blocks are the positions used for motion information of temporally non-adjacent blocks. Temporal motion information is derived based on the motion information at the corresponding positions in the collocated reference image. The specific derivation method is described below. It should be noted that the background grid in FIG4 is only a schematic representation of pixel coordinates and does not represent a specific block division.

[0134] Historical motion information is independent of position. The codec maintains a first-in, first-out list of motion information. Each time a block is encoded or decoded, the codec updates the list with the motion information of that block, ensuring that there is no duplication with existing motion information in the list. Historical motion information is obtained from this list.

[0135] It can also be understood that for the derivation of temporal motion information (vector), temporal motion information prediction is used as a supplement to spatial motion information prediction. Generally speaking, the correlation between adjacent regions in the same image is stronger than the correlation between regions in different images. However, there are also cases where temporal motion information is more useful. For example, if the current block and the surrounding adjacent blocks in the current image belong to different objects and have completely different motions, the motion of the block belonging to the same object as the current block in a reference image can provide better motion information prediction for the current block.

[0136] The motion vector of the co-located block in the co-located reference image (here, the block that obtains temporal motion information is called the co-located block) is the vector from the co-located reference image col_pic to the reference image col_ref of the co-located block. Specifically, as shown in Figure 5, for the current block, the required motion vector is the vector from the current image curr_pic to the reference image curr_ref of the current block. Let the POC distance between col_pic and col_ref be td, and the POC distance between curr_pic and curr_ref be tb. Assuming that the motion from the co-located block to the motion of the current block is unchanged, the scaling ratio can be determined based on td and tb. If the motion vector of the co-located block is (col_mv_x, col_mv_y), the temporal motion vector prediction (tmvp_x, tmvp_y) can be derived as follows: tmvp_x = col_mv_x * tb / td, tmvp_y = col_mv_y * tb / td.

[0137] In VVC, the smallest unit for storing motion information in a co-located reference image is 4×4. That is, each 4×4 sub-block stores a set of motion information. It's understandable that, if hardware implementation costs are not considered, a co-located reference image can also store a set of motion information per pixel.

[0138] It's also understandable that VVC introduces sub-block-based temporal motion vector prediction, or SbTMVP. Conventionally, MVP and TMVP are applied to the entire block, meaning the entire block shares the same MVP. However, SbTMVP is based on sub-blocks, allowing it to generate a unique MVP for each sub-block. This is the fundamental difference between SbTMVP and TMVP.

[0139] On the other hand, TMVP uses the position of the lower right corner of the current block or the position of the current block center to locate the co-located block, while SbTMVP finds a motion offset based on the motion of the surrounding blocks to determine the position. In VVC, if the block at position A1 refers to the co-located reference image, then the motion offset is set to the motion vector of A1 using the co-located reference image. Otherwise, the motion offset is set to (0, 0). As shown in Figure 6, the position is found according to the motion offset, and then the MV corresponding to the position of each sub-block in the "co-located block" is scaled to obtain the MVP of each sub-block.

[0140] It can also be understood that for the fusion technology using motion vector difference, namely MMVD, the motion information in the merge candidate list directly selected by the Merge mode is used as the motion information of the current block. In actual videos, there may sometimes be some differences between the actual motion vector of the current block and the motion vector in the selected merge candidate list. MMVD is a special merge mode in VVC, which uses an efficient method to encode the MVD in this case. Ordinary Merge does not require encoding and decoding of MVD. Ordinary inter mode requires direct encoding and decoding of MVD. MMVD takes advantage of the fact that MVD is more distributed in a single horizontal direction or a single vertical direction, with more MVDs for smaller values ​​and fewer MVDs for larger values, as shown in Figure 7. In Figure 7, circles of different shapes can represent MVDs of different values.

[0141] MMVD can only represent MVDs of specific values ​​in specific directions, and it cannot represent any MVD. It uses mmvd_direction_idx to represent the direction of the MVD, which can also be understood as whether the x and y values ​​of the MVD are non-zero and their positive and negative signs. mmvd_distance_idx represents the absolute value of the non-zero x and y values ​​of the MVD, MmvdDistance.

[0142] Here, Table 1 shows a schematic relationship between mmvd_distance_idx[x0][y0] and MmvdDistance[x0][y0].

[0143] Table 1

[0144] Among them, ph_mmvd_fullpel_only_flag is an image header flag that can set two different combinations of MMVD.

[0145] Here, Table 2 shows a schematic relationship between mmvd_direction_idx[x0][y0] and MmvdSign[x0][y0].

[0146] Table 2

[0147] Among them, the MVD of MMVD is obtained as follows:

[0148] MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][0];

[0149] MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][1].

[0150] It can also be understood that for affine Affine, the simplest and most commonly used translation motion was introduced above. In the real world, there are not only translations, but also many forms of motion such as reduction, enlargement, rotation, perspective (perspective: objects close to the lens appear large, objects far from the lens appear small), and many irregular forms of motion. Affine can be used to represent more complex motion than translation. As shown in Figure 8, Affine uses a linear model to calculate the motion vector of each sub-block or each pixel in the current block based on the motion vectors of 2 control points (4 parameters, a motion vector includes two parameters x and y) or 3 control points (6 parameters). Among them, (a) provides the case of 2 control points, such as v0 and v1; (b) provides the case of 3 control points, such as v0, v1, and v2.

[0151] For the 4-parameter affine model, the motion vector at the (x, y) position in the current block is derived as follows:

[0152] For the 6-parameter affine model, the motion vector at the (x, y) position in the current block is derived as follows:

[0153] Among them, (mv 0x ,mv 0y ) is the motion vector of the control point at the upper left corner of the current block, (mv 1x ,mv 1y ) is the motion vector of the control point in the upper right corner of the current block, (mv 2x ,mv 2y ) is the motion vector of the control point at the lower left corner of the current block.

[0154] To simplify hardware implementation complexity, Affine, used in VVC, divides the current block into 4×4 sub-blocks, calculates an MV for each sub-block, and performs motion compensation. Figure 9 is a schematic diagram of how Affine derives motion vectors based on sub-blocks. It is understandable that with the improvement of hardware processing capabilities, Affine can also perform pixel-based processing. That is, a motion vector is derived for each pixel, and motion compensation is performed on the pixel based on this motion vector. Here, Affine only needs a few control points to derive a separate motion vector for each sub-block or each pixel, allowing for more precise predictions compared to whole-block motion compensation. Furthermore, compared to dividing the CU into smaller units, Affine has much less overhead.

[0155] It can also be understood that for the Geometric Partitioning Mode (GPM), HEVC supports a maximum CTU of 64×64 and can recursively perform quadtree division. VVC supports a more flexible block division method than HEVC, supporting a maximum CTU of 128×128, including quadtree, ternary tree and binary tree division. These division methods. Although block division is becoming more and more flexible, whether it is CU, PU or TU, it can only be divided into rectangular blocks. It should be noted that VVC has weakened the division of PU and TU. The boundaries of texture or motion in natural videos are diverse. For example, when encountering an oblique object boundary, if you simply use rectangular blocks to approach the boundary, you will divide it into many small blocks, which will significantly increase the overhead. The GPM geometric division mode can better handle textures and boundaries in natural videos.

[0156] GPM uses two prediction blocks of the same size as the current block. Some pixel locations in the GPM prediction block use 100% of the pixel values ​​corresponding to the first prediction block, while others use 100% of the pixel values ​​corresponding to the second prediction block. In the boundary region, or transition region, pixel values ​​from both prediction blocks are used in a certain proportion. The weights in the boundary region also gradually transition. Of course, for scenarios such as screen content encoding, the transition region can be omitted. The specific distribution of these weights is determined by the GPM's "partitioning" mode. The weight of each pixel position is determined based on the GPM's partitioning mode. However, in certain situations, such as very small block sizes, some GPM modes may not guarantee that some pixel locations will use 100% of the pixel values ​​corresponding to the first prediction block, while others will use 100% of the pixel values ​​corresponding to the second prediction block. Alternatively, GPM can be considered to use two prediction blocks of different block sizes, taking the required portion from each and discarding any portions with zero weights.

[0157] Figure 10 shows the weights of the 64 GPM modes in VVC on a square block. As shown in Figure 10, black indicates a weight of 0% for the position corresponding to the first prediction block, white indicates a weight of 100%, and gray areas, depending on the depth of the color, indicate a weight greater than 0% and less than 100% for the position corresponding to the first prediction block. The weight of the position corresponding to the second reference block is 100% minus the weight of the position corresponding to the first reference block.

[0158] GPM can be considered a prediction mode or method, as it ultimately produces a prediction block. It can also be considered a "partitioning" mode, simulating a partitioning of the prediction block, similar to implementing PU partitioning without actually partitioning the prediction block. The first and second prediction blocks used in GPM can be prediction blocks generated by intra-frame prediction, inter-frame unidirectional prediction, or inter-frame bidirectional prediction.

[0159] Understandably, the bitrate of general consumer video is limited, so video compression typically seeks a compromise between bitrate overhead and distortion. Taking block partitioning as an example, for the same content, within a certain range, finer the partitioning, the higher the overhead and lower the distortion; coarser the partitioning, the lower the overhead and higher the distortion. Taking motion information encoding as an example, for the same content, within a certain range, more precise motion information leads to higher overhead and lower distortion; coarser motion information leads to lower overhead and higher distortion. Some decoder-side methods utilize decoder-side information for processing and calculation without incurring overhead, thereby improving motion information, enhancing prediction, and reducing distortion. This overhead-free approach means that the encoder does not require instructions based on the original image; processing is performed automatically based on available information. Two typical decoder-side methods in VVC are decoder-side motion vector refinement (DMVR) and bidirectional optical flow (BDOF), which are described in detail below.

[0160] In one possible implementation, for DMVR, one condition for starting DMVR in VVC is that the two reference images of the current block come from the front and back of the current image respectively, and the distances between the two reference images and the current image are equal. Another starting condition is that the current CU uses the whole-block merge mode (including skip), which does not include sub-block-based merges such as SbTMVP and affine merge, because the motion vector in the merge mode is prone to inaccuracy. There are some other conditions that are not described here. DMVR in VVC uses bilateral matching (BM), which is to calculate the matching cost for the reference blocks on both sides, such as the sum of absolute differences (SAD). DMVR searches for the matching cost of the MVs around the original MV. When moving, the MVs of the two reference images are moved in a mirrored manner, that is, the MVs are moved on the basis of their respective original MVs. diff , move the other side - MV diff , as shown in Figure 11. In Figure 11, the two reference images include reference image L0 (refPic in ListL0) and reference image L1 (refPic in ListL1). The search also supports sub-pixel search, so DMVR may find an MV with higher accuracy than the original MV. The search is performed according to certain rules. Generally, the integer pixel MV within a certain range is searched first to find the integer pixel MV with the lowest matching cost, and then the sub-pixel MV is searched based on the integer pixel MV. If an MV with a lower matching cost than the original MV is found, the MV with the lower matching cost is used for motion compensation prediction. The MV improved by DMVR can theoretically be used to store MVs and surrounding blocks. For example, when constructing a merge candidate list for the current block, if the MV of the surrounding blocks is improved by DMVR, using the improved MV to construct the merge candidate list can achieve better compression effect. However, due to hardware implementation considerations, VVC does not do this.

[0161] DMVR can be processed on a sub-block basis. In fact, in VVC, if the horizontal or vertical size of a block is larger than 16 pixels, it will be divided into sub-blocks of 16 pixels. This is partly due to the complexity of hardware implementation, because DMVR needs to be searched at the decoding end, and limiting the size of sub-blocks can reduce the cost of caching. On the other hand, dividing the processing into sub-blocks provides better flexibility. Each sub-block can independently improve the MV, which to a certain extent achieves the effect of improving the division accuracy, which also improves the compression efficiency.

[0162] In another possible implementation, BDOF is also a typical decoder-side method. As its name suggests, BDOF improves MV and prediction based on the principle of optical flow. Optical flow is the instantaneous velocity of pixels moving across the observation imaging plane for a moving object. Optical flow has some basic assumptions, such as constant brightness (i.e., the brightness of the same object does not change when it moves between different images), and temporal continuity or small motion. This means that changes in time do not cause drastic changes in the object's position.

[0163] A condition for starting BDOF in VVC is that the two reference images of the current block come from the previous and next images respectively, and the distance between the two reference images and the current image is equal. For each 4×4 sub-block, VVC will derive a motion vector deviation (v x ,v y ), which is calculated by minimizing the difference between the predicted values ​​in the two directions. This motion vector deviation is also used to adjust the predicted value in the corresponding sub-block. The derivation process is as follows:

[0164] First, calculate the horizontal and vertical gradients of the two prediction blocks and The details are as follows:

[0165] Among them, I (k) (i, j) is the predicted value of the coordinate (i, j) in the reference image list k, k = 0, 1, and shift1 is calculated according to the bit depth bitDepth of the brightness, shift1 = max(6, bitDepth-6).

[0166] Next, calculate S1, S2, S3, S5, and S6 as follows: S1 = ∑ (i,j)∈Ω Abs(ψ x (i,j)) (5) S2=∑ (i,j)∈Ω ψ x (i,j)·Sign(ψ y (i,j)) (6) S3=∑ (i,j)∈Ω θ(i,j)·Sign(ψ x (i,j)) (7) S5=∑ (i,j)∈Ω Abs(ψ y (i,j)) (8) S6=∑ (i,j)∈Ω θ(i,j)·Sign(ψ y (i,j)) (9)

[0167] in, θ(i,j)=(I (1) (i,j)>>n b )-(I (0) (i,j)>>n b ) (12)

[0168] Where Ω is a 6×6 window around the current 4×4 sub-block, n a is min(1,bitDepth-11), n b is min(4,bitDepth-8).

[0169] Again, the motion vector deviation (v x ,v y ) is calculated as follows:

[0170] in, th′ BIO =2max(5,BD-7), is rounded down, BD is bit depth bitDepth.

[0171] Again, based on the motion vector deviation and gradient, each prediction value within the 4×4 sub-block is adjusted as follows:

[0172] The final predicted value of BDOF is calculated as follows: pred BDOF (x,y)=(I (0) (x,y)+I (1) (x,y)+b(x,y)+o offset )>>shift (16)

[0173] Among them, offset and shift are calculated based on the bit depth of brightness. a 、n b Both scalar and shift are processed to reduce the bit width during the calculation process.

[0174] In this way, the motion vector deviation of BDOF can achieve high precision, making the prediction more accurate, and the sub-block-based processing also improves flexibility. These two aspects are similar to DMVR.

[0175] It's also understood that both DMVR and BDOF improve motion vectors. DMVR is based on block matching, while BDOF is based on optical flow. They can be used in combination. For example, this can be referred to as Multi-pass Decoder-side Motion Vector Refinement (MDMVR).

[0176] The first step is to improve the motion vector based on bidirectional matching of the entire block. The second step is to improve the motion vector based on bidirectional matching of sub-blocks. The block size for this step can be 16×16. The third step is to improve the motion vector based on bidirectional optical flow of sub-blocks. The sub-block size for this step can be 8×8. Further steps can be added to this system, such as a fourth step to improve the motion vector based on bidirectional optical flow of 4×4 sub-blocks, or a point-based bidirectional optical flow improvement.

[0177] It's also understandable that the Template Matching (TM) method was first used in inter-frame prediction. It leverages the correlation between adjacent pixels and uses areas surrounding the current block as templates. When the current block is encoded or decoded, its left and upper sides have already been encoded and decoded according to the coding order. Of course, existing hardware decoder implementations don't guarantee that the left and upper sides have already been decoded when decoding begins. This refers to inter-frame blocks. For example, in HEVC, when generating prediction blocks for inter-frame coded blocks, the surrounding reconstructed pixels are not required, allowing the prediction process for inter-frame blocks to proceed in parallel. However, intra-frame coded blocks must use reconstructed pixels on the left and upper sides as reference pixels. Theoretically, the left and upper sides are available, meaning that this can be achieved by adjusting the hardware design accordingly. Comparatively, the right and lower sides are unavailable under current coding standards such as VVC.

[0178] As shown in Figure 12, the rectangular areas to the left and above the current block are used as templates. The height of the left template portion is generally the same as the height of the current block, and the width of the upper template portion is generally the same as the width of the current block, but they can also be different. The best matching position of the template is found in the reference image L0 to determine the motion information, or motion vector, of the current block. This process can be roughly described as starting from a starting position in a reference image and searching within a certain range around it. Search rules, such as the search range and search step size, can be predefined. At each position, the degree of match between the template corresponding to that position and the templates surrounding the current block is calculated. The degree of match can be measured using distortion metrics such as the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), and the mean-square error (MSE). SATD generally uses the Hadamard transform. Smaller values ​​of SAD, SATD, and MSE indicate a higher degree of match. The cost is calculated using the predicted block of the template corresponding to the position and the reconstructed blocks of the template surrounding the current block. In addition to searching at whole-pixel locations, sub-pixel locations can also be searched, with the motion information of the current block determined based on the location with the highest degree of match. By leveraging the correlation between adjacent pixels, the appropriate motion information for the template may also be appropriate for the current block. Of course, template matching may not be applicable to all blocks, so methods can be used to determine whether template matching should be used for the current block, such as using a control switch to indicate whether template matching should be used for the current block. This template matching method is called decoder-side motion vector derivation (DMVD). Both the encoder and decoder can use the template to search to derive motion information or find better motion information based on the existing motion information. This method does not require the transmission of specific motion vectors or motion vector differences. Instead, both the encoder and decoder perform the same search rules to ensure consistent encoding and decoding. Template matching can improve compression performance, but it requires a "search" at the decoder, which introduces a certain degree of decoding complexity.

[0179] (2) Intra-frame prediction.

[0180] There is a strong spatial correlation between adjacent parts or adjacent pixels within the image. Intra-frame prediction is a prediction method that uses the spatial correlation between the coded and decoded pixels around the current block and the pixels within the current block. For example, as shown in Figure 13, the 4×4 white filling pixels are the current block, and the grid filling pixels in the left column and the upper row of the current block are the reference pixels of the current block. Intra-frame prediction uses these reference pixels to predict the current block. These reference pixels may all be available, that is, all have been coded and decoded. Some may also be unavailable. For example, if the current block is the leftmost of the entire frame, then the reference pixels on the left side of the current block are unavailable. Or when encoding and decoding the current block, the lower left part of the current block has not been encoded and decoded, then the reference pixels on the lower left are also unavailable. For the case where reference pixels are unavailable, available reference pixels or certain values ​​or certain methods can be used for filling, or no filling can be performed.

[0181] It should also be noted that the multiple reference line (MRL) intra prediction method can use more reference pixels to improve coding efficiency. As shown in Figure 14, this is a schematic diagram using four reference rows / columns.

[0182] There are multiple prediction modes for intra-frame prediction, as shown in Figure 15. Here are the nine modes for intra-frame prediction of 4×4 blocks in H.264. Among them, mode 0 (vertical mode) copies the pixels above the current block vertically to the current block as the prediction value, mode 1 (horizontal mode) copies the reference pixels on the left side horizontally to the current block as the prediction value, mode 2 (DC mode) uses the average of the eight points A to D and I to L as the prediction value for all points, and modes 3 to 8 copy the reference pixels to the corresponding positions of the current block at a certain angle. Because some positions of the current block cannot correspond exactly to the reference pixels, it may be necessary to use the weighted average of the reference pixels, or the sub-pixels of the interpolated reference pixels.

[0183] In addition, there are modes such as PLANE and PLANAR. With technological advancements and the expansion of block sizes, the number of angular prediction modes is also increasing. For example, HEVC uses intra-frame prediction modes including PLANAR, DC, and 33 angular modes, for a total of 35 prediction modes (see Figure 16 for details). VVC uses intra-frame modes including PLANAR, DC, and 65 angular modes, for a total of 67 prediction modes (see Figure 17 for details). Of course, in addition to the 67 modes mentioned above, VVC also provides wide-angle modes for rectangular blocks with a large difference between length and width. These modes, indicated by the dashed lines in the figure, represent the ranges -14 to -1 and 67 to 80 degrees. These modes replace some conventional modes (see Figure 18 for details).

[0184] It can also be understood that intra block copy (IBC) can significantly improve the compression efficiency of screen content coding (SCC), and therefore IBC has been used for screen content coding from HEVC to VVC. Screen content is different from camera-captured content. It is computer-generated, noise-free, and contains text, computer graphics, and has clear boundaries. There is a large amount of repeated content in the screen content, as shown in Figure 19.

[0185] In the embodiments of this application, IBC can be considered to apply the inter-frame prediction method to intra-frame prediction. Inter-frame prediction uses a reference block in a reference image, which is not the current image, to generate the prediction block for the current block. IBC, on the other hand, uses a reference block from the coded or reconstructed portion of the current image to generate the prediction block for the current block. IBC is also known as intra picture block compensation or current picture referencing (CPR).

[0186] IBC uses a block vector (BV) to represent the position difference between the current block and the reference block, similar to the MV used in inter-frame prediction. The encoder uses block matching within the search range to determine the best matching block for the current block and encodes the BV. There are various methods for encoding the BV, such as the merge mode, which is similar to inter-frame prediction and will not be discussed here.

[0187] IBC can be considered an intra-frame prediction method, or a different type of prediction method independent of intra-frame and inter-frame prediction. IBC is highly efficient for encoding screen content and can also improve compression efficiency in natural sequences captured by cameras.

[0188] Intra Template Matching Prediction (IntraTMP) in ECM10 can be considered a special type of IBC. IntraTMP also searches for a reference block from the coded portion of the current image to generate a prediction block for the current block. IntraTMP uses the reconstructed pixels within a certain range to the left and above the current block as the template for the current block. Each time a BV is searched, the template of the reference block of reconstructed pixels within a certain range to the left and above the corresponding reference block of the same size as the current block is calculated. The matching cost between the current block's template and the reference block's template is calculated, and the prediction block is generated based on the matching cost.

[0189] It can also be understood that for decoder-side intra mode derivation (DIMD), DIMD uses the reconstructed pixels on the left and top sides of the current block to derive the prediction mode, but it does not predict on the template, but analyzes the gradient of the reconstructed pixels.

[0190] As shown in Figure 20, DIMD analyzes the gradient of black points, such as horizontal gradient and vertical gradient, and adapts an intra-frame prediction mode according to its gradient. The analysis of all points that need to be checked can obtain a result similar to the bar graph below. That is, the statistics of the number of points matched by each intra-frame prediction mode. Of course, the so-called bar graph is just to help understanding, and it can be implemented in a variety of simple forms. The current DIMD selects the two highest intra-frame prediction modes in the histogram, plus the PLANAR mode, and the prediction values ​​of a total of three intra-frame prediction modes are weighted. The weights are related to the results of the analysis. For example, as shown in Figure 21, the three intra-frame prediction modes include M1 mode, M2 mode and PLANAR mode. The prediction values ​​obtained for these three intra-frame prediction modes are set to Pred1, Pred2, and Pred3 respectively, and the weight values ​​of these three intra-frame prediction modes are set to w1, w2, and w3 respectively. The specific calculation formulas are as follows:

[0191] The final prediction block can be shown as follows:

[0192] In summary, DIMD uses gradient analysis of reconstructed pixels to select intra prediction modes, and can weight two intra prediction modes plus planar based on the analysis results. The advantage of DIMD is that if DIMD mode is selected for the current block, the decoder does not need to indicate the specific intra prediction mode to be used. Instead, the decoder can derive the selected mode through the above process, which saves a certain amount of overhead.

[0193] It can also be understood that for cross-component prediction (Cross-Component Prediction, CCP), since different components of the same space have a strong correlation, video coding and decoding technology can use this correlation to improve compression efficiency. In some cases, the first component of the same space will be encoded and decoded first, and then the second and third components will be encoded and decoded, so that the second and third components can use some information of the first component. For example, in the YUV format, the pixels of U and V can be predicted using the reconstructed pixels of Y at the corresponding positions. If it is in YUV4:2:0 format, the pixel positions of U and V do not correspond one to one to the position of Y, and the corresponding reconstructed pixels of Y can be found for prediction through methods such as downsampling.

[0194] A typical example of CCP in VVC is Cross-Component Linear Model (CCLM). Other cross-component prediction technologies in ECM include Multi-Model CCLM (MM-CCLM), Convolutional Cross-Component Model (CCCM), Multi-Model CCCM (MM-CCCM), Gradient Linear Model (GLM), inter-CCCM, etc. There are other derivative technologies that are not listed one by one. They are all technologies that use some information of the first color component, such as the reconstruction value, to predict the second / third color component. Cross-component prediction technology can be used for blocks encoded within a frame as well as for blocks encoded between frames. For example, inter-CCCM is used for blocks encoded between frames.

[0195] (1) Cross-component linear model CCLM.

[0196] CCLM, as the name implies, is a technique that uses a linear model of the first color component and the second / third color component for prediction. Specifically, it is as follows: pred C (i,j)=α·rec L ′(i,j)+β (21)

[0197] Among them, pred C (i, j) represents the predicted value of the chrominance pixel at position (i, j), which represents rec L The reconstructed value of the downsampled luminance pixel at the ′(i,j) position. The model parameters (α and β) of CCLM are derived based on the adjacent downsampled luminance pixels and chrominance pixels of the current block. For example, as shown in Figure 22, a sampling diagram of the adjacent reference values ​​of the luminance component and the chrominance component of the current block is shown; wherein, in (a), the larger bold box is used to highlight the chrominance block 21, and the gray solid circle is used to indicate the adjacent reference value of the chrominance block 21; in (b), the larger bold box is used to highlight the luminance block 22, and the gray solid circle is used to indicate the adjacent reference value of the luminance block 22. Among them, the chrominance block 21 is N×N in size, and the luminance block 22 is 2N×2N in size. Here, the adjacent reference values ​​of the chrominance block 21 and the adjacent reference values ​​of the luminance block 22 are both used to derive the model parameters α and β.

[0198] The basic principle is to use the adjacent luminance and chrominance pixels of the current block to derive a linear model, then apply this linear model to the current block, and determine the predicted value of chrominance based on the reconstructed value of luminance and this linear model.

[0199] (2) Multi-model CCLM.

[0200] MM-CCLM is an extension of CCLM. CCLM uses only one linear model in the current block, while MM-CCLM, as the name implies, uses multiple linear models. Specifically, two linear models are used in ECM-10. To derive the two linear models, the adjacent luminance pixels on the left and above sides of the current block used to derive the linear models are divided into two groups, with the grouping threshold being the median of these luminance pixel values. When predicting the current block, the same threshold is used to divide the luminance pixels of the current block into two groups, and prediction is performed using their respective models. Figure 23 is a schematic diagram of MM-CCLM grouping, and Figure 24 is a schematic diagram of MM-CCLM prediction.

[0201] (3) Convolutional Cross-Component Model CCCM.

[0202] Similar to CCLM, CCCM also derives a cross-component model based on the reconstructed portions to the left and above the current block. However, CCCM uses a larger reconstruction area, allowing it to derive a nonlinear model. Another difference is that CCLM predicts a chrominance value from a downsampled luma value, while CCCM uses the reconstructed luma value at the corresponding chrominance pixel, as well as the surrounding pixels above, below, and to the left and right of the pixel.

[0203] Specifically, CCCM uses a 7-tap convolution filter that includes five spatially adjacent pixels. C (center) is the luma pixel corresponding to the current chroma pixel, N (north) is the luma pixel above C, S (south) is the luma pixel below C, W (west) is the luma pixel to the left of C, and E is the luma pixel to the right of C. There is also a nonlinear term P, P = (C × C + midVal) >> bitDepth. Here, midVal means the median, and bitDepth is the bit width. For the 10-bit case, P = (C × C + 512) >> 10. There is also an offset term B, which is set to the median, which is 512 for the 10-bit case. The predicted value of the chroma pixel is predChromaVal = C0C + C1N + C2S + C3E + C4W + C5P + C6B.

[0204] In order to determine C0, C1, C2, C3, C4, C5, and C6, CCCM will apply the same model to predict the chrominance in the reference area as shown in Figure 25 and compare it with the reconstructed chrominance value to solve C0, C1, C2, C3, C4, C5, and C6 that minimizes the mean square error.

[0205] (4) Multi-model convolutional cross-component model MM-CCCM.

[0206] Similar to MM-CCLM, multiple models are applied to the current block. ECM-10 uses two models. The derivation method also uses a threshold to separate the reconstructed luma pixels and the luma pixels of the current block into two groups. CCCM models are derived for each group, and the predictions are combined into a single prediction block.

[0207] (5) Gradient linear model GLM.

[0208] GLM is a method for predicting chromaticity based on the gradient of brightness. There are two GLM modes, one is 2-parameter GLM and the other is 3-parameter GLM.

[0209] The formula for a 2-parameter GLM is pred C (i,j)=α·grad L (i,j)+β;

[0210] The formula for the 3-parameter GLM is pred C (i,j)=α0·grad L (i,j)+α1·rec L ′(i,j)+β.

[0211] Here, the 2-parameter GLM replaces the reconstructed luminance pixel values ​​from the downsampled CCLM with luminance gradients. The 3-parameter GLM adds a luminance gradient to the CCLM. There are four ways to calculate the luminance gradient, as shown in Figure 26. The filter squares correspond to luminance pixels, and the circles correspond to chrominance pixels. This is a set of filters designed for the YUV 4:2:0 format.

[0212] (6) CCCM between frames.

[0213] Intra-frame prediction and inter-frame prediction are different. Intra-frame prediction uses reconstructed pixels around the current block to predict the interior of the current block, while inter-frame prediction uses images of the same object at different times. The most basic principle of inter-frame prediction is translational motion: using a motion vector to find a reference block and use the reference block as the prediction block. Since the same object is found at different times, the reference image contains the same number of components as the current image. For YUV sequences, all three YUV components can be predicted from the reference block. Cross-component models such as CCLM and CCCM follow a similar approach to intra-frame prediction, using surrounding reconstructed pixels to derive the model. However, inter-CCCM uses the original inter-frame prediction block to derive the cross-component model (filter coefficients), as shown in Figure 27. resY, resCb, and resCr represent the luma residual, blue chroma residual, and red chroma residual, respectively. Addition of the corresponding predicted values ​​yields the reconstructed luma value Y, blue chroma reconstructed value Cb, and red chroma reconstructed value Cr. In other words, after luma reconstruction, the derived model (filter coefficients) is applied to the luma reconstruction value to obtain a second chroma prediction value. The original prediction value and the second prediction value obtained by inter-CCCM are weighted to form the new prediction value. Inter-CCCM transforms the reconstruction value containing the luma residual into a chroma prediction through the model, thereby improving compression efficiency.

[0214] Furthermore, the following introduces the transformation technology.

[0215] During encoding, commonly used hybrid coding frameworks first perform a prediction. This prediction leverages spatial or temporal correlation to produce an image identical or similar to the current block. While it's possible for the predicted block to be identical to the current block for a given block, it's difficult to guarantee this for all blocks in a video, especially in natural video or video captured by a camera. Irregular motion, distortion, occlusion, and brightness changes in video are difficult to fully predict. Therefore, hybrid coding frameworks subtract the predicted image from the original image of the current block to produce a residual image, or, in other words, subtract the predicted block from the current block to produce a residual block. This residual block is typically much simpler than the original image, so prediction can significantly improve compression efficiency. The residual block isn't encoded directly; instead, it's usually first transformed. This transform converts the residual image from the spatial domain to the frequency domain to remove correlation. After the residual image is transformed to the frequency domain, since most of the energy is concentrated in the low-frequency region, the non-zero coefficients are concentrated in the upper left corner. Quantization is then used to further compress the image. Furthermore, since the human eye is less sensitive to high frequencies, a larger quantization step size can be used in high-frequency regions.

[0216] Figure 28 is a schematic diagram of a DCT transform. As shown in Figure 28, after the DCT transform of the original image, only the upper left corner has non-zero coefficients. Of course, this example performs a DCT transform on the entire image. In video codecs, images are processed by dividing them into blocks, so the transform is also performed on a block-by-block basis.

[0217] Transforms are very useful in typical video compression, but not all blocks require transforms. In some cases, transforms can even yield worse compression results than non-transforms. Therefore, in some standards, such as VVC, the encoder can choose whether to use transforms for the current block. DCT-II is the most commonly used transform in video compression standards, and its base image is shown in Figure 29.

[0218] In addition, VVC can also use DCT8 type (DCT-VIII) and DST7 type (DST-VII). The basic formulas of these transforms are shown in Table 3, which shows the basic transform formulas of DCT2, DCT8 and DST7 for N-point input.

[0219] Table 3

[0220] Because images are all two-dimensional, the computational complexity and memory overhead of performing a direct two-dimensional transform were prohibitive for the hardware available at the time. Therefore, the DCT2, DCT8, and DST7 transforms used in the standards were split into two steps: horizontal and vertical one-dimensional transforms. For example, the horizontal transform was performed first, followed by the vertical transform, or the vertical transform was performed first, followed by the horizontal transform.

[0221] (1) Multi-transformation selection MTS.

[0222] VVC supports transform kernels such as DCT2, DCT8, and DST7. For a block, the encoder selects the appropriate transform kernel and transmits the index to the bitstream. The decoder then uses the index to determine the transform kernel for the inverse transform. Different transform kernels can be selected for the horizontal and vertical directions, such as using DCT8 horizontally and DST7 vertically. This technique is generally referred to as MTS.

[0223] (2) Low-frequency non-separable transform LFNST.

[0224] The above transformation method is effective for horizontal and vertical textures, but less so for diagonal textures. Indeed, horizontal and vertical textures are the most common, making the above transformation method very useful for improving compression efficiency. As the demand for compression efficiency continues to increase, more efficient processing of diagonal textures could further improve compression efficiency.

[0225] In order to more effectively process the residual of oblique texture, LFNST transform is used in VVC. The above transforms such as DCT2, DCT8, and DST7 are called primary transforms. At the encoding end of VVC, LFNST is used after DCT2 transform and before quantization. At the decoding end of VVC, LFNST is used after inverse quantization and before inverse DCT2 transform. Because it is transformed on the basis of DCT2 (basic transform), LFNST is a secondary transform. Figure 30 is a schematic diagram of the encoding and decoding process without LFNST (secondary transform), and Figure 31 is a schematic diagram of the encoding and decoding process with LFNST (secondary transform). Of course, the encoding end can directly inverse quantize the saved quantization coefficients instead of entropy decoding, because entropy coding is lossless.

[0226] Figure 32 is a detailed schematic diagram of the encoding and decoding process with LFNST (secondary transform). As shown in Figure 32, at the encoding end, the first step is the forward basic transform, and then the LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the basic transform. For example, there are 16 input coefficients for 4×4 LFNST and 64 input coefficients for 8×8 LFNST; the coefficients after the LFNST transform are then quantized and the quantized coefficients are written into the bitstream. At the decoding end, the transform coefficients can be obtained by decoding the bitstream and inverse quantization. Then, in the inverse LFNST transform, there are 8 input coefficients for 4×4 inverse LFNST and 16 input coefficients for 8×8 inverse LFNST. Finally, the inverse basic transform can obtain the residual block.

[0227] That is, at the encoder, LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the base transform. The base transform decorrelates the image, concentrating the energy in the upper left corner. The secondary transform further decorrelates the low-frequency coefficients of the base transform. The result is intuitively shown in Figure 32. At the encoder, 16 coefficients are input to a 4×4 LFNST, and the output is 8 coefficients; 64 coefficients are input to an 8×8 LFNST, and the output is 16 coefficients. At the decoder, 8 coefficients are input to a 4×4 inverse LFNST, and the output is 16 coefficients; 16 coefficients are input to an 8×8 inverse LFNST, and the output is 64 coefficients.

[0228] Figure 33 shows some base images for the LFNST algorithm in VVC. Figure 33 shows some obvious diagonal textures. In addition to kernels optimized for certain diagonal textures, the LFNST algorithm also has kernels optimized for flat, gradient textures, such as kernel group 0 in the LFNST algorithm in VVC.

[0229] LFNST is only applied to intra-coded blocks. Angular prediction tiles the reference pixels at a specified angle onto the current block as the prediction value. This means the predicted block will have a distinct directional texture, and the residual of the current block after angular prediction will also statistically exhibit significant angular characteristics. Therefore, the transform kernel selected by LFNST can be tied to the intra-prediction mode. That is, once the intra-prediction mode is determined, LFNST can only use the set of transform kernels corresponding to that intra-prediction mode.

[0230] Specifically, the LFNST in VVC has a total of 4 groups of transform kernels, and each group can select 2 transform kernels. Table 4 shows the correspondence between intra prediction modes and transform kernel groups. Note that the cross-component prediction modes used for chroma intra prediction are 81 to 83, and there are no such modes for luma intra prediction. The transform kernel of LFNST can be transposed to process more angles with one transform kernel group. For example, modes 13 to 23 and 45 to 55 both correspond to transform kernel group 2, but 13 to 23 is obviously close to the horizontal mode and 45 to 55 is obviously close to the vertical mode.

[0231] Table 4

[0232] VVC's LFNST uses four sets of transform kernels, with the intra-prediction mode specifying which set to use. This leverages the correlation between the intra-prediction mode and the LFNST transform kernel, reducing the transmission of the selected LFNST transform kernel in the bitstream. Whether the current block uses LFNST, and if so, whether to use the first or second set within a set, is determined by the bitstream and certain conditions.

[0233] In the subsequent evolution of ECM technology, LFNST was further expanded. LFNST has more transform kernel groups, 35 in ECM. The correspondence between the transform kernel group index (LFNST set index) and the intra prediction mode (Intra pred.mode) is shown in Table 5. Each transform kernel group is more efficient for textures at the corresponding angle. Here, each transform kernel group can select three transform kernels.

[0234] Table 5

[0235] (3) Non-separable basis transformation NSPT.

[0236] LFNST is a horizontally and vertically inseparable transform. Because it involves a secondary transform, DCT2 can be called the base transform. This approach of performing DCT2 before LFNST is a compromise between performance and complexity. While directly performing the inseparable base transform is more efficient, it also incurs higher complexity, such as increased computational effort and storage space required for the transform kernel.

[0237] In ECM10, some small blocks can use NSPT, while large blocks still use DCT2+LFNST. The sizes of small blocks are 4×4, 4×8, 8×4, 8×8, 4×16, 16×4, 8×16, 16×8, 8×32, and 32×8. In ECM10, NSPT also matches the transform kernel group according to the intra-frame prediction mode. The matching method can refer to the LFNST method. Each transform kernel group has 3 transform kernels to choose from. For example, the 8×8 base image of an NSPT in ECM10 is shown in Figure 34. Figure 34 corresponds to inter-frame angular prediction mode 7, which shows that it handles texture at corresponding angles better. It should be noted that NSPT is only applied to intra-frame coded blocks.

[0238] Furthermore, for the partitioning technology, it can be divided into single tree partitioning and dual tree partitioning. In the Single tree, luminance and chrominance are divided together, and a CU contains a luminance block and a chrominance block at the same position. In the Dual tree, luminance and chrominance are divided separately, and a CU has only luminance blocks or only chrominance blocks. HEVC only has Single tree. Dual tree is introduced in VVC, but the P and B slices in VVC can still only use Single tree. The I slice in VVC can use Dual tree. As shown in Figure 35, luminance and chrominance are divided separately, and it can be seen that luminance has a finer division than chrominance, because luminance has more details than chrominance, and generally luminance requires better quality than chrominance.

[0239] In summary, LFNST and NSPT can be applied to intra-frame coded blocks, or can also be applied to inter-frame coded blocks. As for related technologies, in VVC and ECM, MTS / LFNST / NSPT is not used for the transformation of chroma blocks of single tree. If the current block is a luminance coding unit or a chroma coding unit of a dual tree within a frame, then MTS / LFNST / NSPT can be used; if the current block is a single tree, that is, a coding unit has both luminance blocks and chroma blocks, then MTS / LFNST / NSPT only acts on the transformation of luminance blocks, and cannot act on the transformation of chroma blocks. When MTS / LFNST / NSPT is used for the luminance block, the chroma block uses the default DCT2 transform; it is not conducive to improving the compression efficiency of the chroma component.

[0240] Based on this, an embodiment of the present application provides an encoding method, which determines the mode parameters of the chrominance components of the current block; when the mode parameters meet the first condition, determines the first transform kernel of the chrominance components of the current block; determines the residual block of the chrominance components of the current block, and transforms the residual block of the chrominance components of the current block according to the first transform kernel to determine the transform coefficients of the chrominance components of the current block; encodes the transform coefficients of the chrominance components of the current block, and writes the obtained coded bits into the bitstream. An embodiment of the present application also provides a decoding method, which determines the mode parameters of the chrominance components of the current block; when the mode parameters meet the first condition, determines the first transform kernel of the chrominance components of the current block; determines the transform coefficients of the chrominance components of the current block, and inversely transforms the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine the residual block of the chrominance components of the current block.

[0241] In this way, both the encoding and decoding ends first determine the mode parameters for the current block. When the mode parameters meet the first condition, the first transform kernel for the chroma components of the current block is determined, and the residual block of the chroma components of the current block is transformed based on the first transform kernel. In other words, the determination of the first transform kernel is related to the mode parameters, enabling the application of MTS / LFNST / NSPT transforms to the chroma components of a single tree partition, thereby improving not only compression efficiency, that is, video encoding and decoding efficiency, but also encoding and decoding performance.

[0242] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0243] FIG36 is a schematic diagram of a network architecture for video encoding and decoding provided in an embodiment of the present application. As shown in FIG36 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein electronic devices 13 to 1N can perform video interaction via the communication network 01. During implementation, the electronic devices can be various types of devices with video encoding and decoding capabilities. For example, the electronic devices can include mobile phones, tablet computers, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensor devices, servers, etc., and the embodiments of the present application do not limit this.

[0244] In an embodiment of the present application, a network architecture of a video encoding and decoding system including a decoding method and an encoding method is provided. The decoder or encoder in the embodiment of the present application can be the aforementioned electronic device. In other words, the electronic device in the embodiment of the present application has video encoding and decoding capabilities, and generally includes a video encoder (i.e., encoder) and a video decoder (i.e., decoder).

[0245] FIG37 is a block diagram of the system composition of an encoder provided in an embodiment of the present application. As shown in FIG37 , the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a series of pictures or a single still picture, and the output of the encoder 100 may be a bitstream (also referred to as a "codestream") used to represent a compressed version of the input video.

[0246] Among them, the segmentation unit 101 segments the picture in the input video into one or more Coding Tree Units (CTUs). The segmentation unit 101 divides the picture into multiple tiles (or tiles), and can further divide a tile into one or more bricks. Here, a tile or a brick may include one or more complete and / or partial CTUs. In addition, the segmentation unit 101 can form one or more slices, where a slice can include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture. The segmentation unit 101 can also form one or more sub-pictures, where a sub-picture can include one or more slices, tiles or bricks.

[0247] During the encoding process of encoder 100, segmentation unit 101 transmits the CTU to prediction unit 102. Generally, prediction unit 102 may be composed of block segmentation unit 103, motion estimation (ME) unit 104, motion compensation (MC) unit 105, and intra prediction unit 106. Specifically, block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (CUs). Prediction unit 102 may use ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks for the CU. Intra-frame prediction unit 106 may use various intra-frame prediction modes, including MIP mode, to obtain intra-frame prediction blocks for the CU. In an example, a rate-distortion optimized motion estimation method may be used by ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks, and a rate-distortion optimized mode determination method may be used by intra-frame prediction unit 106 to obtain intra-frame prediction blocks.

[0248] The prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU in the output of the segmentation unit 101 and the prediction block of the CU, i.e., the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels). The inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstructed residual. The second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference. After all CUs in the picture or sub-picture are reconstructed, the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping and chroma scaling (LMCS) filter, and a neural network-based filter. Alternatively, when the filtering unit 113 determines that a CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.

[0249] The output of the filtering unit 113 is a decoded picture or sub-picture, which is cached to the DPB unit 114. The DPB unit 114 outputs the decoded picture or sub-picture based on the timing and control information. Here, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, the entropy coding unit 115 converts the parameters required for decoding the picture from the encoder 100 (such as control parameters and supplementary information, etc.) into binary form and writes this binary form into the code stream according to the syntax structure of each data unit. That is, the encoder 100 finally outputs the code stream.

[0250] Furthermore, the encoder 100 can be a device having a first processor and a first memory for recording a computer program. When the first processor reads and executes the computer program, the encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, the encoder 100 can be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in FIG37 .

[0251] Figure 38 is a block diagram of the system components of a decoder provided in an embodiment of the present application. As shown in Figure 38, the decoder 200 may include: a parsing unit 201, a prediction unit 202, an inverse quantization unit 205, an inverse transform unit 206, an adder 207, a filtering unit 208, and a decoded image buffer unit 209. Here, the input of the decoder 200 is a bitstream representing a compressed version of a video or a still image, and the output of the decoder 200 may be a decoded video consisting of a series of images or a decoded still image.

[0252] The input codestream to decoder 200 may be the codestream generated by encoder 100. Parsing unit 201 parses the input codestream and obtains syntax element values ​​from the input codestream. Parsing unit 201 converts the binary representation of the syntax elements into digital values ​​and sends the digital values ​​to units within decoder 200 to obtain one or more decoded pictures. Parsing unit 201 may also parse one or more syntax elements from the input codestream to display decoded pictures.

[0253] During the decoding process of the decoder 200 , the parsing unit 201 sends the values ​​of the syntax elements and one or more variables set or determined according to the values ​​of the syntax elements and used to obtain one or more decoded pictures to the units in the decoder 200 .

[0254] The prediction unit 202 determines a prediction block for the current decoding block (e.g., CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra-frame prediction unit 204. Specifically, when the inter-frame decoding mode is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the motion compensation unit 203 to obtain an inter-frame prediction block; when the intra-frame prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the intra-frame prediction unit 204 to obtain an intra-frame prediction block.

[0255] The inverse quantization unit 205 has the same functionality as the inverse quantization unit 110 in the encoder 100. The inverse quantization unit 205 performs a scaling operation on the quantized coefficients (i.e., levels) from the parsing unit 201 to obtain reconstructed coefficients. The inverse transform unit 206 has the same functionality as the inverse transform unit 111 in the encoder 100. The inverse transform unit 206 performs one or more transform operations (i.e., the inverse of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual. The adder 207 adds its inputs (the prediction block from the prediction unit 202 and the reconstructed residual from the inverse transform unit 206) to obtain a reconstructed block for the currently decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in intra-frame prediction mode.

[0256] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chroma scaling filter, and a neural network-based filter. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of the filtering unit 208 is a decoded picture or sub-picture, which is cached to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture based on timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0257] Furthermore, the decoder 200 can be a second memory having a second processor and a computer program. When the first processor reads and runs the computer program, the decoder 200 reads the input code stream and generates the corresponding decoded video. In addition, the decoder 200 can also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have connections and data exchange functions similar to those of the corresponding units in Figure 38.

[0258] It should also be noted that when the embodiment of the present application is applied to the encoder 100, the "current block" specifically refers to the current block to be encoded in the video image (which can also be simply referred to as the "encoding block"); when the embodiment of the present application is applied to the decoder 200, the "current block" specifically refers to the current block to be decoded in the video image (which can also be simply referred to as the "decoding block").

[0259] In one embodiment of the present application, referring to FIG39 , a flowchart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG39 , the method may include:

[0260] S3901: Determine the mode parameters of the chrominance components of the current block.

[0261] It should be noted that in the embodiment of the present application, the method is applied to a decoder. Specifically, based on the structure of decoder 200 shown in FIG38 , the decoding method of the embodiment of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, the optimization scheme of the MTS / LFNST / NSPT transform proposed for the chroma component of a single tree partition is mainly to improve compression efficiency.

[0262] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0263] It should also be noted that, in an embodiment of the present application, the partition type of the current block may include single tree partitioning and dual tree partitioning. In a single tree, the luminance component and the chrominance component are partitioned together, and a CU contains a luminance block and a chrominance block at the same position; in a dual tree, the luminance and chrominance are partitioned separately, and a CU contains only luminance blocks or only chrominance blocks. It should be noted that the block of the luminance component of the current block can be called a "luminance block", and the block of the chrominance component of the current block can be called a "chrominance block".

[0264] It should also be noted that although LFNST could originally be used on the chroma components of a single tree during standardization, to reduce hardware complexity, it is more effective to not use LFNST on chroma components. For example, not using LFNST on chroma components can shorten the pipeline compared to using it on chroma components. Based on this, in the embodiments of this application, the following restrictions are added to the use of chroma components in a single tree.

[0265] S3902: When the mode parameter satisfies a first condition, determine a first transform kernel of the chrominance component of the current block.

[0266] It should be noted that in the embodiment of the present application, the restriction condition is that the mode parameters meet the first condition, which may include: the partition type of the current block is single-tree partitioning and the prediction mode of the chrominance component of the current block is a cross-component prediction mode. The cross-component prediction mode may include CCLM, CCCM, inter-CCCM, etc.

[0267] In an embodiment of the present application, if the chroma component of the current block in a single-tree partition uses a cross-component prediction mode, then one possible implementation is to not apply MTS / LFNST / NSPT to the chroma component of the current block. In this case, a preset transform kernel can be determined as the first transform kernel for the chroma component of the current block. The preset transform kernel is the default transform kernel.

[0268] For example, if the chrominance component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT is not applied to the chrominance component of the current block. Instead, the default transform kernel is used, such as separable DCT2 in the horizontal and vertical directions respectively.

[0269] In an embodiment of the present application, if the chroma component of the current block in a single-tree partition uses a cross-component prediction mode, another possible implementation is that MTS / LFNST / NSPT uses a different transform kernel group for the chroma component than for the luma component. In this case, a first transform kernel group for the chroma component of the current block can be determined based on the first transform mode; wherein the first transform kernel group is different from the transform kernel group for the luma component of the current block; and a first transform kernel is determined based on the first transform kernel group.

[0270] It should be noted that, in the embodiment of the present application, the first transform mode includes at least one of the following: multi-transform selection mode (MTS), low-frequency non-separable transform mode (LFNST) and non-separable basic transform mode (NSPT).

[0271] For example, if the chroma component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT can be applied to the chroma component of the current block, but the first transform kernel group used by the chroma component is different from the transform kernel group used by the luminance component.

[0272] In an embodiment of the present application, if the chroma component of the current block under single-tree partitioning uses a cross-component prediction mode, then another possible implementation method is that MTS / LFNST / NSPT uses a different transform kernel for the chroma component than for the luminance component. In this case, a first transform kernel group for the chroma component of the current block can be determined based on the first transform mode; wherein the first transform kernel group is the same as the transform kernel group for the luminance component of the current block; and a first transform kernel is determined based on the first transform kernel group; wherein the first transform kernel is different from the transform kernel for the luminance component of the current block.

[0273] It should be noted that in the embodiments of the present application, MTS / LFNST / NSPT uses different transform kernels for the chroma component and the luma component. This can be done by using the same transform kernel group but using different transform kernels, or by using different transform kernel groups and using different transform kernels.

[0274] It should also be noted that, in the embodiment of the present application, the first transform mode includes at least one of the following: multi-transform selection mode (MTS), low-frequency non-separable transform mode (LFNST) and non-separable basic transform mode (NSPT).

[0275] For example, if the chroma component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT can be applied to the chroma component of the current block, but the first transform kernel used by the chroma component is different from the transform kernel used by the luminance component.

[0276] It should also be noted that when determining the first transform kernel group for the chrominance components of the current block based on the first transform mode, different transform modes may correspond to different transform kernel groups. For example, the multi-transform selection mode may determine the MTS transform kernel group, the low-frequency inseparable transform mode may determine the LFNST transform kernel group, and the inseparable basic transform mode may determine the NSPT transform kernel group. Each transform kernel group may include at least one candidate transform kernel. After determining the first transform kernel group for the chrominance components, the first transform kernel used for the chrominance components of the current block may be determined from the candidate transform kernel group.

[0277] That is to say, in the embodiment of the present application, MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma component than for the luminance component. Among them, LFNST / NSPT in some technologies will determine a transform kernel group based on the intra-frame prediction mode. In fact, this intra-frame prediction mode represents a texture feature, such as the texture at a certain angle. LFNST / NSPT can match the corresponding transform kernel group based on this texture feature. There are multiple transform kernels to choose from in each transform kernel group, for example, 3 transform kernels to choose from. Similarly, intra-frame MTS also matches the transform kernel based on the intra-frame prediction mode. For inter-frame prediction, it itself does not have an intra-frame prediction mode. In the above content, a virtual intra-frame prediction mode is derived using the inter-frame prediction block. This virtual prediction mode can also be called a texture feature index, which is used to match the transform kernel group of LFNST / NSPT, and then the first transform kernel of the current block is determined based on the transform kernel group.

[0278] Understandably, if the chroma and luma components of a single-tree CU are predicted using the same prediction method, their residuals may also conform to the same texture characteristics, such as texture at a specific angle. However, if the chroma and luma components of a single-tree CU are predicted using different prediction methods, their residuals may conform to different texture characteristics. A specific example is cross-component prediction. Cross-component prediction uses the reconstructed block of the luma component to predict the chroma component. The reconstructed block of the luma component already contains the luma component residual, so the prediction block of the chroma component contains information converted from the luma component residual. If the luma component and chroma component have different texture characteristics, different transform kernels or transform kernel groups can be used for them. Another specific example is for LFNST / NSPT: the luma component uses its intra-frame prediction mode or virtual intra-frame prediction mode to match a transform kernel group. If cross-component prediction is used for the chroma component, the transform kernel group corresponding to DC or PLANAR is used for the chroma component, or a transform kernel group specifically optimized for cross-component prediction is used. This is not specifically limited here.

[0279] It should also be noted that the above description is for texture features. For the commonly used YUV4:2:0 format, the number of pixels in the luma component of the same CU is four times the number of pixels in the chroma component, which means that the sizes of the luma and chroma blocks are different. For example, if the luma block is 16×16, then the corresponding chroma block is 8×8, so the transform kernel or transform kernel group must also be of corresponding size.

[0280] In some embodiments, determining the first transform core based on the first transform core group may include: decoding the code stream to determine the value of the first syntax identification information; determining the transform core index of the current block when the first syntax identification information indicates that the current block uses the first transform mode; determining the first transform core based on the first transform core group and the transform core index.

[0281] It should be noted that, in an embodiment of the present application, the first syntax identification information is used to indicate whether the current block uses the first transform mode, such as whether the current block uses MTS / LFNST / NSPT. If the value of the first syntax identification information is equal to the first value, then it can be determined that the current block does not use MTS / LFNST / NSPT; if the value of the first syntax identification information is equal to the second value, then it can be determined that the current block uses MTS / LFNST / NSPT. Among them, the first value can be set to 0, and the second value can be set to non-zero, such as 1, 2, 3, etc. That is, when the current block uses the first transform mode, the first syntax identification information is also used to indicate which transform kernel is used.

[0282] It should also be noted that, in an embodiment of the present application, the first syntax identification information can also be replaced by two other syntax identification information. Among them, one syntax identification information is used to indicate whether the current block uses MTS / LFNST / NSPT, and the other syntax identification information is used to indicate the corresponding transform core index when the current block uses MTS / LFNST / NSPT. That is to say, when determining the transform core index of the current block, the transform core index of the current block can be determined by decoding the code stream. In this way, two syntax elements need to be transmitted in the code stream, resulting in a large code stream overhead. In this case, for the case where the transform core index needs to be transmitted in the code stream, if there is only one transform core in the determined first transform core group, then there is no need to transmit the transform core index at this time. Specifically, when the current block uses MTS / LFNST / NSPT, the only transform core in the first transform core group can be directly used as the first transform core used by the current block.

[0283] It should also be noted that, in the embodiment of the present application, the first syntax identification information can be replaced by three other syntax identification information. Among them, one syntax identification information is used to indicate whether the current block uses NSPT and which NSPT transformation core is used, another syntax identification information is used to indicate whether the current block uses LFNST and which LFNST transformation core is used, and another syntax identification information is used to indicate whether the current block uses MTS and which MTS transformation core is used; these are essentially the same and are not specifically limited here.

[0284] S3903: Determine transform coefficients of chrominance components of the current block, and perform inverse transform on the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine a residual block of the chrominance components of the current block.

[0285] It should be noted that in an embodiment of the present application, the quantization coefficients of the chrominance components of the current block can be determined by decoding the code stream, and then the quantization coefficients of the chrominance components of the current block can be inversely quantized to determine the transform coefficients of the chrominance components of the current block.

[0286] It should also be noted that, in an embodiment of the present application, the method further includes: when the mode parameters do not satisfy the first condition, determining a second transform kernel of the chrominance component of the current block; determining the transform coefficient of the chrominance component of the current block, and performing an inverse transform on the transform coefficient of the chrominance component of the current block according to the second transform kernel to determine the residual block of the chrominance component of the current block.

[0287] That is, in the embodiment of the present application, if the mode parameters satisfy the first condition, a first transform kernel of the chroma components of the current block may be determined, and then the transform coefficients of the chroma components of the current block are inversely transformed according to the first transform kernel to determine the residual block of the chroma components of the current block. If the mode parameters do not satisfy the first condition, a second transform kernel of the chroma components of the current block may be determined, and then the transform coefficients of the chroma components of the current block are inversely transformed according to the second transform kernel to determine the residual block of the chroma components of the current block.

[0288] In an embodiment of the present application, if the mode parameter does not meet the first condition, it may include: the partition type of the current block is dual-tree partitioning or the prediction mode of the chrominance component of the current block is non-cross-component prediction mode. In this case, if the current block is dual-tree partitioning or the chrominance component of the current block does not use cross-component prediction mode, then MTS / LFNST / NSPT can be used directly.

[0289] In some embodiments, determining the second transform kernel of the chrominance component of the current block may include: determining a second transform kernel group of the chrominance component of the current block based on the first transform mode; determining a second transform kernel according to the second transform kernel group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0290] It should be noted that, in an embodiment of the present application, determining the second transform core based on the second transform core group may specifically include: decoding the code stream to determine the value of the first syntax identification information; when the first syntax identification information indicates that the current block uses the first transform mode, then the second transform core can be determined based on the second transform core group and the value of the first syntax identification information.

[0291] It should be noted that in an embodiment of the present application, if the current block uses the multi-transform selection mode, then the MTS transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom. If the current block uses the low-frequency inseparable transform mode, then the LFNST transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom. If the current block uses the inseparable basic transform mode, then the NSPT transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom.

[0292] It should also be noted that in an embodiment of the present application, for the first syntax identification information, if two syntax identification information are used to replace it, one syntax identification information is used to indicate whether the current block uses MTS / LFNST / NSPT, and the other syntax identification information is used to indicate the corresponding transform core index when the current block uses MTS / LFNST / NSPT, then when determining the second transform core of the current block, the second transform core index of the current block can also be determined by decoding the code stream, and then the second transform core is determined based on the second transform core group and the second transform core index. In this way, two syntax elements need to be transmitted in the code stream, resulting in a large code stream overhead. In this case, if there is only one transform core in the determined second transform core group, then there is no need to transmit the transform core index in the case where the transform core index needs to be transmitted in the code stream. Specifically, when the current block uses MTS / LFNST / NSPT, the only transform core in the second transform core group can be directly used as the second transform core.

[0293] In a possible implementation, when the mode parameter satisfies the first condition, the transform coefficients of the chroma components of the current block are inversely transformed according to the first transform core to determine the residual block of the chroma components of the current block, which may include: when the first syntax identification information indicates that the chroma components of the current block use NSPT, the transform coefficients of the chroma components of the current block are inversely transformed by an inseparable basic transform according to the first transform core to determine the residual block of the chroma components of the current block; or, when the first syntax identification information indicates that the chroma components of the current block use LFNST, the transform coefficients of the chroma components of the current block are inversely transformed by a low-frequency inseparable transform according to the first transform core to determine the transform block of the chroma components of the current block; and the transform block of the chroma components is inversely transformed by a discrete cosine transform to determine the residual block of the chroma components of the current block; or, when the first syntax identification information indicates that the chroma components of the current block use MTS, the transform coefficients of the chroma components of the current block are inversely transformed by multi-transform selection according to the first transform core to determine the residual block of the chroma components of the current block.

[0294] In another possible implementation, when the mode parameter does not meet the first condition, the transform coefficients of the chroma components of the current block are inversely transformed according to the second transform core to determine the residual block of the chroma components of the current block, which may include: when the first syntax identification information indicates that the chroma components of the current block use NSPT, the transform coefficients of the chroma components of the current block are inversely transformed by the non-separable basic transform according to the second transform core to determine the residual block of the chroma components of the current block; or, when the first syntax identification information indicates that the chroma components of the current block use LFNST, the transform coefficients of the chroma components of the current block are inversely transformed by the low-frequency non-separable transform according to the second transform core to determine the transform block of the chroma components of the current block; and the transform block of the chroma components is inversely transformed by discrete cosine transform to determine the residual block of the chroma components of the current block; or, when the first syntax identification information indicates that the chroma components of the current block use MTS, the transform coefficients of the chroma components of the current block are inversely transformed by multi-transform selection according to the second transform core to determine the residual block of the chroma components of the current block.

[0295] That is, in the embodiment of the present application, if the chroma component in a single-tree partitioned CU uses a cross-component prediction mode, then MTS / LFNST / NSPT is not used for the chroma component of such a CU. If the chroma component in a single-tree partitioned CU uses a cross-component prediction mode, then MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma component than for the luma component.

[0296] It should also be noted that in the embodiments of the present application, the "inverse transformation" of the transformation coefficients by the decoding end may also be referred to as "transformation" in the standard text. The "transformation" and "inverse transformation" in this article correspond to two opposite processes. For example, if the "transformation" converts the numerical value of the spatial domain to the coefficient of the frequency domain, then the "inverse transformation" converts the coefficient of the frequency domain to the numerical value of the spatial domain. "Inverse" is relative to "positive", and they are essentially both transformations. It should be noted that if the standard only stipulates decoding, then the "transformation" in the standard text is the decoding part, specifically referring to the "inverse transformation" in this article.

[0297] In some embodiments, referring to FIG. 40 , after step S3903, the method may further include:

[0298] S4001: Predict the chrominance component of the current block to determine a predicted block of the chrominance component of the current block.

[0299] S4002: Determine a reconstructed block of the chrominance component of the current block according to the prediction block of the chrominance component of the current block and the residual block of the chrominance component of the current block.

[0300] It should be noted that in the embodiment of the present application, for step S4001, this step can be performed in parallel with steps S3901 to S3902, or it can be performed before steps S3901 to S3902. The order of the steps is not specifically limited here.

[0301] It should also be noted that in this embodiment of the present application, intra-frame prediction can be performed on the chroma components of the current block to determine the predicted block for the chroma components of the current block; alternatively, inter-frame prediction can be performed on the chroma components of the current block to determine the predicted block for the chroma components of the current block. In other words, MTS / LFNST / NSPT is applied to the chroma components of a single tree partition. Optionally, it can be applied to single-tree partitioning within a frame and / or single-tree partitioning between frames.

[0302] It should also be noted that, in an embodiment of the present application, after determining the prediction block of the chrominance component of the current block, the prediction block of the chrominance component of the current block and the residual block of the chrominance component of the current block can be added to determine the reconstructed block of the chrominance component of the current block.

[0303] In an embodiment of the present application, determining the transform coefficients of the chrominance components of the current block may include: decoding the code stream to determine the quantization coefficients of the chrominance components of the current block; and dequantizing the quantization coefficients of the chrominance components of the current block to determine the transform coefficients of the chrominance components of the current block.

[0304] In an embodiment of the present application, determining the transform coefficient of the luminance component of the current block may include: decoding the code stream to determine the quantization coefficient of the luminance component of the current block; and dequantizing the quantization coefficient of the luminance component of the current block to determine the transform coefficient of the luminance component of the current block.

[0305] It should also be noted that, in this embodiment of the present application, the transform coefficients of the current block may include the transform coefficients of the chrominance components of the current block and the transform coefficients of the luminance components of the current block. For the first syntax identification information, parsing may also be performed only when the last non-zero coefficient position of the current block meets certain conditions. Referring to FIG. 41 , the method may include:

[0306] S4101: Decode the code stream and determine the quantization coefficient of the current block.

[0307] S4101: Dequantize the quantization coefficients of the current block to determine the transformation coefficients of the current block.

[0308] S4103: When the position of the last non-zero coefficient in the transformation coefficients of the current block meets a preset condition, decode the code stream and determine the value of the first syntax identification information of the current block.

[0309] It should be noted that, in the embodiment of the present application, for step S4103, decoding the code stream and determining the value of the first syntax identification information of the current block may include: decoding the code stream according to at least one context model to determine the binary symbol string of the first syntax identification information; and performing debinarization processing on the binary symbol string of the first syntax identification information to determine the value of the first syntax identification information.

[0310] It should also be noted that, in an embodiment of the present application, when the first syntax identification information indicates that the current block uses the first transform mode, the transform kernel of the first transform mode used by the current block is determined, and the transform coefficients of the current block are inversely transformed according to the transform kernel of the first transform mode to determine the residual block of the current block. When the first syntax identification information indicates that the current block does not use the first transform mode, the transform kernel of the second transform mode used by the current block is determined, and the transform coefficients of the current block are inversely transformed according to the transform kernel of the second transform mode to determine the residual block of the current block. Exemplarily, the first transform mode can be MTS / LFNST / NSPT, and the second transform mode can be other transform modes (or directly use the default transform kernel), which is not specifically limited here.

[0311] It should also be noted that in the embodiment of the present application, the first syntax identification information can be represented by lfnst_nspt_idx. In related technologies, lfnst_nspt_idx is only parsed if certain conditions are met. One of these conditions is that the last non-zero coefficient position is greater than or equal to 1. Quantized coefficients are scanned using a hierarchical diagonal scan. A coefficient block can be divided into 4×4 sub-blocks. The 4×4 sub-blocks are scanned using a diagonal scan, and the 4×4 sub-blocks are also scanned using a diagonal scan. The position (0, 0) in the upper left corner of the current block is 0 in the scan order. The last non-zero coefficient position is greater than or equal to 1, which means that the current block has coefficients other than (0, 0). For DCT2 transform, (0, 0) is the DC coefficient. In other words, if the current block has only a non-zero coefficient at position (0, 0) or no non-zero coefficients, the syntax element lfnst_nspt_idx does not need to be parsed and defaults to 0. Because LFNST / NSPT is unlikely to be used in this case, this restriction significantly reduces unnecessary lfnst_nspt_idx overhead. Furthermore, related technologies require that the last non-zero coefficient position of all color components must be greater than or equal to 1. For example, in a single-tree block in YUV format, the last non-zero coefficient position of all three color components must be greater than or equal to 1.

[0312] Thus, for inter-frame predicted blocks, the residuals are inherently smaller than those for intra-frame predicted blocks. Furthermore, given that the quality requirements for chrominance components are generally much lower than those for luminance components, inter-frame predicted blocks often have certain luminance coefficients but no chrominance coefficients. Since this LFNST / NSPT condition is designed for intra-frame prediction, and LFNST / NSPT can currently also be used for inter-frame prediction, but their coefficients and residual patterns are different, the present invention can modify this restriction.

[0313] In one possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0314] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0315] It should be noted that, in the embodiment of the present application, the first threshold value may also be referred to as the minimum threshold value, wherein the minimum threshold value may be set to 1, or a larger value such as 2, 3, etc., which is not specifically limited here.

[0316] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the first threshold. In other words, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is greater than the first threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is greater than the first threshold.

[0317] For example, assuming the minimum threshold is 1, the relevant techniques are still used for intra-predicted blocks. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is greater than or equal to 1. For inter-predicted blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is greater than or equal to 1. For single-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required.

[0318] It's also understandable that in related art, lfnst_nspt_idx is only parsed if certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot exceed a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the current block size parameters, namely the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input is the 4x4 DCT2 transformed coefficients, and the output is 8 coefficients. This means that for a 4×4 block, the LFNST output can only have a maximum of 8 coefficients. Therefore, for the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), it is definitely not an LFNST output. This allows the decoder to know that the current block is definitely not using LFNST and therefore does not need to parse its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx. Furthermore, related art requires that the last non-zero coefficient position of any color component cannot exceed its maximum threshold. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three color components Y, U, and V cannot be greater than its maximum threshold.

[0319] In this way, considering that this condition of LFNST / NSPT is designed for intra-frame prediction, and LFNST / NSPT can also be used for inter-frame prediction, but their coefficients and residual rules are different, this restriction condition can be modified.

[0320] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0321] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0322] It should be noted that in the embodiments of the present application, the second threshold may also be referred to as a maximum threshold. In one specific embodiment, the second threshold may be determined based on the size parameters of the current block. For example, the second threshold may be determined based on the length and width of the current block. In another specific embodiment, the second threshold may be set to the difference between the maximum number of coefficients in the transform coefficients of the current block and one. For example, if the LFNST output of the current block may have a maximum of 8 coefficients, the second threshold may be set to 7.

[0323] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the second threshold. That is, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is less than the second threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is less than the second threshold.

[0324] For example, for intra-frame predicted blocks, the relevant technology is still used. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all color components is not greater than its maximum threshold. For inter-frame predicted blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all its components is not greater than the maximum threshold. For single-tree partitioned blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required.

[0325] That is, in the embodiment of the present application, lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is greater than or equal to the minimum threshold, and the U and V components are no longer required. Alternatively, lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the U and V components are no longer required.

[0326] This embodiment provides a decoding method, which determines the mode parameters of the chrominance components of the current block; when the mode parameters meet the first condition, determines the first transform kernel of the chrominance components of the current block; determines the transform coefficients of the chrominance components of the current block, and inversely transforms the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine the residual block of the chrominance components of the current block. In this way, the mode parameters of the current block are first determined; when the mode parameters meet the first condition, the first transform kernel of the chrominance components of the current block is determined, and then the residual block of the chrominance components of the current block is transformed according to the first transform kernel. In other words, the determination of the first transform kernel is related to the mode parameters, which enables the application of the MTS / LFNST / NSPT transform to the chrominance components of a single tree partition, thereby not only improving the compression efficiency, but also improving the encoding and decoding performance.

[0327] In another embodiment of the present application, see Figure 42, which shows a schematic flow chart of an encoding method provided by an embodiment of the present application. As shown in Figure 42, the method may include:

[0328] S4201: Determine the mode parameters of the chrominance components of the current block.

[0329] It should be noted that in the embodiment of the present application, the method is applied to an encoder. Specifically, based on the structure of the encoder 100 shown in Figure 37, the encoding method of the embodiment of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, the optimization scheme of the MTS / LFNST / NSPT transform proposed for the chroma component of a single tree partition is mainly used to improve compression efficiency.

[0330] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0331] It should also be noted that, in an embodiment of the present application, the partition type of the current block may include single tree partitioning and dual tree partitioning. In a single tree, the luminance component and the chrominance component are partitioned together, and a CU contains a luminance block and a chrominance block at the same position; in a dual tree, the luminance and chrominance are partitioned separately, and a CU contains only luminance blocks or only chrominance blocks. It should be noted that the block of the luminance component of the current block can be called a "luminance block", and the block of the chrominance component of the current block can be called a "chrominance block".

[0332] It should also be noted that although LFNST could originally be used on the chroma components of a single tree during standardization, to reduce hardware complexity, it is more effective to not use LFNST on chroma components. For example, not using LFNST on chroma components can shorten the pipeline compared to using it on chroma components. Based on this, in the embodiments of this application, the following restrictions are added to the use of chroma components in a single tree.

[0333] S4202: When the mode parameter satisfies a first condition, determine a first transform kernel of the chrominance component of the current block.

[0334] It should be noted that in the embodiment of the present application, the restriction condition is that the mode parameters meet the first condition, which may include: the partition type of the current block is single-tree partitioning and the prediction mode of the chrominance component of the current block is a cross-component prediction mode. The cross-component prediction mode may include CCLM, CCCM, inter-CCCM, etc.

[0335] In an embodiment of the present application, if the chroma component of the current block in a single-tree partition uses a cross-component prediction mode, then one possible implementation is to not apply MTS / LFNST / NSPT to the chroma component of the current block. In this case, a preset transform kernel can be determined as the first transform kernel for the chroma component of the current block. The preset transform kernel is the default transform kernel.

[0336] For example, if the chrominance component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT is not applied to the chrominance component of the current block. Instead, the default transform kernel is used, such as separable DCT2 in the horizontal and vertical directions respectively.

[0337] In an embodiment of the present application, if the chroma component of the current block in a single-tree partition uses a cross-component prediction mode, another possible implementation is that MTS / LFNST / NSPT uses a different transform kernel group for the chroma component than for the luma component. In this case, a first transform kernel group for the chroma component of the current block can be determined based on the first transform mode; wherein the first transform kernel group is different from the transform kernel group for the luma component of the current block; and a first transform kernel is determined based on the first transform kernel group.

[0338] It should be noted that, in the embodiment of the present application, the first transform mode includes at least one of the following: multi-transform selection mode (MTS), low-frequency non-separable transform mode (LFNST) and non-separable basic transform mode (NSPT).

[0339] For example, if the chroma component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT can be applied to the chroma component of the current block, but the first transform kernel group used by the chroma component is different from the transform kernel group used by the luminance component.

[0340] In an embodiment of the present application, if the chroma component of the current block under single-tree partitioning uses a cross-component prediction mode, then another possible implementation method is that MTS / LFNST / NSPT uses a different transform kernel for the chroma component than for the luminance component. In this case, a first transform kernel group for the chroma component of the current block can be determined based on the first transform mode; wherein the first transform kernel group is the same as the transform kernel group for the luminance component of the current block; and a first transform kernel is determined based on the first transform kernel group; wherein the first transform kernel is different from the transform kernel for the luminance component of the current block.

[0341] It should be noted that in the embodiments of the present application, MTS / LFNST / NSPT uses different transform kernels for the chroma component and the luma component. This can be done by using the same transform kernel group but using different transform kernels, or by using different transform kernel groups and using different transform kernels.

[0342] It should also be noted that, in the embodiment of the present application, the first transform mode includes at least one of the following: multi-transform selection mode (MTS), low-frequency non-separable transform mode (LFNST) and non-separable basic transform mode (NSPT).

[0343] For example, if the chroma component of the single tree uses the cross-component prediction mode, then MTS / LFNST / NSPT can be applied to the chroma component of the current block, but the first transform kernel used by the chroma component is different from the transform kernel used by the luminance component.

[0344] It should also be noted that when determining the first transform kernel group for the chrominance components of the current block based on the first transform mode, different transform modes may correspond to different transform kernel groups. For example, the multi-transform selection mode may determine the MTS transform kernel group, the low-frequency inseparable transform mode may determine the LFNST transform kernel group, and the inseparable basic transform mode may determine the NSPT transform kernel group. Each transform kernel group may include at least one candidate transform kernel. After determining the first transform kernel group for the chrominance components, the first transform kernel used for the chrominance components of the current block may be determined from the candidate transform kernel group.

[0345] That is to say, in the embodiment of the present application, MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma component than for the luminance component. Among them, LFNST / NSPT in some technologies will determine a transform kernel group based on the intra-frame prediction mode. In fact, this intra-frame prediction mode represents a texture feature, such as the texture at a certain angle. LFNST / NSPT can match the corresponding transform kernel group based on this texture feature. There are multiple transform kernels to choose from in each transform kernel group, for example, 3 transform kernels to choose from. Similarly, intra-frame MTS also matches the transform kernel based on the intra-frame prediction mode. For inter-frame prediction, it itself does not have an intra-frame prediction mode. In the above content, a virtual intra-frame prediction mode is derived using the inter-frame prediction block. This virtual prediction mode can also be called a texture feature index, which is used to match the transform kernel group of LFNST / NSPT, and then the first transform kernel of the current block is determined based on the transform kernel group.

[0346] Understandably, if the chroma and luma components of a single-tree CU are predicted using the same prediction method, their residuals may also conform to the same texture characteristics, such as texture at a specific angle. However, if the chroma and luma components of a single-tree CU are predicted using different prediction methods, their residuals may conform to different texture characteristics. A specific example is cross-component prediction. Cross-component prediction uses the reconstructed block of the luma component to predict the chroma component. The reconstructed block of the luma component already contains the luma component residual, so the prediction block of the chroma component contains information converted from the luma component residual. If the luma component and chroma component have different texture characteristics, different transform kernels or transform kernel groups can be used for them. Another specific example is for LFNST / NSPT: the luma component uses its intra-frame prediction mode or virtual intra-frame prediction mode to match a transform kernel group. If cross-component prediction is used for the chroma component, the transform kernel group corresponding to DC or PLANAR is used for the chroma component, or a transform kernel group specifically optimized for cross-component prediction is used. This is not specifically limited here.

[0347] It should also be noted that the above description is for texture features. For the commonly used YUV4:2:0 format, the number of pixels in the luma component of the same CU is four times the number of pixels in the chroma component, which means that the sizes of the luma and chroma blocks are different. For example, if the luma block is 16×16, then the corresponding chroma block is 8×8, so the transform kernel or transform kernel group must also be of corresponding size.

[0348] In some embodiments, determining the first transformation core based on the first transformation core group may include: performing cost value calculations on at least one candidate transformation core in the first transformation core group to determine the cost value of each of the at least one candidate transformation cores; determining the minimum cost value from the cost values ​​of the at least one candidate transformation core, and using the candidate transformation core corresponding to the minimum cost value as the first transformation core.

[0349] In one possible implementation, cost value calculation is performed on at least one candidate transform kernel in the first transform kernel group. Taking the first candidate transform kernel as an example, the method may include: transforming and quantizing the residual block of the chrominance component of the current block based on the first candidate transform kernel, determining the first candidate quantization coefficient of the chrominance component of the current block, and performing entropy coding on the first candidate quantization coefficient to determine the first generation value of the first candidate transform kernel; inverse quantizing and inverse transforming the first candidate quantization coefficient to determine the first candidate residual block of the chrominance component of the current block, and determining the first candidate prediction block of the chrominance component of the current block based on the first candidate residual block; performing cost calculation based on the first candidate prediction block and the original block of the chrominance component of the current block to determine the second generation value of the first candidate transform kernel; and determining the cost value of the first candidate transform kernel based on the first generation value and the second generation value of the first candidate transform kernel.

[0350] It should be noted that, in the embodiment of the present application, the first candidate transformation core is any one of the at least one candidate transformation core, so that the cost value of each of the at least one candidate transformation core can be determined.

[0351] It should also be noted that, in the embodiment of the present application, the first generation value may represent the cost of the first candidate transform core incurred in the codestream, and the second generation value may represent the cost of the distortion of the first candidate transform core. Here, determining the cost value of the first candidate transform core based on the first generation value and the second generation value of the first candidate transform core may include: summing the first generation value and the second generation value of the first candidate transform core, and using the sum as the cost value of the first candidate transform core.

[0352] It should also be noted that in the embodiment of the present application, the cost value calculation here can be determined based on the cost result of Rate Distortion Optimization (RDO), or based on the cost result of Sum of Absolute Difference (SAD), or even based on the cost result of Sum of Absolute Transformed Difference (SATD), but is not specifically limited thereto.

[0353] It should also be noted that, in the embodiments of the present application, the first transform kernel group may include one, two, three, or more candidate transform kernels. If the first transform kernel group includes only one candidate transform kernel, then this candidate transform kernel can be directly used as the first transform kernel for the chrominance component of the current block without performing cost value calculation.

[0354] In some embodiments, the method further includes: determining a value of first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block uses a first transform mode and the corresponding transform core index used; encoding the value of the first syntax identification information, and writing the obtained encoded bits into the bitstream.

[0355] It should be noted that in an embodiment of the present application, the first syntax identification information can be used to indicate whether the current block uses the first transform mode, and when the current block uses the first transform mode, the value of the first syntax identification information can also be used to indicate the transform core index used by the current block.

[0356] It should also be noted that, in an embodiment of the present application, determining the value of the first syntax identification information may include: when the current block does not use the first transformation mode, determining that the value of the first syntax identification information is a first value; when the current block uses the first transformation mode, determining that the value of the first syntax identification information is a second value; wherein the value of the second value is associated with the transformation core index of the first transformation core.

[0357] Here, the first value is different from the second value. In a specific embodiment, the first value can be set to 0, and the second value can be set to a non-zero value, such as 1, 2, 3, etc. That is, when the current block uses the first transform mode, the first syntax identification information is also used to indicate which transform kernel is used.

[0358] It should also be noted that, in the embodiment of the present application, the transform core index here is used to indicate the number of the first transform core of the current block in the first transform core group. For example, if the current block does not use the first transform mode, then the value of the first syntax identification information can be determined to be 0; if the current block uses the first transform mode, and the first transform core is the first transform core of the first transform core group (that is, the transform core index is 0), then the value of the first syntax identification information can be determined to be 1; if the current block uses the first transform mode, and the first transform core is the second transform core of the first transform core group (that is, the transform core index is 1), then the value of the first syntax identification information can be determined to be 2; if the current block uses the first transform mode, and the first transform core is the third transform core of the first transform core group (that is, the transform core index is 2), then the value of the first syntax identification information can be determined to be 3.

[0359] It should also be noted that in this embodiment of the present application, the first syntax identification information can also be replaced by two other syntax identification information. One syntax identification information is used to indicate whether the current block uses MTS / LFNST / NSPT, and the other syntax identification information is used to indicate the corresponding transform core index when MTS / LFNST / NSPT is used in the current block. In other words, after determining the transform core index of the first transform core corresponding to the current block, the transform core index can also be encoded, and the resulting encoded bits can be written into the bitstream.

[0360] This requires transmitting two syntax elements in the bitstream, resulting in high bitstream overhead. In this case, if the transform core index needs to be transmitted in the bitstream, then if the first transform core group contains only one transform core, then the transform core index does not need to be transmitted. Specifically, if MTS / LFNST / NSPT is used in the current block, the only transform core in the first transform core group can be directly used as the first transform core for the current block.

[0361] It should also be noted that, in the embodiment of the present application, the first syntax identification information can be replaced by three other syntax identification information. Among them, one syntax identification information is used to indicate whether the current block uses NSPT and which NSPT transformation core is used, another syntax identification information is used to indicate whether the current block uses LFNST and which LFNST transformation core is used, and another syntax identification information is used to indicate whether the current block uses MTS and which MTS transformation core is used; these are essentially the same and are not specifically limited here.

[0362] S4203: Determine a residual block of the chrominance component of the current block, and transform the residual block of the chrominance component of the current block according to a first transform kernel to determine transform coefficients of the chrominance component of the current block.

[0363] S4204: Encode the transform coefficients of the chrominance components of the current block, and write the obtained coded bits into the bitstream.

[0364] It should be noted that, in an embodiment of the present application, determining the residual block of the chrominance component of the current block may include: determining the predicted block of the chrominance component of the current block; and determining the residual block of the chrominance component of the current block based on the original block of the chrominance component of the current block and the predicted block of the chrominance component of the current block.

[0365] In this embodiment of the present application, intra-frame prediction can be performed on the chroma components of the current block to determine the predicted block for the chroma components of the current block; alternatively, inter-frame prediction can be performed on the chroma components of the current block to determine the predicted block for the chroma components of the current block. In other words, MTS / LFNST / NSPT is applied to the chroma components of a single tree partition. Optionally, it can be applied to single-tree partitioning within a frame and / or single-tree partitioning between frames.

[0366] It should also be noted that, in an embodiment of the present application, after determining the predicted block of the chrominance component of the current block, a subtraction operation can be performed on the original block of the chrominance component of the current block and the predicted block of the chrominance component of the current block to determine the residual block of the chrominance component of the current block.

[0367] In some embodiments, the method further includes: determining a second transform kernel of the chrominance component of the current block when the mode parameter does not satisfy the first condition; and determining a residual block of the chrominance component of the current block, and transforming the residual block of the chrominance component of the current block according to the second transform kernel to determine the transform coefficient of the chrominance component of the current block; encoding the transform coefficient of the chrominance component of the current block, and writing the obtained coded bits into the bitstream.

[0368] That is, in an embodiment of the present application, if the mode parameters satisfy the first condition, a first transform kernel of the chroma components of the current block may be determined, and then the residual block of the chroma components of the current block may be transformed according to the first transform kernel to determine the transform coefficients of the chroma components of the current block. If the mode parameters do not satisfy the first condition, a second transform kernel of the chroma components of the current block may be determined, and then the residual block of the chroma components of the current block may be transformed according to the second transform kernel to determine the transform coefficients of the chroma components of the current block. Finally, the transform coefficients of the chroma components of the current block may be encoded, and the resulting encoded bits may be written into the bitstream.

[0369] In an embodiment of the present application, if the mode parameter does not meet the first condition, it may include: the partition type of the current block is dual-tree partitioning or the prediction mode of the chrominance component of the current block is non-cross-component prediction mode. In this case, if the current block is dual-tree partitioning or the chrominance component of the current block does not use cross-component prediction mode, then MTS / LFNST / NSPT can be used directly.

[0370] In some embodiments, determining the second transform kernel of the chrominance component of the current block may include: determining a second transform kernel group of the chrominance component of the current block based on the first transform mode; determining a second transform kernel according to the second transform kernel group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0371] It should be noted that, in an embodiment of the present application, determining the second transformation core based on the second transformation core group may specifically include: performing cost value calculations on at least one candidate transformation core in the second transformation core group to determine the cost value of each of the at least one candidate transformation cores; determining the minimum cost value from the cost values ​​of each of the at least one candidate transformation cores, and using the candidate transformation core corresponding to the minimum cost value as the second transformation core.

[0372] It should also be noted that in the embodiment of the present application, the cost value calculation here can be determined based on the cost result of RDO, or based on the cost result of SAD, or even based on the cost result of SATD, but is not specifically limited.

[0373] It should also be noted that, in the embodiments of the present application, the second transform core group may include one, two, three, or more candidate transform cores. If the second transform core group includes only one candidate transform core, then this candidate transform core can be directly used as the second transform core of the current block without performing cost calculation.

[0374] It can be understood that in an embodiment of the present application, if the current block uses the multi-transform selection mode, then the MTS transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom. If the current block uses the low-frequency inseparable transform mode, then the LFNST transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom. If the current block uses the inseparable basic transform mode, then the NSPT transform core group of the current block can be determined according to the intra-frame prediction mode or virtual intra-frame prediction mode or texture feature index of the current block, and then the second transform core to be used is determined therefrom.

[0375] It should also be noted that in an embodiment of the present application, for the first syntax identification information, if two syntax identification information are used to replace it, one syntax identification information is used to indicate whether the current block uses MTS / LFNST / NSPT, and the other syntax identification information is used to indicate the corresponding transform core index when the current block uses MTS / LFNST / NSPT, then when determining the second transform core of the current block, the second transform core index of the current block can also be determined by decoding the code stream, and then the second transform core is determined based on the second transform core group and the second transform core index. However, since two syntax elements need to be transmitted in the code stream, the code stream overhead is large. In this case, for the case where the transform core index needs to be transmitted in the code stream, if there is only one transform core in the determined second transform core group, then there is no need to transmit the transform core index. Specifically, when the current block uses MTS / LFNST / NSPT, the only transform core in the second transform core group can be directly used as the second transform core.

[0376] In a possible implementation, when the mode parameter satisfies the first condition, the residual block of the chroma component of the current block is transformed according to the first transform kernel to determine the transform coefficient of the chroma component of the current block, which may include: when the first syntax identification information indicates that the chroma component of the current block uses NSPT, the residual block of the chroma component of the current block is subjected to an inseparable basic transform according to the first transform kernel to determine the transform coefficient of the chroma component of the current block; or, when the first syntax identification information indicates that the chroma component of the current block uses LFNST, the residual block of the chroma component of the current block is subjected to a low-frequency inseparable transform according to the first transform kernel to determine the transform block of the chroma component of the current block; and discrete cosine transform is performed on the transform block of the chroma component to determine the transform coefficient of the chroma component of the current block; or, when the first syntax identification information indicates that the chroma component of the current block uses MTS, the residual block of the chroma component of the current block is subjected to a multi-transform selection transform according to the first transform kernel to determine the transform coefficient of the chroma component of the current block.

[0377] In another possible implementation, when the mode parameter does not meet the first condition, the residual block of the chroma component of the current block is transformed according to the second transform core to determine the transform coefficient of the chroma component of the current block, which may include: when the first syntax identification information indicates that the chroma component of the current block uses NSPT, the residual block of the chroma component of the current block is subjected to an inseparable basic transform according to the second transform core to determine the transform coefficient of the chroma component of the current block; or, when the first syntax identification information indicates that the chroma component of the current block uses LFNST, the residual block of the chroma component of the current block is subjected to a low-frequency inseparable transform according to the second transform core to determine the transform block of the chroma component of the current block; and discrete cosine transform is performed on the transform block of the chroma component to determine the transform coefficient of the chroma component of the current block; or, when the first syntax identification information indicates that the chroma component of the current block uses MTS, the residual block of the chroma component of the current block is subjected to a multi-transform selection transform according to the second transform core to determine the transform coefficient of the chroma component of the current block.

[0378] That is, in the embodiment of the present application, if the chroma component in a single-tree partitioned CU uses a cross-component prediction mode, then MTS / LFNST / NSPT is not used for the chroma component of such a CU. If the chroma component in a single-tree partitioned CU uses a cross-component prediction mode, then MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma component than for the luma component.

[0379] It should also be noted that in the embodiments of this application, the "transformation" of the residual block by the encoder can also be called a "forward transform," specifically referring to the transformation from the spatial domain to the frequency domain to remove residual correlation. It should be noted that if the standard only specifies decoding, then the "transformation" in the standard text refers to the decoding part, specifically referring to the "inverse transform" in this article.

[0380] Furthermore, in some embodiments, referring to FIG43 , the method may further include:

[0381] S4301: Determine the transformation coefficients of the current block.

[0382] S4301: quantize the transform coefficients of the current block to determine the quantization coefficients of the current block.

[0383] S4303: Encode the quantized coefficients of the current block and write the obtained coded bits into the bitstream.

[0384] In an embodiment of the present application, the transform coefficients of the current block may include the transform coefficients of the chrominance components of the current block and the transform coefficients of the luminance components of the current block. In a specific embodiment, for the transform coefficients of the chrominance components of the current block, the method further includes: quantizing the transform coefficients of the chrominance components of the current block to determine the quantization coefficients of the chrominance components of the current block; encoding the quantization coefficients of the chrominance components of the current block, and writing the obtained coded bits into the bitstream. In another specific embodiment, for the transform coefficients of the luminance components of the current block, the method further includes: quantizing the transform coefficients of the luminance components of the current block to determine the quantization coefficients of the luminance components of the current block; encoding the quantization coefficients of the luminance components of the current block, and writing the obtained coded bits into the bitstream.

[0385] It is also understandable that in the embodiment of the present application, the first syntax identification information may be encoded into the bitstream only when the last non-zero coefficient position of the current block meets certain conditions. In some embodiments, referring to FIG. 44 , the method may further include:

[0386] S4401: Determine the transformation coefficients of the current block.

[0387] S4402: When the position of the last non-zero coefficient in the transformation coefficients of the current block meets a preset condition, encoding the value of the first syntax identification information, and writing the obtained encoding bits into the bitstream.

[0388] It should be noted that, in an embodiment of the present application, encoding the value of the first syntax identification information may include: binarizing the value of the first syntax identification information to determine the binary symbol string of the current block; and encoding at least one binary symbol in the binary symbol string according to at least one context model, and writing the obtained encoded bits into the code stream.

[0389] It should also be noted that in the embodiment of the present application, the first syntax identification information can be represented by lfnst_nspt_idx. In related technologies, lfnst_nspt_idx is only encoded if certain conditions are met. One of these conditions is that the last non-zero coefficient position is greater than or equal to 1. The quantized coefficients are scanned using a hierarchical diagonal scan. A coefficient block can be divided into 4×4 sub-blocks. The 4×4 sub-blocks are scanned using a diagonal scan, and the 4×4 sub-blocks are also scanned using a diagonal scan. The position (0, 0) in the upper left corner of the current block is 0 in the scan order. The last non-zero coefficient position is greater than or equal to 1, which means that the current block has coefficients other than (0, 0). For DCT2 transform, (0, 0) is the DC coefficient. In other words, if the current block has only a non-zero coefficient at position (0, 0) or no non-zero coefficients, the syntax element lfnst_nspt_idx does not need to be encoded and defaults to 0. Because LFNST / NSPT is unlikely to be used in this case, this restriction significantly reduces unnecessary lfnst_nspt_idx overhead. Furthermore, related technologies require that the last non-zero coefficient position of all color components must be greater than or equal to 1. For example, in a single-tree block in YUV format, the last non-zero coefficient position of all three color components must be greater than or equal to 1.

[0390] Thus, for inter-frame predicted blocks, the residuals are inherently smaller than those for intra-frame predicted blocks. Furthermore, given that the quality requirements for chrominance components are generally much lower than those for luminance components, inter-frame predicted blocks often have certain luminance coefficients but no chrominance coefficients. Since this LFNST / NSPT condition is designed for intra-frame prediction, and LFNST / NSPT can currently also be used for inter-frame prediction, but their coefficients and residual patterns are different, the present invention can modify this restriction.

[0391] In one possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0392] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0393] It should be noted that, in the embodiment of the present application, the first threshold value may also be referred to as the minimum threshold value, wherein the minimum threshold value may be set to 1, or a larger value such as 2, 3, etc., which is not specifically limited here.

[0394] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the first threshold. In other words, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is greater than the first threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is greater than the first threshold.

[0395] For example, assuming the minimum threshold is equal to 1, the relevant technology is still used for intra-frame predicted blocks. That is, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all color components is greater than or equal to 1; for inter-frame predicted blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all its color components is greater than or equal to 1; for single-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the U and V components are no longer required.

[0396] It's also understandable that in related art, lfnst_nspt_idx is only encoded if certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot exceed a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the current block size parameters, namely the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input is the 4x4 DCT2 transformed coefficients, and the output is 8 coefficients. This means that for a 4×4 block, the LFNST output can only have a maximum of 8 coefficients. Therefore, for the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), it is definitely not an LFNST output. This allows the decoder to know that the current block is definitely not using LFNST and therefore does not need to encode its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx. Furthermore, related art requires that the last non-zero coefficient position of any color component cannot exceed its maximum threshold. For example, in a block divided by a single tree in the YUV format, the last non-zero coefficient position of the three color components Y, U, and V cannot be greater than its maximum threshold.

[0397] In this way, considering that this condition of LFNST / NSPT is designed for intra-frame prediction, and LFNST / NSPT can also be used for inter-frame prediction, but their coefficients and residual rules are different, this restriction condition can be modified.

[0398] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block satisfies a preset condition, which may include: when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0399] In another possible implementation, the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition, which may include: when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to a second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0400] It should be noted that in the embodiments of the present application, the second threshold may also be referred to as a maximum threshold. In one specific embodiment, the second threshold may be determined based on the size parameters of the current block. For example, the second threshold may be determined based on the length and width of the current block. In another specific embodiment, the second threshold may be set to the difference between the maximum number of coefficients in the transform coefficients of the current block and one. For example, if the LFNST output of the current block may have a maximum of 8 coefficients, the second threshold may be set to 7.

[0401] It should also be noted that the embodiments of the present application do not limit the situation where the last non-zero coefficient position is equal to the second threshold. That is, the last non-zero coefficient position in the transform coefficients of the current block meets the preset condition, which may also include: when the prediction mode of the current block is intra-frame prediction mode or dual-tree partitioning, the last non-zero coefficient position of all color components in the transform coefficients of the current block is less than the second threshold; or when the prediction mode of the current block is inter-frame prediction mode or single-tree partitioning, the last non-zero coefficient position of the luminance component in the transform coefficients of the current block is less than the second threshold.

[0402] For example, for intra-frame predicted blocks, the relevant technology is still used. That is, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all color components is not greater than its maximum threshold; for inter-frame predicted blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required. Alternatively, for dual-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of all its color components is not greater than the maximum threshold; for single-tree partitioned blocks, lfnst_nspt_idx is encoded only when the last non-zero coefficient position of the Y component is not greater than its maximum threshold, and the U and V components are no longer required.

[0403] That is, in the embodiment of the present application, lfnst_nspt_idx can be encoded only when the last non-zero coefficient position of the Y component is greater than or equal to the minimum threshold, and the U and V components are no longer required. Alternatively, lfnst_nspt_idx can be encoded only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the U and V components are no longer required.

[0404] In another embodiment of the present application, the embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: a quantization coefficient of the chrominance component of the current block and a value of the first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block is allowed to use the first transform mode and the corresponding transform kernel index used.

[0405] In an embodiment of the present application, the first transform mode may be MTS / LFNST / NSPT. When the mode parameters meet the first condition, for example, if the chroma component in a CU partitioned by a single tree uses a cross-component prediction mode, MTS / LFNST / NSPT is not used for the chroma component of such a CU; if the chroma component in a CU partitioned by a single tree uses a cross-component prediction mode, MTS / LFNST / NSPT may use a different transform kernel or transform kernel group for the chroma component than for the luma component.

[0406] This embodiment provides a coding method, which determines the mode parameters of the chrominance components of the current block; when the mode parameters meet the first condition, determines the first transform kernel of the chrominance components of the current block; determines the residual block of the chrominance components of the current block, and transforms the residual block of the chrominance components of the current block according to the first transform kernel to determine the transform coefficients of the chrominance components of the current block; encodes the transform coefficients of the chrominance components of the current block, and writes the obtained coded bits into the bitstream. In this way, the mode parameters of the current block are first determined; when the mode parameters meet the first condition, the first transform kernel of the chrominance components of the current block is determined, and then the residual block of the chrominance components of the current block is transformed according to the first transform kernel. In other words, the determination of the first transform kernel is related to the mode parameters, and it is possible to apply the MTS / LFNST / NSPT transform to the chrominance components of a single tree partition, thereby not only improving the compression efficiency, but also improving the encoding and decoding performance.

[0407] In another embodiment of the present application, based on the encoding and decoding method described in the above embodiment, the embodiment of the present application can use MTS / LFNST / NSPT for the chrominance component of a single tree. Optionally, it can be used for a single tree within a frame and / or a single tree between frames. However, considering the hardware complexity and pipeline length, the following restrictions need to be met:

[0408] If the chroma component in the CU of the single tree uses cross-component technology, such as CCLM, CCCM, inter-CCCM, etc., one possible implementation method is to not use MTS / LFNST / NSPT for the chroma component of such CU, but use the default transform kernel, such as using separable DCT2 in the horizontal and vertical directions respectively.

[0409] Another possible implementation is that MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma component than for the luminance component. Specifically, in some technologies, LFNST / NSPT will determine a transform kernel group based on the intra-frame prediction mode. In fact, this intra-frame prediction mode represents a texture feature, such as the texture at a certain angle. LFNST / NSPT matches the corresponding transform kernel group based on this feature. Each transform kernel group has multiple transform kernels to choose from, for example, 3 transform kernels to choose from. Similarly, intra-frame MTS also matches the transform kernel based on the intra-frame prediction mode. For inter-frame, it itself does not have an intra-frame prediction mode. In the above content, a virtual intra-frame prediction mode can be derived from the inter-frame prediction block. This virtual prediction mode can also be called a texture feature index, which is used to match the transform kernel group of LFNST / NSPT.

[0410] Furthermore, for a single-tree CU predicted using the same prediction method for the chroma and luma components, their residuals may also conform to the same texture characteristics, such as texture at a specific angle. However, for a single-tree CU predicted using different prediction methods for the chroma and luma components, their residuals may conform to different texture characteristics. A typical example is cross-component prediction. Cross-component prediction involves predicting the chroma component using the reconstructed block of the luma component. The reconstructed block of the luma component already contains the luma residual, so the prediction block of the chroma component contains information converted from the luma residual. If the luma and chroma components conform to different texture characteristics, different transform kernels or transform kernel groups can be used for them. An example is for LFNST / NSPT: the luma component uses its intra prediction mode or virtual intra prediction mode to match a transform kernel group. If cross-component prediction is used for the chroma components, the transform kernel group corresponding to DC or PLANAR can be used for the chroma components, or a transform kernel group specifically optimized for cross-component prediction can be used.

[0411] In the embodiments of this application, these are all for texture features. For the commonly used YUV4:2:0 format, the number of pixels of the surface luma component in the same CU is four times the number of pixels of the chroma component, which means that the sizes of the luma block and the chroma block are different. For example, if the luma block is 16×16, then the corresponding chroma block is 8×8, so the transform kernel or transform kernel group must also be of corresponding size.

[0412] That is, in this embodiment of the present application, MTS / LFNST / NSPT is used for the chroma components of a single tree. If the chroma components of a CU in a single tree use cross-component technology, MTS / LFNST / NSPT is not used for the chroma components of such CU. If the chroma components of a CU in a single tree use cross-component technology, MTS / LFNST / NSPT uses a different transform kernel or transform kernel group for the chroma components than for the luma components.

[0413] It can also be understood that in an embodiment of the present application, the first syntax identification information (such as lfnst_nspt_idx) will only be parsed if certain conditions are met. One of the conditions is that the position of the last non-zero coefficient is greater than or equal to 1. The scanning of the quantized coefficients is performed according to the hierarchical diagonal scan. A coefficient block is divided into 4×4 sub-blocks. The 4×4 sub-blocks are scanned according to the diagonal scan, and the 4×4 sub-blocks are also scanned according to the diagonal scan. The position (0, 0) in the upper left corner of the current block is 0 in the scanning order, and the position of the last non-zero coefficient is greater than or equal to 1, which means that the current block has coefficients other than (0, 0). For DCT2 transform, (0, 0) is the DC coefficient. That is to say, if the coefficients of the current block have only non-zero coefficients at position (0, 0) or no non-zero coefficients, then the syntax element lfnst_nspt_idx does not need to be parsed and defaults to 0. Because LFNST / NSPT is unlikely to be used in this situation, this restriction significantly reduces unnecessary lfnst_nspt_idx overhead. Furthermore, related technologies require that the last non-zero coefficient position of all components be greater than or equal to 1. For example, in a block partitioned by a single tree in YUV format, the last non-zero coefficient position of all three components, Y, U, and V, must be greater than or equal to 1.

[0414] Inter-coded blocks inherently have fewer residuals than intra-coded blocks. Furthermore, given that chroma quality requirements are generally much lower than luma, inter-coded blocks often have certain luma coefficients but no chroma coefficients. While the LFNST / NSPT constraint was designed for intra-coded blocks, inter-coded blocks can now also use LFNST / NSPT. However, their coefficients and residual patterns differ, so this constraint can be modified.

[0415] One possible implementation is to continue using the relevant techniques for intra-coded blocks. That is, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all components is greater than or equal to 1. For inter-coded blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1. The UV component is no longer required.

[0416] Another possible implementation is: for a block partitioned by a dual tree, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of all its components is greater than or equal to 1. For a block partitioned by a single tree, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1. The UV component is no longer required.

[0417] Furthermore, in the related art, the judgment condition is set to be that the last non-zero coefficient position is greater than or equal to 1, that is, the minimum possible threshold is 1, which is called the minimum threshold here. The minimum threshold can also be set to a larger value, such as 2, 3, etc. For example, lfnst_nspt_idx is only parsed when the last non-zero coefficient position is greater than or equal to 2.

[0418] In related art, lfnst_nspt_idx is parsed only when certain conditions are met. One of these conditions is that the position of the last non-zero coefficient cannot be greater than a threshold. This threshold is the maximum possible number of coefficients for LFNST or NSPT minus one, referred to here as the maximum threshold. The maximum threshold can be determined based on the block size, i.e., the length and width. For example, when the encoder performs a secondary transform on a 4×4 block using LFNST, the input to the LFNST is the 4×4 coefficients after the DCT2 transform, and the output is 8 coefficients. This means that for a 4x4 block, the LFNST output can only have a maximum of 8 coefficients. For the decoder, if the last non-zero coefficient position of a 4×4 block exceeds 7, i.e., (8-1), then it is definitely not an LFNST output. The decoder then knows that the current block is definitely not using LFNST and does not need to parse its syntax elements. This restriction significantly reduces unnecessary overhead for lfnst_nspt_idx.

[0419] Furthermore, related technologies require that the last non-zero coefficient position of any component cannot exceed its maximum threshold. For example, in a block partitioned by a single tree in YUV format, the last non-zero coefficient position of each of the Y, U, and V components cannot exceed their maximum thresholds. This constraint is designed for intra-frame coding, and while inter-frame coding can now also use LFNST / NSPT, their coefficients and residual patterns differ, so this constraint can be modified.

[0420] One possible implementation is to continue using the relevant techniques for intra-coded blocks. Specifically, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of any component is no greater than its maximum threshold. For inter-coded blocks, lfnst_nspt_idx is parsed only when the last non-zero coefficient position of the Y component is no greater than its maximum threshold. The UV component is no longer required.

[0421] Another possible implementation is to parse lfnst_nspt_idx for blocks partitioned by dual trees only when the last non-zero coefficient position of all its components is not greater than the maximum threshold. For blocks partitioned by single trees, parse lfnst_nspt_idx only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold. UV components are no longer required.

[0422] That is to say, in the embodiment of the present application, lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is greater than or equal to 1, and the UV component is no longer required; and lfnst_nspt_idx can be parsed only when the last non-zero coefficient position of the Y component is not greater than the maximum threshold, and the UV component is no longer required.

[0423] In the embodiment of the present application, the specific implementation of the aforementioned embodiment is described in detail through the above embodiment. It can be seen that according to the technical solution of the aforementioned embodiment, the use of MTS / LFNST / NSPT for the chroma component can improve the compression efficiency. Among them, considering the pipeline length of the chroma component encoding, if the chroma component uses the cross-component prediction mode, that is, the chroma component can only be predicted after the luminance component obtains the reconstructed block, and the cross-component prediction mode is not used for the chroma component, the prediction of the luminance component and the chroma component can be performed in parallel, that is, the cross-component prediction will make the pipeline of the chroma component longer. LFNST is a secondary transformation, which means that if the current block uses LFNST, the inverse transformation of LFNST will be performed first, and then the inverse transformation of the basic transformation will be performed, which will make the pipeline longer. Therefore, when the chroma uses cross-component prediction, LFNST is not used, thereby reducing the pipeline length, which is beneficial to controlling the complexity of the hardware and thus improving the encoding and decoding performance.

[0424] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see Figure 45, which shows a schematic diagram of the composition structure of an encoder provided by an embodiment of the present application. As shown in Figure 45, the encoder 450 may include a first determination unit 4501, a transformation unit 4502 and an encoding unit 4503, wherein:

[0425] A first determining unit 4501 is configured to determine a mode parameter of a chroma component of a current block; and when the mode parameter satisfies a first condition, determine a first transform kernel of the chroma component of the current block;

[0426] a transform unit 4502 configured to determine a residual block of chrominance components of a current block, and transform the residual block of chrominance components of the current block according to a first transform kernel to determine transform coefficients of the chrominance components of the current block;

[0427] The encoding unit 4503 is configured to perform encoding processing on the transform coefficients of the chrominance components of the current block and write the obtained encoding bits into the bitstream.

[0428] In some embodiments, the mode parameter satisfies a first condition, including: a partition type of the current block is a single-tree partition and a prediction mode of a chroma component of the current block is an inter-component prediction mode.

[0429] In some embodiments, the first determining unit 4501 is further configured to determine a preset transform kernel as a first transform kernel of the chrominance component of the current block.

[0430] In some embodiments, the first determination unit 4501 is further configured to determine a first transform kernel group of the chrominance component of the current block based on the first transform mode; wherein the first transform kernel group is different from the transform kernel group of the luminance component of the current block; and determine the first transform kernel according to the first transform kernel group.

[0431] In some embodiments, the first determination unit 4501 is further configured to determine a first transform kernel group of the chrominance component of the current block based on the first transform mode; wherein the first transform kernel group is the same as the transform kernel group of the luminance component of the current block; and determine a first transform kernel based on the first transform kernel group; wherein the first transform kernel is different from the transform kernel of the luminance component of the current block.

[0432] In some embodiments, the first transform mode includes at least one of the following: a multiple transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0433] In some embodiments, the first determination unit 4501 is further configured to perform cost value calculation on at least one candidate transformation core in the first transformation core group, determine the cost value of each of the at least one candidate transformation core; and determine the minimum cost value from the cost values ​​of each of the at least one candidate transformation core, and use the candidate transformation core corresponding to the minimum cost value as the first transformation core.

[0434] In some embodiments, the first determination unit 4501 is further configured to determine the value of the first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block uses the first transform mode and the corresponding transform core index used; the encoding unit 4503 is further configured to encode the value of the first syntax identification information and write the obtained encoded bits into the bitstream.

[0435] In some embodiments, the first determination unit 4501 is further configured to determine that the value of the first syntax identification information is a first value when the current block does not use the first transformation mode; and to determine that the value of the first syntax identification information is a second value when the current block uses the first transformation mode; wherein the value of the second value is associated with the transformation core index of the first transformation core.

[0436] In some embodiments, the first determination unit 4501 is further configured to determine a second transform kernel of the chrominance component of the current block when the mode parameter does not satisfy the first condition; the transform unit 4502 is further configured to determine a residual block of the chrominance component of the current block, and transform the residual block of the chrominance component of the current block according to the second transform kernel to determine the transform coefficient of the chrominance component of the current block; the encoding unit 4503 is further configured to encode the transform coefficient of the chrominance component of the current block and write the obtained encoded bits into the bitstream.

[0437] In some embodiments, the mode parameter does not satisfy the first condition, including: the partition type of the current block is dual-tree partitioning or the prediction mode of the chroma component of the current block is a non-inter-component prediction mode.

[0438] In some embodiments, the first determination unit 4501 is further configured to determine a second transform kernel group of the chroma component of the current block based on the first transform mode; and determine a second transform kernel group based on the second transform kernel group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0439] In some embodiments, referring to FIG. 45 , the encoder 450 may further include a quantization unit 4504 , wherein:

[0440] The first determining unit 4501 is further configured to determine a transform coefficient of a current block; wherein the transform coefficient of the current block includes a transform coefficient of a chrominance component of the current block and a transform coefficient of a luminance component of the current block;

[0441] a quantization unit 4504 configured to quantize the transform coefficients of the current block to determine the quantization coefficients of the current block;

[0442] The encoding unit 4503 is further configured to perform encoding processing on the quantized coefficients of the current block and write the obtained encoding bits into the bit stream.

[0443] In some embodiments, the encoding unit 4503 is further configured to perform a step of encoding the value of the first syntax identification information and writing the obtained encoding bits into the bitstream when the last non-zero coefficient position in the transformation coefficient of the current block meets a preset condition.

[0444] In some embodiments, the first determination unit 4501 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to a first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0445] In some embodiments, the first determination unit 4501 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0446] In some embodiments, the first determination unit 4501 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0447] In some embodiments, the first determination unit 4501 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0448] In some embodiments, the first determining unit 4501 is further configured to determine the second threshold value according to a size parameter of the current block.

[0449] In some embodiments, the first determining unit 4501 is further configured to set the second threshold to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

[0450] In some embodiments, the first determination unit 4501 is further configured to determine a predicted block of the chrominance component of the current block; and determine a residual block of the chrominance component of the current block based on the original block of the chrominance component of the current block and the predicted block of the chrominance component of the current block.

[0451] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0452] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0453] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 450. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0454] Based on the composition of the encoder 450 and the computer-readable storage medium, refer to Figure 46, which shows a specific hardware structure diagram of the encoder 450 provided in an embodiment of the present application. As shown in Figure 46, the encoder 450 may include: a first communication interface 4601, a first memory 4602 and a first processor 4603; each component is coupled together through a first bus system 4604. It can be understood that the first bus system 4604 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 4604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 4604 in Figure 46. Among them,

[0455] The first communication interface 4601 is used to receive and send signals when sending and receiving information with other external network elements;

[0456] A first memory 4602 is used to store computer programs that can be run on the first processor 4603;

[0457] The first processor 4603 is configured to, when running the computer program, execute:

[0458] Determine a mode parameter of a chroma component of a current block; when the mode parameter satisfies a first condition, determine a first transform kernel of the chroma component of the current block; determine a residual block of the chroma component of the current block, and transform the residual block of the chroma component of the current block according to the first transform kernel to determine transform coefficients of the chroma component of the current block; encode the transform coefficients of the chroma component of the current block, and write the obtained coded bits into a bitstream.

[0459] It is understood that the first memory 4602 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 4602 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0460] The first processor 4603 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 4603. The above-mentioned first processor 4603 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 4602, and the first processor 4603 reads the information in the first memory 4602 and completes the steps of the above method in combination with its hardware.

[0461] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0462] Optionally, as another embodiment, the first processor 4603 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0463] This embodiment provides an encoder that first determines a mode parameter for a current block; when the mode parameter satisfies a first condition, determines a first transform kernel for the chroma component of the current block, and then transforms the residual block of the chroma component of the current block based on the first transform kernel. In other words, the determination of the first transform kernel is dependent on the mode parameter, enabling the application of the MTS / LFNST / NSPT transform to the chroma component of a single tree partition, thereby improving both compression efficiency and encoding / decoding performance.

[0464] Based on the same inventive concept as the above embodiment, referring to FIG47 , a schematic diagram of the structure of a decoder provided by an embodiment of the present application is shown. As shown in FIG47 , the decoder 470 may include a second determination unit 4701 and an inverse transformation unit 4702, wherein:

[0465] The second determining unit 4701 is configured to determine a mode parameter of the chroma component of the current block; and when the mode parameter satisfies a first condition, determine a first transform kernel of the chroma component of the current block;

[0466] The inverse transform unit 4702 is configured to determine transform coefficients of the chrominance components of the current block, and perform inverse transform on the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine a residual block of the chrominance components of the current block.

[0467] In some embodiments, the mode parameter satisfies a first condition, including: a partition type of the current block is a single-tree partition and a prediction mode of a chroma component of the current block is an inter-component prediction mode.

[0468] In some embodiments, the second determining unit 4701 is further configured to determine a preset transform kernel as a first transform kernel of the chrominance component of the current block.

[0469] In some embodiments, the second determination unit 4701 is further configured to determine a first transform kernel group of the chrominance component of the current block based on the first transform mode; wherein the first transform kernel group is different from the transform kernel group of the luminance component of the current block; and determine the first transform kernel according to the first transform kernel group.

[0470] In some embodiments, the second determination unit 4701 is further configured to determine a first transform kernel group of the chrominance component of the current block based on the first transform mode; wherein the first transform kernel group is the same as the transform kernel group of the luminance component of the current block; and determine a first transform kernel based on the first transform kernel group; wherein the first transform kernel is different from the transform kernel of the luminance component of the current block.

[0471] In some embodiments, the first transform mode includes at least one of the following: a multiple transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0472] In some embodiments, referring to FIG. 47 , the decoder 470 may further include a decoding unit 4703 configured to decode the code stream and determine the value of the first syntax identification information; a second determination unit 4701 is further configured to determine the transform core index of the current block when the first syntax identification information indicates that the current block uses the first transform mode; and determine the first transform core based on the first transform core group and the transform core index.

[0473] In some embodiments, the second determination unit 4701 is further configured to determine a second transform kernel of the chrominance component of the current block when the mode parameter does not satisfy the first condition; the inverse transform unit 4702 is further configured to determine the transform coefficient of the chrominance component of the current block, and perform an inverse transform on the transform coefficient of the chrominance component of the current block according to the second transform kernel to determine the residual block of the chrominance component of the current block.

[0474] In some embodiments, the mode parameter does not satisfy the first condition, including: the partition type of the current block is dual-tree partitioning or the prediction mode of the chroma component of the current block is a non-inter-component prediction mode.

[0475] In some embodiments, the second determination unit 4701 is further configured to determine a second transform kernel group of the chroma component of the current block based on the first transform mode; and determine a second transform kernel group based on the second transform kernel group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

[0476] In some embodiments, referring to FIG. 47 , the decoder 470 may further include an inverse quantization unit 4704 , wherein:

[0477] The decoding unit 4703 is further configured to decode the code stream and determine the quantization coefficient of the chrominance component of the current block; the inverse quantization unit 4704 is configured to inverse quantize the quantization coefficient of the chrominance component of the current block and determine the transformation coefficient of the chrominance component of the current block.

[0478] In some embodiments, the transformation coefficients of the current block include the transformation coefficients of the chrominance components of the current block and the transformation coefficients of the luminance components of the current block; accordingly, the decoding unit 4703 is further configured to decode the code stream and determine the quantization coefficients of the luminance components of the current block; the dequantization unit 4704 is further configured to dequantize the quantization coefficients of the luminance components of the current block and determine the transformation coefficients of the luminance components of the current block.

[0479] In some embodiments, the decoding unit 4703 is further configured to perform a step of decoding the code stream and determining a value of the first syntax identification information when the position of the last non-zero coefficient in the transformation coefficients of the current block meets a preset condition.

[0480] In some embodiments, the second determination unit 4701 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0481] In some embodiments, the second determination unit 4701 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is greater than or equal to the first threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to the first threshold.

[0482] In some embodiments, the second determination unit 4701 is further configured to, when the prediction mode of the current block is an intra-frame prediction mode, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the prediction mode of the current block is an inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0483] In some embodiments, the second determination unit 4701 is further configured to, when the partition type of the current block is dual-tree partitioning, the last non-zero coefficient position of all color components in the transformation coefficient of the current block is less than or equal to the second threshold; or, when the partition type of the current block is single-tree partitioning, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

[0484] In some embodiments, the second determining unit 4701 is further configured to determine a second threshold value according to a size parameter of the current block.

[0485] In some embodiments, the second determining unit 4701 is further configured to set the second threshold to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

[0486] In some embodiments, the second determination unit 4701 is further configured to determine a prediction block of the chrominance component of the current block; and determine a reconstructed block of the chrominance component of the current block based on the prediction block of the chrominance component of the current block and the residual block of the chrominance component of the current block.

[0487] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0488] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to decoder 470 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0489] Based on the composition of the decoder 470 and the computer-readable storage medium, refer to Figure 48, which shows a specific hardware structure diagram of the decoder 470 provided in an embodiment of the present application. As shown in Figure 48, the decoder 470 may include: a second communication interface 4801, a second memory 4802 and a second processor 4803; each component is coupled together through a second bus system 4804. It can be understood that the second bus system 4804 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 4804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 4804 in Figure 48. Among them,

[0490] The second communication interface 4801 is used to receive and send signals when sending and receiving information with other external network elements;

[0491] The second memory 4802 is used to store computer programs that can be run on the second processor 4803;

[0492] The second processor 4803 is configured to, when running the computer program, execute:

[0493] Determine a mode parameter of a chroma component of a current block; when the mode parameter satisfies a first condition, determine a first transform kernel of the chroma component of the current block; determine a transform coefficient of the chroma component of the current block, and inversely transform the transform coefficient of the chroma component of the current block according to the first transform kernel to determine a residual block of the chroma component of the current block.

[0494] Optionally, as another embodiment, the second processor 4803 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0495] It can be understood that the hardware functions of the second memory 4802 are similar to those of the first memory 4602, and the hardware functions of the second processor 4803 are similar to those of the first processor 4603; they will not be described in detail here.

[0496] This embodiment provides a decoder that first determines the mode parameters of a current block. When the mode parameters satisfy a first condition, a first transform kernel for the chroma components of the current block is determined, and then the residual block of the chroma components of the current block is transformed based on the first transform kernel. In other words, the determination of the first transform kernel is dependent on the mode parameters, enabling the application of the MTS / LFNST / NSPT transform to the chroma components of a single tree partition, thereby improving both compression efficiency and encoding / decoding performance.

[0497] In yet another embodiment of the present application, referring to FIG49 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG49 , a coding and decoding system 490 may include an encoder 4901 and a decoder 4902 .

[0498] In an embodiment of the present application, the encoder 4901 may be the encoder described in any one of the aforementioned embodiments, and the decoder 4902 may be the decoder described in any one of the aforementioned embodiments.

[0499] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0500] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0501] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0502] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0503] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0504] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0505] In an embodiment of the present application, at the encoding end, the mode parameters of the chrominance components of the current block are determined; when the mode parameters meet the first condition, the first transform kernel of the chrominance components of the current block is determined; the residual block of the chrominance components of the current block is determined, and the residual block of the chrominance components of the current block is transformed according to the first transform kernel to determine the transform coefficients of the chrominance components of the current block; the transform coefficients of the chrominance components of the current block are encoded, and the obtained coded bits are written into the bitstream. At the decoding end, the mode parameters of the chrominance components of the current block are determined; when the mode parameters meet the first condition, the first transform kernel of the chrominance components of the current block is determined; the transform coefficients of the chrominance components of the current block are determined, and the transform coefficients of the chrominance components of the current block are inversely transformed according to the first transform kernel to determine the residual block of the chrominance components of the current block. In this way, whether it is the encoding end or the decoding end, the mode parameters of the current block are first determined; when the mode parameters meet the first condition, the first transform kernel of the chrominance components of the current block is determined, and then the residual block of the chrominance components of the current block is transformed according to the first transform kernel. That is, the determination of the first transform kernel is related to the mode parameters, and it is possible to apply the MTS / LFNST / NSPT transform to the chrominance component of the single tree partition, thereby not only improving the compression efficiency but also improving the encoding and decoding performance.

Claims

1. A decoding method, applied to a decoder, the method comprising: Determining mode parameters for chroma components of the current block; When the mode parameter satisfies a first condition, determining a first transform kernel of a chroma component of the current block; Determine a transform coefficient of the chrominance component of the current block, and inversely transform the transform coefficient of the chrominance component of the current block according to the first transform kernel to determine a residual block of the chrominance component of the current block.

2. The method according to claim 1, wherein: The mode parameters satisfy a first condition, including: the partition type of the current block is single-tree partition and the prediction mode of the chrominance component of the current block is a cross-component prediction mode.

3. The method according to claim 2, wherein: The determining a first transform kernel of the chrominance component of the current block includes: determining a preset transform kernel as the first transform kernel of the chrominance component of the current block.

4. The method according to claim 2, wherein: The determining of a first transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a first transform kernel group of the chrominance component of the current block; wherein the first transform kernel group is different from the transform kernel group of the luminance component of the current block; The first transformation kernel is determined according to the first transformation kernel group.

5. The method according to claim 2, wherein: The determining of a first transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a first transform kernel group of the chrominance component of the current block; wherein the first transform kernel group is the same as the transform kernel group of the luminance component of the current block; The first transform kernel is determined according to the first transform kernel group; wherein the first transform kernel is different from the transform kernel of the brightness component of the current block.

6. The method according to claim 4 or 5, wherein: The first transformation mode includes at least one of the following: a multi-transformation selection mode, a low-frequency inseparable transformation mode and an inseparable basic transformation mode.

7. The method according to claim 4 or 5, wherein: The determining the first transformation core according to the first transformation core group includes: Decoding the code stream to determine a value of the first syntax identification information; When the first syntax identification information indicates that the current block uses a first transform mode, determining a transform core index of the current block; The first transform core is determined according to the first transform core group and the transform core index.

8. The method according to claim 1, wherein: The method further comprises: When the mode parameter does not satisfy the first condition, determining a second transform kernel of the chrominance component of the current block; Determine a transform coefficient of the chrominance component of the current block, and inversely transform the transform coefficient of the chrominance component of the current block according to the second transform kernel to determine a residual block of the chrominance component of the current block.

9. The method according to claim 8, wherein: The mode parameter does not satisfy the first condition, including: the partition type of the current block is dual-tree partition or the prediction mode of the chrominance component of the current block is a non-cross-component prediction mode.

10. The method according to claim 8, wherein: The determining of a second transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a second transform kernel group of the chrominance components of the current block; The second transform core is determined according to the second transform core group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

11. The method according to any one of claims 1 to 10, wherein: The determining of the transform coefficient of the chrominance component of the current block comprises: Decoding a bitstream to determine a quantization coefficient of a chrominance component of the current block; De-quantize the quantized coefficients of the chrominance components of the current block to determine the transform coefficients of the chrominance components of the current block.

12. The method according to claim 7, wherein: The transform coefficient of the current block includes a transform coefficient of a chrominance component of the current block and a transform coefficient of a luminance component of the current block, and the method further includes: Decoding a bit stream to determine a quantization coefficient of a luminance component of the current block; The quantized coefficients of the brightness component of the current block are inversely quantized to determine the transform coefficients of the brightness component of the current block.

13. The method according to claim 12, wherein: The method further comprises: When the position of the last non-zero coefficient in the transform coefficients of the current block meets a preset condition, a step of decoding the code stream and determining a value of the first syntax identification information is performed.

14. The method according to claim 13, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non- The zero coefficient position is greater than or equal to a first threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to a first threshold.

15. The method according to claim 13, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to a first threshold.

16. The method according to claim 13, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to the second threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

17. The method according to claim 13, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to a second threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

18. The method according to claim 16 or 17, wherein: The method further comprises: The second threshold is determined according to a size parameter of the current block.

19. The method according to claim 16 or 17, wherein: The method further comprises: The second threshold is set to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

20. The method according to any one of claims 1 to 19, wherein: The method further comprises: Determining a prediction block for a chrominance component of the current block; A reconstructed block of the chrominance component of the current block is determined according to the predicted block of the chrominance component of the current block and the residual block of the chrominance component of the current block.

21. A coding method, applied to an encoder, the method comprising: Determining mode parameters for chroma components of the current block; When the mode parameter satisfies a first condition, determining a first transform kernel of a chroma component of the current block; Determine a residual block of a chrominance component of the current block, and transform the residual block of the chrominance component of the current block according to the first transform kernel to determine a transform coefficient of the chrominance component of the current block; The transform coefficients of the chrominance components of the current block are coded, and the obtained coded bits are written into a bit stream.

22. The method according to claim 21, wherein: The mode parameters satisfy a first condition, including: the partition type of the current block is single-tree partition and the prediction mode of the chrominance component of the current block is a cross-component prediction mode.

23. The method according to claim 22, wherein: The determining a first transform kernel of the chrominance component of the current block includes: determining a preset transform kernel as the first transform kernel of the chrominance component of the current block.

24. The method according to claim 22, wherein: The determining of a first transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a first transform kernel group of the chrominance component of the current block; wherein the first transform kernel group is different from the transform kernel group of the luminance component of the current block; The first transformation kernel is determined according to the first transformation kernel group.

25. The method according to claim 22, wherein: The determining of a first transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a first transform kernel group of the chrominance component of the current block; wherein the first transform kernel group is the same as the transform kernel group of the luminance component of the current block; The first transform kernel is determined according to the first transform kernel group; wherein the first transform kernel is different from the transform kernel of the brightness component of the current block.

26. The method according to claim 24 or 25, wherein: The first transformation mode includes at least one of the following: a multi-transformation selection mode, a low-frequency inseparable transformation mode and an inseparable basic transformation mode.

27. The method according to claim 24 or 25, wherein: The determining the first transformation core according to the first transformation core group includes: Performing cost value calculation on at least one candidate transform core in the first transform core group to determine a cost value of each of the at least one candidate transform core; A minimum cost value is determined from the cost values ​​of the at least one candidate transformation core, and the candidate transformation core corresponding to the minimum cost value is used as the first transformation core.

28. The method according to claim 27, wherein: The method further comprises: Determine a value of first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block uses a first transform mode and a corresponding transform core index used; The value of the first syntax identification information is encoded, and the obtained encoded bits are written into a bit stream.

29. The method according to claim 28, wherein: The determining a value of the first grammar identification information includes: When the current block does not use the first transform mode, determining that the value of the first syntax identification information is a first value; When the current block uses the first transform mode, it is determined that the value of the first syntax identification information is a second value; wherein the value of the second value is associated with the transform core index of the first transform core.

30. The method of claim 21, wherein: The method further comprises: When the mode parameter does not satisfy the first condition, determining a second transform kernel of the chrominance component of the current block; Determine a residual block of the chrominance component of the current block, and transform the residual block of the chrominance component of the current block according to the second transform kernel to determine a transform coefficient of the chrominance component of the current block; The transform coefficients of the chrominance components of the current block are coded, and the obtained coded bits are written into a bit stream.

31. The method according to claim 30, wherein: The mode parameter does not satisfy the first condition, including: the partition type of the current block is dual-tree partition or the prediction mode of the chrominance component of the current block is a non-cross-component prediction mode.

32. The method of claim 30, wherein: The determining of a second transform kernel of a chrominance component of the current block comprises: Based on the first transform mode, determining a second transform kernel group of the chrominance components of the current block; The second transform core is determined according to the second transform core group; wherein the first transform mode includes at least one of the following: a multi-transform selection mode, a low-frequency inseparable transform mode, and an inseparable basic transform mode.

33. The method of claim 28, wherein: The method further comprises: Determine a transform coefficient of the current block; wherein the transform coefficient of the current block includes a transform coefficient of a chrominance component of the current block and a transform coefficient of a luminance component of the current block; quantizing the transform coefficients of the current block to determine the quantization coefficients of the current block; The quantized coefficients of the current block are coded, and the obtained coded bits are written into a bit stream.

34. The method of claim 33, wherein: The method further comprises: When the last non-zero coefficient position in the transform coefficients of the current block meets a preset condition, a step of encoding the value of the first syntax identification information and writing the obtained encoding bits into a bitstream is performed.

35. The method of claim 34, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to a first threshold.

36. The method of claim 34, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient position of all color components in the transform coefficient of the current block is greater than or equal to a first threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is greater than or equal to a first threshold.

37. The method of claim 34, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the prediction mode of the current block is the intra prediction mode, the last non-zero coefficient position of all color components in the transform coefficient of the current block is less than or equal to the second threshold; or, When the prediction mode of the current block is the inter-frame prediction mode, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

38. The method of claim 34, wherein: The last non-zero coefficient position in the transform coefficient of the current block meets a preset condition, including: When the partition type of the current block is dual-tree partition, the last non-zero coefficient of all color components in the transform coefficients of the current block is The digital position is less than or equal to a second threshold; or, When the partition type of the current block is single-tree partition, the last non-zero coefficient position of the luminance color component in the transformation coefficient of the current block is less than or equal to the second threshold.

39. The method according to claim 37 or 38, wherein: The method further comprises: The second threshold is determined according to a size parameter of the current block.

40. The method according to claim 37 or 38, wherein: The method further comprises: The second threshold is set to a difference between a maximum number of coefficients in the transform coefficients of the current block and one.

41. The method according to any one of claims 21 to 40, wherein: The determining of the residual block of the chrominance component of the current block comprises: Determining a prediction block for a chrominance component of the current block; A residual block of the chrominance component of the current block is determined according to the original block of the chrominance component of the current block and the predicted block of the chrominance component of the current block.

42. A code stream, wherein The code stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: The quantization coefficient of the chrominance component of the current block and the value of the first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block is allowed to use the first transform mode and the corresponding transform kernel index used.

43. An encoder, comprising a first determining unit, a transforming unit and an encoding unit, wherein: The first determination unit is configured to determine a mode parameter of a chrominance component of a current block; and when the mode parameter satisfies a first condition, determine a first transform kernel of the chrominance component of the current block; The transform unit is configured to determine a residual block of the chrominance component of the current block, and transform the residual block of the chrominance component of the current block according to the first transform kernel to determine a transform coefficient of the chrominance component of the current block; The encoding unit is configured to perform encoding processing on the transform coefficients of the chrominance components of the current block, and write the obtained encoding bits into a bit stream.

44. An encoder, comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 21 to 41 when running the computer program.

45. A decoder, comprising a second determination unit and an inverse transformation unit, wherein: The second determination unit is configured to determine a mode parameter of a chrominance component of a current block; and determining a first transform kernel of a chroma component of the current block when the mode parameter satisfies a first condition; The inverse transform unit is configured to determine transform coefficients of the chrominance components of the current block, and inversely transform the transform coefficients of the chrominance components of the current block according to the first transform kernel to determine a residual block of the chrominance components of the current block.

46. ​​A decoder comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 20 when running the computer program.

47. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 20 or the method according to any one of claims 21 to 41 is implemented.

Images