Encoding method, decoding method, bitstream, encoder, decoder, and storage medium

Abstract

Disclosed in the present application are an encoding method, a decoding method, a bitstream, an encoder, a decoder, and a storage medium. The method comprises: decoding a bitstream, so as to determine the value of first syntax identifier information; when the first syntax identifier information indicates that a multi-transform combination mode is applied to the current block, determining at least two first transform coefficients of the current block; determining at least two transform cores of the current block, and performing inverse transform on the at least two first transform coefficients respectively on the basis of the at least two transform cores, so as to determine at least two first residual blocks of the current block; and determining a reconstructed block of the current block on the basis of the at least two first residual blocks. In this way, the processing efficiency can be improved, thereby improving the encoding and decoding performance.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] As demand for video display quality increases, high-resolution video, such as HD and UHD, has emerged. However, high-resolution video typically contains more information and therefore requires more bandwidth. To reduce bandwidth requirements, video coding standards involving video compression have been introduced.

[0003] In video coding standards, the residual block is not encoded directly, but is usually transformed first. The transform can include low-frequency non-separable transform (LFNST) and non-separable primary transform (NSPT). Both LFNST and NSPT have multiple transform kernel groups, each of which includes multiple transform kernels. Although a transform block can have multiple transform kernels to choose from, only one transform kernel will be used in the end. However, for more complex residual blocks, the processing efficiency is low.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve processing efficiency and thus enhance coding and decoding performance.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Decoding the code stream to determine a value of the first syntax identification information;

[0009] When the first syntax identification information indicates that the current block uses a multi-transform combination mode, determining at least two groups of first transform coefficients of the current block;

[0010] Determining at least two transform kernels of the current block, and performing inverse transform on at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block;

[0011] A reconstructed block of the current block is determined according to the at least two first residual blocks.

[0012] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] Determine the residual block of the current block;

[0014] When the current block uses a multi-transform combination mode, determining at least two first residual blocks of the current block according to the residual block of the current block;

[0015] Determining at least two transform kernels of the current block, and transforming at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block;

[0016] Encoding is performed on at least two groups of first transform coefficients of the current block, and the obtained coded bits are written into a bitstream.

[0017] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0018] at least two groups of first transform coefficients of the current block, at least two transform kernel indices of the current block, at least two feature index numbers of the current block, transform mode identification information of the current block, a value of the first syntax identification information, a value of the second syntax identification information, and a value of the third syntax identification information;

[0019] Among them, the value of the first syntax identification information is used to indicate whether the current sequence allows the use of multi-transform combination mode, the second syntax element identification information is used to indicate the topmost residual block among at least two first residual blocks, and the value of the third syntax identification information is used to indicate whether the current slice allows the use of multi-transform combination mode.

[0020] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a first transforming unit, and an encoding unit, wherein:

[0021] a first determining unit configured to determine a residual block of a current block; and when the current block uses a multi-transform combination mode, determine at least two first residual blocks of the current block according to the residual block of the current block;

[0022] a first transform unit configured to determine at least two transform kernels of the current block, and transform the at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block;

[0023] The encoding unit is configured to perform encoding processing on at least two groups of first transform coefficients of the current block, and write the obtained encoding bits into a bit stream.

[0024] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0025] a first memory for storing a computer program capable of running on the first processor;

[0026] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0027] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a decoding unit, a second determining unit, and a second transforming unit, wherein:

[0028] A decoding unit configured to decode the code stream and determine a value of the first syntax identification information;

[0029] A second determining unit is configured to determine at least two groups of first transform coefficients of the current block when the first syntax identification information indicates that the current block uses a multi-transform combination mode;

[0030] a second transform unit configured to determine at least two transform kernels of the current block, and perform inverse transform on the at least two groups of first transform coefficients respectively according to the at least two transform kernels to determine at least two first residual blocks of the current block;

[0031] The second determining unit is further configured to determine a reconstructed block of the current block according to the at least two first residual blocks.

[0032] In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein,

[0033] a second memory for storing a computer program capable of running on the second processor;

[0034] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0035] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method described in the first aspect or the method described in the second aspect.

[0036] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium. At the encoding end, the residual block of the current block is determined; when the current block uses a multi-transform combination mode, at least two first residual blocks of the current block are determined based on the residual block of the current block; at least two transform kernels of the current block are determined, and at least two first residual blocks are transformed according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block; at least two groups of first transform coefficients of the current block are encoded, and the obtained coded bits are written into the code stream. At the decoding end, the code stream is decoded to determine the value of the first syntax identification information; when the first syntax identification information indicates that the current block uses a multi-transform combination mode, at least two groups of first transform coefficients of the current block are determined; at least two transform kernels of the current block are determined, and at least two groups of first transform coefficients are inversely transformed according to the at least two transform kernels to determine at least two first residual blocks of the current block; based on the at least two first residual blocks, a reconstructed block of the current block is determined. In this way, whether it is the encoding end or the decoding end, when determining whether the current block uses a multi-transform combination mode, it is first necessary to determine at least two transform kernels of the current block, and then the encoding end transforms the at least two first residual blocks of the current block according to the at least two transform kernels respectively, while the decoding end performs inverse transform on the at least two groups of first transform coefficients of the current block according to the at least two transform kernels respectively; that is, the current block can use at least two transform kernels to "parallel" process multiple transform / inverse transform processes, use multiple transform kernels to efficiently process a part of the texture respectively, and finally combine multiple residual blocks, thereby improving processing efficiency, and for some more complex residual blocks, its compression efficiency can also be higher than the compression efficiency of using a single transform kernel, thereby improving the encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a flow chart diagram of a hybrid coding framework;

[0038] FIG2 is a schematic diagram of a DCT transform;

[0039] FIG3 is a schematic diagram of a base image of a DCT transformation;

[0040] FIG4 is a schematic diagram of a coding and decoding process without LFNST transformation;

[0041] FIG5 is a schematic diagram of an encoding and decoding process with LFNST transformation;

[0042] FIG6 is a schematic diagram of a detailed encoding and decoding process with LFNST transformation;

[0043] FIG7 is a schematic diagram of a base image of multiple transformation kernel groups;

[0044] FIG8 is a schematic diagram of a base image of NSPT transformation;

[0045] FIG9 is a schematic diagram of a network architecture of a video codec provided in an embodiment of the present application;

[0046] FIG10 is a schematic block diagram of a system composition of an encoder provided in an embodiment of the present application;

[0047] FIG11 is a schematic block diagram of a system composition of a decoder provided in an embodiment of the present application;

[0048] FIG12 is a flowchart diagram of a decoding method provided in an embodiment of the present application;

[0049] FIG13A is a schematic diagram of a single texture;

[0050] FIG13B is a second schematic diagram of a single texture;

[0051] FIG13C is a schematic diagram of a combined texture 1;

[0052] FIG14A is a third schematic diagram of a single texture;

[0053] FIG14B is a fourth schematic diagram of a single texture;

[0054] FIG14C is a second schematic diagram of a combined texture;

[0055] FIG15 is a second flow chart of a decoding method provided in an embodiment of the present application;

[0056] FIG16 is a schematic diagram of the dividing lines of a GPM mode;

[0057] FIG17 is a third flow chart of a decoding method provided in an embodiment of the present application;

[0058] FIG18A is a schematic diagram of a decoding process of a residual block;

[0059] FIG18B is a first schematic diagram of a decoding process with two residual blocks;

[0060] FIG18C is a second schematic diagram of a decoding process with two residual blocks;

[0061] FIG19 is a fourth flow chart of a decoding method provided in an embodiment of the present application;

[0062] FIG20A is a first schematic diagram of a process for determining a reconstruction block according to an embodiment of the present application;

[0063] FIG20B is a second schematic diagram of a process for determining a reconstruction block according to an embodiment of the present application;

[0064] FIG21 is a fifth flow chart of a decoding method provided in an embodiment of the present application;

[0065] FIG22 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0066] FIG23 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0067] FIG24 is a third flow chart of an encoding method provided in an embodiment of the present application;

[0068] FIG25 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0069] FIG26 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0070] FIG27 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0071] FIG28 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0072] Figure 29 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0075] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0076] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0077] In video images, a coding block (CB) is generally represented by a first color component, a second color component, and a third color component. These three color components are a luminance component, a blue chrominance component, and a red chrominance component. Specifically, the luminance component is typically represented by the symbol Y, the blue chrominance component is typically represented by the symbols Cb or U, and the red chrominance component is typically represented by the symbols Cr or V. Thus, video images can be represented in either the YCbCr or YUV format.

[0078] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0079] H.265 / High Efficiency Video Coding (HEVC);

[0080] H.266 / Versatile Video Coding (VVC);

[0081] VVC Test Model (VTM), a reference software testing platform for VVC;

[0082] A platform that improves compression performance after VVC (Enhanced Compression Model, ECM);

[0083] Joint Video Experts Team (JVET);

[0084] Coding Unit (CU);

[0085] Coding Tree Unit (CTU);

[0086] Largest Coding Unit (LCU);

[0087] Prediction Unit (PU);

[0088] Transform Unit (TU);

[0089] Fusion technology (Merge);

[0090] Skip technology (Skip);

[0091] Discrete Cosine Transform (DCT);

[0092] Discrete Sine Transform (DST);

[0093] Multiple Transform Selection (MTS);

[0094] Rate-Distortion Optimized Quantization (RDOQ);

[0095] Low Frequency Non-Separable Transform (LFNST);

[0096] Non-Separable Primary Transform (NSPT);

[0097] Context-based Adaptive Binary Arithmetic Coding (CABAC).

[0098] Currently, common video codec standards all adopt a block-based hybrid coding framework. Each image, sub-image, or frame in a video is divided into square maximum coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 256×256, 128×128, 64×64, etc.). Each LCU or CTU can be divided into rectangular CUs according to a rule. Coding units may also be divided into prediction units (PUs), transform units (TUs), etc. Specifically, as shown in Figure 1, the hybrid coding framework includes a prediction module 11, a transform and quantization module 12, an entropy coding module 13, an inverse quantization and inverse transform module 14, a loop filter module 15, and a decoded picture buffer module 16. The prediction module 11 may include an intra-frame prediction module 11a and an inter-frame prediction module 11b. The inter-frame prediction module 11b may include a motion estimation module and a motion compensation module. Because there is a strong correlation between adjacent pixels in a video image, intra-frame prediction is used in video codec technology to eliminate spatial redundancy between adjacent pixels. In addition, since there is a strong similarity between adjacent images in a video, the inter-image prediction method is used in video coding and decoding technology to eliminate the temporal redundancy between adjacent images, thereby improving coding efficiency.

[0099] The basic process of a video codec is as follows: On the encoder side, an image is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations as the decoder side to obtain a decoded image. The decoded image can serve as a reference image for inter-frame prediction of subsequent images. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoder are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing the bit stream and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The embodiment of the present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0100] In addition, in the embodiments of the present application, the current block (CB) can be the current coding unit, the current prediction unit, or the current transform unit. Due to the need for parallel processing, the image can be divided into slices, etc. Slices in the same image can be processed in parallel, that is, there is no data dependency between them. "Frame" is a commonly used term, and it can generally be understood that a frame is an image. The frame described in the embodiments of the present application can also be replaced by an image or a slice, etc.

[0101] The following is a detailed introduction to the transformation technology in related technologies.

[0102] During encoding, commonly used hybrid coding frameworks first perform a prediction. This prediction leverages spatial or temporal correlation to produce an image identical or similar to the current block. While it's possible for the predicted block to be identical to the current block for a given block, it's difficult to guarantee this for all blocks in a video, especially in natural video or video captured by a camera. Irregular motion, distortion, occlusion, and brightness changes in video are difficult to fully predict. Therefore, hybrid coding frameworks subtract the predicted image from the original image of the current block to produce a residual image, or, in other words, subtract the predicted block from the current block to produce a residual block. This residual block is typically much simpler than the original image, so prediction can significantly improve compression efficiency. The residual block isn't encoded directly; instead, it's usually first transformed. This transform converts the residual image from the spatial domain to the frequency domain to remove correlation. After the residual image is transformed to the frequency domain, since most of the energy is concentrated in the low-frequency region, the non-zero coefficients are concentrated in the upper left corner. Quantization is then used to further compress the image. Furthermore, since the human eye is less sensitive to high frequencies, a larger quantization step size can be used in high-frequency regions.

[0103] Figure 2 is a schematic diagram of a DCT transform. As shown in Figure 2, after the DCT transform of the original image, only the upper left corner contains non-zero coefficients. Of course, this example performs a DCT transform on the entire image. In video codecs, images are processed by dividing them into blocks, so the transform is also performed on a block-by-block basis.

[0104] Transforms are very useful in typical video compression, but not all blocks require transforms. In some cases, transforms can even yield worse compression results than non-transforms. Therefore, in some standards, such as VVC, the encoder can choose whether to use transforms for the current block. DCT-II is the most commonly used transform in video compression standards, and its base image is shown in Figure 3.

[0105] In addition, VVC can also use DCT8 (DCT-VIII) and DST7 (DST-VII). The basic formulas of these transforms are shown in Table 1, which shows the basic transform formulas of DCT2, DCT8 and DST7 for N-point input.

[0106] Table 1

[0107] Because images are all two-dimensional, the computational complexity and memory overhead of performing a direct two-dimensional transform were prohibitive for the hardware available at the time. Therefore, the DCT2, DCT8, and DST7 transforms used in the standards were split into two steps: horizontal and vertical one-dimensional transforms. For example, the horizontal transform was performed first, followed by the vertical transform, or the vertical transform was performed first, followed by the horizontal transform.

[0108] (1) Multi-transformation selection MTS.

[0109] VVC supports transform kernels such as DCT2, DCT8, and DST7. For a block, the encoder selects the appropriate transform kernel and transmits the index to the bitstream. The decoder then uses the index to determine the transform kernel for the inverse transform. Different transform kernels can be selected for the horizontal and vertical directions, such as using DCT8 horizontally and DST7 vertically. This technique is generally referred to as MTS.

[0110] VVC can use a syntax element, mts_idx, to determine the transform kernel of the base transform. As shown in Table 2, trTypeHor represents the transform kernel for the horizontal transform, and trTypeVer represents the transform kernel for the vertical transform. In addition, for trTypeHor and trTypeVer, a value of 0 indicates a DCT2 transform, a value of 1 indicates a DCT7 transform, and a value of 2 indicates a DCT8 transform.

[0111] Table 2

[0112] (2) Low-frequency non-separable transform LFNST.

[0113] The above transformation method is effective for horizontal and vertical textures, but less so for diagonal textures. Indeed, horizontal and vertical textures are the most common, making the above transformation method very useful for improving compression efficiency. As the demand for compression efficiency continues to increase, more efficient processing of diagonal textures could further improve compression efficiency.

[0114] To more effectively process residuals for oblique textures, VVC uses the LFNST transform. These transforms, such as DCT2, DCT8, and DST7, are called primary transforms. On the VVC encoder side, LFNST is applied after the DCT2 transform and before quantization. On the VVC decoder side, LFNST is applied after inverse quantization and before the inverse DCT2 transform.

[0115] Because LFNST is a secondary transform based on DCT2 (the base transform), it is a secondary transform. Figure 4 shows the encoding and decoding process without LFNST (secondary transform), while Figure 5 shows the encoding and decoding process with LFNST (secondary transform). Of course, the encoder can bypass entropy decoding and directly dequantize the stored quantized coefficients, as entropy coding is lossless.

[0116] Figure 6 is a detailed schematic diagram of a coding and decoding process with LFNST (secondary transform). As shown in Figure 6, at the encoding end, the forward basic transform is performed first, and then the LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the basic transform. For example, there are 16 input coefficients for 4×4 LFNST and 64 input coefficients for 8×8 LFNST; the coefficients after the LFNST transform are then quantized and written into the bitstream. At the decoding end, the transform coefficients can be obtained by decoding the bitstream and inverse quantization. Then, in the inverse LFNST transform, there are 8 input coefficients for 4×4 inverse LFNST and 16 input coefficients for 8×8 inverse LFNST. Finally, the inverse basic transform can obtain the residual block.

[0117] That is, on the encoder side, LFNST performs a secondary transform on the low-frequency coefficients in the upper left corner after the base transform. The base transform decorrelates the image, concentrating the energy in the upper left corner. The secondary transform further decorrelates the low-frequency coefficients of the base transform. The result is intuitively shown in Figure 6. On the encoder side, 16 coefficients are input to a 4×4 LFNST, and the output is 8 coefficients; 64 coefficients are input to an 8×8 LFNST, and the output is 16 coefficients. On the decoder side, 8 coefficients are input to a 4×4 inverse LFNST, and the output is 16 coefficients; 16 coefficients are input to an 8×8 inverse LFNST, and the output is 64 coefficients.

[0118] Figure 7 shows some base images for the LFNST algorithm in VVC. Figure 7 shows some obvious diagonal textures. In addition to kernels optimized for certain diagonal textures, the LFNST algorithm also has kernels optimized for flat, gradient textures, such as kernel group 0 in the LFNST algorithm in VVC.

[0119] Angular prediction uses a specified angle to tile reference pixels onto the current block as the prediction value. This means the predicted block will have a distinct directional texture, and the residual of the current block after angular prediction will also statistically exhibit significant angular characteristics. Therefore, the transform kernel selected by LFNST can be tied to the intra prediction mode. That is, once the intra prediction mode is determined, LFNST can only use the set of transform kernels corresponding to the intra prediction mode.

[0120] Specifically, the LFNST in VVC has a total of 4 groups of transform kernels, and each group can select 2 transform kernels. Table 3 shows the correspondence between intra prediction modes and transform kernel groups. Note that the cross-component prediction modes used for chroma intra prediction are 81 to 83, and there are no such modes for luma intra prediction. The transform kernel of LFNST can be transposed to process more angles with one transform kernel group. For example, modes 13 to 23 and 45 to 55 both correspond to transform kernel group 2, but 13 to 23 is obviously close to the horizontal mode and 45 to 55 is obviously close to the vertical mode.

[0121] Table 3

[0122] VVC's LFNST uses four sets of transform kernels, with the intra-prediction mode specifying which set to use. This leverages the correlation between the intra-prediction mode and the LFNST transform kernel, reducing the transmission of the selected LFNST transform kernel in the bitstream. Whether the current block uses LFNST, and if so, whether to use the first or second set within a set, is determined by the bitstream and certain conditions.

[0123] In the subsequent evolution of ECM technology, LFNST was further expanded. LFNST has more transform kernel groups, 35 in ECM. The correspondence between the transform kernel group index (LFNST set index) and the intra prediction mode (Intra pred.mode) is shown in Table 4. Each transform kernel group is more efficient for textures at the corresponding angle. Here, each transform kernel group can select three transform kernels.

[0124] Table 4

[0125] (3) Non-separable basis transformation NSPT.

[0126] LFNST is a horizontally and vertically inseparable transform. Because it involves a secondary transform, DCT2 can be called the base transform. This approach of performing DCT2 before LFNST is a compromise between performance and complexity. While directly performing the inseparable base transform is more efficient, it also incurs higher complexity, such as increased computational effort and storage space required for the transform kernel.

[0127] In ECM10, some small blocks can use NSPT, while large blocks still use DCT2+LFNST. Small block sizes include 4×4, 4×8, 8×4, 8×8, 4×16, 16×4, 8×16, 16×8, 8×32, and 32×8. In ECM10, NSPT also matches the transform kernel group based on the intra prediction mode. The matching method can refer to the LFNST method. Each transform kernel group has three transform kernels to choose from. For example, the 8×8 base image of an NSPT in ECM10 is shown in Figure 8, which corresponds to inter-frame angular prediction mode 7. It can be seen that it better handles texture at corresponding angles.

[0128] It should be noted that in the embodiments of the present application, the "inverse transformation" of the transform coefficients by the decoding end may also be referred to as "transformation" in the standard text. The "transformation" and "inverse transformation" in this article correspond to two opposite processes. For example, if the "transformation" converts the numerical value of the spatial domain to the coefficient of the frequency domain, then the "inverse transformation" converts the coefficient of the frequency domain to the numerical value of the spatial domain. In addition, if the standard text only stipulates decoding, then the "transformation" in the standard text is the decoding part, which refers to the "inverse transformation" in this article. The "inverse transformation" of the transform coefficients by the decoding end may also be referred to as "transformation" in the standard text.

[0129] As can be seen from this, the path for determining the residual value or residual block is very simple. For a transform block, the decoder parses the bitstream to extract a quantized coefficient block. The size of the quantized coefficient block is the same as the transform block, and many positions in the quantized coefficient block may be zero. This is why transform and quantization can improve compression efficiency. Dequantizing the quantized coefficient block yields the transform coefficient block, and then performing an inverse transform on the transform coefficient block to obtain the residual block.

[0130] VVC introduces the secondary transform LFNST, and in this case, transforms such as DCT2 can be called basic transforms. If the current block uses a secondary transform, the transform coefficient block is first subjected to an inverse secondary transform, and the resulting transform coefficient block is then subjected to an inverse basic transform. VVC uses DCT2 when superimposed with LFNST to obtain a residual block. If the current block does not use a secondary transform, the transform coefficient block is subjected to an inverse basic transform, such as DCT2, DCT8, or DST7, to obtain a residual block. The basic transform in VVC uses a separable transform in the horizontal and vertical directions. The choice of DCT2, DCT8, or DST7 in the horizontal and vertical directions is determined by MTS. ECM introduces NSPT, an inseparable basic transform that replaces the DCT2+LFNST combination for certain block sizes.

[0131] Since both LFNST and NSPT have multiple transform kernel sets, each corresponding to a specific texture feature, such as texture at a certain angle, a block can only select one kernel from each set. The specific kernel set selected is determined by the intra prediction mode match of the current block. Although a transform block can choose from multiple kernels, only one kernel is ultimately selected. If LFNST is used, a kernel from LFNST is added to DCT2.

[0132] Based on this, an embodiment of the present application provides an encoding method, which determines the residual block of the current block; when the current block uses a multi-transform combination mode, determines at least two first residual blocks of the current block according to the residual block of the current block; determines at least two transform kernels of the current block, and transforms the at least two first residual blocks according to the at least two transform kernels respectively to determine at least two groups of first transform coefficients of the current block; encodes the at least two groups of first transform coefficients of the current block, and writes the obtained coded bits into the bitstream. An embodiment of the present application also provides a decoding method, which decodes the bitstream and determines the value of the first syntax identification information; when the first syntax identification information indicates that the current block uses a multi-transform combination mode, determines at least two groups of first transform coefficients of the current block; determines at least two transform kernels of the current block, and inversely transforms the at least two groups of first transform coefficients according to the at least two transform kernels respectively to determine at least two first residual blocks of the current block; and determines a reconstructed block of the current block based on the at least two first residual blocks.

[0133] In this way, whether it is the encoding end or the decoding end, when determining whether the current block uses a multi-transform combination mode, it is first necessary to determine at least two transform kernels of the current block, and then the encoding end transforms the at least two first residual blocks of the current block according to the at least two transform kernels respectively, while the decoding end performs inverse transform on the at least two groups of first transform coefficients of the current block according to the at least two transform kernels respectively; that is, the current block can use at least two transform kernels to "parallel" process multiple transform / inverse transform processes, use multiple transform kernels to efficiently process a part of the texture respectively, and finally combine multiple residual blocks, thereby improving processing efficiency, and for some more complex residual blocks, its compression efficiency can also be higher than the compression efficiency of using a single transform kernel, thereby improving the encoding and decoding performance.

[0134] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0135] FIG9 is a schematic diagram of a network architecture for video encoding and decoding provided in an embodiment of the present application. As shown in FIG9 , the network architecture includes one or more electronic devices 31 to 3N and a communication network 01, wherein electronic devices 31 to 3N can perform video interaction via the communication network 01. During implementation, the electronic devices can be various types of devices with video encoding and decoding capabilities. For example, the electronic devices can include mobile phones, tablet computers, personal computers, personal digital assistants, navigation devices, digital phones, video phones, televisions, sensor devices, servers, etc., without limitation herein.

[0136] In an embodiment of the present application, a network architecture of a video encoding and decoding system including a decoding method and an encoding method is provided. The decoder or encoder in the embodiment of the present application can be the aforementioned electronic device. In other words, the electronic device in the embodiment of the present application has video encoding and decoding capabilities, and generally includes a video encoder (i.e., encoder) and a video decoder (i.e., decoder).

[0137] Figure 10 is a schematic block diagram of the system composition of an encoder provided in an embodiment of the present application. As shown in Figure 10, the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a series of pictures or a static picture, and the output of the encoder 100 may be a bitstream (also referred to as a "codestream") used to represent a compressed version of the input video.

[0138] Among them, the segmentation unit 101 segments the picture in the input video into one or more Coding Tree Units (CTUs). The segmentation unit 101 divides the picture into multiple tiles (or tiles), and can further divide a tile into one or more bricks. Here, a tile or a brick may include one or more complete and / or partial CTUs. In addition, the segmentation unit 101 can form one or more slices, where a slice can include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture. The segmentation unit 101 can also form one or more sub-pictures, where a sub-picture can include one or more slices, tiles or bricks.

[0139] During the encoding process of encoder 100, segmentation unit 101 transmits the CTU to prediction unit 102. Generally, prediction unit 102 may be composed of block segmentation unit 103, motion estimation (ME) unit 104, motion compensation (MC) unit 105, and intra prediction unit 106. Specifically, block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (CUs). Prediction unit 102 may use ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks for the CU. Intra-frame prediction unit 106 may use various intra-frame prediction modes, including MIP mode, to obtain intra-frame prediction blocks for the CU. In an example, a rate-distortion optimized motion estimation method may be used by ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks, and a rate-distortion optimized mode determination method may be used by intra-frame prediction unit 106 to obtain intra-frame prediction blocks.

[0140] The prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU in the output of the segmentation unit 101 and the prediction block of the CU, i.e., the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels). The inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstructed residual. The second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference. After all CUs in the picture or sub-picture are reconstructed, the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping and chroma scaling (LMCS) filter, and a neural network-based filter. Alternatively, when the filtering unit 113 determines that a CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.

[0141] The output of the filtering unit 113 is a decoded picture or sub-picture, which is cached to the DPB unit 114. The DPB unit 114 outputs the decoded picture or sub-picture based on the timing and control information. Here, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, the entropy coding unit 115 converts the parameters required for decoding the picture from the encoder 100 (such as control parameters and supplementary information, etc.) into binary form and writes this binary form into the code stream according to the syntax structure of each data unit. That is, the encoder 100 finally outputs the code stream.

[0142] Furthermore, encoder 100 may include a first processor and a first memory storing a computer program. When the first processor reads and executes the computer program, encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, encoder 100 may be a computing device comprising one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in FIG10 .

[0143] Figure 11 is a block diagram of the system components of a decoder provided in an embodiment of the present application. As shown in Figure 11, the decoder 200 may include: a decoding unit (or "parsing unit") 201, a prediction unit 202, an inverse quantization unit 205, an inverse transform unit 206, an adder 207, a filtering unit 208, and a decoded picture buffer unit 209. Here, the input of the decoder 200 is a bitstream representing a compressed version of a video or a still picture, and the output of the decoder 200 may be a decoded video consisting of a series of pictures or a decoded still picture.

[0144] The input codestream to decoder 200 may be the codestream generated by encoder 100. Decoding unit 201 parses the input codestream and obtains syntax element values ​​from the input codestream. Decoding unit 201 converts the binary representation of the syntax elements into digital values ​​and sends the digital values ​​to units within decoder 200 to obtain one or more decoded pictures. Decoding unit 201 may also parse one or more syntax elements from the input codestream to display decoded pictures.

[0145] During the decoding process of the decoder 200 , the decoding unit 201 sends the values ​​of the syntax elements and one or more variables set or determined according to the values ​​of the syntax elements and used to obtain one or more decoded pictures to the units in the decoder 200 .

[0146] The prediction unit 202 determines a prediction block for the current decoding block (e.g., CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra-frame prediction unit 204. Specifically, when the inter-frame decoding mode is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the decoding unit 201 to the motion compensation unit 203 to obtain an inter-frame prediction block; when the intra-frame prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the decoding unit 201 to the intra-frame prediction unit 204 to obtain an intra-frame prediction block.

[0147] The inverse quantization unit 205 has the same functionality as the inverse quantization unit 110 in the encoder 100. The inverse quantization unit 205 performs a scaling operation on the quantized coefficients (i.e., levels) from the decoding unit 201 to obtain reconstructed coefficients. The inverse transform unit 206 has the same functionality as the inverse transform unit 111 in the encoder 100. The inverse transform unit 206 performs one or more transform operations (i.e., the inverse of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual. The adder 207 adds its inputs (the prediction block from the prediction unit 202 and the reconstructed residual from the inverse transform unit 206) to obtain a reconstructed block for the currently decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in intra-frame prediction mode.

[0148] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chroma scaling filter, and a neural network-based filter. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of the filtering unit 208 is a decoded picture or sub-picture, which is cached to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture based on timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0149] Furthermore, the decoder 200 can be a second memory having a second processor and a computer program. When the first processor reads and runs the computer program, the decoder 200 reads the input bit stream and generates the corresponding decoded video. In addition, the decoder 200 can also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in Figure 11.

[0150] It should also be noted that when the embodiment of the present application is applied to the encoder 100, the "current block" specifically refers to the current block to be encoded in the video image (which can also be simply referred to as the "encoding block"); when the embodiment of the present application is applied to the decoder 200, the "current block" specifically refers to the current block to be decoded in the video image (which can also be simply referred to as the "decoding block").

[0151] In one embodiment of the present application, FIG12 is a flowchart of a decoding method provided by the embodiment of the present application. As shown in FIG12 , the method may include:

[0152] S1201: Decode a code stream and determine a value of first syntax identification information.

[0153] It should be noted that in the embodiment of the present application, the method is applied to a decoder. Specifically, based on the structure of the decoder 200 shown in FIG11 , the decoding method of the embodiment of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, it is mainly aimed at the multi-transform kernel scheme proposed in the transformation process to improve compression efficiency.

[0154] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0155] Both NSPT and LFNST are transforms that can efficiently process textures at various angles. They may have multiple transform kernels, and a kernel may be specifically optimized for a specific angular texture. In addition to angular textures, NSPT and LFNST also include transform kernels for gradient textures. These kernels can actually be considered pre-trained KL transforms (Karhunen-Loeve Transforms, KLTs). In other words, both NSPT and LFNST can have multiple transform kernels, each designed for a specific texture. These specific textures include angular textures and gradient textures. Furthermore, gradient textures can be further expanded to include horizontal gradient textures, vertical gradient textures, and diagonal gradient textures. While other basic transforms such as DCT2, DCT8, and DST7 are less specialized than NSPT and LFNST, they each have their own specific applications, which is why MTS can improve compression performance. However, these transform kernels are only efficient for single textures, such as textures that are entirely oriented in a single direction. Here, efficient processing refers to textures that can be fully represented using only a few coefficients in the upper left corner. In other words, the texture can be fully represented by the last few coefficients in the scan order. Transforms like DCT2 can process any texture, but if the coefficients after transformation or quantization are large, it cannot be called efficient processing.

[0156] For example, for the single texture described above, assuming that the lines in Figures 13A and 13B are residual textures, this straight line texture at a certain angle can be called a single texture, and the design of LFNST and NSPT in the related art can efficiently process such residuals. Of course, Figures 13A and 13B are just schematic diagrams. Generally, the residual edges are not very sharp, and sharp edges have a lot of high-frequency information. However, for the texture shown in Figure 13C, it is not a single texture, it is composed of textures in two directions. The transformation kernel currently optimized for a single texture cannot efficiently process such textures.

[0157] If residuals like those in Figure 13C appear frequently in real videos, then it is certainly possible to train kernels for such textures. However, given the ever-changing nature of images and residuals, the number of kernels trained for a wide variety of complex textures would be staggering.

[0158] In one possible implementation, if a complex texture can be split into multiple single textures, then the single textures can be processed separately and then combined. For example, if the texture in Figure 13C is split into two single textures in Figure 13A and Figure 13B, then they can be processed separately. And we know that complex things in nature are composed of simple things. It is in line with the laws of nature to split complex problems into simple problems for processing. Here, splitting can be achieved by dividing into smaller blocks, such as smaller CUs or smaller TUs. For example, Figure 13C can be divided into two CUs or TUs on the left and right for processing. The cost of doing this is the various overheads of division. Dividing a block requires transmitting its various necessary information in the bitstream.

[0159] While textures like those in Figure 13C are relatively easy to resolve by dividing into small blocks, textures like those in Figure 14C require many small blocks to ensure that each block contains only simple textures. In practical applications, Figure 14C may be composed of Figures 14A and 14B. By splitting or decomposing the texture, i.e., decomposing Figure 14C into Figures 14A and 14B, unnecessary divisions can be avoided. This results in a block division approach that favors "more large blocks and fewer small blocks." The technical solution of the embodiments of this application employs this approach to address this problem.

[0160] That is to say, in the embodiment of the present application, the multi-transform kernel scheme here is not limited to inseparable transforms such as NSPT and LFNST, but is also applicable to separable transforms optimized for specific textures, and efficient processing can be achieved by splitting / decomposing textures.

[0161] It can be understood that in this embodiment of the present application, the first syntax identification information is used to indicate whether the current block uses the multi-transform combination mode. The first syntax identification information can be represented by multi_trans_combine_flag. If the value of the first syntax identification information is equal to a first value, it can be determined that the current block uses the multi-transform combination mode; if the value of the first syntax identification information is equal to a second value, it can be determined that the current block does not use the multi-transform combination mode.

[0162] Here, the first value is different from the second value, and the first value and the second value can be in parameter form or in numeric form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here. For example, the first value can be 1 and the second value can be 0; or the first value can be 0 and the second value can be 1; or the first value can be true and the second value can be false; or the first value can be false and the second value can be true.

[0163] In a specific embodiment, the first value is 1 and the second value is 0. That is, if the value of the first syntax identification information obtained through decoding is equal to 1, it can be determined that the current block uses the multi-transform combination mode; if the value of the first syntax identification information obtained through decoding is equal to 0, it can be determined that the current block does not use the multi-transform combination mode.

[0164] S1202: When the first syntax identification information indicates that the current block uses a multi-transform combination mode, determine at least two groups of first transform coefficients of the current block.

[0165] It should be noted that, in the embodiment of the present application, when determining whether the current block uses a multi-transform combination mode, it is necessary to distinguish the transform coefficients of the current block to determine at least two groups of first transform coefficients of the current block.

[0166] In a possible implementation, determining at least two groups of first transform coefficients of the current block may include: decoding a code stream, and determining at least two groups of first transform coefficients of the current block.

[0167] In an embodiment of the present application, at least two groups of first transform coefficients of the current block can be directly obtained by decoding the code stream. Here, if a syntax element (first syntax identification information) is directly used to determine whether the current block uses a multi-transform combination mode, then the value of the first syntax identification information can be parsed before parsing the transform coefficients. For example, when the value of the first syntax identification information is equal to 1, at this time, at least two groups of first transform coefficients can be obtained by decoding the code stream, and the decoding method of each group of transform coefficients is the same as the related art. This method is relatively intuitive. If it is already clear that the current block uses multiple transform combinations, then multiple groups of first transform coefficients can be directly parsed.

[0168] In another possible implementation, determining at least two groups of first transform coefficients for the current block may include: decoding a code stream to determine second transform coefficients for the current block; and determining at least two groups of first transform coefficients for the current block based on the second transform coefficients when the first syntax identification information indicates that the current block uses a multi-transform combination mode.

[0169] In the embodiment of the present application, the transform coefficients may be parsed first, and then whether to use a multi-transform combination may be determined. That is, the bitstream is first decoded to determine the second transform coefficients of the current block; then the bitstream is decoded to determine the value of the first syntax identification information, and based on the value of the first syntax identification information, whether to use a multi-transform combination for the current block is determined.

[0170] In this implementation, when parsing the transform coefficients, it is unclear whether the transform coefficients to be parsed are actually one or multiple groups of transform coefficients. However, in form, a group of transform coefficients is decoded to obtain the second transform coefficients of the current block. If it is determined that the current block uses a multi-transform combination, the obtained second transform coefficients need to be split into at least two groups of first transform coefficients.

[0171] In a specific implementation, determining at least two groups of first transform coefficients for the current block according to the second transform coefficients may include: evenly dividing the second transform coefficients to determine at least two groups of first transform coefficients for the current block.

[0172] In an embodiment of the present application, taking the splitting into two groups of transform coefficients as an example, the second transform coefficient obtained by parsing can be evenly divided, for example, the first half is the first group of transform coefficients, and the second half is the second group of transform coefficients. For example, if there are a total of N coefficients between the first transform coefficient and the last non-zero transform coefficient, then N / 2 coefficients are the first group of transform coefficients, and the remaining N / 2 coefficients are the second group of transform coefficients. This method requires the encoder to meet the requirements when encoding the transform coefficients, such as by forcibly padding with 0 or forcibly quantizing some coefficients to 0 to ensure that the dividing point of the two groups of coefficients is exactly at N / 2.

[0173] In another specific implementation, determining at least two groups of first transform coefficients of the current block based on the second transform coefficient may include: determining the last non-zero coefficient in the second transform coefficient; dividing the second transform coefficient according to the division strategy indicated by the last non-zero coefficient to determine at least two groups of first transform coefficients of the current block.

[0174] In the embodiment of the present application, still taking the splitting into two groups of transform coefficients as an example, the division strategy indicated by the last non-zero coefficient includes: the last non-zero coefficient indicates the last non-zero coefficient position in the first group of transform coefficients, or the last non-zero coefficient indicates the last non-zero coefficient position in the second group of transform coefficients, which is not specifically limited here.

[0175] In an embodiment of the present application, information for splitting the two groups of transform coefficients may be hidden in the coefficients. For example, the last non-zero coefficient is the last non-zero coefficient position in the first group of transform coefficients. That is, the number of coefficients in the first group of transform coefficients from the first coefficient (the upper left corner (0, 0) position) to the last non-zero coefficient in the scanning order is reduced by one. The reason for reducing by one is that each group of transform coefficients should not be all zeros, otherwise the transform will be meaningless.

[0176] For example, the transform coefficients are arranged in scan order, and the parsed coefficients are 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 7. The last non-zero coefficient, 7, is not a true coefficient. Instead, it refers to the eight coefficients in positions 0 through 7, which are the first set of transform coefficients: 12, 5, 5, 6, 0, 2, 0, 1. The remaining coefficients are the second set of transform coefficients: 20, 9, 9, 4. These coefficients are placed in the corresponding positions in the two-dimensional coefficient block, and the values ​​of the other positions are set to 0.

[0177] Alternatively, the last non-zero coefficient may be the number of coefficients from the first to the last non-zero coefficient in the second group of transform coefficients in the scanning order minus 1. Here, the reason for minus 1 is that each group of coefficients should not be all zeros, otherwise the transform will be meaningless.

[0178] For example, arranging the coefficients in scan order yields the following coefficients: 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 3. The last non-zero coefficient, 3, is not a true coefficient; rather, it refers to the fact that the second set of transform coefficients in the above coefficients has four members: 20, 9, 9, 4. The preceding ones represent the first set of transform coefficients: 12, 5, 5, 6, 0, 2, 0, 1. These coefficients are placed in the corresponding positions of the two-dimensional coefficient block, and the values ​​of all other positions are set to 0.

[0179] Thus, when determining that the current block uses the multi-transform combination mode, different partitioning strategies may be used to obtain at least two groups of first transform coefficients of the current block.

[0180] S1203 : Determine at least two transform kernels of the current block, and perform inverse transform on at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block.

[0181] S1204: Determine a reconstructed block of the current block according to the at least two first residual blocks.

[0182] It should be noted that, in the embodiment of the present application, for at least two transform cores of the current block, the method may include: decoding the bitstream to determine the at least two transform cores of the current block. In this manner, the at least two transform cores are directly written into the bitstream.

[0183] It should also be noted that, in the embodiment of the present application, at least two transform kernels of the current block can also be determined by texture feature index. In some embodiments, as shown in FIG15 , the method may include:

[0184] S1501: Determine at least two texture feature indexes of a current block.

[0185] It can be understood that in the embodiment of the present application, the transform core group selected for LFNST and NSPT within a frame is automatically matched according to the intra-frame prediction mode. An intra-frame prediction mode represents a texture feature, for example, an intra-frame angle prediction mode represents the angle of a texture. This derivation can reduce the overhead of the code stream because it no longer needs to use syntax elements to write which transform core group to select in the code stream. In the embodiment of the present application, the transform core group of LFNST / NSPT can be derived from certain information of the current block. An intra-frame prediction mode can determine a transform core group, or it can be said that a texture feature index can correspond to a transform core group. For the convenience of description, a virtual intra-frame prediction mode (or "texture feature index") is proposed here. The so-called virtual intra-frame prediction mode is because this mode does not participate in the prediction, but is only used to select the transform core group of NSPT or LFNST. The derivation of the texture feature index is described in detail below.

[0186] In some embodiments, determining at least two texture feature indexes of the current block may include: determining candidate pixels for deriving the at least two texture feature indexes; and determining the at least two texture feature indexes of the current block based on the candidate pixels.

[0187] In a possible implementation, for candidate pixels, a prediction block of the current block may be determined; and at least part of the pixels in the prediction block are used as candidate pixels.

[0188] In the embodiment of the present application, if a certain texture exists in the prediction block, it can be considered that the residual block has a texture with the same characteristics. In this way, the candidate pixels used to derive at least two texture feature indexes can be all pixels in the prediction block or part of the pixels in the prediction block.

[0189] In another possible implementation, for candidate pixels, adjacent pixels of a reconstructed area of ​​the current block may be determined; and the adjacent pixels of the reconstructed area may be used as candidate pixels.

[0190] In the embodiment of the present application, the candidate pixels used to derive the at least two texture feature indexes may be pixels adjacent to a reconstructed region of the current block, such as the reconstructed regions to the left and right of the current block. Because the reconstructed regions to the left and above, while not the current block, are adjacent to the current block, for example, if the textures are connected, they can be used to estimate the texture of the current block to a certain extent.

[0191] In yet another possible implementation, more pixels are considered. For candidate pixels, adjacent pixels in the reconstructed area and at least part of the pixels in the prediction block may be used as candidate pixels.

[0192] In an embodiment of the present application, as for the candidate pixels used to derive at least two texture feature indexes, the predicted block of the current block and the adjacent pixels of the reconstructed areas to the left and above the current block can also be used simultaneously. In this way, more pixels are used to derive the texture feature index, making the derived texture feature index more accurate.

[0193] In another possible implementation, considering the pipeline length of the hardware, for candidate pixels, a reference block of the current block may be determined; and at least part of the pixels in the reference block may be used as candidate pixels.

[0194] In an embodiment of the present application, for the current block, at least two texture feature indexes can only be derived after the prediction block is obtained, and only after determining the at least two texture feature indexes can the corresponding at least two transform kernels be determined, and then the inverse transform and subsequent processes can be performed. Here, if a reference block is used to derive the at least two texture feature indexes, theoretically, the process of generating the prediction block and the process of deriving the at least two texture feature indexes can be performed in parallel, which is shorter than the pipeline using the prediction block. Therefore, it is also possible to use the reference block to derive the at least two texture feature indexes, or to derive the at least two texture feature indexes synchronously during the process of generating the prediction block, thereby shortening the pipeline length.

[0195] In some embodiments, for the reference block of the current block, the reference block may be determined to be an integer-pixel reference block; or, the reference block may be determined to be a fractional-pixel reference block.

[0196] In one possible implementation, integer-pixel reference blocks can be used directly as reference blocks, as there is no significant difference in texture direction. For example, for a bidirectionally predicted inter-frame block, at least two texture feature indices can be derived from the integer-pixel reference blocks.

[0197] In another possible implementation, the reference block may also use a pixel-by-pixel reference block that has been subjected to an interpolation filter. Specifically, two reference image blocks are determined for bidirectional prediction of the current block; pixel-by-pixel interpolation filtering is performed on the two reference image blocks to determine two pixel-by-pixel reference image blocks; and a weighted combination of the two pixel-by-pixel reference image blocks is performed to determine the reference block for the current block.

[0198] For example, for a bidirectionally predicted inter-frame block, two reference blocks are first subjected to pixel-by-pixel interpolation filtering, and then the two pixel-by-pixel reference blocks are weightedly combined. After obtaining the two interpolated filtered reference blocks, a weighted average of the interpolated filtered reference blocks can be used to obtain a prediction value, and at least two texture feature indices can be derived from the interpolated filtered reference blocks.

[0199] It should also be noted that, in the embodiment of the present application, for the candidate pixels used to derive at least two texture feature indexes, the number of candidate pixels can be at least one, for example, 1, 2, 3, or more. In some embodiments, the number of candidate pixels can be determined based on the size parameter of the current block.

[0200] That is to say, when determining at least two texture feature indexes of the current block, the number of candidate pixels used can be determined by the size parameter of the current block. For example, if the size of the current block is small, then all available pixels can be counted; if the size of the current block is large, then the current block can be downsampled and counted, such as counting one pixel out of every 2, or 4, or 8 pixels in the horizontal and / or vertical directions. Alternatively, if the size of the current block in one of the horizontal or vertical directions is less than or equal to 8, then all available pixels in that direction are counted; otherwise, if the size of the current block in one of the horizontal or vertical directions is less than or equal to 16, then one pixel out of every 2 pixels in that direction is counted; otherwise, one pixel out of every 4 pixels in that direction is counted, and no specific limitation is given here.

[0201] In some embodiments, determining at least two texture feature indexes of the current block based on the candidate pixels may include: determining the horizontal gradient value and the vertical gradient value of the candidate pixels; determining the texture feature index and the gradient intensity value corresponding to the candidate pixels based on the horizontal gradient value and the vertical gradient value of the candidate pixels; constructing a texture feature statistics table based on the texture feature index and the gradient intensity value corresponding to the candidate pixels; and determining at least two texture feature indexes of the current block based on the texture feature statistics table.

[0202] It should be noted that in an embodiment of the present application, when determining the texture feature index and gradient intensity value corresponding to the candidate pixel based on the horizontal gradient value and vertical gradient value of the candidate pixel, it can include: performing angle mapping based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; and performing gradient intensity calculation based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel.

[0203] In a specific embodiment, performing angle mapping based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel may include: determining the texture feature index corresponding to the candidate pixel using a preset lookup table based on the horizontal gradient value and the vertical gradient value of the candidate pixel.

[0204] In the embodiment of the present application, the horizontal gradient value of the candidate pixel can be expressed as grad x Indicates that the vertical gradient value of the candidate pixel can be expressed as grad yIn this way, according to grad x and grad y Deriving the virtual intra prediction mode can be achieved by looking up a table.

[0205] For example, if abs(grad x ) is equal to 0 and abs(grad y ) is not equal to 0, then there is horizontal texture, corresponding to intra prediction mode 18 in some technologies. y ) is equal to 0 and abs(grad x ) is not equal to 0, then there is vertical texture, corresponding to the intra prediction mode 50 in some technologies. x ) and abs(grad y ) are not equal to 0, if abs(grad x ) is equal to abs(grad y ), and grad x and grad y The symbols are the same, corresponding to intra prediction mode 34 in some technologies. x ) is equal to 2 times abs(grad y ), and grad x and grad y The symbols are the same, corresponding to the intra-frame prediction mode 40 in some technologies. In addition, other situations can be determined by looking up the table according to the same principle.

[0206] In a specific embodiment, performing gradient strength calculation based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient strength value corresponding to the candidate pixel may include: performing an addition operation on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient strength value corresponding to the candidate pixel.

[0207] Here, the gradient intensity value corresponding to the candidate pixel can be recorded as amp. For example, amp=abs(grad x )+abs(grad y ).

[0208] It should be noted that, in the embodiment of the present application, the horizontal gradient value and the vertical gradient value of the candidate pixel can be calculated using the Sobel operator. For example, for the Sobel operator, the specific calculation is as follows:

[0209] Operator for horizontal gradient value:

[0210] Operator for vertical gradient value:

[0211] So, suppose the pixel value at pixel position (x, y) is P x,y , then the horizontal gradient value grad x And the vertical gradient value grad y The calculation of grad is as follows: x =P x+1,y-1 +2*P x+1,y +P x+1,y+1 -P x-1,y-1 -2*P x-1,y -P x-1,y+1 (22) grad y =P x-1,y+1 +2*P x,y+1 +P x+1,y+1 -P x-1,y-1 -2*P x,y-1 -P x+1,y-1 (twenty three)

[0212] In some embodiments, constructing a texture feature statistics table based on the texture feature index and gradient intensity value corresponding to the candidate pixel may include: determining at least one texture feature index and at least one corresponding gradient intensity value when the number of candidate pixels is at least one; determining at least one reference texture feature index with mutually different characteristics based on the at least one texture feature index, and accumulating the gradient intensity values ​​belonging to the same reference texture feature index based on the at least one gradient intensity value to determine the gradient intensity cumulative value corresponding to the at least one reference texture feature index; constructing a texture feature statistics table based on the at least one reference texture feature index and the gradient intensity cumulative value corresponding to the at least one reference texture feature index.

[0213] That is to say, in an embodiment of the present application, taking at least some pixels in a prediction block as candidate pixels as an example, the gradient values ​​of all or some pixels in the prediction block are calculated. Generally speaking, the horizontal gradient value and the vertical gradient value can be calculated. Here, the Sobel operator can be used to calculate the gradient value. For a certain pixel, the texture direction of the pixel can be inferred based on its horizontal gradient value and vertical gradient value. For example, if the horizontal gradient value is non-zero and the vertical gradient value is zero, then the texture of the pixel is in the vertical direction. Conversely, if the horizontal gradient value is zero and the vertical gradient value is non-zero, then the texture of the pixel is in the horizontal direction. For example, if the horizontal gradient value and the vertical gradient value are equal and not zero, then the texture of the pixel is 45 degrees. Of course, there are many other cases in the embodiment of the present application where the horizontal gradient value and the vertical gradient value are not zero, and the texture direction of the pixel can be determined based on their ratio. In this way, the gradient intensity value of each pixel can be corresponded to the corresponding texture feature index. A texture feature statistical table is thus constructed, and the gradient intensity value of each calculated pixel is added to the corresponding texture feature index item in the statistical table to obtain the final texture feature statistical table, and then at least two texture feature indexes of the current block can be determined based on the texture feature statistical table.

[0214] In one possible implementation, when determining at least two texture feature indexes of the current block based on a texture feature statistics table, the method may include: sorting the texture feature statistics table from high to low according to the gradient intensity cumulative value, and determining at least two gradient intensity cumulative values ​​with the highest sorting level; and determining the reference texture feature indexes corresponding to the at least two gradient intensity cumulative values ​​with the highest sorting level as the at least two texture feature indexes of the current block.

[0215] In embodiments of the present application, a prediction block can be used to derive a texture feature index. This applies to blocks decoded using intra, inter, or IBC methods. If a certain texture exists in the prediction block, it can be assumed that the residual block contains a texture with the same characteristics. For example, the gradient values ​​of all or part of the pixels in the prediction block can be calculated. Generally, horizontal and vertical gradient values ​​can be calculated, and the Sobel operator can be used to calculate the gradient values. For a particular pixel, the texture direction of the pixel can be inferred based on its horizontal and vertical gradient values. For example, if the horizontal gradient value is non-zero and the vertical gradient value is zero, then the texture of the pixel is vertical. Conversely, if the horizontal gradient value is zero and the vertical gradient value is non-zero, then the texture of the pixel is horizontal. For example, if the horizontal gradient value and the vertical gradient value are equal and non-zero, then the texture of the pixel is 45 degrees. Of course, there are many other cases where the horizontal and vertical gradient values ​​are non-zero, and the texture direction of the pixel can be determined based on their ratio. In this way, the corresponding gradient intensity value can be determined based on the horizontal gradient value and the vertical gradient value, and then the corresponding texture feature index can be mapped based on the gradient intensity value. To construct a texture feature index statistics table (i.e., the aforementioned "texture feature statistics table"), the gradient intensity value of each calculated candidate pixel needs to be accumulated to the corresponding texture feature index item in the statistics table. After the gradient statistics are completed, the texture feature index statistics table can display the texture features reflected by the predicted block.

[0216] In addition, similar statistics can be performed on DIMD to derive texture feature indices, but the difference is that DIMD uses the surrounding reconstruction area of ​​the current block, while this solution uses the predicted block of the current block. In terms of implementation, this solution can reuse some logic with DIMD.

[0217] It should also be noted that, regarding the pixels used to derive the texture feature index, the reconstructed areas to the left and above the current block can also be used, similar to the DIMD approach. Because the reconstructed areas to the left and above are not the current block but are adjacent to the current block, for example, when the textures are connected, they can be used to estimate the texture of the current block to a certain extent. Another possibility is to use the predicted block of the current block and the reconstructed areas to the left and above the current block at the same time, so that more pixels can be used to infer the texture feature index.

[0218] Another possibility is to consider the length of the hardware pipeline. For the current block, the texture feature index can only be derived after the prediction block is obtained, and the transformation kernel of LFNST / NSPT can be determined after the texture feature index is determined, and the inverse transformation and subsequent processes can be performed. If a reference block is used to derive the texture feature index, theoretically, the process of generating the prediction block and the process of deriving the texture feature index can be carried out in parallel, which is shorter than the above-mentioned design pipeline. Therefore, the embodiment of the present application can also use the reference block to derive the texture feature index, or synchronously derive the texture feature index in the process of generating the prediction block. Among them, when using the reference block, the whole pixel reference block can be directly used because there will be no obvious difference in the texture direction. For example, a bidirectionally predicted inter-frame block, on the one hand, derives the texture feature index based on the whole pixel reference block, and on the other hand, performs pixel-by-pixel interpolation filtering on the two reference blocks, and then performs weighted combination on the two pixel-by-pixel reference blocks. Specifically, pixel-by-pixel reference blocks that have been interpolated can also be used. For example, for a bidirectionally predicted inter-frame block, it is necessary to first perform pixel-by-pixel interpolation filtering on two reference blocks and then perform a weighted combination of the two pixel-by-pixel reference blocks. After obtaining the two interpolated filtered reference blocks, a weighted average of the interpolated filtered reference blocks can be used to obtain a prediction value, and a texture feature index can be derived based on the interpolated filtered reference blocks.

[0219] In some embodiments, at least two texture feature indexes of the current block are determined, and the method may further include: determining an angle index when the current block is subjected to a geometric partitioning mode; and determining one of the texture feature indexes of the current block according to the angle index.

[0220] In the embodiment of the present application, for the texture feature index corresponding to the geometric partitioning mode (GPM), the GPM prediction block is obtained by combining prediction blocks of different motions. Therefore, it may contain the content of two objects. Generally, the boundary is difficult to predict perfectly, so there may be a large residual at the boundary between the two objects. The direction of this residual is consistent with the "partitioning" direction of the GPM. Therefore, for GPM, the GPM "partitioning" mode can be used to determine the corresponding texture feature index.

[0221] In the embodiment of the present application, the dividing line can be considered as a line composed of points with the same weight in the area where the weight changes in the weight matrix of the GPM. In other words, it is a line composed of points with the median weight, and the point with the median weight may not be at an integer pixel position. Taking the weight of 0 to 8 as an example, the median can be 4. Figure 16 is a schematic diagram of the dividing line of a GPM mode. As shown in Figure 16, the bold solid line is the dividing line described here. In fact, the weight of the GPM is derived based on the dividing line.

[0222] It is also understandable that in embodiments of the present application, the texture feature index of the current block may also directly copy the texture feature index of the reference block. In some embodiments, determining at least two texture feature indexes of the current block may include: determining a reference block for the current block; and determining at least two texture feature indexes of the current block based on at least two intra-frame prediction modes or virtual intra-frame prediction modes of the reference block. In other words, determining a reference block for the current block; and determining at least two texture feature indexes of the current block based on the at least two texture feature indexes of the reference block.

[0223] It should be noted that if the reference block of the current block has an intra-frame decoded block, then the intra-frame decoded block has an intra-frame prediction mode, and at least two intra-frame prediction modes or virtual intra-frame prediction modes of the reference block can be used as at least two texture feature indexes of the current block. If the reference block of the current block contains multiple intra-frame decoded blocks, the intra-frame prediction mode of a certain position can be determined based on the coordinates, for example, the intra-frame prediction mode of the reference block corresponding to the center position of the current block can be used as a texture feature index of the current block. A (virtual) intra-frame prediction mode can be saved for each inter-frame decoded block. If the reference block of the current block is an inter-frame decoded block, then the corresponding texture feature index can be found. In addition, the texture feature index can also be stored at a certain granularity, for example, each 4×4 pixel is a minimum storage unit, and each minimum storage unit shares the same intra-frame prediction mode or virtual intra-frame prediction mode. If the reference block of the current block contains multiple 4×4 minimum storage units, the (virtual) intra-frame prediction mode of a certain position can be determined according to the coordinates, for example, the (virtual) intra-frame prediction mode of the minimum storage unit of the reference block corresponding to the center position of the current block is used as a texture feature index of the current block.

[0224] It is also understandable that in embodiments of the present application, a candidate list may be constructed to determine at least two texture feature indexes for the current block. In some embodiments, the method may further include: constructing a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; decoding the bitstream to determine at least two feature index numbers for the current block; and determining at least two texture feature indexes for the current block based on the candidate list and the at least two feature index numbers. Each feature index number represents a corresponding number of the texture feature index of the current block in the candidate list.

[0225] It should be noted that the aforementioned embodiment mentions deriving texture feature indices using a prediction block or a reference block of the current block or a reconstructed image around the current block. In a specific embodiment, one or more texture feature indices with the largest gradient strength values ​​can be selected to derive at least two transform kernel groups. In addition, a step can be added to construct a candidate list, i.e., a candidate list of texture feature indices, and then use a syntax element written in the bitstream (i.e., the feature index sequence number of the current block) to indicate which candidates are ultimately selected.

[0226] In some embodiments, constructing a candidate list may include: sorting the texture feature statistics table from high to low according to the gradient intensity cumulative value, determining the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; and constructing a candidate list based on the N reference texture feature indexes.

[0227] In some embodiments, constructing a candidate list may include: determining a first texture feature index corresponding to a geometric partitioning mode of the current block, and determining a reference texture feature index corresponding to the top (N-1) gradient intensity accumulation values; and constructing a candidate list based on the first texture feature index and the (N-1) reference texture feature indexes.

[0228] That is to say, in an embodiment of the present application, an example of constructing a candidate list is to still follow the above-mentioned method of calculating the gradient strength, and select the top N texture feature indexes according to the gradient strength from large to small to form a candidate list. Among them, the value of N may be 2, 3, 4, etc. Another example is to assume that the value of N is 4, that is, the texture feature index with the largest gradient strength is the first element of the candidate list, the texture feature index with the second largest gradient strength is the second element of the candidate list, and so on. Another example is that for GPM, the first element of the candidate list can be set to the texture feature index corresponding to the "division" mode of GPM, the second element to the texture feature index with the largest gradient strength, the third element to the texture feature index with the second largest gradient strength, and so on; or vice versa, which is not specifically limited here.

[0229] In some embodiments, constructing the candidate list may further include: determining a reference block of the current block; and constructing the candidate list according to texture feature indexes corresponding to at least two candidate positions in the reference block.

[0230] That is, in an embodiment of the present application, a candidate list is constructed based on at least two texture feature indexes of a reference block. For example, a candidate list is constructed based on the texture feature indexes of a 4×4 minimum storage unit corresponding to certain positions in the reference block. The coordinates of the upper left corner of the current block are (x, y), the width of the current block is nCbW, the height of the current block is nCbH, the center point posC of the current block is (x+nCbW / 2, y+nCbH / 2), the upper left corner is posTL(x, y), the upper right corner is posTR(x+nCbW-1, y), the lower left corner is posBL(x, y+nCbH-1), and the lower right corner is posBR(x+nCbW-1, y+nCbH-1). The candidate list is constructed using the intra-frame prediction modes (texture feature indexes) of the minimum storage units corresponding to posC, posTL, posTR, posBL, and posBR in sequence. Here, the candidate position can be at least two of posC, posTL, posTR, posBL, and posBR, but is not limited to this.

[0231] In this way, after constructing the candidate list, at least two texture feature indexes of the current block are determined based on the syntax elements in the bitstream and the candidate list. The syntax elements used here are the feature index numbers of the current block. Assuming that there are two feature index numbers for the current block, these two feature index numbers can be represented by lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1, respectively. The values ​​of lfnst_nspt_feature_idx0 or lfnst_nspt_feature_idx1 can be integers such as 0, 1, 2, or 3. For example, if the value of lfnst_nspt_feature_idx0 is 0, the first element of the candidate list can be selected; if lfnst_nspt_feature_idx0 is 1, the second element of the candidate list can be selected to obtain one of the texture feature indexes of the current block; similarly, according to the value of lfnst_nspt_feature_idx1, the other texture feature index of the current block can be obtained.

[0232] It is also understandable that in the embodiment of the present application, for at least two feature index numbers of the current block, decoding is performed only when it is determined that the current block uses LFNST or NSPT. In some embodiments, the method may further include: decoding the bitstream to determine transform mode identification information of the current block; and when the transform mode identification information indicates that the current block uses the first transform mode, performing the steps of decoding the bitstream to determine the at least two feature index numbers of the current block.

[0233] It should be noted that, in the embodiment of the present application, the transform mode identification information can be represented by lfnst_nspt_idx, and the transform mode identification information can indicate whether the current block uses the first transform mode. If the value of the transform mode identification information is the first value, it is determined that the current block uses the first transform mode; if the value of the transform mode identification information is the second value, it is determined that the current block does not use the first transform mode.

[0234] Here, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. For example, the first value can be 1 and the second value can be 0.

[0235] It should also be noted that in this embodiment of the present application, the first transform mode can be LFNST transform or NSPT transform. That is, lfnst_nspt_idx is used to indicate whether the current block uses LFNST transform or NSPT transform. If it is determined that the current block uses LFNST transform or NSPT transform, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are further parsed. If the current block does not use LFNST transform or NSPT transform, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are no longer parsed.

[0236] For example, lfnst_nspt_idx can be used to indicate whether the current block uses LFNST / NSPT. If lfnst_nspt_idx is 0, it indicates that the current block does not use LFNST / NSPT. In this case, there is no need to parse lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1. Otherwise, if lfnst_nspt_idx is not 0, it indicates that the current block uses LFNST / NSPT. In this case, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 need to be parsed.

[0237] It should also be noted that this method is applicable not only to inter-frame prediction but also to intra-frame prediction. In addition, the two transform kernel groups in the embodiment of the present application cannot be the same. If N is 4, lfnst_nspt_feature_idx0 has four options, while lfnst_nspt_feature_idx1 has only three options.

[0238] It should also be noted that when constructing the list, the decoder needs to ensure that each item in the candidate list is not repeated. If the available texture feature indexes cannot fill the list, default texture feature indexes can be added, such as horizontal and vertical texture feature indexes.

[0239] In this way, after the candidate list is constructed, at least two texture feature indexes of the current block can be determined according to the at least two feature index numbers obtained from the decoded bitstream.

[0240] S1502: Determine at least two transformation kernel groups according to at least two texture feature indexes.

[0241] S1503 : Determine at least two transform cores of the current block according to the at least two transform core groups.

[0242] It should be noted that after determining the texture feature index of the current block, the transform kernel group of the current block can be determined based on the correspondence between the texture feature index and the transform kernel group. For example, some technologies use 4 transform kernel groups for LFNST. The correspondence between intra prediction modes and transform kernel groups is detailed in Table 3. Some technologies use more transform kernel groups for LFNST, such as 35. The correspondence between intra prediction modes and transform kernel groups is detailed in Table 4.

[0243] It should also be noted that each transform core group may include at least two transform cores. For example, a transform core group may include two transform cores, three transform cores, or even four optional transform cores, or a greater number of optional transform cores, all of which are not specifically limited herein.

[0244] In some embodiments, referring to FIG. 17 , after step S1502 , the method may further include:

[0245] S1701: Decode a code stream and determine at least two transform core indexes of a current block.

[0246] S1702 : Determine at least two transform cores of the current block according to at least two transform core groups and corresponding at least two transform core indexes.

[0247] It should be noted that, in an embodiment of the present application, when the current block uses the first transform mode, it is also necessary to decode the code stream to determine at least two transform core indexes of the current block, and then determine at least two transform cores of the current block based on the transform core group and the at least two transform core indexes; wherein the transform core index is used to indicate the number of the transform core of the current block in the corresponding transform core group.

[0248] Exemplarily, taking the first transform core group and the second transform core group as an example, the code stream is decoded to determine the first transform core index and the second transform core index of the current block; based on the first transform core group and the first transform core index, the first transform core of the current block is determined; based on the second transform core group and the second transform core index, the second transform core of the current block is determined; wherein, the first transform core index is used to indicate the number of the first transform core of the current block in the first transform core group, and the second transform core index is used to indicate the number of the second transform core of the current block in the second transform core group.

[0249] It should also be noted that in the embodiment of the present application, for the case where the transform core index needs to be transmitted in the code stream, the code stream overhead is large because at least two transform core indices need to be transmitted. In this case, if there is only one transform core in a certain transform core group, then there is no need to transmit the corresponding transform core index at this time. Specifically, when the current block uses LFNST / NSPT, the only transform core in a certain transform core group can be directly used as one of the transform cores of the current block. Exemplarily, if the first transform core group includes only one transform core, then the first transform core index of the current block is no longer transmitted in the code stream, and the only transform core in the first transform core group is used as the first transform core of the current block; or, if the second transform core group includes only one transform core, then the second transform core index of the current block is no longer transmitted in the code stream, and the only transform core in the second transform core group is used as the second transform core of the current block.

[0250] In this way, after determining at least two transform kernels of the current block, inverse transforms can be performed on at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block.

[0251] It should be noted that, in an embodiment of the present application, when performing inverse transformation on at least two groups of first transformation coefficients of the current block according to at least two transformation cores to determine at least two residual blocks of the current block, it can include: if the size parameter of the current block meets the first condition, then performing inverse transformation of the inseparable basic transformation on the at least two groups of first transformation coefficients of the current block according to the at least two transformation cores to determine at least two first residual blocks of the current block; if the size parameter of the current block meets the second condition, then performing inverse transformation of the low-frequency inseparable transformation on the at least two groups of first transformation coefficients of the current block according to the at least two transformation cores to determine at least two transformation blocks of the current block; and performing inverse transformation of the discrete cosine transform on the at least two transformation blocks of the current block to determine at least two first residual blocks of the current block.

[0252] Here, the size parameter of the current block satisfies the first condition, including: the size parameter of the current block is relatively small, for example, the size parameter of the current block is less than a certain threshold. In other words, for relatively small blocks, an NSPT transform kernel is used. That is, an inverse NSPT transform is performed on at least two groups of first transform coefficients of the current block based on the NSPT transform kernel to determine at least two first residual blocks of the current block.

[0253] Here, the size parameter of the current block satisfies the second condition, including: the size parameter of the current block is relatively large, for example, the size parameter of the current block is greater than a certain threshold. That is, for relatively large blocks, an LFNST transform kernel is used. Specifically, an inverse LFNST transform is performed on at least two groups of first transform coefficients of the current block according to the LFNST transform kernel to determine at least two transform blocks of the current block; and an inverse DCT2 transform is performed on the at least two transform blocks of the current block to determine at least two first residual blocks of the current block.

[0254] It should also be noted that in the embodiments of the present application, the "inverse transformation" of the transformation coefficients by the decoding end may also be referred to as "transformation" in the standard text. The "transformation" and "inverse transformation" in this article correspond to two opposite processes. For example, if the "transformation" converts the numerical value of the spatial domain to the coefficient of the frequency domain, then the "inverse transformation" converts the coefficient of the frequency domain to the numerical value of the spatial domain. "Inverse" is relative to "positive", and they are essentially both transformations. It should be noted that if the standard only stipulates decoding, then the "transformation" in the standard text is the decoding part, specifically referring to the "inverse transformation" in this article.

[0255] That is to say, in an embodiment of the present application, the final residual block can be composed of a combination of multiple first residual blocks. The multiple first residual blocks can be called sub-residual blocks, and the size of the sub-residual blocks and the final residual block is the same. Here, at least two sub-residual blocks can be included. Each sub-residual block is obtained by inverse transformation of its own transform kernel (if a secondary transform is used, it is a secondary transform kernel and a basic transform kernel). In terms of process, there can be multiple inverse transformation processes to obtain sub-residual blocks, and the core is the "parallel" multiple inverse transformation process, which is the multi-transformation combination mode described in the embodiment of the present application.

[0256] For example, Figure 18A is a schematic diagram of a decoding process of a residual block, Figure 18B is a schematic diagram of a decoding process with two sub-residual blocks, and Figure 18C is a schematic diagram of a decoding process with two sub-residual blocks. As shown in Figures 18B and 18C, two sub-residual blocks are schematically provided here, but there can also be multiple. Among them, the technical solution of the embodiment of the present application can be optional at the block (CU) level, that is, some blocks will use the technical solution of the embodiment of the present application, and some blocks will use related technical solutions. In other words, in the entire system, these two processes exist.

[0257] In a possible implementation, each residual block corresponds to its own transform coefficient block and quantization coefficient block. Each quantization coefficient block can be parsed from the bitstream, as shown in FIG18B .

[0258] In another possible implementation, the code stream can be decoded to obtain a quantized coefficient block, which is then divided into two transform coefficient blocks for inverse transformation after inverse quantization, as specifically shown in FIG18C .

[0259] In this way, when the current block uses a multi-transform combination mode, at least two groups of first transform coefficients are inversely transformed according to at least two transform kernels, and at least two first residual blocks of the current block can be determined, thereby determining a reconstructed block of the current block. In some embodiments, to determine the reconstructed block of the current block, as shown in FIG19, after step S1203, the method may include:

[0260] S1901: Determine a prediction block for the current block.

[0261] S1902: Determine a reconstructed block of the current block according to the prediction block and at least two first residual blocks.

[0262] It should be noted that determining the reconstructed block of the current block according to the prediction block and the at least two first residual blocks may include: performing addition processing on the prediction block and the at least two first residual blocks to determine the reconstructed block of the current block.

[0263] It should also be noted that determining the reconstructed block of the current block based on the prediction block and at least two first residual blocks may also include: determining the second residual block based on the at least two first residual blocks; and determining the reconstructed block of the current block based on the prediction block and the second residual block.

[0264] That is to say, in an embodiment of the present application, at least two sub-residual blocks (i.e., the first residual block) and the prediction block can be directly added together to obtain a reconstructed block, as shown in FIG20A ; or, at least two sub-residual blocks (i.e., the first residual block) can be combined first to determine the residual block of the current block (i.e., the second residual block), and then the residual block of the current block and the prediction block are added together to obtain a reconstructed block, as shown in FIG20B . The essence of the two is the same.

[0265] In a possible implementation, determining the second residual block according to the at least two first residual blocks may include: performing an addition operation on the at least two first residual blocks to determine the second residual block.

[0266] It should be noted that, in the embodiment of the present application, it is assumed that the present technical solution supports two residual blocks, and the residual value of the first residual block at the coordinate (x, y) is rec0 x,y, the residual value of the second residual block at the coordinate (x, y) is rec1 x,y The final residual block has a residual value of rec at coordinate (x, y) x,y In this implementation, for each coordinate (x, y) of the current block, rec x,y =rec0 x,y +rec1 x,y .

[0267] In another possible implementation, determining the second residual block based on at least two first residual blocks may include: when there is a non-zero overlapping area in the at least two first residual blocks, determining the topmost residual block in the at least two first residual blocks; and determining the residual value of the second residual block in the non-zero overlapping area based on the topmost residual block.

[0268] It should be noted that in an embodiment of the present application, if the problem of image overlap is considered, as shown in Figures 14A, 14B, and 14C, the textures in Figures 14A and 14B are combined into the texture in Figure 14C, but there are some intersections (ie, there are non-zero overlapping areas). One understanding is that one texture blocks another texture, which is very common in the real world, such as when one object blocks part of another object. In this implementation, a processing method is that if there is occlusion, it can always be assumed that a certain residual block is on top. For example, assuming that two residual blocks are supported in this technical solution, and the first residual block is on top, then the residual value of the final residual block in the occluded area can be determined based on the first residual block above, specifically:

[0269] If abs(rec0 x,y )>0,rec x,y =rec0 x,y ;

[0270] Otherwise, rec x,y =rec1 x,y .

[0271] If there are multiple residual blocks, it can be assumed that the texture of the previous residual block is on top of the subsequent residual block.

[0272] Another processing method is to use some syntax elements to indicate which sub-residual block is on top and which sub-residual block is on the bottom. In other words, determining the topmost residual block among the at least two first residual blocks may include: decoding the bitstream, determining the value of the second syntax identification information; and determining the topmost residual block among the at least two first residual blocks based on the value of the second syntax identification information.

[0273] It should be noted that, in the embodiment of the present application, the value of the second syntax identification information can be used to indicate the topmost residual block among the at least two first residual blocks. In this way, by decoding the bitstream, the topmost residual block among the at least two first residual blocks can be determined, and then the residual value of the final residual block in the occluded area can be determined based on the topmost residual block.

[0274] In some embodiments, regarding whether the current block uses a multi-transform combination mode, the method further includes: determining the transform coefficients of the current block; and when the non-zero coefficient information in the transform coefficients meets a preset condition, executing a decoding code stream to determine the value of the first syntax identification information.

[0275] It should be noted that, in an embodiment of the present application, one implementation method for determining whether the current block uses the multi-transform combination mode is to directly use a syntax element (such as the first syntax identification information) to determine whether the current block uses the multi-transform combination mode. Exemplarily, a CU-level or TU-level multi_trans_combine_flag is used. If the value of multi_trans_combine_flag is 1, it indicates that the current block uses the multi-transform combination mode; otherwise, that is, the value of multi_trans_combine_flag is 0, it indicates that the current block does not use the multi-transform combination mode.

[0276] It should also be noted that in this embodiment of the present application, the multi_trans_combine_flag can also be conditionally parsed, using some known conditions to exclude situations where multiple transform combinations will not be used, thereby reducing the overhead of the multi_trans_combine_flag. For example, whether the non-zero coefficient information in the transform coefficients meets a preset condition is used to exclude situations where multiple transform combinations will not be used. The multi_trans_combine_flag is parsed only when the non-zero coefficient information in the transform coefficients meets the preset condition.

[0277] That is, in the embodiments of the present application, if the texture of the residual block is complex, there will still be a large number of non-zero coefficients after transform quantization. In other words, if the transform kernel cannot efficiently process the texture features of the residual block, there will also be a large number of non-zero coefficients after transform quantization. For example, a transform that is separable horizontally and vertically is not very efficient in processing diagonal textures. A transform kernel optimized for a single texture feature may not be very efficient in processing the residual of a composite texture feature, but splitting the residual of a composite texture feature into single texture features and processing it with multiple transform kernels optimized for single texture features may be more efficient. This is also the original intention of the multi-transform combination mode. An intuitive reflection of so-called efficiency is that there are few non-zero coefficients after transform quantization, or the sum of the values ​​of the non-zero coefficients after transform quantization is small. Therefore, the embodiments of the present application can exclude some situations where multi-transform combination is not necessary based on the number of non-zero coefficients, the last non-zero position, or the value of the non-zero coefficient.

[0278] In the design of the LFNST / NSPT transform core, the number of inputs and outputs of the transform core can be different. Taking NSPT as an example, the transform core information of the NSPT decoder in ECM10 is shown in Table 5.

[0279] Table 5

[0280] For example, taking a 4x16 block as an example, the number of NSPT inputs is 24, and the number of outputs is 64. The output number is easy to understand, as a 4x16 block has 64 pixels, so the inverse NSPT transform outputs 64 pixel values. The 24 inputs mean that if the inverse transform of the current block uses NSPT, the number of coefficients between the last non-zero coefficient and the first coefficient (the upper-left corner (0, 0) position) in scan order cannot exceed 24. In other words, if there are more than 24 coefficients, the current block definitely does not use NSPT. LFNST has a similar design: for a given block size, if the position of the last non-zero coefficient is less than (or equal to) a threshold determined by LFNST / NSPT, then LFNST / NSPT is likely used. In this case, the LFNST / NSPT syntax elements need to be parsed to determine whether the current block uses LFNST / NSPT. Otherwise, it is inferred that the current block does not use LFNST / NSPT, and no LFNST / NSPT syntax elements need to be parsed.

[0281] In a specific embodiment, the non-zero coefficient information in the transform coefficients satisfies a preset condition, which may include: the number of coefficients between the first coefficient and the last non-zero coefficient in the transform coefficients does not exceed a first threshold.

[0282] That is, in this embodiment of the present application, a threshold (e.g., a first threshold) may be set for the multi-trans combination mode. If the number of coefficients between the last non-zero coefficient and the first coefficient (at the upper left corner (0, 0)) in scanning order does not exceed the first threshold, then the current block may use the multi-trans combination mode, and it is necessary to analyze the multi_trans_combine_flag. Otherwise, it is not necessary to analyze the multi_trans_combine_flag, and it is inferred that the current block does not use the multi-trans combination mode.

[0283] It should also be noted that the first threshold can be the same as, larger than, or smaller than the threshold of a single LFNST / NSPT. For example, taking two transform kernels as an example, on the one hand, the maximum number of inputs for these two transform kernels is larger than that for a single transform kernel; on the other hand, multiple transform combinations are more efficient than a single transform, as intuitively reflected in fewer nonzero coefficients.

[0284] It is also understandable that the embodiment of the present application can use a high-level syntax to control the switch of the technical solution. In some embodiments, as shown in Figure 21, for step S1201, the method further includes:

[0285] S2101: Decode a code stream and determine a value of third syntax identification information.

[0286] S2102: When the third syntax identification information indicates that the current sequence allows the use of the multi-transformation combination mode, decode the code stream and determine the value of the first syntax identification information.

[0287] It should be noted that, in the embodiment of the present application, the current sequence may include the current block. The third syntax identification information may be represented by sps_lfnst_nspt_enabled_flag, and the third syntax identification information is a syntax element in a sequence parameter set (SPS).

[0288] It should also be noted that, in an embodiment of the present application, if the value of the third grammar identification information is the first value, it is determined that the third grammar identification information indicates that the current sequence allows the use of a multi-transformation combination mode; if the value of the third grammar identification information is the second value, it is determined that the third grammar identification information indicates that the current sequence does not allow the use of a multi-transformation combination mode.

[0289] In an embodiment of the present application, the first value is different from the second value, and the first value and the second value may be in parameter form or in digital form. Specifically, the third syntax identification information may be a parameter written in the profile or the value of a flag, which is not specifically limited here. Exemplarily, the first value may be 1 and the second value may be 0; or, the first value may be 0 and the second value may be 1; or, the first value may be true and the second value may be false; or, the first value may be false and the second value may be true. In a specific embodiment, the first value is 1 and the second value is 0.

[0290] Furthermore, in some embodiments, the method further includes: decoding the code stream to determine a value of the third syntax identification information; when the third syntax identification information indicates that the current sequence allows the use of the multi-transform combination mode, decoding the code stream to determine a value of the fifth syntax identification information; when the fifth syntax identification information indicates that the current image allows the use of the multi-transform combination mode, performing the steps of decoding the code stream to determine a value of the first syntax identification information; wherein, the current sequence includes the current image, and the current image includes the current block.

[0291] It should be noted that, in the embodiment of the present application, the fifth syntax identification information can be represented by ph_inter_lfnst_nspt_enabled_flag, and the fifth syntax identification information is a syntax element at the picture level.

[0292] It should also be noted that, in an embodiment of the present application, if the value of the fifth grammar identification information is the first value, it is determined that the fifth grammar identification information indicates that the current image allows the use of the multi-transform combination mode; if the value of the fifth grammar identification information is the second value, it is determined that the fifth grammar identification information indicates that the current image does not allow the use of the multi-transform combination mode.

[0293] Furthermore, in some embodiments, the method further includes: decoding the code stream to determine a value of third syntax identification information; when the third syntax identification information indicates that the current sequence allows the use of a multi-transform combination mode, decoding the code stream to determine a value of a fourth syntax identification information; when the fourth syntax identification information indicates that the current slice allows the use of a multi-transform combination mode, performing the steps of decoding the code stream to determine a value of the first syntax identification information; wherein the current sequence includes the current slice, and the current slice includes the current block.

[0294] It should be noted that, in the embodiment of the present application, the fourth syntax identification information can be represented by sh_lfnst_nspt_enabled_flag, and the fourth syntax identification information is a syntax element at the slice level.

[0295] It should also be noted that, in an embodiment of the present application, if the value of the fourth grammar identification information is the first value, it is determined that the fourth grammar identification information indicates that the current slice allows the use of the multi-transform combination mode; if the value of the fourth grammar identification information is the second value, it is determined that the fourth grammar identification information indicates that the current slice does not allow the use of the multi-transform combination mode.

[0296] In an embodiment of the present application, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. Specifically, both the fourth grammar identification information and the fifth grammar identification information can be parameters written in the profile, or can be the value of a flag, and this is not specifically limited here. Exemplarily, the first value can be 1 and the second value can be 0; or, the first value can be 0 and the second value can be 1; or, the first value can be true and the second value can be false; or, the first value can be false and the second value can be true. In a specific embodiment, the first value is 1 and the second value is 0.

[0297] That is to say, in an embodiment of the present application, a high-level syntax can be used as a switch for flexibly controlling the present technical solution, such as a sequence-level flag, an image-level flag, or a slice-level flag. For example, taking the use of a sequence-level flag as an example, a syntax element sps_lfnst_nspt_enabled_flag is added to the sequence parameter set SPS. If the value of sps_lfnst_nspt_enabled_flag is 1, the current sequence allows the use of the present technical solution. If the value of sps_lfnst_nspt_enabled_flag is 0, the current sequence does not allow the use of the present technical solution. If the present technical solution is allowed, the decoder also needs to decode the block-level multi_trans_combine_flag when decoding the current block to determine whether the current block uses the multi-transform combination mode.

[0298] An embodiment of the present application provides a decoding method that decodes a bitstream, determines the value of first syntax identification information; when the first syntax identification information indicates that the current block uses a multi-transform combination mode, determines at least two groups of first transform coefficients of the current block; determines at least two transform kernels of the current block, and performs inverse transforms on the at least two groups of first transform coefficients based on the at least two transform kernels to determine at least two first residual blocks of the current block; and determines a reconstructed block of the current block based on the at least two first residual blocks. Thus, when determining that the current block uses a multi-transform combination mode, at least two transform kernels of the current block are first determined, and then inverse transforms are performed on the at least two groups of first transform coefficients of the current block based on the at least two transform kernels. In other words, the current block can use at least two transform kernels to "parallel" process multiple transform / inverse transform processes, using multiple transform kernels to efficiently process a portion of the texture, and finally combining the multiple residual blocks, thereby improving processing efficiency. Furthermore, for some more complex residual blocks, the compression efficiency can also be higher than that of using a single transform kernel, thereby improving encoding and decoding performance.

[0299] In another embodiment of the present application, based on the decoding method described in the above embodiment, FIG22 is a flow chart of an encoding method provided by the embodiment of the present application. As shown in FIG22 , the method may include:

[0300] S2201: Determine a residual block of the current block.

[0301] It should be noted that in the embodiment of the present application, the method is applied to an encoder. Specifically, based on the structure of the encoder 100 shown in FIG10 , the encoding method of the embodiment of the present application can be applied to intra-frame prediction mode and / or inter-frame prediction mode. Here, it is mainly aimed at the multi-transform kernel scheme proposed in the transformation process to improve compression efficiency.

[0302] It should also be noted that, in the embodiments of the present application, the prediction mode of the current block may include: an inter-frame prediction mode and / or an intra-frame prediction mode. The intra-frame prediction mode mainly predicts the current block based on the reconstructed area surrounding the current block, while the inter-frame prediction mode mainly predicts the current block based on the reference image of the current block.

[0303] In some embodiments, determining the residual block of the current block may include: determining a prediction block of the current block; and determining the residual block of the current block according to an original image of the current block and the prediction block of the current block.

[0304] It should be noted that, in the embodiment of the present application, after determining the prediction block of the current block, a subtraction operation may be performed on the original image of the current block and the prediction block of the current block to determine the residual block of the current block.

[0305] S2202 : When the current block uses a multi-transform combination mode, determine at least two first residual blocks of the current block according to the residual block of the current block.

[0306] In the embodiments of the present application, both NSPT and LFNST are transforms that can efficiently process textures at various angles. There may be multiple transform kernels, and one transform kernel may be specifically optimized for a specific angular texture. In addition to angular textures, NSPT and LFNST also include transform kernels for processing gradient textures. In fact, these transform kernels can also be said to be trained KL transforms (Karhunen-Loeve Transform, KLT). In other words, both NSPT and LFNST can have multiple transform kernels, each designed for a specific texture. Specific textures include angular textures, gradient textures, etc. In addition, gradient textures can be further expanded to include horizontal gradient textures, vertical gradient textures, and diagonal gradient textures. Compared to NSPT and LFNST, other basic transforms such as DCT2, DCT8, and DST7 are not as specialized, but they also have their own suitable situations, which is why MTS can improve compression performance. However, these transform kernels can only efficiently process single textures, such as textures that are all oriented in one direction. The efficient processing mentioned here refers to textures that can be fully represented using only a few coefficients in the upper left corner. In other words, the texture can be fully represented by the last few coefficients in the scan order. Transforms like DCT2 can process any texture, but if the coefficients after transformation or quantization are large, it cannot be called efficient processing.

[0307] For example, for the single texture described above, assuming that the lines in Figures 13A and 13B are residual textures, this straight line texture at a certain angle can be called a single texture, and the design of LFNST and NSPT in the related art can efficiently process such residuals. Of course, Figures 13A and 13B are just schematic diagrams. Generally, the residual edges are not very sharp, and sharp edges have a lot of high-frequency information. However, for the texture shown in Figure 13C, it is not a single texture, it is composed of textures in two directions. The transformation kernel currently optimized for a single texture cannot efficiently process such textures.

[0308] If residuals like those in Figure 13C appear frequently in real videos, then it is certainly possible to train kernels for such textures. However, given the ever-changing nature of images and residuals, the number of kernels trained for a wide variety of complex textures would be staggering.

[0309] In one possible implementation, if a complex texture can be split into multiple single textures, then the single textures can be processed separately and then combined. For example, if the texture in Figure 13C is split into two single textures in Figure 13A and Figure 13B, then they can be processed separately. And we know that complex things in nature are composed of simple things. It is in line with the laws of nature to split complex problems into simple problems for processing. Here, splitting can be achieved by dividing into smaller blocks, such as smaller CUs or smaller TUs. For example, Figure 13C can be divided into two CUs or TUs on the left and right for processing. The cost of doing this is the various overheads of division. Dividing a block requires transmitting its various necessary information in the bitstream.

[0310] While textures like those in Figure 13C are relatively easy to resolve by dividing into small blocks, textures like those in Figure 14C require many small blocks to ensure that each block contains only simple textures. In practical applications, Figure 14C may be composed of Figures 14A and 14B. By splitting or decomposing the texture, i.e., decomposing Figure 14C into Figures 14A and 14B, unnecessary divisions can be avoided. This results in a block division approach that favors "more large blocks and fewer small blocks." The technical solution of the embodiments of this application employs this approach to address this problem.

[0311] That is to say, in an embodiment of the present application, the current block can use a multi-transformation combination technical solution, and this technical solution is not limited to inseparable transforms such as NSPT and LFNST, but is also applicable to separable transforms optimized for specific textures, and efficient processing can be achieved by splitting / decomposing the texture.

[0312] It can be understood that in the embodiment of the present application, whether the current block uses the multi-transform combination mode can be determined by first calculating the coding cost of the current block using the multi-transform combination mode and the coding cost of the current block not using the multi-transform combination mode (i.e., the current block uses a single transform kernel), and then determining the difference between the two coding costs. If the coding cost of the current block using the multi-transform combination mode is less than the coding cost of the current block not using the multi-transform combination mode, then it can be determined that the current block uses the multi-transform combination mode; if the coding cost of the current block using the multi-transform combination mode is greater than or equal to the coding cost of the current block not using the multi-transform combination mode, then it can be determined that the current block does not use the multi-transform combination mode.

[0313] It should be noted that, in embodiments of the present application, whether the current block uses the multi-transform combination mode can be indicated by first syntax identification information. In some embodiments, the method further includes: determining a value of the first syntax identification information; encoding the value of the first syntax identification information, and writing the resulting encoded bits into the bitstream.

[0314] In an embodiment of the present application, the first syntax identification information is used to indicate whether the current block uses the multi-transform combination mode. The first syntax identification information can be represented by multi_trans_combine_flag. If the current block uses the multi-transform combination mode, the value of the first syntax identification information can be determined to be a first value; if the current block does not use the multi-transform combination mode, the value of the first syntax identification information can be determined to be a second value.

[0315] Here, the first value is different from the second value, and the first value and the second value can be in parameter form or in numeric form. Specifically, the first syntax identification information can be a parameter written in the profile or a flag value, which is not specifically limited here. For example, the first value can be 1 and the second value can be 0; or the first value can be 0 and the second value can be 1; or the first value can be true and the second value can be false; or the first value can be false and the second value can be true.

[0316] In a specific embodiment, the first value is 1 and the second value is 0. That is, if the current block uses the multi-transform combination mode, the value of the first syntax identification information is determined to be equal to 1; if the current block does not use the multi-transform combination mode, the value of the first syntax identification information is determined to be equal to 0; so that the decoding end can determine whether the current block uses the multi-transform combination mode based on the value of the first syntax identification information.

[0317] In an embodiment of the present application, when the current block uses a multi-transform combination mode, the residual block of the current block may be composed of multiple residual blocks, and these multiple residual blocks may be referred to as divided residual blocks (i.e., first residual blocks). In some embodiments, determining at least two first residual blocks of the current block based on the residual block of the current block may include: pre-analyzing the residual block of the current block using a preset method to determine whether the current block uses a multi-transform combination mode, and, when the current block uses the multi-transform combination mode, dividing the residual block of the current block into the at least two first residual blocks.

[0318] In the embodiment of the present application, the decoder needs to decode the mode selected by the encoder from the bitstream and then decode according to the decoded mode. The encoder needs to select the mode or parameters it considers best from multiple possible modes or parameters and pass them to the bitstream. For the multi-transform combination mode, it is necessary to divide a residual block into at least two transform cores for encoding. Whether the residual block of the current block is suitable for the multi-transform combination mode and which residuals each transform core should encode can be determined using a preset method. Among them, the preset method may include a statistical gradient method, a neural network method, an RDOQ method, etc.

[0319] For example, one possible implementation is for the encoder to perform a preliminary analysis before attempting a multi-transform combination to determine whether the residual block of the current block is suitable for the multi-transform combination mode. This can include detecting whether there are two textures in the residual block and whether the texture in the residual block is consistent with the texture suitable for the transform kernel. For example, this detection method can use a statistical gradient method, similar to the method used to derive texture feature indices. This can eliminate some cases that are not suitable for multi-transform combinations and reduce the burden on the encoder.

[0320] Another possible implementation method is to use some neural network methods to perform pre-analysis to determine whether the residual block of the current block is suitable for the multi-transform combination mode, and then decompose the residual block into two parts, for example, decomposing Figure 14C into Figure 14A and Figure 14B. In this way, the divided residuals can be processed by different transform kernels.

[0321] In another possible implementation, an RDOQ method may be used. As shown in FIG23 , the method may include:

[0322] S2301: Split the residual block of the current block using multiple combination methods to determine multiple combined residual blocks.

[0323] S2302: Calculate the costs of the residual blocks of the various combinations respectively to determine the cost results of the various combinations.

[0324] S2303: Determine a target combination method based on the respective cost results of the multiple combination methods.

[0325] S2304: Split the residual block of the current block by using a target combination method to determine at least two first residual blocks of the current block.

[0326] In a specific embodiment, determining the target combination method based on the cost results corresponding to each of the multiple combination methods may include: determining the minimum cost result from the cost results corresponding to each of the multiple combination methods, and determining the combination method corresponding to the minimum cost result as the target combination method.

[0327] It should be noted that each combined residual block includes at least two sub-residual blocks. Here, first, based on the multiple combinations of the sub-residual blocks, multiple combined residual blocks for the current block are determined. Then, cost calculation, such as RDOQ cost calculation, is performed on each of these multiple combined residual blocks. The combination corresponding to the minimum cost value is selected. Then, based on the combination corresponding to the minimum cost value, at least two sub-residual blocks for the current block are determined. In this case, the coding cost is minimized.

[0328] It's also worth noting that the first two implementations can be described as the encoder actively determining and splitting the residual texture. The latter implementation doesn't require analyzing the image to determine whether the residual texture is suitable for splitting between two transform kernels, or how to split it. Instead, the encoder considers various possible scenarios, determining whether to use a multi-transform combination mode based on the coding cost, and then selecting the combination of residual blocks that minimizes the coding cost. These methods can be combined, and are not specifically limited here.

[0329] It can also be understood that in an embodiment of the present application, if the problem of image overlap is considered, as shown in Figures 14A, 14B, and 14C, the textures in Figures 14A and 14B are combined into the texture in Figure 14C, but there are some intersections (i.e., there are non-zero overlapping areas). One understanding is that one texture blocks another texture, which is very common in the real world, such as when one object blocks part of another object. In this implementation, a possible implementation is that if there is occlusion, it can always be assumed that a certain residual block is on top. For example, assuming that two residual blocks are supported in this technical solution, and the first residual block is on top, the decoding end can determine the residual value of the final residual block in the occluded area based on the first residual block.

[0330] In another possible implementation, some syntax elements may be used to indicate which sub-residual block is on top and which is on the bottom. Accordingly, the method further includes: determining a value of second syntax identification information; encoding the value of the second syntax identification information, and writing the resulting coded bits into the bitstream.

[0331] It should be noted that, in an embodiment of the present application, the second syntax element identification information can be used to indicate the topmost residual block among the at least two first residual blocks; in this way, the decoding end can subsequently determine the topmost residual block among the at least two first residual blocks by decoding the code stream, and then determine the residual value of the final residual block in the occluded area based on the topmost residual block.

[0332] S2203 : Determine at least two transformation kernels of the current block, and transform at least two first residual blocks respectively according to the at least two transformation kernels to determine at least two groups of first transformation coefficients of the current block.

[0333] It should be noted that in the embodiment of the present application, the encoder does not directly consider which part of the residual is assigned to which transform kernel. Instead, it adjusts the values ​​of the two sets of transform coefficients and uses the coding cost (bit rate and distortion cost) to determine whether it has been done well enough. This is similar to the RDOQ process in the encoder, but now it optimizes the two sets of transform coefficients together. First, the entire residual block is transformed and quantized using two transform kernels respectively. In theory, the coefficients obtained by the two transform kernels can reconstruct a complete residual block. Of course, due to the existence of quantization, there will be distortion. Then, try adjusting the two sets of transform coefficients, that is, try multiple combinations of the two sets of transform coefficients, such as reducing the values ​​of the transform coefficients from high frequency to low frequency, or setting the values ​​of the transform coefficients to 0. Calculate the cost of encoding these transform coefficients, that is, the rate portion of the rate-distortion. Dequantize and inversely transform the transform coefficients, combine the residual blocks, and subtract them from the original residual blocks to obtain the distortion cost, that is, the distortion portion of the rate-distortion. Adding the rate and distortion in a given ratio is the coding cost of the current mode. Try adjusting the coefficients, and finally select the coefficient combination with the lowest coding cost as the multi-transform combination scheme. When adjusting the coefficients, you can adjust the first and second sets of coefficients in sequence, optimizing alternately. That is, first use RDOQ to optimize the first set of transform coefficients to a better level. Because a second transform is added here, the adjustment standard can be different from the method of using only one transform. For example, a higher weight is set for the rate so that the first transform uses fewer codewords to produce a decent residual block. Then, the first set of transform coefficients is fixed, the second set of coefficients is adjusted, and RDOQ is used to optimize the second set of transform coefficients, so as to determine at least two sets of transform coefficients for the current block.

[0334] It should also be noted that in the embodiments of the present application, at least two transform kernels for the current block need to be determined first, which can be determined using texture feature indices. In some embodiments, the method may include: determining at least two texture feature indices for the current block; determining at least two transform kernel groups based on the at least two texture feature indices; and determining at least two transform kernels for the current block based on the at least two transform kernel groups.

[0335] Furthermore, in some embodiments, the method further includes encoding the at least two transform cores of the current block and writing the obtained coded bits into the bitstream. In this way, the decoder can obtain the at least two transform cores of the current block by decoding the bitstream.

[0336] It can be understood that in the embodiment of the present application, the transform core group selected for LFNST and NSPT within a frame is automatically matched according to the intra-frame prediction mode. An intra-frame prediction mode represents a texture feature, for example, an intra-frame angle prediction mode represents the angle of a texture. This derivation can reduce the overhead of the code stream because it no longer needs to use syntax elements to write which transform core group to select in the code stream. In the embodiment of the present application, the transform core group of LFNST / NSPT can be derived from certain information of the current block. An intra-frame prediction mode can determine a transform core group, or it can be said that a texture feature index can correspond to a transform core group. For the convenience of description, a virtual intra-frame prediction mode (or "texture feature index") is proposed here. The so-called virtual intra-frame prediction mode is because this mode does not participate in the prediction, but is only used to select the transform core group of NSPT or LFNST. The derivation of the texture feature index is described in detail below.

[0337] In some embodiments, determining at least two texture feature indexes of the current block may include: determining candidate pixels for deriving the at least two texture feature indexes; and determining the at least two texture feature indexes of the current block based on the candidate pixels.

[0338] In a possible implementation, for candidate pixels, a prediction block of the current block may be determined; and at least part of the pixels in the prediction block are used as candidate pixels.

[0339] In the embodiment of the present application, if a certain texture exists in the prediction block, it can be considered that the residual block has a texture with the same characteristics. In this way, the candidate pixels used to derive at least two texture feature indexes can be all pixels in the prediction block or part of the pixels in the prediction block.

[0340] In another possible implementation, for candidate pixels, adjacent pixels of a reconstructed area of ​​the current block may be determined; and the adjacent pixels of the reconstructed area may be used as candidate pixels.

[0341] In the embodiment of the present application, the candidate pixels used to derive the at least two texture feature indexes may be pixels adjacent to a reconstructed region of the current block, such as the reconstructed regions to the left and right of the current block. Because the reconstructed regions to the left and above, while not the current block, are adjacent to the current block, for example, if the textures are connected, they can be used to estimate the texture of the current block to a certain extent.

[0342] In yet another possible implementation, more pixels are considered. For candidate pixels, adjacent pixels in the reconstructed area and at least part of the pixels in the prediction block may be used as candidate pixels.

[0343] In an embodiment of the present application, as for the candidate pixels used to derive at least two texture feature indexes, the predicted block of the current block and the adjacent pixels of the reconstructed areas to the left and above the current block can also be used simultaneously. In this way, more pixels are used to derive the texture feature index, making the derived texture feature index more accurate.

[0344] In another possible implementation, considering the pipeline length of the hardware, for candidate pixels, a reference block of the current block may be determined; and at least part of the pixels in the reference block may be used as candidate pixels.

[0345] In an embodiment of the present application, for the current block, at least two texture feature indexes can only be derived after the prediction block is obtained, and only after determining the at least two texture feature indexes can the corresponding at least two transform kernels be determined, and then the inverse transform and subsequent processes can be performed. Here, if a reference block is used to derive the at least two texture feature indexes, theoretically, the process of generating the prediction block and the process of deriving the at least two texture feature indexes can be performed in parallel, which is shorter than the pipeline using the prediction block. Therefore, it is also possible to use the reference block to derive the at least two texture feature indexes, or to derive the at least two texture feature indexes synchronously during the process of generating the prediction block, thereby shortening the pipeline length.

[0346] In some embodiments, for the reference block of the current block, the reference block may be determined to be an integer-pixel reference block; or, the reference block may be determined to be a fractional-pixel reference block.

[0347] In one possible implementation, integer-pixel reference blocks can be used directly as reference blocks, as there is no significant difference in texture direction. For example, for a bidirectionally predicted inter-frame block, at least two texture feature indices can be derived from the integer-pixel reference blocks.

[0348] In another possible implementation, the reference block may also use a pixel-by-pixel reference block that has been subjected to an interpolation filter. Specifically, two reference image blocks are determined for bidirectional prediction of the current block; pixel-by-pixel interpolation filtering is performed on the two reference image blocks to determine two pixel-by-pixel reference image blocks; and a weighted combination of the two pixel-by-pixel reference image blocks is performed to determine the reference block for the current block.

[0349] For example, for a bidirectionally predicted inter-frame block, two reference blocks are first subjected to pixel-by-pixel interpolation filtering, and then the two pixel-by-pixel reference blocks are weightedly combined. After obtaining the two interpolated filtered reference blocks, a weighted average of the interpolated filtered reference blocks can be used to obtain a prediction value, and at least two texture feature indices can be derived from the interpolated filtered reference blocks.

[0350] It should also be noted that, in the embodiment of the present application, for the candidate pixels used to derive at least two texture feature indexes, the number of candidate pixels can be at least one, for example, 1, 2, 3, or more. In some embodiments, the number of candidate pixels can be determined based on the size parameter of the current block.

[0351] That is to say, when determining at least two texture feature indexes of the current block, the number of candidate pixels used can be determined by the size parameter of the current block. For example, if the size of the current block is small, then all available pixels can be counted; if the size of the current block is large, then the current block can be downsampled and counted, such as counting one pixel out of every 2, or 4, or 8 pixels in the horizontal and / or vertical directions. Alternatively, if the size of the current block in one of the horizontal or vertical directions is less than or equal to 8, then all available pixels in that direction are counted; otherwise, if the size of the current block in one of the horizontal or vertical directions is less than or equal to 16, then one pixel out of every 2 pixels in that direction is counted; otherwise, one pixel out of every 4 pixels in that direction is counted, and no specific limitation is given here.

[0352] In some embodiments, determining at least two texture feature indexes of the current block based on the candidate pixels may include: determining the horizontal gradient value and the vertical gradient value of the candidate pixels; determining the texture feature index and the gradient intensity value corresponding to the candidate pixels based on the horizontal gradient value and the vertical gradient value of the candidate pixels; constructing a texture feature statistics table based on the texture feature index and the gradient intensity value corresponding to the candidate pixels; and determining at least two texture feature indexes of the current block based on the texture feature statistics table.

[0353] It should be noted that in an embodiment of the present application, when determining the texture feature index and gradient intensity value corresponding to the candidate pixel based on the horizontal gradient value and vertical gradient value of the candidate pixel, it can include: performing angle mapping based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; and performing gradient intensity calculation based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel.

[0354] In a specific embodiment, performing angle mapping based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel may include: determining the texture feature index corresponding to the candidate pixel using a preset lookup table based on the horizontal gradient value and the vertical gradient value of the candidate pixel.

[0355] In the embodiment of the present application, the horizontal gradient value of the candidate pixel can be expressed as grad x Indicates that the vertical gradient value of the candidate pixel can be expressed as grad y In this way, according to gradx and grad y Deriving the virtual intra prediction mode can be achieved by looking up a table.

[0356] For example, if abs(grad x ) is equal to 0 and abs(grad y ) is not equal to 0, then there is horizontal texture, corresponding to intra prediction mode 18 in some technologies. y ) is equal to 0 and abs(grad x ) is not equal to 0, then there is vertical texture, corresponding to the intra prediction mode 50 in some technologies. x ) and abs(grad y ) are not equal to 0, if abs(grad x ) is equal to abs(grad y ), and grad x and grad y The symbols are the same, corresponding to intra prediction mode 34 in some technologies. x ) is equal to 2 times abs(grad y ), and grad x and grad y The symbols are the same, corresponding to the intra-frame prediction mode 40 in some technologies. In addition, other situations can be determined by looking up the table according to the same principle.

[0357] In a specific embodiment, performing gradient strength calculation based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient strength value corresponding to the candidate pixel may include: performing an addition operation on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient strength value corresponding to the candidate pixel.

[0358] Here, the gradient intensity value corresponding to the candidate pixel can be recorded as amp. For example, amp=abs(grad x )+abs(grad y ).

[0359] It should be noted that, in the embodiment of the present application, the horizontal gradient value and the vertical gradient value of the candidate pixel can be calculated using the Sobel operator. For example, for the Sobel operator, the specific calculation is as follows:

[0360] Operator for horizontal gradient value:

[0361] Operator for vertical gradient value:

[0362] So, suppose the pixel value at pixel position (x, y) is P x,y , then the horizontal gradient value grad x And the vertical gradient value grad y The calculation of grad is as follows: x =P x+1,y-1 +2*P x+1,y +P x+1,y+1 -P x-1,y-1 -2*P x-1,y -P x-1,y+1 (22) grad y =P x-1,y+1 +2*P x,y+1 +P x+1,y+1 -P x-1,y-1 -2*P x,y-1 -P x+1,y-1 (twenty three)

[0363] In some embodiments, constructing a texture feature statistics table based on the texture feature index and gradient intensity value corresponding to the candidate pixel may include: determining at least one texture feature index and at least one corresponding gradient intensity value when the number of candidate pixels is at least one; determining at least one reference texture feature index with mutually different characteristics based on the at least one texture feature index, and accumulating the gradient intensity values ​​belonging to the same reference texture feature index based on the at least one gradient intensity value to determine the gradient intensity cumulative value corresponding to the at least one reference texture feature index; constructing a texture feature statistics table based on the at least one reference texture feature index and the gradient intensity cumulative value corresponding to the at least one reference texture feature index.

[0364] That is to say, in an embodiment of the present application, taking at least some pixels in a prediction block as candidate pixels as an example, the gradient values ​​of all or some pixels in the prediction block are calculated. Generally speaking, the horizontal gradient value and the vertical gradient value can be calculated. Here, the Sobel operator can be used to calculate the gradient value. For a certain pixel, the texture direction of the pixel can be inferred based on its horizontal gradient value and vertical gradient value. For example, if the horizontal gradient value is non-zero and the vertical gradient value is zero, then the texture of the pixel is in the vertical direction. Conversely, if the horizontal gradient value is zero and the vertical gradient value is non-zero, then the texture of the pixel is in the horizontal direction. For example, if the horizontal gradient value and the vertical gradient value are equal and not zero, then the texture of the pixel is 45 degrees. Of course, there are many other cases in the embodiment of the present application where the horizontal gradient value and the vertical gradient value are not zero, and the texture direction of the pixel can be determined based on their ratio. In this way, the gradient intensity value of each pixel can be corresponded to the corresponding texture feature index. A texture feature statistical table is thus constructed, and the gradient intensity value of each calculated pixel is added to the corresponding texture feature index item in the statistical table to obtain the final texture feature statistical table, and then at least two texture feature indexes of the current block can be determined based on the texture feature statistical table.

[0365] In one possible implementation, when determining at least two texture feature indexes of the current block based on a texture feature statistics table, the method may include: sorting the texture feature statistics table from high to low according to the gradient intensity cumulative value, and determining at least two gradient intensity cumulative values ​​with the highest sorting level; and determining the reference texture feature indexes corresponding to the at least two gradient intensity cumulative values ​​with the highest sorting level as the at least two texture feature indexes of the current block.

[0366] In embodiments of the present application, a prediction block can be used to derive a texture feature index. This applies to blocks decoded using intra, inter, or IBC methods. If a certain texture exists in the prediction block, it can be assumed that the residual block contains a texture with the same characteristics. For example, the gradient values ​​of all or part of the pixels in the prediction block can be calculated. Generally, horizontal and vertical gradient values ​​can be calculated, and the Sobel operator can be used to calculate the gradient values. For a particular pixel, the texture direction of the pixel can be inferred based on its horizontal and vertical gradient values. For example, if the horizontal gradient value is non-zero and the vertical gradient value is zero, then the texture of the pixel is vertical. Conversely, if the horizontal gradient value is zero and the vertical gradient value is non-zero, then the texture of the pixel is horizontal. For example, if the horizontal gradient value and the vertical gradient value are equal and non-zero, then the texture of the pixel is 45 degrees. Of course, there are many other cases where the horizontal and vertical gradient values ​​are non-zero, and the texture direction of the pixel can be determined based on their ratio. In this way, the corresponding gradient intensity value can be determined based on the horizontal gradient value and the vertical gradient value, and then the corresponding texture feature index can be mapped based on the gradient intensity value. To construct a texture feature index statistics table (i.e., the aforementioned "texture feature statistics table"), the gradient intensity value of each calculated candidate pixel needs to be accumulated to the corresponding texture feature index item in the statistics table. After the gradient statistics are completed, the texture feature index statistics table can display the texture features reflected by the predicted block.

[0367] In addition, similar statistics can be performed on DIMD to derive texture feature indices, but the difference is that DIMD uses the surrounding reconstruction area of ​​the current block, while this solution uses the predicted block of the current block. In terms of implementation, this solution can reuse some logic with DIMD.

[0368] It should also be noted that, regarding the pixels used to derive the texture feature index, the reconstructed areas to the left and above the current block can also be used, similar to the DIMD approach. Because the reconstructed areas to the left and above are not the current block but are adjacent to the current block, for example, when the textures are connected, they can be used to estimate the texture of the current block to a certain extent. Another possibility is to use the predicted block of the current block and the reconstructed areas to the left and above the current block at the same time, so that more pixels can be used to infer the texture feature index.

[0369] Another possibility is to consider the length of the hardware pipeline. For the current block, the texture feature index can only be derived after the prediction block is obtained, and the transformation kernel of LFNST / NSPT can be determined after the texture feature index is determined, and the inverse transformation and subsequent processes can be performed. If a reference block is used to derive the texture feature index, theoretically, the process of generating the prediction block and the process of deriving the texture feature index can be carried out in parallel, which is shorter than the above-mentioned design pipeline. Therefore, the embodiment of the present application can also use the reference block to derive the texture feature index, or synchronously derive the texture feature index in the process of generating the prediction block. Among them, when using the reference block, the whole pixel reference block can be directly used because there will be no obvious difference in the texture direction. For example, a bidirectionally predicted inter-frame block, on the one hand, derives the texture feature index based on the whole pixel reference block, and on the other hand, performs pixel-by-pixel interpolation filtering on the two reference blocks, and then performs weighted combination on the two pixel-by-pixel reference blocks. Specifically, pixel-by-pixel reference blocks that have been interpolated can also be used. For example, for a bidirectionally predicted inter-frame block, it is necessary to first perform pixel-by-pixel interpolation filtering on two reference blocks and then perform a weighted combination of the two pixel-by-pixel reference blocks. After obtaining the two interpolated filtered reference blocks, a weighted average of the interpolated filtered reference blocks can be used to obtain a prediction value, and a texture feature index can be derived based on the interpolated filtered reference blocks.

[0370] In some embodiments, at least two texture feature indexes of the current block are determined, and the method may further include: determining an angle index when the current block is subjected to a geometric partitioning mode; and determining one of the texture feature indexes of the current block according to the angle index.

[0371] In the embodiment of the present application, for the texture feature index corresponding to the geometric partitioning mode (GPM), the GPM prediction block is obtained by combining prediction blocks of different motions. Therefore, it may contain the content of two objects. Generally, the boundary is difficult to predict perfectly, so there may be a large residual at the boundary between the two objects. The direction of this residual is consistent with the "partitioning" direction of the GPM. Therefore, for GPM, the GPM "partitioning" mode can be used to determine the corresponding texture feature index.

[0372] In the embodiment of the present application, the dividing line can be considered as a line composed of points with the same weight in the area where the weight changes in the weight matrix of the GPM. In other words, it is a line composed of points with the median weight, and the point with the median weight may not be at an integer pixel position. Taking the weight of 0 to 8 as an example, the median can be 4. Figure 16 is a schematic diagram of the dividing line of a GPM mode. As shown in Figure 16, the bold solid line is the dividing line described here. In fact, the weight of the GPM is derived based on the dividing line.

[0373] It is also understandable that in embodiments of the present application, the texture feature index of the current block may also directly copy the texture feature index of the reference block. In some embodiments, determining at least two texture feature indexes of the current block may include: determining a reference block for the current block; and determining at least two texture feature indexes of the current block based on at least two intra-frame prediction modes or virtual intra-frame prediction modes of the reference block. In other words, determining a reference block for the current block; and determining at least two texture feature indexes of the current block based on the at least two texture feature indexes of the reference block.

[0374] It should be noted that if the reference block of the current block has an intra-frame coded block, then the intra-frame coded block has an intra-frame prediction mode, and at least two intra-frame prediction modes or virtual intra-frame prediction modes of the reference block can be used as at least two texture feature indexes of the current block. If the reference block of the current block contains multiple intra-frame coded blocks, the intra-frame prediction mode of a certain position can be determined based on the coordinates, for example, the intra-frame prediction mode of the reference block corresponding to the center position of the current block can be used as a texture feature index of the current block. A (virtual) intra-frame prediction mode can be saved for each inter-frame coded block, and if the reference block of the current block is an inter-frame coded block, the corresponding texture feature index can be found. In addition, the texture feature index can also be stored at a certain granularity, for example, each 4×4 pixel is a minimum storage unit, and each minimum storage unit shares the same intra-frame prediction mode or virtual intra-frame prediction mode. If the reference block of the current block contains multiple 4×4 minimum storage units, the (virtual) intra-frame prediction mode of a certain position can be determined according to the coordinates, for example, the (virtual) intra-frame prediction mode of the minimum storage unit of the reference block corresponding to the center position of the current block is used as a texture feature index of the current block.

[0375] It is also understandable that in the embodiments of the present application, a candidate list may be constructed to determine at least two texture feature indexes of the current block. In some embodiments, the method may further include: constructing a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; selecting at least two candidate texture feature indexes from the N candidate texture feature indexes, and determining the at least two candidate texture feature indexes as the at least two texture feature indexes of the current block.

[0376] In a specific embodiment, selecting at least two candidate texture feature indexes from N candidate texture feature indexes may include: performing cost value calculations on the N candidate texture feature indexes respectively to determine the cost results of each of the N candidate texture feature indexes; determining at least two target cost values ​​based on the cost results of each of the N candidate texture feature indexes, and determining the candidate texture feature indexes corresponding to the at least two target cost values ​​as at least two texture feature indexes.

[0377] It should also be noted that, in the embodiment of the present application, cost values ​​are calculated for each of the N candidate texture feature indexes to determine a cost result for each of the N candidate texture feature indexes. Here, the cost result can be determined based on a cost result of rate distortion optimization (RDO), a cost result of sum of absolute differences (SAD), or even a cost result of sum of absolute transformed differences (SATD), but this is not limited to any of the above.

[0378] For example, taking the rate-distortion optimization method as an example, the rate-distortion cost calculation can be performed for each of the N candidate texture feature indexes in the candidate list, and the cost results of each of the N candidate texture feature indexes can be determined. Then, at least two target cost results can be selected from them, and then at least two texture feature indexes can be determined. For example, with respect to the determination of two texture feature indexes, after calculating the cost results of each of the N candidate texture feature indexes, the candidate texture feature index corresponding to the minimum cost result and the candidate texture feature index corresponding to the second minimum cost result can be selected from these cost results as the two texture feature indexes of the current block, thereby improving coding efficiency.

[0379] In some embodiments, the method further includes: determining at least two feature index serial numbers of the current block based on at least two texture feature indexes of the current block; wherein the at least two feature index serial numbers are respectively used to indicate the numbers of the at least two texture feature indexes of the current block in the candidate list; encoding the at least two feature index serial numbers of the current block, and writing the obtained encoded bits into the bitstream.

[0380] It should be noted that the aforementioned embodiment mentions the use of a prediction block or a reference block of the current block or a reconstructed image around the current block to derive a texture feature index. In a specific embodiment, one or more texture feature indexes with the largest gradient intensity value can be selected to derive at least two transform kernel groups. In addition, a step can be added to construct a candidate list, that is, a candidate list of texture feature indexes, and then use the syntax elements written in the bitstream (that is, the feature index sequence number of the current block) to indicate which candidates are finally selected, so that at the decoding end, the decoder can obtain at least two texture feature indexes of the current block by decoding the bitstream.

[0381] It should also be noted that, in an embodiment of the present application, for the determination of two texture feature indexes, if the candidate list includes only two candidate texture feature indexes, then not only can these two candidate texture feature indexes be directly used as the two texture feature indexes required for the current block without the need to calculate the cost result; but also there is no need to determine the corresponding texture feature index in the candidate list, nor to write it into the bitstream, thereby saving coding bits and improving coding efficiency.

[0382] In some embodiments, constructing a candidate list may include: sorting the texture feature statistics table from high to low according to the gradient intensity cumulative value, determining the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; and constructing a candidate list based on the N reference texture feature indexes.

[0383] In some embodiments, constructing a candidate list may include: determining a first texture feature index corresponding to a geometric partitioning mode of the current block, and determining a reference texture feature index corresponding to the top (N-1) gradient intensity accumulation values; and constructing a candidate list based on the first texture feature index and the (N-1) reference texture feature indexes.

[0384] That is to say, in an embodiment of the present application, an example of constructing a candidate list is to still follow the above-mentioned method of calculating the gradient strength, and select the top N texture feature indexes according to the gradient strength from large to small to form a candidate list. Among them, the value of N may be 2, 3, 4, etc. Another example is to assume that the value of N is 4, that is, the texture feature index with the largest gradient strength is the first element of the candidate list, the texture feature index with the second largest gradient strength is the second element of the candidate list, and so on. Another example is that for GPM, the first element of the candidate list can be set to the texture feature index corresponding to the "division" mode of GPM, the second element to the texture feature index with the largest gradient strength, the third element to the texture feature index with the second largest gradient strength, and so on; or vice versa, which is not specifically limited here.

[0385] In some embodiments, constructing the candidate list may further include: determining a reference block of the current block; and constructing the candidate list according to texture feature indexes corresponding to at least two candidate positions in the reference block.

[0386] That is, in an embodiment of the present application, a candidate list is constructed based on at least two texture feature indexes of a reference block. For example, a candidate list is constructed based on the texture feature indexes of a 4×4 minimum storage unit corresponding to certain positions in the reference block. The coordinates of the upper left corner of the current block are (x, y), the width of the current block is nCbW, the height of the current block is nCbH, the center point posC of the current block is (x+nCbW / 2, y+nCbH / 2), the upper left corner is posTL(x, y), the upper right corner is posTR(x+nCbW-1, y), the lower left corner is posBL(x, y+nCbH-1), and the lower right corner is posBR(x+nCbW-1, y+nCbH-1). The candidate list is constructed using the intra-frame prediction modes (texture feature indexes) of the minimum storage units corresponding to posC, posTL, posTR, posBL, and posBR in sequence. Here, the candidate position can be at least two of posC, posTL, posTR, posBL, and posBR, but is not limited to this.

[0387] In this way, after constructing the candidate list, at least two texture feature indexes of the current block are determined based on the syntax elements in the bitstream and the candidate list. The syntax elements used here are the feature index numbers of the current block. Assuming that there are two feature index numbers for the current block, these two feature index numbers can be represented by lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1, respectively. The values ​​of lfnst_nspt_feature_idx0 or lfnst_nspt_feature_idx1 can be integers such as 0, 1, 2, or 3. For example, if the value of lfnst_nspt_feature_idx0 is 0, the first element of the candidate list can be selected; if lfnst_nspt_feature_idx0 is 1, the second element of the candidate list can be selected to obtain one of the texture feature indexes of the current block; similarly, according to the value of lfnst_nspt_feature_idx1, the other texture feature index of the current block can be obtained.

[0388] It is also understandable that in the embodiment of the present application, encoding is performed only when it is determined that the current block uses LFNST or NSPT. In some embodiments, the method may further include: when the current block uses the first transform mode, encoding the at least two feature index numbers of the current block, and writing the resulting coded bits into the bitstream.

[0389] In some embodiments, the method may further include: determining transform mode identification information of the current block; encoding the transform mode identification information of the current block, and writing the obtained encoded bits into a bitstream.

[0390] It should be noted that, in the embodiment of the present application, the transform mode identification information may be represented by lfnst_nspt_idx, and the transform mode identification information may indicate whether the current block uses the first transform mode. If the current block uses the first transform mode, the value of the transform mode identification information is determined to be the first value; if the current block does not use the first transform mode, the value of the transform mode identification information is determined to be the second value.

[0391] Here, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. For example, the first value can be 1 and the second value can be 0.

[0392] It should also be noted that in this embodiment of the present application, the first transform mode can be LFNST transform or NSPT transform. That is, lfnst_nspt_idx is used to indicate whether the current block uses LFNST transform or NSPT transform. If it is determined that the current block uses LFNST transform or NSPT transform, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are further encoded. If the current block does not use LFNST transform or NSPT transform, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are not encoded.

[0393] For example, lfnst_nspt_idx can be used to indicate whether the current block uses LFNST / NSPT. If lfnst_nspt_idx is 0, it indicates that the current block does not use LFNST / NSPT. In this case, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 do not need to be encoded. Otherwise, if lfnst_nspt_idx is not 0, it indicates that the current block uses LFNST / NSPT. In this case, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 need to be encoded.

[0394] It should also be noted that this method is applicable not only to inter-frame prediction but also to intra-frame prediction. In addition, the two transform kernel groups in the embodiment of the present application cannot be the same. If N is 4, lfnst_nspt_feature_idx0 has four options, while lfnst_nspt_feature_idx1 has only three options.

[0395] It should also be noted that when the encoder constructs the list, it needs to ensure that each item in the candidate list is not repeated. If the available texture feature indexes cannot fill the list, default texture feature indexes can be added, such as horizontal and vertical texture feature indexes.

[0396] In this way, after the candidate list is constructed, at least two texture feature indexes of the current block can be determined according to the at least two feature index numbers obtained from the decoded bitstream.

[0397] It is also understandable that after determining the texture feature index of the current block, the transform kernel group of the current block can be determined based on the correspondence between the texture feature index and the transform kernel group. For example, some technologies use 4 transform kernel groups in LFNST. The correspondence between intra prediction modes and transform kernel groups is detailed in Table 3. Some technologies use LFNST with more transform kernel groups, such as 35. The correspondence between intra prediction modes and transform kernel groups is detailed in Table 4.

[0398] In the embodiment of the present application, each transform core group may include at least two transform cores. For example, the transform core group may include two transform cores, three transform cores, or even four optional transform cores, or a larger number of optional transform cores, which are not specifically limited here.

[0399] In an embodiment of the present application, for at least two transform core groups, each transform core group includes at least one candidate transform core. Accordingly, taking the first transform core group as an example, determining at least two transform cores for the current block based on the at least two transform core groups may include: performing cost value calculations on the at least one candidate transform core based on the at least one candidate transform core in the first transform core group to determine a cost result for each of the at least one candidate transform core; determining a minimum cost result from the cost results of the at least one candidate transform core, and using the candidate transform core corresponding to the minimum cost result as the first transform core for the current block; wherein the first transform core group is one of the at least two transform core groups, and the first transform core is one of the at least two transform cores.

[0400] In one possible implementation, for the first transform core group, a cost value is calculated for at least one candidate transform core in the first transform core group. Taking the first candidate transform core as an example, the method may include: transforming and quantizing the residual block of the current block based on the first candidate transform core, determining the first candidate quantization coefficient of the current block, and performing entropy coding on the first candidate quantization coefficient to determine the first cost result of the first candidate transform core; inverse quantizing and inverse transforming the first candidate quantization coefficient to determine the first candidate residual block of the current block, and determining the first candidate prediction block of the current block based on the first candidate residual block; performing cost calculation based on the first candidate prediction block and the original image of the current block to determine the second cost result of the first candidate transform core; and determining the cost result of the first candidate transform core based on the first cost result and the second cost result of the first candidate transform core.

[0401] It should be noted that, in the embodiment of the present application, the first candidate transformation core is any transformation core in the first transformation core group, so that the cost results of each of the first transformation core group can be determined.

[0402] It should also be noted that, in the embodiment of the present application, the first cost result may represent the cost of the first candidate transform core incurred in the bitstream, and the second cost result may represent the cost of the distortion of the first candidate transform core. Here, determining the cost result of the first candidate transform core based on the first cost result and the second cost result of the first candidate transform core may include: performing a sum operation on the first cost result and the second cost result of the first candidate transform core, and using the sum of the two as the cost result of the first candidate transform core.

[0403] It should also be noted that in the embodiment of the present application, the cost result calculation here can be determined based on the cost result of RDO, or based on the cost result of SAD, or even based on the cost result of SATD, but is not specifically limited.

[0404] It should also be noted that, in the embodiment of the present application, still taking the first transform core group as an example, the first transform core group may include one, two, three, or more candidate transform cores. If the first transform core group includes only one candidate transform core, then this candidate transform core can be directly used as the first transform core of the current block without performing cost calculation.

[0405] In some embodiments, the method further includes: determining at least two transform core indexes of the current block based on at least two transform cores and at least two transform core groups of the current block; encoding the at least two transform core indexes and writing the obtained encoded bits into the bitstream.

[0406] It should be noted that in this embodiment of the present application, the at least two transform core indices are used to indicate the numbers of the at least two transform cores of the current block in the corresponding transform core group. When the current block uses a multi-transform combination mode, the encoder also needs to encode the at least two transform core indices and write them into the bitstream. This allows the decoder to obtain the at least two transform core indices by decoding the bitstream and then directly determine the at least two transform cores of the current block, thereby improving decoding efficiency.

[0407] For example, using the first and second transform core groups as examples, the first and second transform core indices of the current block are first determined. These indices are then encoded, and the resulting encoded bits are written into the bitstream. Subsequently, at the decoding end, the first transform core of the current block is determined based on the first transform core group and the first transform core index; and the second transform core of the current block is determined based on the second transform core group and the second transform core index. The first transform core index indicates the number of the first transform core of the current block in the first transform core group, and the second transform core index indicates the number of the second transform core of the current block in the second transform core group.

[0408] It should also be noted that in the embodiment of the present application, for the case where the transform core index needs to be transmitted in the code stream, the code stream overhead is large because at least two transform core indices need to be transmitted. In this case, if there is only one transform core in a certain transform core group, then there is no need to transmit the corresponding transform core index at this time. Specifically, when the current block uses LFNST / NSPT, the only transform core in a certain transform core group can be directly used as one of the transform cores of the current block. Exemplarily, if the first transform core group includes only one transform core, then the first transform core index of the current block is no longer transmitted in the code stream, and the only transform core in the first transform core group is used as the first transform core of the current block; or, if the second transform core group includes only one transform core, then the second transform core index of the current block is no longer transmitted in the code stream, and the only transform core in the second transform core group is used as the second transform core of the current block.

[0409] In this way, after determining at least two transform kernels of the current block, at least two first residual blocks may be transformed respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block.

[0410] It should be noted that in the embodiments of this application, the "transformation" of the residual block by the encoder can also be called a "forward transform," specifically referring to the transformation from the spatial domain to the frequency domain to remove residual correlation. It should be noted that if the standard only specifies decoding, then the "transformation" in the standard text refers to the decoding part, specifically referring to the "inverse transform" in this article.

[0411] It should also be noted that, in an embodiment of the present application, when transforming at least two first residual blocks according to at least two transform cores respectively to determine at least two groups of first transform coefficients of the current block, it can include: if the size parameter of the current block meets the first condition, performing an inseparable basic transform on the at least two first residual blocks according to the at least two transform cores to determine at least two groups of first transform coefficients of the current block; if the size parameter of the current block meets the second condition, performing a discrete cosine transform on the at least two first residual blocks of the current block to determine at least two transform blocks of the current block; and performing a low-frequency inseparable transform on the at least two transform blocks of the current block according to the at least two transform cores to determine at least two groups of first transform coefficients of the current block.

[0412] Here, the size parameter of the current block satisfies the first condition, including: the size parameter of the current block is relatively small, for example, the size parameter of the current block is less than a certain threshold. In other words, for relatively small blocks, an NSPT transform kernel is used. That is, an NSPT transform is performed on at least two first residual blocks of the current block according to the NSPT transform kernel to determine at least two sets of first transform coefficients for the current block.

[0413] Here, the size parameter of the current block satisfies the second condition, including: the size parameter of the current block is relatively large, for example, the size parameter of the current block is greater than a certain threshold. That is, for relatively large blocks, an LFNST transform kernel is used. Specifically, a DCT2 base transform is first performed on each of the at least two first residual blocks of the current block. An LFNST transform is then performed on each of the at least two transform blocks of the current block based on the LFNST transform kernel to determine at least two sets of first transform coefficients for the current block.

[0414] S2204: Perform encoding processing on at least two groups of first transform coefficients of the current block, and write the obtained encoding bits into a bitstream.

[0415] It should be noted that in the embodiment of the present application, when the current block uses a multi-transform combination mode, it is necessary to encode at least two groups of first transform coefficients of the current block. In one possible implementation, encoding the at least two groups of first transform coefficients of the current block and writing the resulting coded bits into the bitstream may include: encoding the at least two groups of first transform coefficients of the current block separately, and writing the resulting coded bits into the bitstream.

[0416] It should also be noted that, in an embodiment of the present application, if the current block uses a multi-transform combination mode, then the at least two groups of first transform coefficients of the current block can be encoded separately, and the resulting coded bits written into the bitstream. In other words, if the current block uses a multi-transform combination mode, then the value of the first syntax identification information of the current block can be encoded first, and the resulting coded bits written into the bitstream; then, the at least two groups of first transform coefficients of the current block can be encoded separately, and the resulting coded bits written into the bitstream.

[0417] Thus, in an embodiment of the present application, at least two groups of first transform coefficients of the current block can be directly obtained by decoding the code stream. Here, if a syntax element (first syntax identification information) is directly used to determine whether the current block uses a multi-transform combination mode, then the value of the first syntax identification information can be parsed before parsing the transform coefficients. For example, when the value of the first syntax identification information is equal to 1, at this time, at least two groups of first transform coefficients can be obtained respectively by decoding the code stream, and the decoding method of each group of transform coefficients is the same as that of the related art. This method is relatively intuitive. If it is already clear that the current block uses multiple transform combinations, then multiple groups of first transform coefficients can be directly parsed.

[0418] In another possible implementation, encoding processing is performed on at least two groups of first transform coefficients of the current block. As shown in FIG. 24 , after step S2203, the method further includes:

[0419] S2401 : Determine a second transform coefficient of the current block according to at least two groups of first transform coefficients of the current block.

[0420] S2402 , performing encoding processing on the second transform coefficients of the current block respectively, and writing the obtained encoding bits into a bitstream.

[0421] It should be noted that in this embodiment of the present application, the last non-zero coefficient in the second transform coefficients is used to indicate the division strategy of the at least two groups of first transform coefficients. In this implementation, when encoding the transform coefficients, it is not clear whether the encoding actually requires one or multiple groups of transform coefficients, but it is formally a group of transform coefficients, namely, the second transform coefficients of the current block. If it is determined that the current block uses a multi-transform combination, then the subsequent decoding end can split the sum of the second transform coefficients of the current block into at least two groups of first transform coefficients after determining it.

[0422] In a specific implementation, the second transform coefficients may be evenly divided to determine at least two groups of first transform coefficients for the current block. In an embodiment of the present application, taking the splitting into two groups of transform coefficients as an example, the second transform coefficient written into the code stream may be evenly divided, for example, the first half is the first group of transform coefficients, and the second half is the second group of transform coefficients. For example, if there are a total of N coefficients between the first transform coefficient and the last non-zero transform coefficient, then N / 2 coefficients are the first group of transform coefficients, and the remaining N / 2 coefficients are the second group of transform coefficients. This method requires that the encoder must meet the requirements when encoding the transform coefficients, such as by forcibly padding with 0 or forcibly quantizing certain coefficients to 0 to ensure that the dividing point of the two groups of coefficients is exactly at N / 2.

[0423] In another specific implementation, the last non-zero coefficient in the second transform coefficients may be determined; the second transform coefficients may be divided according to a division strategy indicated by the last non-zero coefficient to determine at least two groups of first transform coefficients for the current block. In the embodiment of the present application, still taking the division into two groups of transform coefficients as an example, the division strategy indicated by the last non-zero coefficient includes: the last non-zero coefficient indicates the position of the last non-zero coefficient in the first group of transform coefficients, or the last non-zero coefficient indicates the position of the last non-zero coefficient in the second group of transform coefficients, which is not specifically limited here.

[0424] It should be noted that in the embodiment of the present application, the information used to split the two groups of transform coefficients can be hidden in the coefficients. For example, the last non-zero coefficient is the last non-zero coefficient position in the first group of transform coefficients. That is, the number of coefficients in the first group of transform coefficients from the first coefficient (the upper left corner (0, 0) position) to the last non-zero coefficient in the scanning order is reduced by one. Here, the reason for reducing by one is that each group of transform coefficients should not be all zeros, otherwise the transformation will be meaningless.

[0425] For example, the transform coefficients are arranged in scan order, and the coefficients written into the bitstream are 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 7. The last non-zero coefficient, 7, is not a true coefficient. Instead, it refers to the eight coefficients in positions 0 through 7, which are the first set of transform coefficients: 12, 5, 5, 6, 0, 2, 0, 1. The remaining coefficients are the second set of transform coefficients: 20, 9, 9, 4. These coefficients are placed in the corresponding positions of the two-dimensional coefficient block, and the values ​​of the other positions are set to 0.

[0426] Alternatively, the last non-zero coefficient may be the number of coefficients from the first to the last non-zero coefficient in the second group of transform coefficients in the scanning order minus 1. Here, the reason for minus 1 is that each group of coefficients should not be all zeros, otherwise the transform will be meaningless.

[0427] For example, the coefficients are arranged in scan order, and the coefficients written into the bitstream are 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 3. The last non-zero coefficient, 3, is not a true coefficient; rather, it refers to the fact that the second group of transform coefficients in the above coefficients has four, namely, 20, 9, 9, 4. The preceding ones are the first group of transform coefficients, namely, 12, 5, 5, 6, 0, 2, 0, 1. These coefficients are placed in the corresponding positions in the two-dimensional coefficient block, and the values ​​of the other positions are 0.

[0428] In this way, when the current block uses a multi-transform combination mode, the embodiment of the present application can also use a set of transform coefficients to write into the code stream, and the subsequent decoding end can also obtain at least two sets of first transform coefficients of the current block according to different division strategies.

[0429] It can also be understood that in the embodiment of the present application, regarding whether the value of the first syntax identification information is written into the bitstream, the method further includes: determining the transform coefficient of the current block; when the non-zero coefficient information in the transform coefficient meets the preset conditions, performing encoding processing on the value of the first syntax identification information, and writing the obtained encoded bits into the bitstream.

[0430] It should be noted that, in an embodiment of the present application, one implementation method is to directly use a syntax element (such as the first syntax identification information) to determine whether the current block uses the multi-transform combination mode. Exemplarily, a CU-level or TU-level multi_trans_combine_flag is used. If the value of multi_trans_combine_flag is 1, it indicates that the current block uses the multi-transform combination mode; otherwise, that is, the value of multi_trans_combine_flag is 0, it indicates that the current block does not use the multi-transform combination mode.

[0431] It should also be noted that in this embodiment of the present application, the multi_trans_combine_flag can also be conditionally encoded, using some known conditions to exclude situations where multiple transform combinations will not be used, thereby saving the overhead of the multi_trans_combine_flag. For example, whether the non-zero coefficient information in the transform coefficients meets the preset conditions is used to exclude situations where multiple transform combinations will not be used. The multi_trans_combine_flag is only encoded when the non-zero coefficient information in the transform coefficients meets the preset conditions.

[0432] That is, in the embodiments of the present application, if the texture of the residual block is complex, there will still be a large number of non-zero coefficients after transform quantization. In other words, if the transform kernel cannot efficiently process the texture features of the residual block, there will also be a large number of non-zero coefficients after transform quantization. For example, a transform that is separable horizontally and vertically is not very efficient in processing diagonal textures. A transform kernel optimized for a single texture feature may not be very efficient in processing the residual of a composite texture feature, but splitting the residual of a composite texture feature into single texture features and processing it with multiple transform kernels optimized for single texture features may be more efficient. This is also the original intention of the multi-transform combination mode. An intuitive reflection of so-called efficiency is that there are few non-zero coefficients after transform quantization, or the sum of the values ​​of the non-zero coefficients after transform quantization is small. Therefore, the embodiments of the present application can exclude some situations where multi-transform combination is not necessary based on the number of non-zero coefficients, the last non-zero position, or the value of the non-zero coefficient.

[0433] In the design of the LFNST / NSPT transform core, the number of inputs and outputs of the transform core can be different. Taking NSPT as an example, the transform core information of the NSPT decoder in ECM10 is shown in Table 5.

[0434] For example, taking a 4x16 block as an example, the number of NSPT inputs is 24, and the number of outputs is 64. The output number is easy to understand, as a 4x16 block has 64 pixels, so the inverse NSPT transform outputs 64 pixel values. The 24 inputs mean that if the inverse transform of the current block uses NSPT, the number of coefficients between the last non-zero coefficient and the first coefficient (the upper-left corner (0, 0) position) in scan order cannot exceed 24. In other words, if there are more than 24 coefficients, the current block definitely does not use NSPT. LFNST has a similar design: for a given block size, if the position of the last non-zero coefficient is less than (or equal to) a threshold determined by LFNST / NSPT, then LFNST / NSPT is likely used. In this case, the LFNST / NSPT syntax elements need to be parsed to determine whether the current block uses LFNST / NSPT. Otherwise, it is inferred that the current block does not use LFNST / NSPT, and no LFNST / NSPT syntax elements need to be parsed.

[0435] In a specific embodiment, the non-zero coefficient information in the transform coefficients satisfies a preset condition, which may include: the number of coefficients between the first coefficient and the last non-zero coefficient in the transform coefficients does not exceed a first threshold.

[0436] That is, in this embodiment of the present application, a threshold (e.g., a first threshold) may be set for the multi-trans combination mode. If the number of coefficients between the last non-zero coefficient and the first coefficient (at the upper left corner (0, 0)) in scanning order does not exceed the first threshold, then the current block may use the multi-trans combination mode, and it is necessary to analyze the multi_trans_combine_flag. Otherwise, it is not necessary to analyze the multi_trans_combine_flag, and it is inferred that the current block does not use the multi-trans combination mode.

[0437] It should also be noted that the first threshold can be the same as, larger than, or smaller than the threshold of a single LFNST / NSPT. For example, taking two transform kernels as an example, on the one hand, the maximum number of inputs for these two transform kernels is larger than that for a single transform kernel; on the other hand, multiple transform combinations are more efficient than a single transform, as intuitively reflected in fewer nonzero coefficients.

[0438] It is also understandable that embodiments of the present application may use a high-level syntax to control the on / off of the present technical solution. In some embodiments, the method further includes: determining a value of third syntax identification information; encoding the value of the third syntax identification information, and writing the resulting encoded bits into the bitstream.

[0439] In this embodiment of the present application, the third syntax identification information is used to indicate whether the current sequence allows the use of the multi-transformation combination mode. Specifically, if the current sequence uses the multi-transformation combination mode, the value of the third syntax identification information can be determined to be the first value; if the current sequence does not use the multi-transformation combination mode, the value of the third syntax identification information can be determined to be the second value.

[0440] Furthermore, in some embodiments, the method further includes: when the current sequence allows the use of the multi-transform combination mode, performing a step of determining whether the current block uses the multi-transform combination mode.

[0441] It should be noted that, in the embodiment of the present application, the third syntax identification information can be represented by sps_lfnst_nspt_enabled_flag, and the third syntax identification information is a syntax element in a sequence parameter set (SPS).

[0442] It should also be noted that in the embodiment of the present application, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. Specifically, the third syntax identification information can be a parameter written in the profile, or it can be the value of a flag, which is not specifically limited here. Exemplarily, the first value can be 1 and the second value can be 0; or, the first value can be 0 and the second value can be 1; or, the first value can be true and the second value can be false; or, the first value can be false and the second value can be true. In a specific embodiment, the first value is 1 and the second value is 0.

[0443] Furthermore, in some embodiments, the method further includes: determining a value of third syntax identification information and a value of fifth syntax identification information; wherein the third syntax identification information is used to indicate whether the current sequence allows the use of the multi-transform combination mode, and the fifth syntax identification information is used to indicate whether the current image allows the use of the multi-transform combination mode; encoding the values ​​of the third syntax identification information and the fifth syntax identification information, and writing the obtained coded bits into the bitstream. The current sequence includes the current image, and the current image includes the current block.

[0444] It should be noted that, in this embodiment of the present application, the fifth syntax flag information may be represented by ph_inter_lfnst_nspt_enabled_flag, and the fifth syntax flag information is a picture-level syntax element. Specifically, if the current picture allows the use of the multi-transform combination mode, the value of the fifth syntax flag information is determined to be a first value; if the current picture does not allow the use of the multi-transform combination mode, the value of the fifth syntax flag information is determined to be a second value.

[0445] Furthermore, in some embodiments, the method further includes: determining a value of third grammar identification information and a value of fourth grammar identification information; wherein the third grammar identification information is used to indicate whether the current sequence allows the use of a multi-transformation combination mode, and the fourth grammar identification information is used to indicate whether the current slice allows the use of a multi-transformation combination mode, the current sequence includes the current slice, and the current slice includes the current block; encoding the value of the third grammar identification information and the value of the fourth grammar identification information, and writing the obtained coded bits into the bitstream.

[0446] It should be noted that, in this embodiment of the present application, the fourth syntax identification information may be represented by sh_lfnst_nspt_enabled_flag, and the fourth syntax identification information is a slice-level syntax element. Specifically, if the current slice allows the use of the multi-transform combination mode, the value of the fourth syntax identification information is determined to be a first value; if the current slice does not allow the use of the multi-transform combination mode, the value of the fourth syntax identification information is determined to be a second value.

[0447] In an embodiment of the present application, the first value is different from the second value, and the first value and the second value can be in parameter form or in digital form. Specifically, both the fourth syntax identification information and the fifth syntax identification information can be parameters written in the profile, or can be the value of a flag, and this is not specifically limited here. Exemplarily, the first value can be 1 and the second value can be 0; or, the first value can be 0 and the second value can be 1; or, the first value can be true and the second value can be false; or, the first value can be false and the second value can be true. In a specific embodiment, the first value is 1 and the second value is 0.

[0448] That is to say, in an embodiment of the present application, a high-level syntax can be used as a switch for flexibly controlling the present technical solution, such as a sequence-level flag, an image-level flag, or a slice-level flag. For example, taking the use of a sequence-level flag as an example, a syntax element sps_lfnst_nspt_enabled_flag is added to the sequence parameter set SPS. If the value of sps_lfnst_nspt_enabled_flag is 1, the current sequence allows the use of the present technical solution. If the value of sps_lfnst_nspt_enabled_flag is 0, the current sequence does not allow the use of the present technical solution. If the present technical solution is allowed, the encoder also needs to write the block-level multi_trans_combine_flag into the bitstream when encoding the current block, so that the decoding end can determine whether the current block uses the multi-transform combination mode through decoding.

[0449] In another embodiment of the present application, the embodiment of the present application further provides a code stream, which is generated by bit encoding based on the information to be encoded; wherein the information to be encoded may include at least one of the following: at least two groups of first transform coefficients of the current block, at least two transform core indexes of the current block, at least two feature index serial numbers of the current block, transform mode identification information of the current block, the value of the first syntax identification information, the value of the second syntax identification information and the value of the third syntax identification information.

[0450] In an embodiment of the present application, the value of the first syntax identification information is used to indicate whether the current sequence allows the use of a multi-transform combination mode, the second syntax element identification information is used to indicate the topmost residual block among at least two first residual blocks, and the value of the third syntax identification information is used to indicate whether the current slice allows the use of a multi-transform combination mode.

[0451] An embodiment of the present application provides an encoding method, which determines a residual block of a current block; when the current block uses a multi-transform combination mode, determines at least two first residual blocks of the current block based on the residual block of the current block; determines at least two transform kernels of the current block, and transforms the at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block; encodes the at least two groups of first transform coefficients of the current block, and writes the obtained coded bits into a bitstream. In this way, when determining that the current block uses a multi-transform combination mode, at least two transform kernels of the current block are first determined, and then at least two first residual blocks of the current block are transformed respectively according to the at least two transform kernels, and the decoding end performs inverse transform on at least two groups of first transform coefficients of the current block respectively according to the at least two transform kernels; that is, the current block can use at least two transform kernels to "parallel" process multiple transform / inverse transform processes, use multiple transform kernels to efficiently process a part of the texture respectively, and finally combine multiple residual blocks, thereby improving processing efficiency, and for some more complex residual blocks, its compression efficiency can also be higher than the compression efficiency of using a single transform kernel, thereby improving encoding and decoding performance.

[0452] In another embodiment of the present application, based on the encoding and decoding methods described in the aforementioned embodiments, for NSPT and LFNST, both NSPT and LFNST are transforms that process textures at various angles. They may have multiple transform kernels, and one transform kernel may be specifically optimized for a specific angular texture. Of course, in addition to angular textures, NSPT and LFNST also include transform kernels for processing gradient textures. In fact, these transform kernels can also be said to be trained KLT transforms. It can also be summarized as NSPT and LFNST having multiple transform kernels, each designed for a specific texture, including angular textures, gradient textures, etc. Of course, gradient textures can be further expanded to include horizontal gradient textures, vertical gradient textures, diagonal gradient textures, etc. Compared to NSPT and LFNST, other basic transforms such as DCT2, DCT8, DST7, etc., although not as specialized, also have their own suitable situations, which is why MTS can improve compression performance.

[0453] However, these transform kernels can only efficiently process single textures, such as textures that are entirely oriented in a single direction. Efficient processing here refers to textures that can be fully represented using only the coefficients in the upper left corner, or textures that can be fully represented using the last few coefficients in the scan order. Transforms like DCT2 can process any texture, but they cannot be considered efficient if the transformed or quantized coefficients are very large.

[0454] As shown in Figures 13A, 13B, and 13C, assuming that the lines in Figures 13A and 13B are residual textures, this straight line texture at a certain angle can be called a single texture, and the design of LFNST and NSPT in related technologies can efficiently process such residuals. Of course, Figures 13A and 13B are just schematic diagrams. Generally, the residual edges will not be very sharp, and sharp edges have a lot of high-frequency information. However, for the texture shown in Figure 13C, it is not a single texture, it is composed of textures in two directions. The transformation kernel currently optimized for a single texture cannot efficiently process such textures.

[0455] If residuals like those in Figure 13C appear frequently in real videos, then it is certainly possible to train kernels for such textures. However, given the ever-changing nature of images and residuals, the number of kernels trained for a wide variety of complex textures would be staggering.

[0456] In one possible implementation, if a complex texture can be split into multiple single textures, then the single textures can be processed separately and then combined. For example, if the texture in Figure 13C is split into two single textures in Figure 13A and Figure 13B, then they can be processed separately. And we know that complex things in nature are composed of simple things. It is in line with the laws of nature to split complex problems into simple problems for processing. Here, splitting can be achieved by dividing into smaller blocks, such as smaller CUs or smaller TUs. For example, Figure 13C can be divided into two CUs or TUs on the left and right for processing. The cost of doing this is the various overheads of division. Dividing a block requires transmitting its various necessary information in the bitstream.

[0457] While textures like those in Figure 13C are relatively easy to resolve by dividing into small blocks, textures like those in Figure 14C require many small blocks to ensure that each block contains only simple textures. In practical applications, Figure 14C may be composed of Figures 14A and 14B. By splitting or decomposing the texture, i.e., decomposing Figure 14C into Figures 14A and 14B, unnecessary divisions can be avoided. This results in a block division approach that favors "more large blocks and fewer small blocks." The technical solution of the embodiments of this application employs this approach to address this problem.

[0458] The following describes how this technical solution is implemented from the perspectives of a decoder and an encoder.

[0459] (1)Decoder.

[0460] In related technologies, a block of quantized coefficients is dequantized to obtain a transform coefficient block, which is then inversely transformed to a residual block corresponding to a transform kernel (a secondary transform kernel and a base transform kernel if a quadratic transform is used). This residual block is then added to the prediction block to obtain the reconstructed block.

[0461] In the present technical solution, the final residual block can be composed of a plurality of residual blocks. The plurality of residual blocks can be called sub-residual blocks, and the size of the sub-residual blocks and the final residual block is the same. There are at least two sub-residual blocks here. Each sub-residual block is obtained by inverse transformation of its own transform kernel (if a secondary transform is used, it is a secondary transform kernel and a basic transform kernel). In terms of process, there are multiple inverse transformation processes to obtain residual blocks and a process to combine residual blocks. The core of the present technical solution is the "parallel" process of multiple inverse transformations. In addition, for the convenience of description, the technical solution of the embodiment of the present application can be referred to as a multi-transformation combination technology (i.e., the aforementioned "multi-transformation combination mode").

[0462] For example, Figure 18A is a schematic diagram of a decoding process of a residual block, Figure 18B is a schematic diagram of a decoding process with two sub-residual blocks, and Figure 18C is a schematic diagram of a decoding process with two sub-residual blocks. As shown in Figures 18B and 18C, two sub-residual blocks are schematically provided here, but there can also be multiple. Among them, the technical solution of the embodiment of the present application can be optional at the block (CU) level, that is, some blocks will use the technical solution of the embodiment of the present application, and some blocks will use related technical solutions. In other words, in the entire system, these two processes exist.

[0463] In a possible implementation, each residual block corresponds to its own transform coefficient block and quantization coefficient block. Each quantization coefficient block can be parsed from the bitstream, as shown in FIG18B .

[0464] In another possible implementation, the code stream can be decoded to obtain a quantized coefficient block, which is then divided into two transform coefficient blocks for inverse transformation after inverse quantization, as specifically shown in FIG18C .

[0465] Of course, in an embodiment of the present application, two sub-residual blocks can be combined into a final residual block, and the residual block plus the prediction block equals the reconstructed block, as shown in FIG20A ; alternatively, several sub-residual blocks and the prediction block can be directly added together to obtain a reconstructed block, as shown in FIG20B , and the essence of the two is the same.

[0466] (a) Residual block combination method.

[0467] Assume that this technical solution supports two residual blocks, and the residual value of the first residual block at the coordinate (x, y) is rec0 x,y , the residual value of the second residual block at the coordinate (x, y) is rec1 x,y The final residual block has a residual value of rec at coordinate (x, y) x,y One possible method is to calculate the coordinates (x, y) of the current block, rec x,y =rec0 x,y +rec1 x,y .

[0468] If we consider the issue of image overlap, as shown in Figures 14A, 14B, and 14C, the textures in Figures 14A and 14B combine to form the texture in Figure 14C, but there are some overlapping areas. One way to understand this is that one texture occludes another texture, which is very common in the real world, such as when one object occludes part of another object. One possible approach is to always assume that a certain residual block is on top if there is occlusion, for example, assuming that the first residual block is on top.

[0469] If abs(rec0 x,y )>0,rec x,y =rec0 x,y ;

[0470] Otherwise, rec x,y =rec1 x,y .

[0471] If there are multiple residual blocks, it can be assumed that the texture of the previous residual block is on top of the subsequent residual block.

[0472] Another possible method is to use some syntax elements to indicate which sub-residual block is on top and which is on the bottom. For example, the value of the second syntax identification information can be used to indicate the topmost residual block among the at least two first residual blocks.

[0473] (b) Transformation kernel set and selection of transformation kernel.

[0474] In the embodiment of the present application, a transform kernel is required for the process of decoding each quantized coefficient block to obtain a residual block. A special case is transform skip. For the convenience of description, transform skip can be directly regarded as a special transform. Under this premise, a transform kernel is required for the process of decoding each quantized coefficient block to obtain a residual block. Among them, known transforms include horizontally and vertically separable transforms, horizontally and vertically inseparable transforms, etc. Separable transforms include DCT2, DCT8, DST7, etc., and inseparable transforms include LFNST, NSPT, etc. Of course, there is also the transform skip mentioned above, which can all be used in this technical solution. For example, one quantized coefficient block uses DCT2, and one quantized coefficient block uses NSPT; or both quantized coefficient blocks use LFNST, etc.

[0475] In one possible implementation, to allow the decoder to clearly identify the transform kernel group and transform kernel, a basic method is to use some syntax elements in the bitstream. The decoder parses the corresponding syntax elements to determine the transform kernel used for each quantized coefficient block. For example, if the current block can use two residual blocks, then two MTS indexed syntax elements, mts_idx0 and mts_idx1, are used to represent the transform kernel for the inverse transform of the first quantized coefficient block and the transform kernel for the inverse transform of the second quantized coefficient block, respectively.

[0476] In another possible implementation, the transform core group selected for LFNST and NSPT within a frame is automatically matched according to the intra-frame prediction mode. An intra-frame prediction mode represents a texture feature, for example, an intra-frame angle prediction mode represents the angle of a texture. This derivation can reduce the overhead of the bitstream because it no longer needs to use syntax elements to write which transform core group to select in the bitstream. In this technical solution, the transform core group of LFNST / NSPT can be derived from certain information of the current block. An intra-frame prediction mode can determine a transform core group, or it can be said that a texture feature index can correspond to a transform core group. For the convenience of description later, the texture feature index is used here for description.

[0477] Here, the texture feature index can be derived using the prediction block of the current block, or the reference block, or the reconstructed area around the current block, and the texture feature index is used to determine the transformation kernel group of LFNST / NSPT.

[0478] Using the prediction block to derive texture feature indices is applicable to blocks coded intra, inter, and IBC. If the prediction block has a certain texture, it can be assumed that the residual block contains the same texture. One method is to calculate the gradients of all or part of the pixels in the prediction block. Generally, horizontal and vertical gradients can be calculated, and the Sobel operator can be used to calculate the gradients. For a particular pixel, the texture direction is inferred based on its horizontal and vertical gradients. For example, if the horizontal gradient is non-zero and the vertical gradient is zero, the texture at that point is vertical. Conversely, if the horizontal gradient is zero and the vertical gradient is non-zero, the texture at that point is horizontal. For example, if the horizontal and vertical gradients are equal and non-zero, the texture at that point is 45 degrees. Of course, there are many other cases where both the horizontal and vertical gradients are non-zero, and the texture direction of the pixel can be determined based on their ratio. This allows the gradient to be mapped to the corresponding texture feature index. A texture feature index statistics table is constructed, and the gradient strength of each calculated pixel is accumulated into the corresponding texture feature index entry in the statistics table. After the gradient statistics are completed, the texture feature index statistics table can show the texture features reflected by the prediction block.

[0479] The following describes the gradient calculation using the Sobel operator as an example.

[0480] Horizontal gradient operator:

[0481] Operator for vertical gradient:

[0482] The pixel value of the predicted block is (x, y) x,y , then the horizontal gradient value grad x And the vertical gradient value grad y The calculation is as follows: grad x =P x+1,y-1 +2*P x+1,y +P x+1,y+1 -P x-1,y-1 -2*P x-1,y -P x-1,y+1 grad y =P x-1,y+1 +2*P x,y+1 +P x+1,y+1 -P x-1,y-1 -2*P x,y-1 -P x+1,y-1

[0483] The gradient strength is denoted as amp, an example is amp=abs(grad x )+abs(grad y ).

[0484] According to grad x and grad y The texture feature index can be derived by looking up the table, for example,

[0485] If abs(grad x ) is equal to 0 and abs(grad y ) is not equal to 0, there is horizontal texture, corresponding to intra prediction mode 18 in VVC.

[0486] If abs(grad y ) is equal to 0 and abs(grad x ) is not equal to 0, there is vertical texture, corresponding to intra prediction mode 50 in VVC.

[0487] If abs(grad x ) and abs(grad y ) are not equal to 0:

[0488] If abs(grad x ) is equal to abs(grad y ), and grad x and grad y The symbols are the same, corresponding to intra prediction mode 34 in VVC.

[0489] If abs(grad x ) is equal to 2 times abs(grad y ), and grad x and grad y The symbols are the same, corresponding to intra prediction mode 40 in VVC.

[0490] Other situations can be determined by looking up the table according to the same principle.

[0491] On the other hand, another possibility is to use the reconstructed areas to the left and above the current block, similar to the current DIMD approach. Because the reconstructed areas to the left and above, while not the current block, are adjacent to it, for example, if the texture is connected, they can be used to estimate the texture of the current block to some extent. Another possibility is to use both the predicted block of the current block and the reconstructed areas to the left and above the current block simultaneously, which allows for more pixels to be used for inference of texture feature indices.

[0492] On the other hand, the hardware pipeline length should be considered. For the current block, the texture feature index can only be derived after the prediction block is obtained. Once the texture feature index is determined, the LFNST / NSPT transform kernel can be determined, and the inverse transform and subsequent processes can be performed. If a reference block is used to derive the texture feature index, theoretically, the process of generating the prediction block and the process of deriving the texture feature index can be performed in parallel, which is shorter than the above design pipeline. Therefore, it is also possible to use the reference block to derive the texture feature index, or to derive the texture feature index synchronously during the process of generating the prediction block.

[0493] When using reference blocks, you can directly use integer pixel reference blocks because there will be no obvious difference in texture direction. For example, for a bidirectionally predicted inter-frame block, on the one hand, the texture feature index is derived based on the integer pixel reference block, and on the other hand, the two reference blocks are interpolated and filtered at the sub-pixel level, and then the two sub-pixel reference blocks are weightedly combined. Of course, you can also use sub-pixel reference blocks that have been interpolated (interpolation filter). For example, for a bidirectionally predicted inter-frame block, you need to first perform sub-pixel interpolation filtering on the two reference blocks, and then the two sub-pixel reference blocks are weightedly combined. Then, after obtaining the two interpolation filtered reference blocks, on the one hand, the prediction value can be obtained by weighted averaging the interpolation filtered reference blocks, and on the other hand, the texture feature index can be derived based on the interpolation filtered reference blocks.

[0494] In this way, a statistical table of texture features is obtained using the above method. If the current block does not use the multi-transform combination mode, the texture feature index with the largest statistical amplitude in the texture feature statistical table can be used to derive the LFNST / NSPT transform kernel group. If the current block uses the multi-transform combination mode, for example, two LFNST / NSPT transform kernel groups are required, the first two texture feature indexes with the largest statistical amplitude in the texture feature statistical table can be used to derive the LFNST / NSPT transform kernel group. Among them, the texture feature index with the largest statistical amplitude is used to derive the first LFNST / NSPT transform kernel group, and the texture feature index with the second largest statistical amplitude is used to derive the second LFNST / NSPT transform kernel group.

[0495] In another possible implementation, texture feature indices are derived using a predicted block, a reference block, or a reconstructed image surrounding the current block. In a specific embodiment, one or more texture feature indices with the strongest gradient strength are selected to derive the transform kernel set for LFNST / NSPT. Another approach is to construct a candidate list of texture feature indices and write a syntax element in the bitstream to indicate which candidates are ultimately selected.

[0496] An example of constructing a candidate list is to continue using the above method for calculating gradient strength and select the top texture feature indexes from largest to smallest by statistical magnitude to form the candidate list. N can be 2, 3, 4, etc. Assuming N is 4, the texture feature index with the largest statistical magnitude is the first element in the candidate list, the texture feature index with the second largest statistical magnitude is the second element in the candidate list, and so on.

[0497] Another example of constructing a candidate list is that for GPM, the first element of the candidate list can be set to the texture feature index corresponding to the "division" mode of GPM, the second element to the texture feature index with the largest statistical amplitude, the third element to the texture feature index with the second largest statistical amplitude, and so on.

[0498] In an embodiment of the present application, syntax elements such as lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 may also be used to select texture feature indexes, respectively. For example, if lfnst_nspt_feature_idx0 is 0, the first transform selects the first element of the candidate list; if lfnst_nspt_feature_idx0 is 1, the first transform selects the second element of the candidate list. lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 may be parsed if the current block uses LFNST or NSPT. If the current block does not use LFNST or NSPT, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are not parsed. For example, lfnst_nspt_idx may also be used to indicate whether the current block uses LFNST / NSPT and which transform kernel is selected in the LFNST / NSPT transform kernel group. If lfnst_nspt_idx is 0, it indicates that the current block does not use LFNST / NSPT. In this case, there is no need to parse lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1. Otherwise, if lfnst_nspt_idx is not 0, it indicates that the current block uses LFNST / NSPT. In this case, lfnst_nspt_feature_idx0 and lfnst_nspt_feature_idx1 are parsed.

[0499] In addition, because two transform kernel groups cannot be the same, if N is 4, lfnst_nspt_feature_idx0 has 4 options, and lfnst_nspt_feature_idx1 has only 3 options.

[0500] In simple terms, one possible implementation method is to rely entirely on the bitstream indication, another possible implementation method is to rely entirely on the decoder to derive it itself, and yet another possible implementation method is a compromise between the two, relying on the decoder to derive the candidate list, and then relying on the bitstream indication to determine the transform core group of the current block from a small range of candidates.

[0501] (c) Determine whether the current block uses the multi-transform combination mode.

[0502] In one possible implementation, a syntax element is directly used to determine whether the current block uses the multi-transform combination mode, such as a CU-level or TU-level multi_trans_combine_flag. If the value of multi_trans_combine_flag is 1, it indicates that the current block uses the multi-transform combination mode. Otherwise, that is, the value of multi_trans_combine_flag is 0, it indicates that the current block does not use the multi-transform combination mode.

[0503] In another possible implementation, further, multi_trans_combine_flag may be conditionally parsed, and some known conditions may be used to exclude situations where the multi-transform combination mode will not be used, thereby saving the overhead of multi_trans_combine_flag.

[0504] In the embodiment of the present application, if the texture of the residual block is complex, there will still be a large number of non-zero coefficients after transform quantization. In other words, the transform kernel cannot efficiently process the texture features of the residual block, and there will also be a large number of non-zero coefficients after transform quantization. For example, a horizontally and vertically separable transform is not very efficient in processing diagonal textures. A transform kernel optimized for a single texture feature may not be very efficient in processing the residual of a composite texture feature, but splitting the residual of the composite texture feature into a single texture feature and using multiple transform kernels optimized for single texture features may be more efficient. This is also the original intention of the multi-transform combination mode. An intuitive reflection of so-called efficiency is that there are few non-zero coefficients after transform quantization, or the sum of the values ​​of the non-zero coefficients after transform quantization is small. Therefore, some situations where the multi-transform combination mode is not used can be excluded based on the number of non-zero coefficients, the last non-zero position, or the value of the non-zero coefficient.

[0505] In the LFNST / NSPT transform kernel design, the number of inputs and outputs can vary. Taking NSPT as an example, the transform kernel information for the NSPT decoder in ECM10 is shown in Table 5. For a 4x16 block, the NSPT has 24 inputs and 64 outputs. The output number is self-explanatory, as a 4x16 block has 64 pixels, so the NSPT inverse transform outputs 64 pixel values. The 24 inputs mean that if the NSPT is used for the inverse transform of the current block, the number of coefficients between the last non-zero coefficient and the first coefficient (at the upper left corner (0, 0)) in scan order will not exceed 24. In other words, if the number exceeds 24, then the NSPT is definitely not used for the current block. LFNST also has a similar design, that is, for a given block size, if the position of the last non-zero coefficient is less than (or equal to) a threshold, which is determined by LFNST / NSPT, then it may use LFNST / NSPT. Then it is necessary to parse the syntax elements of LFNST / NSPT to determine whether the current block uses LFNST / NSPT. Otherwise, it is inferred that the current block does not use LFNST / NSPT and there is no need to parse the syntax elements of LFNST / NSPT.

[0506] Similarly, a threshold (the aforementioned first threshold) can be set for multi-transform combinations. If the number of coefficients between the last non-zero coefficient and the first coefficient (at the top-left corner (0,0)) in scan order does not exceed the first threshold, then the current block may use multi-transform combinations, and the multi_trans_combine_flag needs to be parsed. Otherwise, the multi_trans_combine_flag does not need to be parsed, and it is inferred that the current block does not use multi-transform combinations. The first threshold can be the same as, larger than, or smaller than the threshold for a single LFNST / NSPT. For example, using two transform kernels, the maximum number of inputs to two transform kernels is greater than that to a single transform kernel. On the other hand, we expect multi-transform combinations to be more efficient than a single transform, as intuitively reflected by the fewer non-zero coefficients.

[0507] (d) Distinguish multiple groups of transform coefficients.

[0508] In related technologies, a TB (transform block) for a component in a TU has only one set of transform coefficients. However, in a multi-transform combination mode, a TB for a component can have multiple sets of transform coefficients, such as two, each supplying a different transform kernel. Therefore, it is necessary to distinguish these multiple sets of transform coefficients from the bitstream.

[0509] In one possible implementation, if the syntax element multi_trans_combine_flag is used directly to determine whether the current block uses the multi-transform combining mode, then the multi_trans_combine_flag can be parsed before the coefficients are parsed. Different strategies can be used to parse the transform coefficients based on the multi_trans_combine_flag. For example, if the value of multi_trans_combine_flag is 1, two sets of transform coefficients can be parsed separately, each using the same parsing method as in the related art. This method is relatively straightforward. Since two transform kernels are known, two sets of transform coefficients can be parsed.

[0510] In another possible implementation, the transform coefficients may be parsed first, and then the decision on whether to use the multi-transform combination mode may be made. This means that when parsing the transform coefficients, it is not known whether one or two sets of transform coefficients are actually being parsed, but formally one set of transform coefficients is present. If it is determined that the multi-transform combination mode is to be used for the current block, the parsed coefficients are split into two sets of transform coefficients.

[0511] One possible approach is to hide the information that distinguishes the two groups of transform coefficients within the coefficients. For example, the last non-zero coefficient is the position of the last non-zero coefficient in the first group of coefficients. That is, the number of coefficients in the first group of transform coefficients, from the first coefficient (the upper left corner (0, 0) position) to the last non-zero coefficient, is calculated in scanning order minus one. This minus one is necessary because each group of coefficients should not be entirely zero, otherwise the transform would be meaningless.

[0512] For example, arranging the coefficients in scan order yields the following coefficients: 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 7. The last non-zero coefficient, 7, is not a true coefficient; rather, it refers to the eight coefficients in positions 0 through 7, which are the first set of transform coefficients: 12, 5, 5, 6, 0, 2, 0, 1. The remaining coefficients are the second set of transform coefficients: 20, 9, 9, 4. These coefficients are placed in the corresponding positions of the two-dimensional coefficient block, leaving the values ​​of all other positions as 0.

[0513] Alternatively, the last non-zero coefficient is the number of coefficients from the first coefficient to the last non-zero coefficient in the second group of coefficients in the scanning order minus 1. The reason for minus one is that each group of coefficients should not be all zero, otherwise the transformation is meaningless.

[0514] For example, arranging the coefficients in scan order yields the following coefficients: 12, 5, 5, 6, 0, 2, 0, 1, 20, 9, 9, 4, 3. The last non-zero coefficient, 3, is not a true coefficient; rather, it refers to the fact that the second set of transform coefficients in the above coefficients has four members: 20, 9, 9, 4. The preceding ones represent the first set of transform coefficients: 12, 5, 5, 6, 0, 2, 0, 1. These coefficients are placed in the corresponding positions of the two-dimensional coefficient block, and the values ​​of all other positions are set to 0.

[0515] In another possible implementation, the first half of the parsed coefficients can be forced to be the first set of transform coefficients, and the second half to be the second set of transform coefficients. For example, if there are N coefficients from the first to the last non-zero coefficient, then N / 2 coefficients are the first set of transform coefficients, and the remaining coefficients are the second set of transform coefficients. This method requires the encoder to meet requirements when encoding the coefficients, such as by forcibly padding with zeros or forcibly quantizing some coefficients to zero to ensure that the dividing point between the two sets of coefficients is exactly at N / 2.

[0516] (2)Encoder.

[0517] For many encoding tools, the decoder decodes the bitstream to determine the mode selected by the encoder and then decodes according to the decoded mode. The encoder, on the other hand, selects the best mode or parameters from multiple possible modes or parameters and transmits them to the bitstream. For example, in the case of a multi-transform combination mode using two LFNST / NSPT transform kernels, a residual block needs to be divided between the two transform kernels for encoding. Whether the residual block of the current block is suitable for the multi-transform combination mode and which residuals each transform kernel should encode can be determined based on the following method.

[0518] In one possible implementation, the encoder performs a preliminary analysis before attempting a multi-transform combination mode to determine whether the residual block of the current block is suitable for multi-transform combination. For example, this involves checking whether the residual block contains two textures and whether the texture in the residual block is consistent with the texture suitable for the transform kernel. This detection method can use a statistical gradient method, similar to the method used to derive texture feature indices. This can eliminate some cases that are not suitable for multi-transform combination and reduce the burden on the encoder.

[0519] In another possible implementation, neural network techniques can be used for pre-analysis to determine whether the residual block of the current block is suitable for multi-transform combination, and to decompose the residual block into two parts, for example, decomposing Figure 14C into Figure 14A and Figure 14B. In this way, the decomposed residuals can be processed by different transform kernels.

[0520] In another possible implementation, rather than directly considering which portion of the residual is assigned to which transform, the values ​​of the two sets of transform coefficients can be adjusted, using the coding cost (rate and distortion) to determine whether the performance is good enough. This is similar to the RDOQ process in current encoders, but now the two sets of transform coefficients are optimized together. First, the entire residual block is transformed and quantized using two transform kernels. In theory, the transform coefficients generated by each of the two transform kernels can reconstruct a complete residual block. Of course, due to quantization, there will be distortion. Then, the two sets of transform coefficients are adjusted, that is, various combinations of the two sets of transform coefficients are tried, such as reducing the values ​​of the coefficients from high to low frequencies or setting them to zero. The cost of encoding these coefficients (the rate portion of the rate-distortion) is calculated. The coefficients are then dequantized and inversely transformed, and the resulting residual block is combined. The distortion cost (the distortion portion of the rate-distortion) is subtracted from the original residual block. The rate and distortion are summed in a given ratio to obtain the coding cost for the current mode. The coefficients are adjusted repeatedly, and the coefficient combination with the lowest coding cost is selected as the multi-transform combination. When adjusting coefficients, the first and second sets of transform coefficients can be adjusted sequentially, optimizing them alternately. Specifically, the first set of transform coefficients is first optimized to a relatively good level using RDOQ. Because a second transform is involved, the adjustment criteria can differ from those used with only one transform. For example, a higher weight can be assigned to the rate so that the first transform uses fewer codewords to produce a decent residual block. Then, the first set of transform coefficients is fixed, the second set is adjusted, and RDOQ is used to optimize the second set.

[0521] In simple terms, the first two implementations can be said to be active judgments and splitting by the encoder. The latter implementation means that the encoder does not need to analyze from the perspective of the image whether the residual texture is suitable for being divided into two transform kernels and how to divide it. Instead, it tries some possible situations and determines whether to use the multi-transform combination mode based on the coding cost. If, after trying, the coding cost of the multi-transform combination mode is less than the coding cost of a single transform in the related technology, then the multi-transform combination mode is used; otherwise, the multi-transform combination mode is not used. It should be noted that the above implementations can be used in combination.

[0522] In the embodiments of the present application, the specific implementation of the aforementioned embodiments is described in detail through the aforementioned embodiments. It can be seen that according to the technical solutions of the aforementioned embodiments, for the multi-transform combination mode, its core is the "parallel" process of multiple inverse transforms, using multiple transform cores to efficiently process a portion of the texture, and then combining multiple residuals. In this way, by using the multi-transform combination mode, multiple transforms can efficiently process a certain texture separately, and the compression efficiency of certain more complex residual blocks can be higher than that of using a single transform core, thereby improving processing efficiency and further enhancing encoding and decoding performance.

[0523] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG25 is a schematic diagram of the composition structure of an encoder provided in an embodiment of the present application. As shown in FIG25 , the encoder 250 may include a first determination unit 2501, a first transformation unit 2502, and an encoding unit 2503, wherein:

[0524] A first determining unit 2501 is configured to determine a residual block of a current block; and when the current block uses a multi-transform combination mode, determine at least two first residual blocks of the current block according to the residual block of the current block;

[0525] A first transform unit 2502 is configured to determine at least two transform kernels of a current block, and transform at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block;

[0526] The encoding unit 2503 is configured to perform encoding processing on at least two groups of first transform coefficients of the current block, and write the obtained encoding bits into the bitstream.

[0527] In some embodiments, the encoding unit 2503 is further configured to perform encoding processing on at least two groups of first transform coefficients of the current block respectively, and write the obtained encoding bits into the bitstream.

[0528] In some embodiments, the first determination unit 2501 is further configured to determine the second transform coefficient of the current block based on at least two groups of first transform coefficients of the current block; the encoding unit 2503 is further configured to encode the second transform coefficients of the current block separately and write the obtained encoded bits into the bit stream.

[0529] In some embodiments, the last non-zero coefficient in the second transform coefficients is used to indicate a partitioning strategy of the at least two groups of first transform coefficients.

[0530] In some embodiments, the first determining unit 2501 is further configured to determine a prediction block of the current block; and determine a residual block of the current block according to the original image of the current block and the prediction block of the current block.

[0531] In some embodiments, the first determination unit 2501 is further configured to pre-analyze the residual block of the current block using a preset method to determine whether the current block uses a multi-transform combination mode, and when the current block uses a multi-transform combination mode, divide the residual block of the current block into at least two first residual blocks.

[0532] In some embodiments, the first determination unit 2501 is further configured to split the residual block of the current block using multiple combination methods to determine multiple combined residual blocks; perform cost calculations on the multiple combined residual blocks respectively to determine the cost results of each of the multiple combination methods; determine the target combination method based on the cost results of each of the multiple combination methods; and split the residual block of the current block using the target combination method to determine at least two first residual blocks of the current block.

[0533] In some embodiments, the first determining unit 2501 is further configured to determine a minimum cost result from the cost results corresponding to the multiple combination modes, and determine the combination mode corresponding to the minimum cost result as the target combination mode.

[0534] In some embodiments, the first determination unit 2501 is further configured to determine a value of second syntax identification information; wherein the second syntax element identification information is used to indicate the topmost residual block among at least two first residual blocks; the encoding unit 2503 is further configured to encode the value of the second syntax identification information and write the obtained coded bits into the bitstream.

[0535] In some embodiments, the first determination unit 2501 is further configured to determine at least two texture feature indexes of the current block; determine at least two transform core groups based on the at least two texture feature indexes; and determine at least two transform cores of the current block based on the at least two transform core groups.

[0536] In some embodiments, the encoding unit 2503 is further configured to perform encoding processing on at least two transform cores of the current block, and write the obtained encoded bits into the bitstream.

[0537] In some embodiments, the first determination unit 2501 is further configured to perform cost value calculation on at least one candidate transformation core based on at least one candidate transformation core in the first transformation core group, and determine the cost result of each of the at least one candidate transformation core; and determine the minimum cost result from the cost results of each of the at least one candidate transformation core, and use the candidate transformation core corresponding to the minimum cost result as the first transformation core of the current block; wherein the first transformation core group is one of the at least two transformation core groups, and the first transformation core is one of the at least two transformation cores.

[0538] In some embodiments, the first determination unit 2501 is further configured to determine at least two transform core indexes of the current block based on at least two transform cores and at least two transform core groups of the current block; wherein the at least two transform core indexes are respectively used to indicate the numbers of the at least two transform cores of the current block in the corresponding transform core groups; the encoding unit 2503 is further configured to encode the at least two transform core indexes and write the obtained encoded bits into the bitstream.

[0539] In some embodiments, the first determining unit 2501 is further configured to determine candidate pixels for deriving at least two texture feature indexes; and determine at least two texture feature indexes of the current block according to the candidate pixels.

[0540] In some embodiments, the first determination unit 2501 is further configured to determine the horizontal gradient value and the vertical gradient value of the candidate pixel; determine the texture feature index and the gradient intensity value corresponding to the candidate pixel based on the horizontal gradient value and the vertical gradient value of the candidate pixel; construct a texture feature statistics table based on the texture feature index and the gradient intensity value corresponding to the candidate pixel; and determine at least two texture feature indexes of the current block based on the texture feature statistics table.

[0541] In some embodiments, the first determining unit 2501 is further configured to determine a prediction block of the current block; and use at least part of the pixels in the prediction block as candidate pixels.

[0542] In some embodiments, the first determining unit 2501 is further configured to determine adjacent pixels of a reconstructed area of ​​the current block; and use the adjacent pixels of the reconstructed area as candidate pixels.

[0543] In some embodiments, the first determining unit 2501 is further configured to use adjacent pixels in the reconstructed area and at least part of the pixels in the prediction block as candidate pixels.

[0544] In some embodiments, the first determining unit 2501 is further configured to determine a reference block of the current block; and use at least part of the pixels in the reference block as candidate pixels.

[0545] In some embodiments, the first determining unit 2501 is further configured to determine that the reference block is an integer-pixel reference block; or, determine that the reference block is a fractional-pixel reference block.

[0546] In some embodiments, the first determination unit 2501 is further configured to determine two reference image blocks when the current block is bidirectionally predicted; perform pixel-by-pixel interpolation filtering on the two reference image blocks to determine two pixel-by-pixel reference image blocks; and perform weighted combination on the two pixel-by-pixel reference image blocks to determine the reference block of the current block.

[0547] In some embodiments, the first determination unit 2501 is further configured to perform angle mapping based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; and perform gradient intensity calculation based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel.

[0548] In some embodiments, the first determining unit 2501 is further configured to determine the texture feature index corresponding to the candidate pixel using a preset lookup table according to the horizontal gradient value and the vertical gradient value of the candidate pixel.

[0549] In some embodiments, the first determining unit 2501 is further configured to perform an addition operation on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient intensity value corresponding to the candidate pixel.

[0550] In some embodiments, the first determination unit 2501 is further configured to determine at least one texture feature index and at least one corresponding gradient intensity value when the number of candidate pixels is at least one; determine at least one reference texture feature index with different characteristics based on the at least one texture feature index, and perform cumulative calculation on the gradient intensity values ​​belonging to the same reference texture feature index based on the at least one gradient intensity value to determine the gradient intensity cumulative value corresponding to the at least one reference texture feature index; and construct a texture feature statistical table based on the at least one reference texture feature index and the gradient intensity cumulative value corresponding to the at least one reference texture feature index.

[0551] In some embodiments, the first determination unit 2501 is further configured to sort the texture feature statistics table from high to low according to the gradient intensity cumulative value, determine at least two gradient intensity cumulative values ​​at the top of the sort; and determine the reference texture feature indexes corresponding to the at least two gradient intensity cumulative values ​​at the top of the sort as at least two texture feature indexes of the current block.

[0552] In some embodiments, the first determining unit 2501 is further configured to determine a reference block of the current block; and determine at least two texture feature indexes of the current block according to the at least two texture feature indexes of the reference block.

[0553] In some embodiments, the first determination unit 2501 is further configured to construct a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; and select at least two candidate texture feature indexes from the N candidate texture feature indexes, and determine the at least two candidate texture feature indexes as at least two texture feature indexes of the current block.

[0554] In some embodiments, the first determination unit 2501 is further configured to determine at least two feature index numbers of the current block based on at least two texture feature indexes of the current block; wherein the at least two feature index numbers are respectively used to indicate the numbers of the at least two texture feature indexes of the current block in the candidate list; the encoding unit 2503 is further configured to encode the at least two feature index numbers of the current block and write the obtained encoding bits into the bitstream.

[0555] In some embodiments, the first determination unit 2501 is further configured to perform cost value calculations on the N candidate texture feature indexes respectively to determine the cost results of each of the N candidate texture feature indexes; and determine at least two target cost values ​​based on the cost results of each of the N candidate texture feature indexes, and determine the candidate texture feature indexes corresponding to the at least two target cost values ​​as at least two texture feature indexes.

[0556] In some embodiments, the first determination unit 2501 is further configured to sort the texture feature statistics table from high to low according to the gradient intensity cumulative value, determine the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; and construct a candidate list based on the N reference texture feature indexes.

[0557] In some embodiments, the first determination unit 2501 is further configured to determine the first texture feature index corresponding to the current block when the geometric division mode is performed, and determine the reference texture feature index corresponding to the top (N-1) gradient intensity accumulation values; and construct a candidate list based on the first texture feature index and the (N-1) reference texture feature indexes.

[0558] In some embodiments, the encoding unit 2503 is further configured to, when the current block uses the first transform mode, perform encoding processing on at least two feature index numbers of the current block and write the obtained encoding bits into the bitstream.

[0559] In some embodiments, the first determination unit 2501 is further configured to determine the transformation mode identification information of the current block; wherein the transformation mode identification information is used to indicate whether the current block uses the first transformation mode; the encoding unit 2503 is further configured to encode the transformation mode identification information of the current block and write the obtained encoded bits into the bitstream.

[0560] In some embodiments, the first determination unit 2501 is further configured to determine the value of the first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block uses a multi-transformation combination mode; the encoding unit 2503 is further configured to encode the value of the first syntax identification information and write the obtained encoded bits into the bitstream.

[0561] In some embodiments, the first determination unit 2501 is further configured to determine the transformation coefficient of the current block; the encoding unit 2503 is further configured to perform encoding processing on the value of the first syntax identification information and write the obtained encoding bits into the bit stream when the non-zero coefficient information in the transformation coefficient meets the preset conditions.

[0562] In some embodiments, the non-zero coefficient information in the transform coefficients satisfies a preset condition, including: the number of coefficients between the first coefficient and the last non-zero coefficient in the transform coefficients does not exceed a first threshold.

[0563] In some embodiments, the first determination unit 2501 is further configured to determine a value of third syntax identification information; wherein the third syntax identification information is used to indicate whether the current sequence allows the use of a multi-transformation combination mode, and the current sequence includes the current block; the encoding unit 2503 is further configured to perform encoding processing on the value of the third syntax identification information and write the obtained encoded bits into the bitstream.

[0564] In some embodiments, the first determination unit 2501 is further configured to determine a value of the third grammar identification information and a value of the fourth grammar identification information; wherein the third grammar identification information is used to indicate whether the current sequence allows the use of the multi-transform combination mode, and the fourth grammar identification information is used to indicate whether the current slice allows the use of the multi-transform combination mode, the current sequence includes the current slice, and the current slice includes the current block; the encoding unit 2503 is further configured to encode the value of the third grammar identification information and the value of the fourth grammar identification information, and write the obtained coded bits into the bitstream.

[0565] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0566] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0567] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 250. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0568] Based on the composition of the above-mentioned encoder 250 and the computer-readable storage medium, Figure 26 is a schematic diagram of the specific hardware structure of an encoder provided by an embodiment of the present application. As shown in Figure 26, the encoder 250 may include: a first communication interface 2601, a first memory 2602 and a first processor 2603; each component is coupled together through a first bus system 2604. It can be understood that the first bus system 2604 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 2604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 2604 in Figure 26. Among them,

[0569] The first communication interface 2601 is used to receive and send signals when sending and receiving information with other external network elements;

[0570] A first memory 2602 is used to store computer programs that can be run on the first processor 2603;

[0571] The first processor 2603 is configured to, when running the computer program, execute:

[0572] Determine a residual block of the current block; when the current block uses a multi-transform combination mode, determine at least two first residual blocks of the current block based on the residual block of the current block; determine at least two transform kernels of the current block, and transform the at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block; encode the at least two groups of first transform coefficients of the current block, and write the obtained coded bits into a bitstream.

[0573] It is understood that the first memory 2602 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 2602 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0574] The first processor 2603 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 2603. The above-mentioned first processor 2603 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 2602 , and the first processor 2603 reads the information in the first memory 2602 and completes the steps of the above method in combination with its hardware.

[0575] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0576] Optionally, as another embodiment, the first processor 2603 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0577] This embodiment provides an encoder, which first needs to determine at least two transform kernels of the current block, and then transform at least two first residual blocks of the current block respectively according to the at least two transform kernels; that is, the current block can use at least two transform kernels to "parallel" process multiple transform / inverse transform processes, use multiple transform kernels to efficiently process a part of the texture respectively, and finally combine the multiple residual blocks, thereby improving the processing efficiency. Moreover, for some more complex residual blocks, the compression efficiency can also be higher than the compression efficiency of using a single transform kernel, thereby improving the encoding and decoding performance.

[0578] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG27 is a schematic diagram of the composition structure of a decoder provided in an embodiment of the present application. As shown in FIG27 , the decoder 270 may include a decoding unit 2701, a second determination unit 2702, and a second transformation unit 2703, wherein:

[0579] The decoding unit 2701 is configured to decode the code stream and determine the value of the first syntax identification information;

[0580] The second determining unit 2702 is configured to determine at least two groups of first transform coefficients of the current block when the first syntax identification information indicates that the current block uses a multi-transform combination mode;

[0581] The second transform unit 2703 is configured to determine at least two transform kernels of the current block, and perform inverse transform on the at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block;

[0582] The second determining unit 2702 is further configured to determine a reconstructed block of the current block according to the at least two first residual blocks.

[0583] In some embodiments, the second determining unit 2702 is further configured to determine a prediction block of the current block; and determine a reconstructed block of the current block according to the prediction block and the at least two first residual blocks.

[0584] In some embodiments, the second determining unit 2702 is further configured to determine a second residual block according to at least two first residual blocks; and determine a reconstructed block of the current block according to the predicted block and the second residual block.

[0585] In some embodiments, the second determining unit 2702 is further configured to perform an addition operation on at least two first residual blocks to determine a second residual block.

[0586] In some embodiments, the second determination unit 2702 is further configured to determine the topmost residual block among the at least two first residual blocks when there is a non-zero overlapping area in the at least two first residual blocks; and determine the residual value of the second residual block in the non-zero overlapping area based on the topmost residual block.

[0587] In some embodiments, the decoding unit 2701 is further configured to decode the code stream and determine the value of the second syntax identification information; the second determination unit 2702 is further configured to determine the topmost residual block among the at least two first residual blocks based on the value of the second syntax identification information.

[0588] In some embodiments, the decoding unit 2701 is further configured to decode the code stream to determine at least two groups of first transform coefficients of the current block.

[0589] In some embodiments, the decoding unit 2701 is further configured to decode the code stream and determine the second transform coefficient of the current block; the second determination unit 2702 is further configured to determine at least two groups of first transform coefficients of the current block according to the second transform coefficient when the first syntax identification information indicates that the current block uses a multi-transform combination mode.

[0590] In some embodiments, the second determining unit 2702 is further configured to evenly divide the second transform coefficients to determine at least two groups of first transform coefficients for the current block.

[0591] In some embodiments, the second determination unit 2702 is further configured to determine the last non-zero coefficient in the second transform coefficients; and divide the second transform coefficients according to the division strategy indicated by the last non-zero coefficient to determine at least two groups of first transform coefficients of the current block.

[0592] In some embodiments, the encoding unit 2701 is further configured to decode the code stream and determine at least two transform kernels of the current block.

[0593] In some embodiments, the second determining unit 2702 is further configured to determine at least two texture feature indexes of the current block; determine at least two transform kernel groups according to the at least two texture feature indexes; and determine at least two transform kernels of the current block according to the at least two transform kernel groups.

[0594] In some embodiments, the encoding unit 2701 is further configured to decode the code stream and determine at least two transform core indexes of the current block; the second determination unit 2702 is further configured to determine at least two transform cores of the current block based on at least two transform core groups and corresponding at least two transform core indexes.

[0595] In some embodiments, the second determining unit 2702 is further configured to determine candidate pixels for deriving at least two texture feature indexes; and determine at least two texture feature indexes of the current block according to the candidate pixels.

[0596] In some embodiments, the second determination unit 2702 is further configured to determine the horizontal gradient value and the vertical gradient value of the candidate pixel; and determine the texture feature index and the gradient intensity value corresponding to the candidate pixel based on the horizontal gradient value and the vertical gradient value of the candidate pixel; construct a texture feature statistics table based on the texture feature index and the gradient intensity value corresponding to the candidate pixel; and determine at least two texture feature indexes of the current block based on the texture feature statistics table.

[0597] In some embodiments, the second determining unit 2702 is further configured to determine a prediction block of the current block; and use at least some pixels in the prediction block as candidate pixels.

[0598] In some embodiments, the second determining unit 2702 is further configured to determine adjacent pixels of a reconstructed area of ​​the current block; and use the adjacent pixels of the reconstructed area as candidate pixels.

[0599] In some embodiments, the second determining unit 2702 is further configured to use adjacent pixels in the reconstructed area and at least part of the pixels in the prediction block as candidate pixels.

[0600] In some embodiments, the second determining unit 2702 is further configured to determine a reference block of the current block; and use at least part of the pixels in the reference block as candidate pixels.

[0601] In some embodiments, the second determining unit 2702 is further configured to determine that the reference block is an integer-pixel reference block; or, determine that the reference block is a fractional-pixel reference block.

[0602] In some embodiments, the second determination unit 2702 is further configured to determine two reference image blocks when the current block is bidirectionally predicted; perform pixel-by-pixel interpolation filtering on the two reference image blocks to determine two pixel-by-pixel reference image blocks; and perform weighted combination on the two pixel-by-pixel reference image blocks to determine the reference block of the current block.

[0603] In some embodiments, the second determination unit 2702 is further configured to perform angle mapping based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; and to perform gradient intensity calculation based on the horizontal gradient value and vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel.

[0604] In some embodiments, the second determining unit 2702 is further configured to determine the texture feature index corresponding to the candidate pixel using a preset lookup table according to the horizontal gradient value and the vertical gradient value of the candidate pixel.

[0605] In some embodiments, the second determining unit 2702 is further configured to perform an addition operation on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient intensity value corresponding to the candidate pixel.

[0606] In some embodiments, the second determination unit 2702 is further configured to determine at least one texture feature index and at least one corresponding gradient intensity value when the number of candidate pixels is at least one; determine at least one reference texture feature index with different characteristics based on the at least one texture feature index, and perform cumulative calculation on the gradient intensity values ​​belonging to the same reference texture feature index based on the at least one gradient intensity value to determine the gradient intensity cumulative value corresponding to the at least one reference texture feature index; and construct a texture feature statistical table based on the at least one reference texture feature index and the gradient intensity cumulative value corresponding to the at least one reference texture feature index.

[0607] In some embodiments, the second determination unit 2702 is further configured to sort the texture feature statistics table from high to low according to the gradient intensity cumulative value, and determine at least two gradient intensity cumulative values ​​with the highest sorting level; and determine the reference texture feature indexes corresponding to the at least two gradient intensity cumulative values ​​with the highest sorting level as the at least two texture feature indexes of the current block.

[0608] In some embodiments, the second determining unit 2702 is further configured to determine a reference block for the current block; and determine at least two texture feature indexes for the current block according to the at least two texture feature indexes of the reference block.

[0609] In some embodiments, the second determination unit 2702 is further configured to construct a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; the encoding unit 2701 is further configured to decode the bitstream and determine at least two feature index numbers of the current block; the second determination unit 2702 is further configured to determine at least two texture feature indexes of the current block based on the candidate list and the at least two feature index numbers.

[0610] In some embodiments, the second determination unit 2702 is further configured to sort the texture feature statistics table from high to low according to the gradient intensity cumulative value, determine the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; and construct a candidate list based on the N reference texture feature indexes.

[0611] In some embodiments, the second determination unit 2702 is further configured to determine a first texture feature index corresponding to the geometric partitioning mode of the current block, and to determine a reference texture feature index corresponding to the top (N-1) gradient intensity accumulation values; and to construct a candidate list based on the first texture feature index and the (N-1) reference texture feature indexes.

[0612] In some embodiments, the encoding unit 2701 is further configured to decode the code stream and determine the transformation mode identification information of the current block; and when the transformation mode identification information indicates that the current block uses the first transformation mode, perform the step of decoding the code stream and determining at least two feature index numbers of the current block.

[0613] In some embodiments, the second determination unit 2702 is further configured to determine the transformation coefficient of the current block; the encoding unit 2701 is further configured to execute the decoding code stream and determine the value of the first syntax identification information when the non-zero coefficient information in the transformation coefficient meets the preset conditions.

[0614] In some embodiments, the non-zero coefficient information in the transform coefficients satisfies a preset condition, including: the number of coefficients between the first coefficient and the last non-zero coefficient in the transform coefficients does not exceed a first threshold.

[0615] In some embodiments, the encoding unit 2701 is further configured to decode the code stream and determine the value of the third syntax identification information; and when the third syntax identification information indicates that the current sequence allows the use of a multi-transformation combination mode, perform the steps of decoding the code stream and determining the value of the first syntax identification information; wherein the current sequence includes the current block.

[0616] In some embodiments, the encoding unit 2701 is further configured to decode the code stream and determine the value of the third syntax identification information; when the third syntax identification information indicates that the current sequence allows the use of the multi-transform combination mode, decode the code stream and determine the value of the fourth syntax identification information; and when the fourth syntax identification information indicates that the current slice allows the use of the multi-transform combination mode, perform the steps of decoding the code stream and determining the value of the first syntax identification information; wherein the current sequence includes the current slice, and the current slice includes the current block.

[0617] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0618] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium for use in decoder 270. The computer-readable storage medium stores a computer program that, when executed by a second processor, implements any of the methods described in the aforementioned embodiments.

[0619] Based on the compo...

Claims

1. A decoding method, applied to a decoder, comprising: Decoding the code stream to determine a value of the first syntax identification information; When the first syntax identification information indicates that the current block uses a multi-transform combination mode, determining at least two groups of first transform coefficients of the current block; Determining at least two transform kernels of the current block, and performing inverse transforms on the at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block; A reconstructed block of the current block is determined according to the at least two first residual blocks.

2. The method according to claim 1, wherein The determining, according to the at least two first residual blocks, a reconstructed block of the current block includes: Determining a prediction block for the current block; A reconstructed block of the current block is determined according to the prediction block and the at least two first residual blocks.

3. The method according to claim 2, wherein: The determining, according to the prediction block and the at least two first residual blocks, a reconstructed block of the current block includes: Determining a second residual block according to the at least two first residual blocks; A reconstructed block of the current block is determined according to the prediction block and the second residual block.

4. The method according to claim 3, wherein: The determining the second residual block according to the at least two first residual blocks includes: An addition operation is performed on the at least two first residual blocks to determine the second residual block.

5. The method according to claim 3, wherein The determining the second residual block according to the at least two first residual blocks includes: When a non-zero overlapping area exists in the at least two first residual blocks, determining an uppermost residual block in the at least two first residual blocks; Determine a residual value of the second residual block in the non-zero overlapping area according to the uppermost residual block.

6. The method according to claim 5, wherein: The determining the uppermost residual block among the at least two first residual blocks includes: Decoding the code stream to determine a value of the second syntax identification information; The uppermost residual block among the at least two first residual blocks is determined according to a value of the second syntax identification information.

7. The method according to claim 1, wherein The determining of at least two groups of first transform coefficients of the current block includes: The code stream is decoded to determine at least two groups of first transform coefficients of the current block.

8. The method according to claim 1, wherein The determining of at least two groups of first transform coefficients of the current block includes: Decoding a code stream to determine a second transform coefficient of the current block; When the first syntax identification information indicates that the current block uses a multi-transform combination mode, at least two groups of first transform coefficients of the current block are determined according to the second transform coefficients.

9. The method according to claim 8, wherein The determining, according to the second transform coefficient, at least two groups of first transform coefficients of the current block includes: The second transform coefficients are evenly divided to determine at least two groups of first transform coefficients of the current block.

10. The method according to claim 8, wherein The determining, according to the second transform coefficient, at least two groups of first transform coefficients of the current block includes: determining a last non-zero coefficient in the second transform coefficients; The second transform coefficients are divided according to the division strategy indicated by the last non-zero coefficient to determine at least two groups of first transform coefficients of the current block.

11. The method according to any one of claims 1 to 10, wherein The determining of at least two transform kernels of the current block includes: The code stream is decoded to determine at least two transform kernels of the current block.

12. The method according to any one of claims 1 to 10, wherein The determining of at least two transform kernels of the current block includes: Determining at least two texture feature indexes of the current block; determining the at least two transformation kernel groups according to the at least two texture feature indexes; At least two transform cores of the current block are determined according to the at least two transform core groups.

13. The method according to claim 12, wherein: The determining, according to the at least two transform core groups, at least two transform cores of the current block includes: Decoding a code stream, and determining at least two transform kernel indices of the current block; At least two transform cores of the current block are determined according to the at least two transform core groups and the corresponding at least two transform core indexes.

14. The method according to claim 12, wherein: The determining of at least two texture feature indexes of the current block includes: determining candidate pixels for deriving the at least two texture feature indexes; At least two texture feature indexes of the current block are determined according to the candidate pixels.

15. The method according to claim 14, wherein The determining, based on the candidate pixels, at least two texture feature indexes of the current block includes: Determining a horizontal gradient value and a vertical gradient value of the candidate pixel; Determining a texture feature index and a gradient intensity value corresponding to the candidate pixel according to the horizontal gradient value and the vertical gradient value of the candidate pixel; Constructing a texture feature statistics table according to the texture feature index and gradient intensity value corresponding to the candidate pixel; At least two texture feature indexes of the current block are determined according to the texture feature statistics table.

16. The method according to claim 14, wherein The method further comprises: Determining a prediction block for the current block; At least part of the pixels in the prediction block are used as the candidate pixels.

17. The method according to claim 16, wherein: The method further comprises: Determining adjacent pixels of a reconstructed area of ​​the current block; Adjacent pixels of the reconstructed area are used as the candidate pixels.

18. The method according to claim 17, wherein: The method further comprises: Adjacent pixels of the reconstructed area and at least part of pixels in the prediction block are used as the candidate pixels.

19. The method according to claim 14, wherein The method further comprises: Determining a reference block for the current block; At least part of the pixels in the reference block are used as the candidate pixels.

20. The method according to claim 19, wherein The determining a reference block of the current block includes: Determine that the reference block is an integer-pixel reference block; or, The reference block is determined to be a sub-pixel reference block.

21. The method according to claim 20, wherein When the reference block is a sub-pixel reference block, determining the reference block of the current block includes: Determining two reference image blocks when performing bidirectional prediction on the current block; Performing pixel-by-pixel interpolation filtering on the two reference image blocks to determine two pixel-by-pixel reference image blocks; A weighted combination is performed on the reference image blocks of the two sub-pixels to determine a reference block of the current block.

22. The method according to claim 15, wherein The determining, based on the horizontal gradient value and the vertical gradient value of the candidate pixel, a texture feature index and a gradient intensity value corresponding to the candidate pixel includes: Performing angle mapping according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; A gradient strength calculation is performed based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient strength value corresponding to the candidate pixel.

23. The method according to claim 22, wherein The performing angle mapping according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel includes: According to the horizontal gradient value and the vertical gradient value of the candidate pixel, a texture feature index corresponding to the candidate pixel is determined using a preset lookup table.

24. The method according to claim 22, wherein The performing gradient intensity calculation according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel includes: An addition operation is performed on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient intensity value corresponding to the candidate pixel.

25. The method according to claim 15, wherein The step of constructing a texture feature statistics table according to the texture feature index and the gradient intensity value corresponding to the candidate pixel includes: When the number of the candidate pixels is at least one, determining at least one texture feature index and at least one corresponding gradient intensity value; Determine at least one reference texture feature index having mutually different characteristics according to the at least one texture feature index, and less one gradient intensity value, accumulating the gradient intensity values ​​belonging to the same reference texture feature index to determine the gradient intensity accumulated value corresponding to the at least one reference texture feature index; The texture feature statistics table is constructed according to the at least one reference texture feature index and the gradient intensity accumulation value corresponding to the at least one reference texture feature index.

26. The method according to claim 25, wherein The determining, according to the texture feature statistics table, at least two texture feature indexes of the current block includes: Sorting the texture feature statistics table from high to low according to the gradient intensity cumulative values, and determining at least two gradient intensity cumulative values ​​that are ranked top; The reference texture feature indexes corresponding to the at least two gradient intensity accumulated values ​​that are ranked first are determined as the at least two texture feature indexes of the current block.

27. The method according to claim 12, wherein The determining of at least two texture feature indexes of the current block includes: Determining a reference block for the current block; At least two texture feature indexes of the current block are determined according to the at least two texture feature indexes of the reference block.

28. The method according to claim 12, wherein The determining of at least two texture feature indexes of the current block includes: Constructing a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; Decoding a code stream to determine at least two feature index numbers of the current block; At least two texture feature indexes of the current block are determined according to the candidate list and the at least two feature index serial numbers.

29. The method according to claim 28, wherein The candidate list is constructed, including: Sorting the texture feature statistics table from high to low according to the gradient intensity cumulative values, and determining the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; The candidate list is constructed according to the N reference texture feature indexes.

30. The method according to claim 29, wherein The candidate list is constructed, including: Determining a first texture feature index corresponding to a geometric partitioning mode of the current block, and determining a reference texture feature index corresponding to the top (N-1) gradient intensity accumulated values; The candidate list is constructed according to the first texture feature index and the (N-1) reference texture feature indexes.

31. The method of claim 28, wherein The method further comprises: Decoding a code stream to determine transform mode identification information of the current block; When the transform mode identification information indicates that the current block uses the first transform mode, a step of decoding a code stream and determining at least two feature index sequence numbers of the current block is performed.

32. The method according to any one of claims 1 to 31, wherein The method further comprises: determining a transform coefficient of the current block; When the non-zero coefficient information in the transform coefficient meets a preset condition, a step of decoding the code stream and determining a value of the first syntax identification information is performed.

33. The method according to claim 32, wherein The non-zero coefficient information in the transform coefficients satisfies a preset condition, including: The number of coefficients between a first coefficient and a last non-zero coefficient in the transform coefficients does not exceed a first threshold.

34. The method according to any one of claims 1 to 31, wherein The method further comprises: Decoding the code stream to determine a value of the third syntax identification information; When the third syntax identification information indicates that the current sequence allows the use of a multi-transformation combination mode, a step of decoding a code stream and determining a value of the first syntax identification information is performed; wherein the current sequence includes the current block.

35. The method according to any one of claims 1 to 31, wherein The method further comprises: Decoding the code stream to determine a value of the third syntax identification information; When the third syntax identification information indicates that the current sequence allows the use of the multi-transformation combination mode, decoding the code stream and determining the value of the fourth syntax identification information; When the fourth syntax identification information indicates that the current slice allows the use of a multi-transform combination mode, a step of decoding the code stream and determining the value of the first syntax identification information is performed; wherein the current sequence includes the current slice, and the current slice includes the current block.

36. A coding method, applied to an encoder, comprising: Determine the residual block of the current block; When the current block uses a multi-transform combination mode, determining at least two first residual blocks of the current block according to the residual block of the current block; Determining at least two transform kernels of the current block, and transforming the at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block; Encoding is performed on at least two groups of first transform coefficients of the current block, and the obtained coded bits are written into a bitstream.

37. The method according to claim 36, wherein The encoding process is performed on at least two groups of first transform coefficients of the current block, and the obtained encoding bits are written into a bitstream, comprising: Encoding is performed on at least two groups of first transform coefficients of the current block respectively, and the obtained coded bits are written into a bitstream.

38. The method of claim 36, wherein: The encoding process is performed on at least two groups of first transform coefficients of the current block, and the obtained encoding bits are written into a bitstream, comprising: determining a second transform coefficient of the current block according to at least two groups of first transform coefficients of the current block; The second transform coefficients of the current block are respectively coded, and the obtained coded bits are written into a bitstream.

39. The method according to claim 38, wherein The last non-zero coefficient in the second transform coefficients is used to indicate a partitioning strategy of the at least two groups of first transform coefficients.

40. The method of claim 36, wherein The determining of the residual block of the current block includes: Determining a prediction block for the current block; A residual block of the current block is determined according to the original image of the current block and the prediction block of the current block.

41. The method of claim 36, wherein: The determining, according to the residual block of the current block, at least two first residual blocks of the current block includes: Pre-analyzing the residual block of the current block in a preset manner to determine whether the current block uses a multi-transform combination mode, and dividing the residual block of the current block into the at least two first residual blocks when the current block uses the multi-transform combination mode.

42. The method of claim 36, wherein: The determining, according to the residual block of the current block, at least two first residual blocks of the current block includes: Splitting the residual block of the current block using multiple combination methods to determine multiple combined residual blocks; Performing cost calculations on the multiple combined residual blocks respectively to determine cost results of the multiple combination methods; Determining a target combination method based on the cost results of each of the multiple combination methods; The residual block of the current block is split using the target combination method to determine at least two first residual blocks of the current block.

43. The method according to claim 42, wherein The determining of a target combination method according to the cost results corresponding to each of the plurality of combination methods includes: A minimum cost result is determined from the cost results corresponding to the multiple combination methods, and the combination method corresponding to the minimum cost result is determined as the target combination method.

44. The method of claim 36, wherein The method further comprises: Determining a value of second syntax element identification information; wherein the second syntax element identification information is used to indicate the uppermost residual block of the at least two first residual blocks; The value of the second syntax identification information is coded, and the obtained coded bits are written into a bitstream.

45. The method according to any one of claims 36 to 44, wherein The determining of at least two transform kernels of the current block includes: Determining at least two texture feature indexes of the current block; determining the at least two transformation kernel groups according to the at least two texture feature indexes; At least two transform cores of the current block are determined according to the at least two transform core groups.

46. ​​The method of claim 45, wherein The method further comprises: Perform encoding processing on at least two transform cores of the current block, and write the obtained encoding bits into a bitstream.

47. The method of claim 45, wherein The determining, according to the at least two transform core groups, at least two transform cores of the current block includes: Based on at least one candidate transformation core in the first transformation core group, respectively calculating a cost value for the at least one candidate transformation core to determine a cost result for each of the at least one candidate transformation core; Determining a minimum cost result from the respective cost results of the at least one candidate transformation core, and using the candidate transformation core corresponding to the minimum cost result as the first transformation core of the current block; The first transformation core group is one of the at least two transformation core groups, and the first transformation core is one of the at least two transformation cores.

48. The method of claim 47, wherein The method further comprises: Determining at least two transform core indexes of the current block according to the at least two transform cores of the current block and the at least two transform core groups; wherein the at least two transform core indexes are respectively used to indicate numbers of the at least two transform cores of the current block in the corresponding transform core groups; The at least two transform core indexes are coded, and the obtained coded bits are written into a bitstream.

49. The method of claim 45, wherein The determining of at least two texture feature indexes of the current block includes: determining candidate pixels for deriving the at least two texture feature indexes; At least two texture feature indexes of the current block are determined according to the candidate pixels.

50. The method of claim 49, wherein The determining, based on the candidate pixels, at least two texture feature indexes of the current block includes: Determining a horizontal gradient value and a vertical gradient value of the candidate pixel; Determining a texture feature index and a gradient intensity value corresponding to the candidate pixel according to the horizontal gradient value and the vertical gradient value of the candidate pixel; Constructing a texture feature statistics table according to the texture feature index and gradient intensity value corresponding to the candidate pixel; At least two texture feature indexes of the current block are determined according to the texture feature statistics table.

51. The method of claim 49, wherein The method further comprises: Determining a prediction block for the current block; At least part of the pixels in the prediction block are used as the candidate pixels.

52. The method of claim 51, wherein The method further comprises: Determining adjacent pixels of a reconstructed area of ​​the current block; Adjacent pixels of the reconstructed area are used as the candidate pixels.

53. The method of claim 52, wherein: The method further comprises: Adjacent pixels of the reconstructed area and at least part of pixels in the prediction block are used as the candidate pixels.

54. The method of claim 49, wherein The method further comprises: Determining a reference block for the current block; At least part of the pixels in the reference block are used as the candidate pixels.

55. The method of claim 54, wherein The determining a reference block of the current block includes: Determine that the reference block is an integer-pixel reference block; or, The reference block is determined to be a sub-pixel reference block.

56. The method of claim 55, wherein: When the reference block is a sub-pixel reference block, determining the reference block of the current block includes: Determining two reference image blocks when performing bidirectional prediction on the current block; Performing pixel-by-pixel interpolation filtering on the two reference image blocks to determine two pixel-by-pixel reference image blocks; A weighted combination is performed on the reference image blocks of the two sub-pixels to determine a reference block of the current block.

57. The method of claim 50, wherein: The determining, based on the horizontal gradient value and the vertical gradient value of the candidate pixel, a texture feature index and a gradient intensity value corresponding to the candidate pixel includes: Performing angle mapping according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel; A gradient strength calculation is performed based on the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient strength value corresponding to the candidate pixel.

58. The method of claim 57, wherein The performing angle mapping according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the texture feature index corresponding to the candidate pixel includes: According to the horizontal gradient value and the vertical gradient value of the candidate pixel, a texture feature index corresponding to the candidate pixel is determined using a preset lookup table.

59. The method of claim 57, wherein The performing gradient intensity calculation according to the horizontal gradient value and the vertical gradient value of the candidate pixel to determine the gradient intensity value corresponding to the candidate pixel includes: An addition operation is performed on the absolute value of the horizontal gradient value and the absolute value of the vertical gradient value to determine the gradient intensity value corresponding to the candidate pixel.

60. The method of claim 50, wherein: The step of constructing a texture feature statistics table according to the texture feature index and the gradient intensity value corresponding to the candidate pixel includes: When the number of the candidate pixels is at least one, determining at least one texture feature index and at least one corresponding gradient intensity value; Determining at least one reference texture feature index having mutually different characteristics based on the at least one texture feature index, and accumulating gradient intensity values ​​belonging to the same reference texture feature index based on the at least one gradient intensity value to determine a gradient intensity accumulation value corresponding to the at least one reference texture feature index; The texture feature statistics table is constructed according to the at least one reference texture feature index and the gradient intensity accumulation value corresponding to the at least one reference texture feature index.

61. The method of claim 60, wherein: The determining, according to the texture feature statistics table, at least two texture feature indexes of the current block includes: The texture feature statistics table is sorted from high to low according to the cumulative value of gradient strength, and at least two gradient strengths that are ranked first are determined. Accumulated value; The reference texture feature indexes corresponding to the at least two gradient intensity accumulated values ​​that are ranked first are determined as the at least two texture feature indexes of the current block.

62. The method of claim 45, wherein The determining of at least two texture feature indexes of the current block includes: Determining a reference block for the current block; At least two texture feature indexes of the current block are determined according to the at least two texture feature indexes of the reference block.

63. The method of claim 45, wherein The determining of at least two texture feature indexes of the current block includes: Constructing a candidate list, wherein the candidate list includes N candidate texture feature indexes, where N is an integer greater than 1; At least two candidate texture feature indexes are selected from the N candidate texture feature indexes, and the at least two candidate texture feature indexes are determined as the at least two texture feature indexes of the current block.

64. The method of claim 63, wherein The method further comprises: Determining at least two feature index serial numbers of the current block according to the at least two texture feature indexes of the current block; wherein the at least two feature index serial numbers are respectively used to indicate the numbers of the at least two texture feature indexes of the current block in the candidate list; Encoding is performed on at least two feature index numbers of the current block, and the obtained coded bits are written into a bitstream.

65. The method of claim 63, wherein The selecting at least two candidate texture feature indexes from the N candidate texture feature indexes comprises: Calculating cost values ​​for each of the N candidate texture feature indexes to determine a cost result for each of the N candidate texture feature indexes; At least two target cost values ​​are determined based on the respective cost results of the N candidate texture feature indexes, and the candidate texture feature indexes corresponding to the at least two target cost values ​​are determined as the at least two texture feature indexes.

66. The method of claim 63, wherein The candidate list is constructed, including: Sorting the texture feature statistics table from high to low according to the gradient intensity cumulative values, and determining the reference texture feature indexes corresponding to the top N gradient intensity cumulative values; The candidate list is constructed according to the N reference texture feature indexes.

67. The method of claim 66, wherein The candidate list is constructed, including: Determining a first texture feature index corresponding to a geometric partitioning mode of the current block, and determining a reference texture feature index corresponding to the top (N-1) gradient intensity accumulated values; The candidate list is constructed according to the first texture feature index and the (N-1) reference texture feature indexes.

68. The method of claim 64, wherein The method further comprises: When the current block uses the first transform mode, the steps of encoding at least two feature index numbers of the current block and writing the obtained encoding bits into a bitstream are performed.

69. The method of claim 68, wherein The method further comprises: Determining transform mode identification information of the current block; wherein the transform mode identification information is used to indicate whether the current block uses a first transform mode; The transformation mode identification information of the current block is coded, and the obtained coded bits are written into a bitstream.

70. The method according to any one of claims 36 to 69, wherein The method further comprises: Determining a value of first syntax identification information; wherein the first syntax identification information is used to indicate whether the current block uses a multi-transform combination mode; The value of the first syntax identification information is encoded, and the obtained encoded bits are written into a bitstream.

71. The method of claim 70, wherein The method further comprises: determining a transform coefficient of the current block; When the non-zero coefficient information in the transform coefficient meets a preset condition, the step of encoding the value of the first syntax identification information and writing the obtained encoding bits into the bitstream is performed.

72. The method of claim 71, wherein The non-zero coefficient information in the transform coefficients satisfies a preset condition, including: The number of coefficients between a first coefficient and a last non-zero coefficient in the transform coefficients does not exceed a first threshold.

73. The method according to any one of claims 36 to 69, wherein The method further comprises: Determining a value of third syntax identification information; wherein the third syntax identification information is used to indicate whether a current sequence allows the use of a multi-transform combination mode, the current sequence including the current block; The value of the third syntax identification information is coded, and the obtained coded bits are written into a bitstream.

74. The method according to any one of claims 36 to 69, wherein The method further comprises: Determine the value of the third syntax identification information and the value of the fourth syntax identification information; wherein the third syntax identification information is used to indicate whether the current sequence allows the use of the multi-transform combination mode, and the fourth syntax identification information is used to indicate whether the current slice allows the use of the multi-transform combination mode. In a combined mode, the current sequence includes a current slice, and the current slice includes the current block; The value of the third syntax identification information and the value of the fourth syntax identification information are coded, and the obtained coded bits are written into a bitstream.

75. A code stream, wherein The code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: at least two groups of first transform coefficients of a current block, at least two transform kernel indices of the current block, at least two feature index sequences of the current block, transform mode identification information of the current block, a value of the first syntax identification information, a value of the second syntax identification information, and a value of the third syntax identification information; The value of the first syntax identification information is used to indicate whether the current sequence allows the use of a multi-transform combination mode, the second syntax element identification information is used to indicate the uppermost residual block among the at least two first residual blocks, and the value of the third syntax identification information is used to indicate whether the current slice allows the use of a multi-transform combination mode.

76. An encoder, comprising a first determining unit, a first transforming unit, and an encoding unit, wherein: The first determining unit is configured to determine a residual block of the current block; and determining at least two first residual blocks of the current block according to the residual block of the current block when the current block uses a multi-transform combination mode; The first transform unit is configured to determine at least two transform kernels of the current block, and transform the at least two first residual blocks respectively according to the at least two transform kernels to determine at least two groups of first transform coefficients of the current block; The encoding unit is configured to perform encoding processing on at least two groups of first transform coefficients of the current block, and write the obtained encoding bits into a bitstream.

77. An encoder comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 36 to 74 when running the computer program.

78. A decoder comprising a decoding unit, a second determining unit, and a second transforming unit, wherein: The decoding unit is configured to decode the code stream and determine the value of the first syntax identification information; The second determining unit is configured to determine at least two groups of first transform coefficients of the current block when the first syntax identification information indicates that the current block uses a multi-transform combination mode; The second transform unit is configured to determine at least two transform kernels of the current block, and perform inverse transform on the at least two groups of first transform coefficients according to the at least two transform kernels to determine at least two first residual blocks of the current block; The second determining unit is further configured to determine a reconstructed block of the current block according to the at least two first residual blocks.

79. A decoder comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 35 when running the computer program.

80. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, which, when executed by at least one processor, implements the method according to any one of claims 1 to 35 or the method according to any one of claims 36 to 74.

Images