Transform methods, apparatus, coder, decoder and readable storage medium

Abstract

The present application belongs to the technical field of video processing. Disclosed are transform methods, an apparatus, a coder, a decoder, and a readable storage medium, which can improve the accuracy of determining transform kernels for chrominance blocks, thereby enhancing the transform effect of chrominance blocks. A transform method in the embodiments of the present application comprises: a coder performing intra prediction on a chrominance block to obtain a prediction block of the chrominance block; determining a transform kernel of the chrominance block on the basis of the prediction block; and transforming a residual block of the chrominance block or a transform block of the chrominance block on the basis of the transform kernel. The transform methods in the embodiments of the present application may be applied to video coding and decoding.

Description

Transformation method, apparatus, encoder, decoder and readable storage medium Technical Field

[0001] This application belongs to the field of video processing technology, specifically relating to a transformation method, apparatus, encoder, decoder, and readable storage medium. Background Technology

[0002] Currently, video encoding and decoding technologies can encode and decode video frames through multiple modules, including block partitioning, intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, loop filtering, and post-processing filtering. Among these, the transform module can select the transform kernel to use for transforming or inverse transforming based on information from the intra-frame prediction mode.

[0003] Typically, different intra-frame prediction modes use different transform kernels. Taking cross-component prediction mode as an example, when transforming the prediction residual of a chroma block, a conventional prediction mode can usually be derived based on the prediction mode of the corresponding luma block. Then, the corresponding transform set is determined based on the conventional prediction mode and the transform set mapping table. The transform kernel for the chroma block is then determined from the transform set, and the prediction residual of the chroma block is transformed based on the transform kernel, or the inverse transform of the dequantized coefficients is performed.

[0004] According to the above method, the transformation kernel of the chroma block is usually determined based on the traditional prediction mode derived from the prediction mode of the luminance block corresponding to the chroma block. Since the luminance block and the chroma block represent two different dimensions of luminance and chrominance respectively, the determined transformation kernel of the chroma block may not be accurate enough, resulting in poor transformation effect of the chroma block.

[0005] Summary of the Invention

[0006] This application provides a transformation method, apparatus, encoder, decoder, and readable storage medium, which can improve the accuracy of determining the transformation kernel of chroma blocks, thereby improving the transformation effect of chroma blocks.

[0007] In a first aspect, embodiments of this application provide a transformation method applied to an encoder. The method may include: performing intra-frame prediction on a chroma block to obtain a prediction block; determining a transform kernel for the chroma block based on the prediction block; and transforming a residual block or a transform block of the chroma block based on the transform kernel.

[0008] In one possible implementation, the above-mentioned determination of the transform kernel of the chroma block based on the prediction block may include: determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block; and determining the transform kernel of the chroma block based on the first intra-frame prediction mode.

[0009] In one possible implementation, determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block may include: determining the gradient histogram (HoG) of the prediction block based on the gradient information of the prediction block, wherein the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude; and determining the first intra-frame prediction mode based on the HoG of the prediction block.

[0010] In one possible implementation, determining the first intra-frame prediction mode based on the HoG of the prediction block may include: determining the intra-frame prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-frame prediction mode; wherein the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among at least one gradient magnitude.

[0011] In one possible implementation, determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode may include: if there is a non-zero gradient magnitude in at least one gradient magnitude, determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0012] In one possible implementation, the above transformation method may further include: determining a preset intra-frame prediction mode as a first intra-frame prediction mode when at least one gradient magnitude is 0.

[0013] In one possible implementation, the chroma block includes a first chroma block and a second chroma block, and the prediction block includes a first prediction block of the first chroma block and a second prediction block of the second chroma block. Determining the HoG of the prediction block based on the gradient information of the prediction block may include: determining a first HoG of the first prediction block based on the gradient information of the first prediction block; determining a second HoG of the second prediction block based on the gradient information of the second prediction block; and summing the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0014] In one possible implementation, the process of determining the transform kernel of the chroma block based on the first intra-frame prediction mode may include: determining a first transform set based on the first intra-frame prediction mode and a first mapping relationship, wherein the first mapping relationship is a mapping relationship between the first intra-frame prediction mode and the first transform set; and determining the transform kernel of the chroma block from the first transform set; wherein the first mapping relationship is one of a plurality of mapping relationships, and each of the plurality of mapping relationships is a mapping relationship between an intra-frame prediction mode and a transform set.

[0015] In one possible implementation, the above transformation method may further include: writing first index information into the bitstream, the first index information being used to indicate the transformation kernel of the chroma block.

[0016] In one possible implementation, the above transformation method may further include: using a transformation kernel to perform an inverse transformation on the first coefficient; the first coefficient is the transformation coefficient of the chroma block or the inverse quantization coefficient of the chroma block.

[0017] In one possible implementation, the above-mentioned intra-frame prediction of the chroma block to obtain the prediction block of the chroma block may include: performing intra-frame prediction of the chroma block based on the reconstructed value of the luma block corresponding to the chroma block to obtain the prediction block of the chroma block.

[0018] The transformation method provided in this application allows the encoder to determine the transform kernel of a chroma block by combining it with the prediction block of that chroma block. In other words, the encoder can combine the information of the chroma block to determine the transform kernel. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0019] Secondly, embodiments of this application provide a transformation method applied to a decoder. The method may include: performing intra-frame prediction on a chroma block to obtain a prediction block of the chroma block; determining a transform kernel of the chroma block based on the prediction block; and performing an inverse transform on a first coefficient based on the transform kernel. The first coefficient may be a transform coefficient of the chroma block; or, the first coefficient may be obtained by inverse quantization of the quantization coefficient of the chroma block.

[0020] In one possible implementation, the above-mentioned transformation kernel for determining the chroma block based on the prediction block may include: determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block; and determining the transformation kernel for the chroma block based on the first intra-frame prediction mode.

[0021] In one possible implementation, determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block may include: determining the HoG of the prediction block based on the gradient information of the prediction block, wherein the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude; and determining the first intra-frame prediction mode based on the HoG of the prediction block.

[0022] In one possible implementation, determining the first intra-frame prediction mode based on the HoG of the prediction block may include: determining the intra-frame prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-frame prediction mode; wherein the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among at least one gradient magnitude.

[0023] In one possible implementation, determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode may include: if there is a non-zero gradient magnitude in at least one gradient magnitude, determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0024] In one possible implementation, the above transformation method may further include: determining a preset intra-frame prediction mode as a first intra-frame prediction mode when at least one gradient magnitude is 0.

[0025] In one possible implementation, the chroma block includes a first chroma block and a second chroma block, and the prediction block includes a first prediction block of the first chroma block and a second prediction block of the second chroma block; the above-mentioned determination of the HoG of the prediction block based on the gradient information of the prediction block may include: determining a first HoG of the first prediction block based on the gradient information of the first prediction block; determining a second HoG of the second prediction block based on the gradient information of the second prediction block; and accumulating the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0026] In one possible implementation, the above-mentioned determination of the transform kernel of the chroma block based on the first intra-frame prediction mode may include: determining a first transform set based on the first intra-frame prediction mode and a first mapping relationship, wherein the first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set; and determining the transform kernel indicated by the first index information in the first transform set as the transform kernel of the chroma block based on the first index information in the bitstream.

[0027] In one possible implementation, the above-mentioned intra-frame prediction of the chroma block to obtain the prediction block of the chroma block may include: performing intra-frame prediction of the chroma block based on the reconstructed value of the luma block corresponding to the chroma block to obtain the prediction block.

[0028] The transformation method provided in this application, during the process of the decoder determining the transform kernel of a chroma block, can combine the prediction block of the chroma block for determination. That is, the encoder can combine the information of the chroma block to determine the transform kernel. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0029] Thirdly, embodiments of this application provide a transformation apparatus, which may include: a prediction module and a transformation module; the prediction module is configured to perform intra-frame prediction on a chroma block to obtain a prediction block of the chroma block; and to determine a transformation kernel of the chroma block based on the prediction block; the transformation module is configured to transform the residual block of the chroma block or the transformation block of the chroma block based on the transformation kernel obtained by the prediction module.

[0030] Fourthly, embodiments of this application provide a transformation apparatus, which includes: a prediction module and a transformation module; the prediction module is configured to perform intra-frame prediction on a chroma block to obtain a prediction block of the chroma block; and to determine a transformation kernel of the chroma block based on the prediction block; the transformation module is configured to perform inverse transformation on a first coefficient based on the transformation kernel obtained by the prediction module; the first coefficient is a transformation coefficient of the chroma block; or, the first coefficient is obtained by inverse quantization of the quantization coefficient of the chroma block.

[0031] Fifthly, embodiments of this application provide an encoder, the decoder including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the transformation method as described in the first aspect and its various possible implementations above.

[0032] In a sixth aspect, embodiments of this application provide a decoder that includes a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the transformation method as described in the second aspect above and its various possible implementations.

[0033] In a seventh aspect, embodiments of this application provide a chip including a processor and a communication interface coupled to the processor. The processor is used to run programs or instructions to implement the steps of the transformation method as described in the first aspect and its various possible implementations, or to implement the steps of the transformation method as described in the second aspect and its various possible implementations.

[0034] Eighthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the transformation method as described in the first aspect and its various possible implementations, or implement the steps of the transformation method as described in the second aspect and its various possible implementations.

[0035] Ninthly, embodiments of this application provide a readable storage medium storing a bitstream, the bitstream being a bitstream encoded based on the transformation method described in the first aspect above and its various possible implementations.

[0036] In a tenth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the transformation method as described in the first aspect and its various possible implementations, or to implement the steps of the transformation method as described in the second aspect and its various possible implementations.

[0037] The technical effects of the third to tenth aspects mentioned above can be found in the detailed descriptions of the beneficial effects of the first aspect and its various possible implementations, as well as the second aspect and its various possible implementations. To avoid repetition, these descriptions will not be repeated here. Attached Figure Description

[0038] Figure 1 is a schematic diagram of the architecture of an encoding / decoding system provided in an embodiment of this application;

[0039] Figure 2 is a schematic block diagram of an encoder provided in an embodiment of this application;

[0040] Figure 3 is a schematic block diagram of a decoder provided in an embodiment of this application;

[0041] Figure 4 is a mapping diagram of the direction of angular pattern prediction and the prediction pattern index in WAIP mode provided by an embodiment of this application;

[0042] Figure 5 is a flowchart of an LFNST provided in an embodiment of this application;

[0043] Figure 6 is a schematic diagram of an application of NSPT provided in an embodiment of this application;

[0044] Figure 7 is a flowchart illustrating one of the transformation methods provided in an embodiment of this application;

[0045] Figure 8 is a schematic diagram of CCLM pattern prediction provided in an embodiment of this application;

[0046] Figure 9 is a schematic diagram of a CCLM_SLOPE mode adjustment parameter provided in an embodiment of this application;

[0047] Figure 10 is a schematic diagram of the spatial components of a convolutional filter provided in an embodiment of this application;

[0048] Figure 11 is a second schematic flowchart of a transformation method provided in an embodiment of this application;

[0049] Figure 12 is a schematic diagram of a Sobel operator provided in an embodiment of this application;

[0050] Figure 13 is a schematic diagram of a 3x3 window sliding in a prediction block according to an embodiment of this application;

[0051] Figure 14 is a schematic diagram of the transformation set derivation of a joint chromaticity prediction block provided in an embodiment of this application;

[0052] Figure 15 is a schematic diagram of the transformation set derivation of an independent chromaticity prediction block provided in an embodiment of this application;

[0053] Figure 16 is a third schematic flowchart of a transformation method provided in an embodiment of this application;

[0054] Figure 17 is a schematic diagram of a conversion device provided in an embodiment of this application;

[0055] Figure 18 is a schematic diagram of another conversion device provided in an embodiment of this application;

[0056] Figure 19 is a schematic diagram of a conversion device provided in an embodiment of this application;

[0057] Figure 20 is a schematic diagram of another conversion device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0059] In the description of the embodiments of this application, terms such as "first" and "second" are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" covers three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. The character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0060] In the description of the embodiments of this application, "instruction" can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result in the instruction sent. An indirect instruction can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result based on the judgment result.

[0061] In the description of the embodiments of this application, "at least one (item)," "at least one of," etc., refer to any one, any two, or a combination of two or more of the included objects. For example, at least one (item) of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)."

[0062] In the description of the embodiments of this application, "multiple" means two or more. For example, multiple prediction units refer to two or more prediction units, and multiple coding units refer to two or more coding units. The meanings of "at least two" and "multiple" are similar, and in some embodiments, the two can be used interchangeably.

[0063] In the description of embodiments of this application, the terms "including," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0064] The encoding and decoding system used in the embodiments of this application will be described below with reference to Figure 1.

[0065] Referring to Figure 1, Figure 1 shows a schematic diagram of the architecture of an encoding / decoding system 10 used in an embodiment of this application. As shown in Figure 1, the encoding / decoding system 10 may include a source device 11 and a destination device 12. The source device 11 is used to encode images; therefore, the source device 11 may be referred to as an image encoding apparatus (device) or a video encoding apparatus (device). The destination device 12 is used to decode the encoded image data generated by the source device 11; therefore, the destination device 12 may be referred to as an image decoding apparatus (device) or a video decoding apparatus (device).

[0066] Source device 11 and destination device 12 can support various prediction unit (PU) sizes. Assuming a specific coding unit (CU) size is 2M×2M, source device 11 and destination device 12 can support PU sizes of 2M×2M or M×M for intra-frame prediction, and support symmetric PUs of 2M×2M, 2M×M, M×2M, M×M, or similar sizes for inter-frame prediction. Source device 11 and destination device 12 can also support asymmetric PUs of 2M×MU, 2M×MD, ML×2M, and MR×2M for inter-frame prediction.

[0067] The source device 11 and the destination device 12 can take various forms, and this application embodiment does not specifically limit them. For example, the source device 11 and the destination device 12 can be a desktop computer, a mobile computing device, a laptop (e.g., a laptop computer), a tablet computer, a set-top box, a handset such as a so-called "smartphone", a television, a camera, a display device, a digital media player, a video game console, a vehicle computer, or other similar devices.

[0068] Optionally, the source device 11 and the destination device 12 shown in FIG1 can be two separate devices. Alternatively, the source device 11 and the destination device 12 can also be a single device, that is, the source device 11 or its corresponding function and the destination device 12 or its corresponding function can be integrated into the same device.

[0069] Optionally, the source device 11 and the destination device 12 can communicate with each other. For example, the destination device 12 can receive encoded image data from the source device 11. In one example, the source device 11 and the destination device 12 may include one or more communication devices that can be used to transmit the encoded image data from the source device 11 to the destination device 12. The one or more communication devices may include routers, switches, base stations, or any other possible devices that facilitate communication from the source device 11 to the destination device 12, which can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0070] As shown in Figure 1, the source device 11 may include an encoder 112. Optionally, the source device 11 may also include an image preprocessor 111 and a communication interface 113. The image preprocessor 111 can be used to perform preprocessing on the received image to be encoded. For example, the preprocessing performed by the image preprocessor 111 may include trimming, color format conversion (e.g., from RGB to YUV format), color correction, or noise reduction, or any other possible processing. The encoder 112 can be used to receive the image preprocessed by the image preprocessor 111, process the preprocessed image using a correlation prediction mode, and output encoded image data. In some embodiments, the encoder 112 can be used to perform the encoding process described in the various embodiments below. For example, the encoder 112 can predict the current block and perform transformation, quantization, and other operations on the residual blocks. The communication interface 113 can be used to transmit the encoded image data output by the encoder 112 to the destination device 12 or any other device (such as a storage device) for storage or direct reconstruction. The other device can be any device used for decoding or storage. Of course, in actual implementation, the communication interface 113 can also encapsulate the encoded image data output by the encoder 112 into a suitable format before transmission.

[0071] Optionally, the image preprocessor 111, encoder 112, and communication interface 113 may be hardware components in the source device 11, software programs in the source device 11, or a combination of hardware components and software programs in the source device 11. The specific details can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0072] The destination device 12 may include a decoder 122. Optionally, the destination device 12 may also include a communication interface 121 and an image post-processor 123. The communication interface 121 may be used to receive encoded image data from the source device 11 or any other source device, such as a storage device. The communication interface 121 may also decapsulate the data transmitted by the communication interface 113 to obtain encoded image data. The decoder 122 is used to receive the encoded image data and output decoded image data (also referred to as reconstructed image data or reconstructed image data). In some embodiments, the decoder 122 may be used to perform the decoding processes described in the various embodiments below; for example, the decoder 122 may predict the current block and perform inverse transform, inverse quantization, and other operations on the residual block. The image post-processor 123 may be used to perform post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 may include any possible processing such as color format conversion (e.g., from YUV format to RGB format), color correction, retouching, or resampling. The image post-processor 123 may also be used to transfer the post-processed image data to a display device for display.

[0073] Optionally, the aforementioned communication interface 121, decoder 122, and image post-processor 123 may be hardware components in the target device 12, software programs in the target device 12, or a combination of hardware components and software programs in the target device 12. The specific details can be determined according to actual usage requirements, and this application embodiment does not limit this.

[0074] To better understand the encoder 112 and decoder 122 shown in Figure 1, the structure of the encoder 112 and decoder 122 shown in Figure 1 will be briefly introduced below.

[0075] Referring to Figure 2, Figure 2 shows a possible structural schematic block diagram of the encoder 112 used to implement the encoding method of the embodiments of this application. It is understood that the encoder 112 can be used for lossy compression of images, or for lossless compression of images. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0076] The encoder 112 can be applied to image data in luminance / chrominance (YCbCr, YUV) format. For example, the encoder 112 can be applied to image data with a YUV ratio of 4:2:0, 4:2:2, or 4:4:4. Here, Y can represent luminance (Luma), Cb(U) can represent blue chrominance, Cr(V) can represent red chrominance, and U and V can be used to represent chrominance (Chroma), describing the color and saturation of the image data. For example, in color format, 4:2:0 can represent 4 luminance components and 2 chrominance components per 4 samples (i.e., YYYYCbCr); 4:2:2 can represent 4 luminance components and 4 chrominance components per 4 samples (i.e., YYYYCbCrCbCrCbCr); and 4:4:4 can represent full sample display (i.e., YYYYCbCrCbCrCbCrCbCr).

[0077] It is understood that in the embodiments of this application, luminance can also be referred to as luminance or luminance component; chrominance can also be referred to as chrominance component. For example, Cb(U) can be referred to as blue chrominance component, and Cr(v) can be referred to as red chrominance component.

[0078] As shown in Figure 2, the encoder 112 may include a prediction processing unit 201, a residual calculation unit 202, a residual transformation unit 203, a quantization unit 204, an encoding unit 205, an inverse quantization unit (also called an inverse quantization unit) 206, an inverse residual transformation unit 207, a reconstruction unit (or reconstruction unit) 208, and a filter unit 209. Optionally, the encoder 112 may also include a buffer and a decoded image buffer. The buffer can be used to buffer the reconstructed image block (or reconstructed image block) output by the reconstruction unit 208; the decoded image buffer can be used to buffer the filtered image block output by the filter unit 209.

[0079] In one example, the input to encoder 112 is an image block (i.e., a block to be encoded, an encoding unit, or an encoding tree unit) of the image to be encoded. In another example, the input to encoder 112 is the image to be encoded, and encoder 20 may include a segmentation unit (not shown in Figure 2) that can be used to segment the image to be encoded into multiple image blocks. Encoder 112 can complete the encoding of the image to be encoded by encoding multiple image blocks one by one. For example, encoder 112 can perform the encoding process for each image block separately to complete the encoding of the image to be encoded.

[0080] Related technologies define Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU), and Transform Unit (TU). CTU, CU, PU, ​​and TU can all be considered as image blocks obtained after partitioning. PU and TU are both based on CU partitioning. However, regardless of whether it's CU, PU, ​​or TU, they are essentially all concepts of blocks (or image blocks).

[0081] In some embodiments, the prediction processing unit 201 may include an inter-frame prediction unit (not shown in FIG2) and an intra-frame prediction unit. Since there is a strong correlation between adjacent samples within a frame of a video or image, intra-frame prediction can be used in encoding and decoding techniques to eliminate spatial redundancy between adjacent samples. Simultaneously, since there is a strong similarity between adjacent frames in a video, inter-frame prediction can be used in encoding and decoding techniques to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0082] It is understandable that the prediction processing unit 201 can predict image patches based on redundancy characteristics such as temporal redundancy and spatial redundancy in the video sequence. For example, in the same frame, due to the correlation between adjacent image patches, the value of the current image patch can be inferred (predicted) based on adjacent image patches. Similarly, in different frames, for an image patch at a certain coordinate position, the value of the image patch at the previous moment is correlated with the value at the next moment; therefore, the value of the image patch at the next moment can be predicted based on the value at the previous moment. The image patch being predicted can be either a PU or a CTU as mentioned above. The prediction method can be called a prediction mode. A prediction mode based on the correlation between adjacent image patches in the same frame is called intra-prediction mode; a prediction mode based on the correlation between image patches at a certain coordinate position in different frames is called inter-prediction mode.

[0083] The inter-frame prediction unit of prediction processing unit 201 can be used for inter-frame prediction. Inter-frame prediction can include motion estimation and motion compensation. By referencing image information from different frames and using the motion information corresponding to that image information, a reference block can be found in the reference frame, and a prediction block can be generated based on the reference block to eliminate temporal redundancy. The frames used for inter-frame prediction can be at least one of P-frames and B-frames. P-frames refer to forward prediction frames, and B-frames refer to bidirectional prediction frames. In inter-frame prediction, motion information can include a list of reference frames, a reference frame index, and motion vectors. Motion vectors can be integer pixels (samples) or fractional pixels. If the motion vector is fractional, interpolation filtering needs to be used in the reference frame to create the required fractional pixel blocks. Here, the integer pixel or fractional pixel blocks in the reference frame found based on the motion vector can be called reference blocks. In some examples, the above reference block can also be called a prediction block, that is, the reference block is directly used as the prediction block. In some examples, a prediction block can be generated by further processing the reference block; this can also be understood as taking the reference block as a prediction block and then processing it to generate a new prediction block, which is the final prediction block.

[0084] A common inter-frame prediction method is translational prediction. Translational prediction assumes that the content to be predicted is translated between the current image and the reference image. For example, if the content of the current block (CU, PU, ​​or CTU) is translated between the current image and the reference image, then this content can be found in the reference image using a motion vector (MV) and used as the predicted block for the current block. Translational motion accounts for a large proportion of video or image encoding and decoding. Stationary backgrounds, objects that are translated as a whole, and camera panning can all be handled using translational prediction.

[0085] Because some content in videos or images is not simply translated—for example, there are subtle changes during the translation process, including changes in shape and color—bidirectional prediction can be used. This employs the sub-mode GPM in merge mode to find two reference blocks from the reference image and perform a weighted average of these two reference blocks to obtain a predicted block that is as similar as possible to the current block. For example, in some scenes, a weighted average of one reference block from before and one from after the current frame may be more similar to the current block than a single reference block. Based on this, bidirectional prediction further improves compression performance compared to unidirectional prediction.

[0086] The translation prediction described above is block-based, such as coding units, prediction units, or coding tree units. That is, a sample matrix is ​​used as the unit for prediction. The most basic block is a rectangular block, such as a square block or a rectangular block. Video encoding and decoding technologies allow the encoder to determine the size and partitioning of coding units and prediction units based on the video content. Regions with simple textures or motion tend to use larger blocks, while regions with complex textures or motion tend to use smaller blocks. The deeper the block partitioning, the more complex and closer the blocks are to the actual texture or motion, but the overhead for representing these partitions is correspondingly greater. Motion information may also need to be transmitted in the bitstream. And generally, the finer the block partitioning, the greater the overhead of motion information.

[0087] Intra-frame prediction units (IMUs) refer to units that predict sample information within the current image block by referencing information from only the same frame, thereby eliminating spatial redundancy. The frames used for intra-frame prediction can be I-frames (keyframes).

[0088] Intra-frame prediction aims to predict the value of a current sample or block by leveraging the similarity within an image, and then encodes the difference (residual) between the predicted and actual values. This method can effectively reduce redundant information in image data, thereby improving compression efficiency.

[0089] Specifically, in the embodiments of this application, encoder 112 can be used to implement the encoding method described in the embodiments below.

[0090] In one example, an encoding process implemented by encoder 112 may include the following steps 11-15:

[0091] Step 11: The prediction processing unit 201 determines the prediction mode and predicts the block to be encoded based on the determined prediction mode and the reconstructed block of the encoded image block to obtain the prediction block of the block to be encoded.

[0092] The reconstructed block of the encoded image block is obtained by the inverse quantization unit 206, the inverse residual transform unit 207, and the reconstruction unit 208 sequentially processing the quantized residual coefficient block of the encoded image block.

[0093] Step 12: The residual calculation unit 202 obtains the residual block of the block to be encoded based on the prediction block and the original sample value of the block to be encoded.

[0094] Step 13: The residual transformation unit 203 transforms the residual block of the block to be encoded to obtain the residual coefficient block.

[0095] Step 14: Quantization unit 204 quantizes the residual coefficient block to obtain the quantized residual coefficient block.

[0096] Step 15: The encoding unit 205 encodes the quantized residual coefficient block and the relevant syntax elements (e.g., prediction mode, encoding mode) to obtain the bitstream of the block to be encoded.

[0097] Referring to Figure 3, a possible structural schematic block diagram of the decoder 122 used to implement the decoding method of the embodiments of this application is shown.

[0098] Decoder 122 can be used to receive, for example, image data encoded by encoder 112 (i.e., an encoded bitstream, for example, an encoded bitstream including image blocks and associated syntax elements) to obtain decoded image blocks.

[0099] As shown in Figure 3, the decoder 122 may include a bitstream parsing unit 301, an inverse quantization unit 302, a residual inverse transform unit 303, a prediction processing unit 304, a reconstruction unit 305, and a filter unit 306. In some instances, the decoder 122 may perform a decoding process that is generally the inverse of the encoding process described in the encoder 112 shown in Figure 2. Optionally, the decoder 122 may also include a buffer and a filtered image buffer. The buffer can be used to buffer the reconstructed image blocks output by the reconstruction unit 305, and the filtered image buffer can be used to buffer the filtered image blocks output by the filter unit 306.

[0100] The bitstream parsing unit 301 can be used to decode the encoded bitstream to obtain quantized residual coefficients (or quantized residual values) and / or decoding parameters (e.g., decoding parameters may include any one or more of inter-frame prediction parameters, intra-frame prediction parameters, filter parameters, and / or other syntax elements performed on the encoding side). The bitstream parsing unit 301 can also be used to forward the aforementioned decoding parameters to the prediction processing unit 304, so that the prediction processing unit 304 can perform a prediction process based on the decoding parameters.

[0101] The function of the dequantization unit 302 is the same as that of the dequantization unit 206 of the encoder 112, used to dequantize (i.e., reverse quantization) the quantized residual coefficients output by the code stream parsing unit 301.

[0102] The function of the residual inverse transform unit 303 is the same as that of the residual inverse transform unit 207 of the encoder 112, which is used to perform an inverse transform on the above-mentioned inverse quantized residual coefficients (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to obtain the reconstructed residual value. The block obtained after the inverse transform is the residual block of the reconstructed block to be decoded in the sample domain.

[0103] The function of reconstruction unit 305 (e.g., a summer) can be the same as that of reconstruction unit 208 of encoder 112.

[0104] The prediction processing unit 304 is used to receive or acquire encoded image data (e.g., the encoded bitstream of the current image block) and reconstructed image data. The prediction processing unit 304 can also receive or acquire relevant parameters of the prediction mode and / or information about the selected prediction mode (i.e., the decoding parameters mentioned above) from, for example, the bitstream parsing unit 302, and make a prediction of the current image block based on the relevant data and decoding parameters in the reconstructed image data to obtain the prediction block of the current image block.

[0105] The reconstruction unit 305 can be used to add the reconstructed residual block to the prediction block to obtain the reconstructed block of the image to be decoded in the sample domain, for example, by adding the residual value in the reconstructed residual block to the predicted value in the prediction block.

[0106] The filter unit 306 can be used to filter the reconstructed block to obtain a filtered block, which is the decoded image block.

[0107] Specifically, in the embodiments of this application, the decoder 122 can be used to implement the decoding method described in the embodiments below.

[0108] It is understood that in the encoder 112 and decoder 122 provided in the embodiments of this application, the processing result of a certain stage may be further processed before being output to the next stage. For example, after the prediction, transformation or filtering stages, the processing result of the corresponding stage may be further processed by Clip or shift operations.

[0109] In one example, a decoding process implemented by decoder 122 may include the following steps:

[0110] Step 21: The code stream parsing unit 301 parses the prediction mode and the residual coding mode.

[0111] Step 22: The code stream parsing unit 301 parses the quantization-related values ​​(such as near values ​​or QP values) based on the prediction mode and residual coding mode.

[0112] Step 23: The inverse quantization unit 302 analyzes the residual coefficients based on the prediction mode and quantization correlation value.

[0113] Step 24: The prediction processing unit 304 obtains the predicted values ​​of each sample in the current image block based on the prediction mode.

[0114] Step 25: The inverse residual transformation unit 303 performs an inverse transformation on the residual coefficients to reconstruct the residual values ​​of each sample in the current image block.

[0115] Step 26: The reconstruction unit 305 obtains the reconstructed value based on the predicted value and residual value of each sample in the current image patch.

[0116] It should be noted that the block partitioning information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and in-loop filtering, can be carried in the bitstream when necessary. The decoder determines the same block partitioning information, prediction, transform, quantization, entropy coding, and in-loop filtering mode information or parameter information as the encoder by parsing the bitstream and analyzing existing information, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0117] The above describes the basic flow of a codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or flow may be optimized. This application applies to the basic flow of a codec under this block-based hybrid coding framework, but is not limited to this framework and flow.

[0118] In some embodiments, the current block can be the current CU, the current PU, or the current CTU, etc. Due to the need for parallel processing, images can be divided into slices, blocks, etc. Slices within the same image can be processed in parallel, meaning there is no data dependency between them. A "frame" is a commonly used term, generally understood as one image. In the embodiments of this application, the term "frame" can also be replaced with image, slice, or block, etc.

[0119] Currently, most video compression technologies are block-based, where a video clip, an image, or a series of images can be divided into CTUs (Cost Units). During prediction, intra-frame blocks can be predicted using surrounding samples as references, while inter-frame blocks can reference spatially neighboring blocks and reference information from other frames. In contrast to the prediction information, residual information can be written into the bitstream through block-based transforms, quantization, and entropy coding. Therefore, video encoding / decoding devices can implement these techniques to achieve more efficient video encoding / decoding, transmission, and storage.

[0120] When performing a transform operation on the current block, the transform kernel to be used can be selected for transforming or inverse transforming based on the information of the intra-prediction mode. Typically, different intra-prediction modes select different transform kernels.

[0121] Commonly used transform operations include Multiple Transform Selection (MTS), Non-Separable Primary Transform (NSPT), and Low-Frequency Non-Separable Transform (LFNST). MTS includes traditional transforms such as the Discrete Cosine Transform (DCT) and Discrete Sine Transform (DST); while NSPT and LFNST are a series of transform coefficients obtained from a universal training set based on the optimal transform. NSPT can typically be directly used to transform the residual coefficients; LFNST is often used to further transform the transform coefficients after a two-dimensional DCT (DCT-II) transform.

[0122] Understandably, both NSPT and LFNST have multiple transform kernels. Therefore, when performing a transform operation on the current block, to determine which NSPT / LFNST transform kernel to use for a given prediction mode, the Enhanced Compression Model (ECM) establishes a mapping relationship that associates traditional prediction modes with NSPT / LFNST transforms. This allows the selection of the appropriate NSPT / LFNST transform kernel to use based on the prediction mode corresponding to the current block during the transform process.

[0123] It should be noted that ECM is a reference software model developed based on the reference software (VVC Test Model, VTM) for the purpose of exploring digital video compression technology. In related technologies, traditional intra-frame prediction modes can include: Planar mode, Direct-Current (DC) mode, and Angle mode. Specifically, the intra-frame prediction mode index for Planar mode is 0; for DC mode it is 1; and for Angle mode it ranges from 2 to 66.

[0124] Figure 4 shows a mapping between the directions of angle pattern prediction and the prediction mode indices in Wide-angle intra-prediction (WAIP) mode. The arrows in Figure 4 point to the directions of angle pattern prediction present in related technologies. These directions use prediction mode indices from 2 to 66 during encoding and decoding. When the current block is a non-square block, some angle directions can be replaced with wide angles, such as the prediction mode indices from -1 to -14 in Figure 4, and the prediction mode indices from 67 to 80 in Figure 4.

[0125] In related technologies, NsPT and LFNsT divide the traditional intra-prediction modes into 35 groups, each with 3 selectable transform kernels. The mapping relationship between the traditional intra-prediction modes and the transform sets is shown in Table 1 below:

[0126] Table 1

[0127] In related technologies, LFNST is a method based on direct matrix multiplication to apply inseparable transforms, enabling them to be implemented in a single iteration without requiring multiple iterations. As shown in Figure 5, LFNST can typically be applied between the main transform and quantization, and between the inverse transform and the inverse main transform. For some coding units using traditional intra-frame prediction modes, the selection of the LFNST transform kernel is based on the intra-frame prediction mode and the mapping relationship in Table 1 above to find the corresponding transform set, and then the final transform kernel to be used is determined from the transform set according to the transform kernel index. In related technologies, the use of LFNST can be indicated by the transform index identifier (lfnstIdx). Specifically, when lfnstIdx > 0, it indicates that the current block uses LFNST, and the specific transform kernel is determined from the selected transform set according to the specific value of lfnstIdx; when lfnstIdx = 0, it indicates that the current block does not use LFNST.

[0128] In related technologies, NSPT is a non-separable main transform used to replace the combination of separable DCT-II main transform and LFNST transform. That is, when the current block uses LFNST and the main transform is DCT-II, if the current block meets specific block size conditions, an NSPT transform scheme can be directly used to replace the original transform scheme. Figure 6 illustrates the cases where NSPT can be applied. In Figure 6, 4×N, N×4, 8×N, N×8, and M×N represent block sizes. Specifically, when the current block uses LFNST, the main transform is DCT-II, and the block size is 4×N or N×4, if N≥4 (e.g., the block size is 4×8 or 8×4), then NSPT can be used for the current block. When the current block uses LFNST, the main transform is DCT-II, and the block size is 8×N or N×8, if N≥8 (e.g., the block size is 8×8 or 8×16), then NSPT can be used for the current block. It is understandable that when determining the transform kernel, conditions such as lfnstIdx and block size can be used to indicate whether NSPT should be used. That is, when lfnstIdx > 0 and the NSPT replacement condition is met, the current block can use NSPT, and the specific transform kernel to be used can be determined from the selected transform set based on the specific value of lfnstIdx.

[0129] For some non-traditional angle prediction modes, since it is not possible to directly use Table 1 to determine which set of NSPT / LFNST transform kernels to use for the chrominance components of the current block, a traditional intra-frame prediction mode can usually be derived through Direct Export Mode (DM), and then the corresponding transform set can be determined based on the exported traditional intra-frame prediction mode and Table 1.

[0130] In related technologies, DM is a commonly used chromaticity prediction mode. When predicting chromaticity blocks, luminance blocks have usually already been predicted. At this time, the chromaticity prediction mode of the current block can borrow the already completed luminance prediction mode to determine the prediction mode of the luminance block where the center point of the corresponding luminance block is located, so as to be used for the prediction of the current chromaticity block.

[0131] When using non-traditional intra-prediction modes for prediction, it is usually necessary to first derive a traditional intra-prediction mode through the DM method in order to determine the corresponding transform set.

[0132] For example, the prediction mode of the corresponding luma block can be obtained first using the DM method, and then the traditional prediction mode of the chroma block used to guide the transform set can be derived based on the intra-frame prediction mode of the luma block. The specific derivation method can be described as follows (A1-A5):

[0133] A1. If the intra-prediction mode of the same brightness block is the extrapolation filter-based intra-prediction (EIP) mode, then the traditional prediction mode saved by the EIP technology can be derived.

[0134] A2. If the intra-prediction mode of the same brightness block is a template-based intra-mode derivation (TIMD) mode, then the high-precision angle prediction mode saved by TIMD technology can be exported and converted into a traditional prediction mode according to certain rules.

[0135] A3. If the intra-frame prediction mode of the same brightness block is template-based multiple reference line intra-frame prediction (TMRL) mode, then the high-precision angle prediction mode saved by TMRL technology can be exported and converted into a traditional prediction mode according to certain rules.

[0136] A4. If the intra-prediction mode of the same brightness block is matrix weighted intra-prediction (MIP) mode, then the PLANA mode can be directly derived.

[0137] A5. If the intra-prediction mode of the same luma block is intra-template matching prediction (intraTMP) mode, then the PLANA mode can be directly exported.

[0138] It is understandable that after deriving the traditional intra-frame prediction mode using the above method, the mapping relationship shown in Table 1 can be used to determine which set of NSPT / LFNST transform kernels to use for the chroma block transformation.

[0139] However, in the above method, the transformation kernel of the chroma block is usually determined based on the traditional prediction mode derived from the prediction mode of the luminance block corresponding to the chroma block. Since the luminance block and the chroma block represent luminance and chrominance as two different dimensions, the determined transformation kernel of the chroma block may not be accurate enough, resulting in poor transformation effect of the chroma block.

[0140] To address the aforementioned problems, embodiments of this application provide a transformation method. During the process of determining the transform kernel of a chroma block using an encoder or decoder, the method can be combined with the prediction block of that chroma block. That is, the encoder can determine the transform kernel of the chroma block by incorporating its information. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby enhancing the transformation effect of the chroma block.

[0141] The transformation method provided in the embodiments of this application will be described exemplarily below with reference to the accompanying drawings.

[0142] It should be noted that the encoder in any embodiment of this application can be the encoder 112 in Figure 1 or Figure 2 above, or the source device 11 in Figure 1 above. The decoder in any embodiment of this application can be the decoder 122 in Figure 1 or Figure 3 above, or the destination device 12 in Figure 1 above. This application does not limit the specific implementation of the decoder.

[0143] This application provides a transformation method that can be applied to an encoder.

[0144] In the process of encoding video data, the encoder first divides each frame of the image into blocks, then predicts each unit block to obtain a prediction block, and generates a residual block based on the prediction block. The encoder then transforms the residual block to obtain residual transform coefficients, and finally quantizes the residual transform coefficients to obtain residual quantization coefficients. Finally, the encoder entropy-encodes the residual quantization coefficients, motion information, prediction information, etc., to obtain the video bitstream output.

[0145] It should be noted that the encoder can first divide a video into several sequences, and then divide a sequence into several Groups of Pictures (GOPs), each GOP representing a group of consecutive video frames. When performing predictive coding and transform coding on the images, the encoder can first use a quadtree to divide the image. During quadtree division, the encoder can divide the entire image into several square coding tree blocks (CTBs). CTBs can be further divided into coding blocks (CBs), and CBs can be further divided into prediction blocks (PBs) and transform blocks (TBs).

[0146] It is understandable that a luma CB and two chroma CBs at the same location, along with their corresponding syntax elements, can form a coding unit (CU); a luma CTB and two chroma CTBs at the same location, along with their corresponding syntax elements and corresponding CUs, can form a coding tree unit (CTU).

[0147] During the transformation process, the encoder can map and transform the spatial domain signals in the image to the frequency domain, and then encode the generated transform coefficients. It's understandable that in the spatial domain, the correlation between data is relatively high, resulting in significant data redundancy, especially noticeable in flat areas of the image where brightness values ​​change slowly. Transforming to the frequency domain converts the spatially dispersed data into a more concentrated distribution, thereby reducing correlation and data redundancy, ultimately achieving the goal of eliminating spatial redundancy.

[0148] Understandably, since transform coding only converts image data from the spatial domain to the frequency domain, the amount of data is not reduced. Therefore, if further data compression is desired, quantization and entropy coding can be applied to the data in the frequency domain.

[0149] Figure 7 shows a flowchart of a transformation method provided in an embodiment of this application. As shown in Figure 7, the transformation method provided in this embodiment may include the following steps 101 to 103.

[0150] Step 101: The encoder performs intra-frame prediction on the chroma block to obtain the prediction block of the chroma block.

[0151] In some embodiments of this application, the aforementioned chroma block may refer to the block currently being processed during the encoding process.

[0152] It is understandable that the aforementioned chroma block can be a dynamically changing block, and as the encoder processes the image block by block, the aforementioned chroma block will also change accordingly.

[0153] In some embodiments of this application, the chromaticity component of the chromaticity block may refer to at least one chromaticity component of the chromaticity block in the color space.

[0154] For example, the chromaticity component of the aforementioned chromaticity block can refer to the Cb component, or the Cr component, or both Cb and Cr components in the YCbCr color space. It is understood that both the Cb and Cr components can be used to represent the chromaticity and saturation information of a color. In video compression technology, the chromaticity component is often encoded together with the luminance component to improve compression efficiency and image quality.

[0155] It should be noted that the following embodiments all use the example of the chromaticity components of the chromaticity block referring to the Cb and Cr components. In actual implementation, the chromaticity components of the chromaticity block may also include other possible chromaticity components.

[0156] In some embodiments of this application, the intra prediction mode is a prediction mode that makes predictions based on the correlation between adjacent image blocks in the same frame.

[0157] In some embodiments of this application, the prediction block of the chromaticity block may include the prediction block of the chromaticity block of chromaticity component Cb and the prediction block of the chromaticity block of chromaticity component Cr.

[0158] It should be noted that the prediction blocks for chromaticity component Cb and chromaticity component Cr mentioned above can also be expressed as the predicted values ​​for chromaticity component Cb and chromaticity component Cr, respectively. They have the same meaning and can be used interchangeably.

[0159] In some embodiments of this application, the encoder can utilize the correlation between different color components in an image to reduce the amount of data that needs to be encoded by prediction, thereby reducing redundancy between different color components and improving the efficiency of video encoding.

[0160] In some embodiments of this application, the prediction mode for intra-frame prediction of chroma blocks in step 101 above can be a cross-component prediction mode.

[0161] For example, step 101 above can be implemented by step 101a below.

[0162] Step 101a: The encoder performs intra-frame prediction on the chroma block based on the reconstructed value of the luminance block corresponding to the chroma block, and obtains the prediction block of the chroma block.

[0163] In some embodiments of this application, the reconstructed value of the above-mentioned luminance block can be a sample value (pixel value) obtained by transforming, quantizing, dequantizing and inverse transforming after intra-frame prediction or inter-frame prediction.

[0164] In some embodiments of this application, the encoder may employ a conventional chroma intra-frame prediction mode or a non-conventional chroma intra-frame prediction mode. Based on the reconstructed value of the luma block, it performs intra-frame prediction on the chroma block to obtain the prediction block of the chroma component, such as obtaining the prediction block of the chroma block of the chroma component Cb and the prediction block of the chroma block of the chroma component Cr.

[0165] In some embodiments of this application, the aforementioned conventional chroma intra-frame prediction modes may include, but are not limited to: PLANA mode, DC mode, and angle mode.

[0166] In some embodiments of this application, when the encoder performs intra-frame prediction of chroma blocks using the traditional chroma intra-frame prediction mode, the encoder may also choose whether to perform intra-frame prediction of chroma blocks in a prediction mode that fuses multiple intra-frame prediction results.

[0167] For example, the encoder can choose whether to perform intra-frame prediction of the chroma block in chroma fusion prediction mode to obtain the predicted block of the chroma block.

[0168] In some embodiments of this application, the aforementioned non-traditional chroma intra-frame prediction modes may include, but are not limited to: cross-component prediction mode and decoder-side block vector (DBV) mode.

[0169] It is understandable that traditional chroma intra-frame prediction modes mainly rely on neighboring samples in the current block space for prediction; while non-traditional chroma intra-frame prediction modes can utilize repetitive or related elements within the image to perform intra-frame prediction using a method similar to inter-frame prediction.

[0170] In some embodiments of this application, the aforementioned cross-component prediction mode is a prediction mode that generates predicted values ​​for chroma blocks using information from luma blocks to reduce cross-component redundancy and perform cross-component prediction. It is a non-traditional prediction mode. Therefore, when using the cross-component prediction mode to predict chroma components, the encoder needs to first derive a traditional intra-frame prediction mode using the DM method before determining the corresponding transform set based on that traditional intra-frame prediction mode.

[0171] In some embodiments of this application, when performing cross-component prediction in Cross-Component Linear Mode (CCLM), the linear relationship between the chroma block and the luma block can be expressed as shown in the following formula (1): pred C (i, j) = a·rec L Formula (1) is (i, j) + b.

[0172] Where i and j can represent the sample (pixel) coordinates of the current CU; pred C (i, j) can represent the current chromaticity prediction sample of the CU; rec L ′(i,j) can represent the downsampled reconstructed luminance sample of the current CU; a can be the scaling parameter of the linear model, and b can be the offset parameter of the linear model. a and b can be calculated and derived from the adjacent chroma and luminance samples of the current CU.

[0173] Figure 8 shows an example of the current block and adjacent reconstructed chromaticity samples and corresponding reconstructed luminance samples in CCLM mode.

[0174] Understandably, since the linear model for cross-component prediction can be computed in the encoder and decoder, it does not need to be written into the bitstream, thus saving transmission resources.

[0175] In related technologies, besides using all the upper and left reference samples to jointly calculate the parameters of the linear model, there are two other ways to calculate the linear model parameters. That is, CCLM has two other modes, which can be called CCLM-T and CCLM-L modes. In CCLM-T mode, only the upper reference samples can be used to calculate the linear model parameters; in CCLM-L mode, only the left reference samples can be used to calculate the linear model parameters.

[0176] To further improve the coding efficiency of CCLM, chroma blocks can be predicted using CCLM_SLOPE, Multi-Model Local Matching (MMLM) mode, Convolutional cross-component intra prediction model (CCCM) mode, etc., to obtain the predicted blocks of chroma blocks.

[0177] In CCLM_SLOPE, the calculated linear model parameters can be adjusted using the following formula (2): a′=a+u,b′=bu*yr Formula (2)

[0178] Where u can be the adjustment magnitude of the linear model parameters a and b; yr can be the average value of the reference brightness sample. Thus, as shown in Figure 9, by using the above formula (2), the mapping function in the linear model can be tilted or rotated around the point with brightness value yr, thereby obtaining a better prediction value.

[0179] In MMLM, multiple linear model parameters can be provided between luma and chroma blocks within the same CU. Specifically, adjacent luma and chroma samples can be classified into different categories based on a classification threshold, and samples in each category are calculated using different linear model parameters.

[0180] In CCCM, a set of 7-tap convolutional filters can be used to filter luminance samples to obtain predicted values. Typically, a 7-tap convolutional filter contains 5 spatial components, one nonlinear term, and one bias term. Figure 10 shows the 5 spatial components of this convolutional filter. In Figure 10, C represents the luminance block sample at the corresponding position of the chroma block sample, and N, S, W, and E represent the samples above, below, to the left, and to the right of the corresponding sample, respectively. In CCCM, the predicted value of the chroma block sample can be calculated using the following formula (3). C (i, j) = c0C + c1N + c2S + c3E + c4W + c5P + c6B Formula (3)

[0181] Where c0, c1, c2, c3, c4, c5, and c6 can be the filter coefficients of the convolutional filter; P can represent the nonlinear term, which can be the square power of the center luminance sample C, and can be scaled to the corresponding sample value range by the following formula (4); B can represent the bias term, which can usually be the median value of the chromaticity value, and can be calculated by the following formula (5). P=(C*C+midVal)>>bitDepth Formula (4) B=midVal Formula (5)

[0182] Here, bitDepth can represent the bit depth; the value of midVal can be determined based on the bit depth value. For example, when the bit depth is 10, the value of midVal can be (2^10) / 2 = 512.

[0183] Understandably, the parameters of the convolutional filter are also obtained through adjacent reconstructed samples. Unlike CCLM, the reference template region of CCCM typically includes 6 rows and 6 columns of reconstructed samples surrounding the current block, as well as extended regions to the upper right and lower left. The filter coefficients can be calculated by minimizing the mean-square error (MSE) between the predicted and reconstructed chroma samples in the reference region. Furthermore, MSE minimization can be achieved by calculating the autocorrelation matrix of the input of the luminance component corresponding to the current block and the cross-correlation vector between the input of the luminance component and the output of the chroma component.

[0184] It is understandable that there are various derivative variations of cross-component derivation modes based on convolutional models, including the Generalized Linear Model (GLM). Similar to CCLM, CCCM can choose to use different reference region shapes, i.e., there are CCCM-T and CCCM-L modes. Likewise, CCCM can also choose to use multiple linear parameter models, i.e., the MM-CCCM mode. In the MM-CCCM mode, the current input sample can determine the convolutional filter to be used based on a threshold, thereby obtaining the final predicted value through that convolutional filter. The embodiments in this application are not specifically limited.

[0185] Step 102: The encoder determines the transform kernel of the chroma block based on the prediction block.

[0186] In some embodiments of this application, the transform kernel of the chroma block described above can be used to optimize the coding efficiency of the chroma block in order to obtain a chroma that is more in line with human visual perception.

[0187] In some embodiments of this application, the encoder can determine the transform kernel of the chroma component based on the intra-prediction mode corresponding to the prediction block.

[0188] In some embodiments of this application, the encoder can determine the transform kernel of the chroma block based on the prediction block, provided that the current intra-frame prediction mode is determined to be the chroma intra-frame prediction mode and the current transform technique uses LFNST or NSPT.

[0189] In some embodiments of this application, after the encoder performs intra-frame prediction on the chroma block to obtain the prediction block of the chroma block, it can first determine the intra-frame prediction mode corresponding to the prediction block based on the prediction block, and then determine the transform kernel of the chroma block based on the intra-frame prediction mode.

[0190] It should be noted that, in the embodiments of this application, the intra-prediction mode corresponding to the prediction block of the chroma block determined by the encoder is a different concept from the intra-prediction mode used when performing intra-prediction on the chroma block. The intra-prediction mode corresponding to the prediction block of the chroma block can be used for selecting the transform kernel of the chroma block, while the intra-prediction mode used when performing intra-prediction on the chroma block is used for performing intra-prediction on the chroma block.

[0191] For example, referring to FIG7, as shown in FIG11, the above step 102 can be implemented by the following steps 102a and 102b.

[0192] Step 102a: The encoder determines the first intra-frame prediction mode corresponding to the prediction block based on the prediction block.

[0193] In some embodiments of this application, the first intra-frame prediction mode described above can be an intra-frame prediction mode derived by the encoder based on the prediction block.

[0194] In some embodiments of this application, for the prediction block of the chrominance block of chrominance component Cb and the prediction block of the chrominance block of chrominance component Cr, the encoder can determine the intra prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and the intra prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cr, respectively; or, the encoder can determine the intra prediction mode that is commonly corresponding to the prediction blocks of the chrominance blocks of chrominance component Cb and the prediction blocks of the chrominance blocks of chrominance component Cr. In other words, the aforementioned first intra-frame prediction mode may include an intra-frame prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and an intra-frame prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cr. That is, the first intra-frame prediction mode may include two intra-frame prediction modes, which correspond to chrominance component Cb and chrominance component Cr respectively; or, the aforementioned first intra-frame prediction mode may also be an intra-frame prediction mode that corresponds to both the prediction block of the chrominance block of chrominance component Cb and the prediction block of the chrominance block of chrominance component Cr. That is, the first intra-frame prediction mode may include one intra-frame prediction mode, which may correspond to both chrominance component Cb and chrominance component Cr.

[0195] In some embodiments of this application, the encoder may use the Decoder-side Chroma Intra Mode Derivation (DIMD) method to determine the first intra-prediction mode corresponding to the prediction block.

[0196] It should be noted that the above-mentioned DIMD is a method that uses the reconstructed information of the surrounding area of ​​the current block to form an input template, and uses the gradient of the samples in the input template to derive the intra-prediction mode, thereby saving the need to represent the intra-prediction mode of the current block through bitstream transmission.

[0197] In some embodiments of this application, in conjunction with the above-described DIMD, step 102a can be specifically implemented through the following steps 102a1 and 102a2.

[0198] Step 102a1: The encoder determines the HoG of the prediction block based on the gradient information of the prediction block.

[0199] The histogram of gradient (HoG) of the aforementioned prediction block may include at least one gradient direction and at least one gradient magnitude.

[0200] In some embodiments of this application, the gradient direction described above can be used to indicate the angle of the angle prediction mode corresponding to the sample located in the non-edge region of the prediction block.

[0201] In some embodiments of this application, a gradient direction can correspond to a gradient magnitude, that is, the above-mentioned at least one gradient direction can correspond to the above-mentioned at least one gradient magnitude.

[0202] In some embodiments of this application, the gradient magnitude described above can be used to indicate the cumulative intensity in its corresponding gradient direction.

[0203] In some embodiments of this application, the encoder may first obtain the gradient information of the prediction block of the chroma block, and then determine the HoG of the prediction block of the chroma block based on the gradient information of the prediction block of the chroma block.

[0204] In some embodiments of this application, the encoder may employ a filter to calculate the gradient information of the predicted block of the chroma block.

[0205] For example, the encoder can employ DIMD, using a 3x3 horizontal Sobel filter and a vertical Sobel filter as shown in Figure 12. It calculates the horizontal gradient Gx and vertical gradient Gy for the sample at the center of the input template, and then calculates arctan(Gx / Gy) to obtain the angle of the sample at the center of the input template. This angle is then converted into the angle of the corresponding angle prediction mode, i.e., the gradient direction corresponding to that angle. Next, the magnitude value of that angle is obtained by calculating the sum of the absolute values ​​of the horizontal gradient Gx and the vertical gradient Gy, i.e., |Gx|+|Gy|, i.e., the gradient magnitude corresponding to the gradient direction of that angle. It can be understood that by performing the above operation for each sample in the input template and accumulating the magnitude value corresponding to each angle, a HoG containing the accumulated magnitude values ​​for different angles can be obtained.

[0206] It should be noted that arctan is a function that calculates the angle between two legs of a right triangle using the ratio of the opposite side to the adjacent side. Gx and Gy can be obtained by multiplying the horizontal Sobel operator and the vertical Sobel operator by the predicted value within the window position, respectively.

[0207] In some embodiments of this application, the input template for DIMD may include, but is not limited to: reconstructed samples adjacent to the chroma block, and predicted samples in the prediction block.

[0208] In some embodiments of this application, in the DIMD prediction mode of the chroma block, a prediction block can be obtained by using the reconstructed samples adjacent to the chroma block and the partial reconstructed samples of the co-located luminance block. Then, the HoG of the chroma component is constructed based on the prediction block, and the traditional intra-frame prediction angle mode corresponding to the chroma component is determined according to the HoG.

[0209] In some embodiments of this application, when the encoder determines the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cr, the encoder can calculate the gradient information of the prediction block of the chrominance block of chrominance component Cb, determine the HoG of the prediction block of the chrominance block of chrominance component Cb, and calculate the gradient information of the prediction block of the chrominance block of chrominance component Cr, determine the HoG of the prediction block of the chrominance block of chrominance component Cr.

[0210] For example, taking cross-component prediction as an example, assume that the predicted block of the chrominance block of chrominance component Cb obtained by cross-component prediction is a block with a width and height of (W, H), and the predicted block of the chrominance block of chrominance component Cr is a block with a width and height of (W, H). The encoder can slide a 3x3 window 131 as shown in Figure 13 in the predicted block of the chrominance block of chrominance component Cb, calculate the gradient values ​​and corresponding amplitude values ​​of the (W-2)×(H-2) sample positions at the center of the predicted block of the chrominance block of chrominance component Cb, and accumulate the amplitude value of the gradient at each position on its derived conventional angle category to obtain a gradient amplitude histogram HoG, that is, the HoG of the predicted block of the chrominance block of chrominance component Cb. The encoder can also slide a 3x3 window 131 as shown in Figure 13 in the prediction block of the chromaticity block of the chromaticity component Cr, calculate the gradient values ​​and corresponding amplitude values ​​of the (W-2)×(H-2) sample positions at the center of the prediction block of the chromaticity block of the chromaticity component Cr, and accumulate the amplitude value of the gradient at each position on its derived conventional angle category to obtain a gradient amplitude value histogram HoG, which is the HoG of the prediction block of the chromaticity block of the chromaticity component Cr.

[0211] In some embodiments of this application, when the intra-frame prediction mode is the same as the prediction block of the chroma block of chroma component Cb and the prediction block of the chroma block of chroma component Cr, the chroma component may include a first chroma block and a second chroma block, and the prediction block may include a first prediction block of the first chroma block and a second prediction block of the second chroma block.

[0212] The chromaticity components of the first chromaticity block and the second chromaticity block may be different.

[0213] In this case, step 102a1 can be implemented by steps 102a11 to 102a13 as described below, for example.

[0214] Step 102a11: The encoder determines the first HoG of the first prediction block based on the gradient information of the first prediction block.

[0215] Step 102a12: The encoder determines the second HoG of the second prediction block based on the gradient information of the second prediction block.

[0216] In some embodiments of this application, when the chromaticity component of the first chromaticity block is chromaticity component Cb, the chromaticity component of the second chromaticity block can be chromaticity component Cr; when the chromaticity component of the first chromaticity block is chromaticity component Cr, the chromaticity component of the second chromaticity block can be chromaticity component Cb.

[0217] In some embodiments of this application, the encoder can first calculate the gradient information of the first prediction block and the gradient information of the second prediction block respectively, determine the first HoG of the first prediction block and the second HoG of the second prediction block, and then obtain the HoG of the prediction block based on the first HoG and the second HoG.

[0218] Step 102a13: The encoder sums the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0219] In some embodiments of this application, after obtaining the first HoG and the second HoG, the encoder can accumulate the gradient magnitudes with the same gradient direction in the first HoG and the second HoG to obtain the HoG of the prediction block.

[0220] In this way, when the chroma block contains a first chroma block and a second chroma block, the encoder can first determine the first HoG of the first prediction block of the first chroma block and the second HoG of the second prediction block of the second chroma block, respectively, and then accumulate the two HoGs according to the gradient direction to obtain a HoG containing the information of the first chroma block and the information of the second chroma block, thereby making the intra-frame prediction mode determined based on the HoG more accurate.

[0221] Step 102a2: The encoder determines the intra-frame prediction mode based on the HoG of the prediction block.

[0222] In some embodiments of this application, after obtaining the HoG of the prediction block, the encoder can select a suitable gradient direction based on the gradient magnitude in the HoG, and thus determine the first intra-frame prediction mode based on the gradient direction.

[0223] In this way, the encoder can derive appropriate angle information based on the HoG of the prediction block, thereby determining the corresponding intra-prediction mode. Subsequently, in subsequent steps, a suitable transform kernel can be determined based on the derived intra-prediction mode. This improves the accuracy of determining the transform kernel for the chroma block, thus enhancing the transform effect of the chroma block.

[0224] In some embodiments of this application, step 102a2 can be specifically implemented by step 102a21 as described below.

[0225] Step 102a21: The encoder determines the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0226] The first gradient direction can be one of the at least one gradient directions. Specifically, the first gradient direction can be the gradient direction corresponding to the first gradient magnitude among the at least one gradient directions, and the first gradient magnitude can be the maximum gradient magnitude among the at least one gradient magnitude.

[0227] In some embodiments of this application, the maximum gradient magnitude among the at least one gradient magnitude can be represented as follows: in the HoG of the prediction block, the sample intensity change is most significant in the gradient direction corresponding to this maximum gradient magnitude. That is, the edge change in this gradient direction contributes the most to the image features. Therefore, the intra prediction mode corresponding to the maximum gradient magnitude among the at least one gradient magnitude is a more suitable intra prediction mode for selecting the transform kernel.

[0228] Understandably, when constructing a HoG (Hope of Gaussian) for prediction blocks, the image is typically divided into several small grids (e.g., 8x8 sample units), and the gradient magnitude and direction of all samples are calculated within each small grid. The gradient direction and magnitude of each sample can be used to weight the histogram channels in a specific direction, with the cumulative magnitude value correspondingly higher in the gradient direction with the largest gradient magnitude. This statistical method allows the histogram to capture the main directional features of local regions in the image. In other words, the gradient direction with the largest gradient magnitude in the HoG reflects the most dominant edge direction within a local region of the image, and is an important part of the image feature description.

[0229] Thus, by determining the intra-prediction mode corresponding to the gradient direction of the maximum gradient magnitude in at least one gradient magnitude in the HoG of the prediction block as the first intra-prediction mode, the derived first intra-prediction mode can be a more suitable intra-prediction mode for selecting the transform kernel, thereby making the transform kernel selected based on the first intra-prediction mode more suitable.

[0230] In some embodiments of this application, step 102a21 described above can be specifically implemented through step A as follows.

[0231] Step A: If there is a non-zero gradient magnitude in at least one gradient magnitude, the encoder determines the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0232] In some embodiments of this application, the transformation method provided in this application may further include the following step B.

[0233] Step B: When at least one gradient magnitude is 0, the encoder determines the preset intra-frame prediction mode as the first intra-frame prediction mode.

[0234] In some embodiments of this application, the presence of a non-zero gradient magnitude among the at least one gradient magnitude can indicate that the image exhibits edge or texture transformation within that local region. Therefore, in this case, the encoder can determine the intra-prediction mode corresponding to the gradient direction of the maximum gradient magnitude among the at least one gradient magnitude as the first intra-prediction mode, which is used to determine the transform kernel of the chroma block.

[0235] In some embodiments of this application, the fact that at least one gradient magnitude is 0 can indicate that the image has no edge or texture changes in the local region, that is, the local region is visually uniform, such as a flat region or image noise. Therefore, in this case, the encoder can directly determine the preset intra-frame prediction mode as the first intra-frame prediction mode to determine the transform kernel of the chroma component.

[0236] For example, when at least one gradient magnitude is 0, the encoder can directly determine the PLANAR mode as the first intra-frame prediction mode for determining the transform kernel of the chroma components.

[0237] Thus, for different cases where at least one gradient magnitude is 0, the encoder can determine the first intra-frame prediction mode in different ways to determine the transform kernel of the chroma block, thereby ensuring that the encoder can determine the appropriate first intra-frame prediction mode in any case, and then determine the appropriate transform kernel for the chroma block based on the first intra-frame prediction mode.

[0238] Step 102b: The encoder determines the transform kernel of the chroma component based on the intra-frame prediction mode of the first frame.

[0239] In some embodiments of this application, when the chroma block transformation technique uses LFNST or NSPT, the encoder can determine the transformation kernel of the chroma components based on the determined first intra-frame prediction mode, using Table 1 above.

[0240] It should be noted that a detailed description of how the encoder determines the transform kernel of the chroma block based on the determined intra-frame prediction mode using Table 1 above can be found in the relevant descriptions of the LFNST and NSPT technologies above. To avoid repetition, it will not be repeated here.

[0241] In this way, the encoder can determine the transform kernel of a chroma block by combining it with the prediction block of that chroma block. That is, the encoder can determine the transform kernel of a chroma block by incorporating the information of the chroma block. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby improving the transform effect of the chroma block.

[0242] In some embodiments of this application, step 102b can be specifically implemented by the following steps 102b1 and 102b2.

[0243] Step 102b1: The encoder determines the first transform set based on the first intra-frame prediction mode and the first mapping relationship.

[0244] The first mapping relationship mentioned above can be the mapping relationship between the first intra-frame prediction mode and the first transform set.

[0245] In some embodiments of this application, the first mapping relationship described above may be one of a plurality of mapping relationships, each of which may be a mapping relationship between an intra-frame prediction mode and a transform set.

[0246] In some embodiments of this application, the above-mentioned multiple mapping relationships may include, but are not limited to, the mapping relationship between intra-prediction modes and transform sets shown in Table 1 above.

[0247] It is understandable that the first mapping relationship mentioned above can be one of the mapping relationships shown in Table 1 above.

[0248] For example, as shown in Table 1 above, assuming the mode index of the first intra-frame prediction mode is 24, then the index of the LFNST transform set (i.e., the first transform set) is 24. That is, the first mapping relationship is the mapping relationship between the intra-frame prediction mode with mode index 24 and the LFNST transform set with index 24.

[0249] Understandably, after determining the LFNST transform set corresponding to the intra-prediction mode in the first frame, the encoder can then determine whether the current block uses NSPT technology based on whether the main transform of the current block is DCT-II and whether the current block meets specific block size conditions.

[0250] Step 102b2: The encoder determines the transform kernel of the chroma block from the first transform set.

[0251] In some embodiments of this application, the encoder may attempt to transform using each transform kernel in the first transform set to select a transform kernel that is more in line with human vision, and then use that transform kernel as the transform kernel of the chroma block.

[0252] It should be noted that the transformation kernel of the chroma block in the embodiments of this application can be understood as the transformation kernel used when transforming the chroma block.

[0253] In this way, the encoder can first determine the transform set based on the mapping relationship between the intra-frame prediction mode and the transform set, and then determine the appropriate transform kernel from the transform set for the transform corresponding to the chroma block, thereby improving the transform effect of the chroma block.

[0254] In some embodiments of this application, the transformation method provided in this application may further include the following step C.

[0255] Step C: The encoder writes the first index information into the bitstream.

[0256] The aforementioned first index information can be used to indicate the transform kernel of the chroma block. That is, the aforementioned first index information can be used to indicate the transform kernel used by the encoder to transform the chroma block.

[0257] It can be understood that the aforementioned first index information can be the transform index identifier of the transform kernel determined by the encoder for the chroma block from the first transform set.

[0258] It is understandable that in the bitstream transmitted from the encoder to the decoder, each intra-frame CU can transmit an LFNST transform index identifier once. This transform index identifier can be used to indicate whether LFNST is used and the index of the transform kernel when LFNST is used.

[0259] It should be noted that for a description of how the transform index identifier indicates whether LFNST is used, and the index of the transform kernel when LFNST is used, please refer to the detailed description of LFNST above. To avoid repetition, it will not be repeated here.

[0260] Thus, by writing the index information of the chroma block's transform kernel into the bitstream, the decoder can directly determine whether to use LFNST and the transform kernel when using LFNST based on this index information after parsing the bitstream. This simplifies the decoder's steps in determining the chroma block's transform kernel and improves the chroma block's transform effect.

[0261] Step 103: The encoder transforms the residual block or the transform block of the chroma block based on the transform kernel.

[0262] In some embodiments of this application, the encoder can apply the transform kernel to the residual block (i.e., the chroma residual block) or the transform block of the chroma block to transform the residual block of the chroma components from the spatial domain to the frequency domain.

[0263] It is understandable that when the encoder performs a primary transformation on the chroma block, the encoder can transform the residual block of the chroma block based on the transformation kernel to obtain the corresponding transformation coefficients; when the encoder performs a secondary transformation on the chroma block, the encoder can transform the transformed block of the chroma block based on the transformation kernel to obtain the corresponding transformation coefficients.

[0264] In some embodiments of this application, when the encoder determines the intra-prediction mode that is commonly corresponding to the prediction block of the chrominance block of chrominance component Cb and the prediction block of the chrominance block of chrominance component Cr, as shown in FIG14, the encoder can select the transform set commonly corresponding to chrominance component Cb and chrominance component Cr based on the intra-prediction mode corresponding to the gradient direction corresponding to the maximum amplitude value in the HoG of the prediction block jointly determined by the prediction blocks of the chrominance blocks of chrominance component Cb and the prediction blocks of the chrominance blocks of chrominance component Cr, and Table 1 above, and determine the common transform kernel of chrominance component Cb and chrominance component Cr from the commonly corresponding transform set, and then transform the residual block or transform block of the chrominance block of chrominance component Cb and the residual block or transform block of the chrominance block of chrominance component Cr based on the transform kernel.

[0265] In some embodiments of this application, the test results of the scheme for the encoder to determine the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and the prediction block of the chrominance block of chrominance component Cr are shown in Table 2 below:

[0266] Table 2

[0267] In Table 2 above, EncT represents encoding complexity, and DecT represents decoding complexity. It should be noted that when EncT is 100%, it means the encoding complexity neither increases nor decreases; when EncT is greater than 100%, it means the encoding complexity increases; and when EncT is less than 100%, it means the encoding complexity decreases. Similarly, when DecT is 100%, it means the decoding complexity neither increases nor decreases; when DecT is greater than 100%, it means the decoding complexity increases; and when DecT is less than 100%, it means the decoding complexity decreases.

[0268] It is understandable that the scheme of determining the intra-prediction mode corresponding to the prediction blocks of the chrominance blocks of the chrominance component Cb and the chrominance blocks of the chrominance component Cr through the above encoder can more accurately determine the corresponding intra-prediction mode without increasing the complexity of encoding or decoding, thereby determining a suitable transform kernel for transform.

[0269] In some embodiments of this application, when the encoder determines the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cr, as shown in FIG15, the encoder can, after determining the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cb and the intra-prediction mode corresponding to the prediction block of the chrominance block of chrominance component Cr, determine the transform set corresponding to chrominance component Cb and the transform set corresponding to chrominance component Cr, and then determine the transform kernel of the chrominance block of chrominance component Cb from the transform set corresponding to chrominance component Cb, so as to transform the residual block or transform block of the chrominance block of chrominance component Cb, and determine the transform kernel of the chrominance block of chrominance component Cr from the transform set corresponding to chrominance component Cr, so as to transform the residual block or transform block of the chrominance block of chrominance component Cr.

[0270] It is understandable that during the transformation of the chroma block, the encoder can generate a set of transformation coefficients, which can indicate the representation of the residual block of the chroma block in the frequency domain.

[0271] In some embodiments of this application, after obtaining the transform coefficients of the chroma block, the encoder can also quantize the transform coefficients to reduce the precision of the coefficients, reduce the amount of data, and facilitate entropy coding.

[0272] The transformation method provided in this application allows the encoder to determine the transform kernel of a chroma block by combining it with the prediction block of that chroma block. In other words, the encoder can combine the information of the chroma block to determine the transform kernel. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0273] In some embodiments of this application, the transformation method provided in this application may further include the following step 104.

[0274] Step 104: The encoder uses a transform kernel to perform an inverse transform on the first coefficient.

[0275] The first coefficient mentioned above can be the transformation coefficient of the chroma block, or the first coefficient mentioned above can also be the inverse quantization coefficient of the chroma block.

[0276] It is understandable that, in the case where the first coefficient is the inverse quantization coefficient of the chroma block, the inverse quantization coefficient can be obtained by inverse quantization of the chroma block's quantization coefficient.

[0277] In some embodiments of this application, the encoder can dequantize the quantized coefficients to reconstruct the transform coefficients.

[0278] It is understandable that when the encoder performs encoding in lossless mode, it can directly perform inverse transformation on the transform coefficients of the chroma block to obtain the residual block of the chroma block.

[0279] In some embodiments of this application, during the process of encoder reconstructing chroma blocks, the encoder can combine the residual block after inverse transformation with the prediction block to reconstruct the chroma image block, that is, obtain the reconstructed block of the chroma components.

[0280] Thus, by using the same transform kernel as the decoder during reconstruction, the first coefficients are inversely transformed, which improves the transformation effect of the chroma components and reconstructs the same chroma blocks as the decoder.

[0281] The above is a description of the transformation method provided in the embodiments of this application, taking the encoder as an example. The following is an exemplary description of the transformation method provided in the embodiments of this application, taking the decoder as an example.

[0282] This application provides a transformation method that can be applied to a decoder.

[0283] During video data decoding, the decoder can inverse-quantize the quantization coefficients obtained from parsing the bitstream to obtain inverse-quantized coefficients, which are then used to reconstruct the transform coefficients. Next, the inverse-quantized coefficients are inversely transformed to obtain residual blocks, which are then used to reconstruct the residual data. Finally, for inter-frame prediction, the decoder can add the predicted blocks to the reconstructed residual data to obtain the current reconstructed block; for intra-frame prediction, the decoder can add the intra-frame predicted blocks to the reconstructed residual data to obtain the current reconstructed block. It can be understood that by reconstructing the chroma block and its corresponding luma block, the current reconstructed block can be obtained.

[0284] Figure 16 shows a flowchart of a transformation method provided in an embodiment of this application. As shown in Figure 16, the transformation method provided in this embodiment may include the following steps 201 to 203.

[0285] Step 201: The decoder performs intra-frame prediction on the chroma block to obtain the prediction block of the chroma block.

[0286] In some embodiments of this application, the decoder can perform intra-frame prediction on the chroma blocks after parsing the bitstream from the encoder to obtain the predicted blocks of the chroma blocks.

[0287] In some embodiments of this application, the aforementioned chroma block may refer to the block being processed during the decoding process. For example, a chroma block may be any possible form such as an image block, image frame, slice, CTU, CU, etc., and this application does not specifically limit this.

[0288] It is understandable that the chroma block mentioned above is a dynamically changing block, and the chroma block will change as the decoder processes the image block by block.

[0289] In some embodiments of this application, the chromaticity component of the chromaticity block may refer to at least one chromaticity component of the chromaticity block in the color space.

[0290] It should be noted that the following embodiments all use the example of the chromaticity components of the chromaticity block referring to the Cb and Cr components. In actual implementation, the chromaticity components of the chromaticity block may also include other possible chromaticity components.

[0291] In some embodiments of this application, the prediction block of the chromaticity block may include the prediction block of the chromaticity block of chromaticity component Cb and the prediction block of the chromaticity block of chromaticity component Cr.

[0292] In some embodiments of this application, the prediction mode for intra-frame prediction of chroma blocks in step 101 above can be a cross-component prediction mode.

[0293] For example, step 201 above can be implemented by step 201a below.

[0294] Step 201a: The decoder performs intra-frame prediction on the chroma block based on the reconstructed value of the luminance block corresponding to the chroma block, and obtains the prediction block of the chroma block.

[0295] In some embodiments of this application, the decoder may employ a traditional chroma intra-frame prediction mode or a non-traditional chroma intra-frame prediction mode. Based on the reconstructed value of the luminance block corresponding to the chroma block, intra-frame prediction is performed on the chroma block to obtain the prediction block of the chroma block. For example, the prediction block of the chroma block of chroma component Cb and the prediction block of the chroma block of chroma component Cr can be obtained. That is, the prediction block of the chroma block can be obtained.

[0296] In some embodiments of this application, the aforementioned conventional chroma intra-frame prediction modes may include, but are not limited to: PLANA mode, DC mode, and angle mode.

[0297] In some embodiments of this application, the aforementioned non-traditional chroma intra-frame prediction modes may include, but are not limited to: cross-component prediction mode and DBV mode.

[0298] It should be noted that for a detailed description of steps 201 and 201a above, please refer to the relevant descriptions of steps 101 and 101a above. To avoid repetition, they will not be repeated here.

[0299] Step 202: The decoder determines the transform kernel of the chroma block based on the prediction block.

[0300] In some embodiments of this application, the transform kernel of the chroma block described above can be used to optimize the coding efficiency of the chroma block in order to obtain a chroma that is more in line with human visual perception.

[0301] In some embodiments of this application, the decoder can determine the transform kernel of the chroma block based on the prediction block, provided that the current intra-frame prediction mode is the chroma intra-frame prediction mode and the current transform technique uses LFNST or NSPT.

[0302] In some embodiments of this application, after the decoder performs intra-frame prediction on the chroma block to obtain the prediction block of the chroma block, it can first determine the intra-frame prediction mode corresponding to the prediction block based on the prediction block, and then determine the transform kernel of the chroma block based on the intra-frame prediction mode.

[0303] It should be noted that, in the embodiments of this application, the intra-prediction mode corresponding to the prediction block of the chroma block determined by the decoder is a different concept from the intra-prediction mode used when performing intra-prediction on the chroma block. The intra-prediction mode corresponding to the prediction block of the chroma block can be used for selecting the transform kernel of the chroma block, while the intra-prediction mode used when performing intra-prediction on the chroma block is used for performing intra-prediction on the chroma block.

[0304] For example, step 202 above can be implemented by steps 202a and 202b below.

[0305] Step 202a: The decoder determines the first intra-frame prediction mode corresponding to the prediction block based on the prediction block.

[0306] In some embodiments of this application, the first intra-frame prediction mode described above can be an intra-frame prediction mode derived by the decoder based on the prediction block.

[0307] In some embodiments of this application, step 202a can be specifically implemented by the following steps 202a1 and 202a2.

[0308] Step 202a1: The decoder determines the HoG of the prediction block based on the gradient information of the prediction block.

[0309] The HoG of the aforementioned prediction block may include at least one gradient direction and at least one gradient magnitude.

[0310] In some embodiments of this application, the gradient direction described above can be used to indicate the angle of the angle prediction mode corresponding to the angle of a sample located in the non-edge region of the prediction block.

[0311] In some embodiments of this application, a gradient direction may correspond to a gradient magnitude.

[0312] In some embodiments of this application, the gradient magnitude described above can be used to indicate the cumulative intensity in its corresponding gradient direction.

[0313] In some embodiments of this application, the chroma block may include a first chroma block and a second chroma block, and the prediction block may include a first prediction block of the first chroma block and a second prediction block of the second chroma block.

[0314] The chromaticity components of the first chromaticity block and the second chromaticity block may be different.

[0315] In this case, step 202a1 can be implemented by steps 202a11 to 202a13 as described below.

[0316] Step 202a11: The decoder determines the first HoG of the first prediction block based on the gradient information of the first prediction block.

[0317] Step 202a12: The decoder determines the second HoG of the second prediction block based on the gradient information of the second prediction block.

[0318] Step 202a13: The decoder sums the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0319] In some embodiments of this application, after obtaining the first HoG and the second HoG, the decoder can accumulate the gradient magnitudes with the same gradient direction in the first HoG and the second HoG to obtain the HoG of the prediction block.

[0320] In this way, when the chroma block contains a first chroma block and a second chroma block, the decoder can first determine the first HoG of the first prediction block of the first chroma block and the second HoG of the second prediction block of the second chroma block, and then accumulate the two HoGs according to the gradient direction to obtain a HoG containing the information of the first chroma block and the information of the second chroma block, thereby making the intra-frame prediction mode determined based on the HoG more accurate.

[0321] Step 202a2: The decoder determines the intra-frame prediction mode based on the HoG of the prediction block.

[0322] In some embodiments of this application, after obtaining the HoG of the prediction block, the decoder can select a suitable gradient direction based on the gradient magnitude in the HoG, and thus determine the first intra-frame prediction mode based on the gradient direction.

[0323] In this way, the decoder can derive appropriate angle information based on the HoG of the prediction block, and then determine the appropriate transform kernel in subsequent steps based on the derived angle. This improves the accuracy of determining the transform kernel for the chroma block, thereby improving the transform effect of the chroma block.

[0324] In some embodiments of this application, step 202a2 can be specifically implemented by step 202a21 as described below.

[0325] Step 202a21: The decoder determines the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0326] Wherein, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among at least one gradient magnitude.

[0327] In some embodiments of this application, the maximum gradient magnitude among the at least one gradient magnitude can be represented as follows: in the HoG of the prediction block, the sample intensity change is most significant in the gradient direction corresponding to this maximum gradient magnitude. That is, the edge change in this gradient direction contributes the most to the image features. Therefore, the intra prediction mode corresponding to the maximum gradient magnitude among the at least one gradient magnitude is a more suitable intra prediction mode for selecting the transform kernel.

[0328] Thus, by determining the intra-prediction mode corresponding to the gradient direction of the maximum gradient magnitude in at least one gradient magnitude in the HoG of the prediction block as the first intra-prediction mode, the derived first intra-prediction mode can be a more suitable intra-prediction mode for selecting the transform kernel, thereby making the transform kernel selected based on the first intra-prediction mode more suitable.

[0329] In some embodiments of this application, step 202a21 described above can be implemented by step D as follows.

[0330] Step D: If the decoder has a non-zero gradient magnitude in at least one gradient magnitude, it determines the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode.

[0331] In some embodiments of this application, the transformation method provided in this application may further include the following step E.

[0332] Step E: When at least one gradient magnitude is 0, the decoder determines the preset intra-frame prediction mode as the first intra-frame prediction mode.

[0333] In some embodiments of this application, the presence of a non-zero gradient magnitude among the aforementioned at least one gradient magnitude can indicate that the image exhibits edge or texture transformation within that local region. Therefore, the decoder can determine the intra-frame prediction mode corresponding to the gradient direction of the maximum gradient magnitude as the first intra-frame prediction mode.

[0334] In some embodiments of this application, the fact that at least one gradient magnitude is 0 can indicate that the image has no edge or texture changes in the local region, that is, the local region is visually uniform, such as a flat region or image noise. Therefore, the decoder can directly determine the preset intra-frame prediction mode as the first intra-frame prediction mode.

[0335] Thus, for different cases where at least one gradient magnitude is 0, the decoder can determine the first intra-frame prediction mode in different ways, thereby ensuring that the decoder can determine the appropriate first intra-frame prediction mode in any case, and then determine the appropriate transform kernel based on the first intra-frame prediction mode.

[0336] Step 202b: The decoder determines the transform kernel of the chroma block based on the intra-frame prediction mode of the first frame.

[0337] In some embodiments of this application, when the chroma block transformation technique uses LFNST or NSPT, the decoder can determine the chroma block transformation kernel based on the derived first intra-frame prediction mode, using Table 1 above.

[0338] In this way, the decoder can derive appropriate angle information based on the predicted information of the chroma blocks, and then determine the appropriate transform kernel based on the derived angle. This improves the accuracy of determining the transform kernel for the chroma blocks, thereby improving the transform effect of the chroma blocks.

[0339] In some embodiments of this application, step 202b can be specifically implemented by the following steps 202b1 and 202b2.

[0340] Step 202b1: The decoder determines the first transform set based on the first intra-frame prediction mode and the first mapping relationship.

[0341] The first mapping relationship mentioned above can be the mapping relationship between the first intra-frame prediction mode and the first transform set.

[0342] Step 202b2: Based on the first index information in the bitstream, the decoder determines the transform kernel indicated by the first index information in the first transform set as the transform kernel of the chroma block.

[0343] In this way, the decoder can determine the transform set based on the mapping relationship between the intra-frame prediction mode and the transform set, and then determine a suitable transform kernel from the transform set for transformation, thereby improving the transformation effect of the chroma component.

[0344] It should be noted that detailed descriptions of steps 202, 202a, 202b, 202a1, 202a2, 202a11 to 202a13, 202a21, D, E, 202b1, and 202b2 can be found in the aforementioned description of the encoder side. To avoid repetition, these details will not be repeated here.

[0345] Step 203: The decoder performs an inverse transform on the first coefficients based on the transform kernel.

[0346] The first coefficient mentioned above can be the transformation coefficient of the chroma block; or, the first coefficient mentioned above can be obtained by inverse quantization of the quantization coefficient of the chroma block.

[0347] In some embodiments of this application, in the case of lossy compression, the decoder can first perform entropy decoding on the bitstream to obtain the quantization coefficients of the residual, and then perform dequantization (scaling) on ​​the quantization coefficients to obtain the first coefficients.

[0348] In some embodiments of this application, under lossless compression, the decoder can directly obtain the transform coefficients of the chroma block after entropy decoding of the bitstream. That is, when the decoder performs decoding in lossless mode, it can directly perform inverse transform on the transform coefficients of the chroma block to obtain the inverse transform coefficients of the chroma block, and then obtain the reconstructed block of the chroma block based on the inverse transform coefficients of the chroma block.

[0349] In this embodiment of the application, after the decoder receives the bitstream sent by the encoder, it can obtain the transform index identifier of the transform kernel and the first coefficient by parsing the bitstream. Then, based on the transform index identifier of the transform kernel, and if it is determined that the current block uses NSPT / LFNST, the decoder can determine the transform kernel indicated by the transform index identifier from the transform set determined according to the first intra-frame prediction mode, and perform an inverse transform on the first coefficient based on the transform kernel to obtain the inverse transform coefficient.

[0350] In this embodiment of the application, after the decoder obtains the inverse transform coefficients, the coefficients in the transform domain can be converted back to the sample values ​​in the spatial domain based on the inverse transform coefficients.

[0351] The transformation method provided in this application allows the decoder to determine the transform kernel of a chroma block by combining it with the prediction block of that chroma block. In other words, the decoder can combine the information of the chroma block to determine the transform kernel. Since the information of the chroma block better reflects the chroma components, a more suitable transform kernel can be determined based on this information. This improves the accuracy of determining the transform kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0352] Accordingly, this application provides a conversion device, which can be any conversion device in the video encoding process, and can be an encoder or a device within an encoder. Based on the above method example, the conversion device can be divided into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0353] With each functional module divided according to its corresponding function, Figure 17 shows a possible structural schematic diagram of the transformation device involved in the above embodiments. As shown in Figure 17, the transformation device 170 may include a prediction module 171 and a transformation module 172.

[0354] The prediction module 171 can be used to perform intra-frame prediction on the chroma block to obtain the prediction block of the chroma block; and determine the transform kernel of the chroma block based on the prediction block; the transformation module 172 can be used to transform the residual block or the transform block of the chroma block based on the transform kernel determined by the prediction module 171.

[0355] In one possible implementation, the prediction module 171 described above can be used to: determine the first intra-frame prediction mode corresponding to the prediction block based on the prediction block; and determine the transform kernel of the chroma block based on the first intra-frame prediction mode.

[0356] In one possible implementation, the prediction module 171 described above can be specifically used to: determine the HoG of the prediction block based on the gradient information of the prediction block, wherein the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude; and determine the prediction mode of the first frame based on the HoG of the prediction block.

[0357] In one possible implementation, the prediction module 171 can be specifically used to: determine the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode. Here, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among at least one gradient magnitude.

[0358] In one possible implementation, the prediction module 171 described above can be specifically used to: determine the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode when there is a non-zero gradient magnitude in at least one gradient magnitude.

[0359] In one possible implementation, the prediction module 171 can also be used to determine the preset intra-frame prediction mode as the first intra-frame prediction mode when at least one gradient magnitude is 0.

[0360] In one possible implementation, the chroma block may include a first chroma block and a second chroma block, and the prediction block may include a first prediction block of the first chroma block and a second prediction block of the second chroma block.

[0361] The prediction module 171 described above can be specifically used to: determine the first HoG of the first prediction block based on the gradient information of the first prediction block; determine the second HoG of the second prediction block based on the gradient information of the second prediction block; and accumulate the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0362] In one possible implementation, the prediction module 171 described above can be specifically used to: determine a first transform set based on a first intra-frame prediction mode and a first mapping relationship; and determine the transform kernel of the chroma components from the first transform set. The first mapping relationship is a mapping relationship between the first intra-frame prediction mode and the first transform set. The first mapping relationship is one of multiple mapping relationships, and each of the multiple mapping relationships is a mapping relationship between an intra-frame prediction mode and a transform set.

[0363] In one possible implementation, the transformation device 170 may further include an encoding module. This encoding module can be used to write first index information into the bitstream, the first index information being used to indicate the transformation kernel of the chroma block.

[0364] In one possible implementation, the transformation module 172 can also be used to perform an inverse transformation on the first coefficient using a transformation kernel; wherein the first coefficient is the transformation coefficient of the chroma block or the inverse quantization coefficient of the chroma block.

[0365] In one possible implementation, the prediction module 171 can be used to perform intra-frame prediction of the chroma block of the current block based on the reconstruction value of the luminance block corresponding to the chroma block, so as to obtain the prediction block of the chroma block.

[0366] This application provides a transformation device that, in determining the transformation kernel of a chroma block, can combine it with the prediction block of the chroma block. That is, the transformation device can determine the transformation kernel of the chroma block by combining the information of the chroma block. Since the information of the chroma block better reflects the chroma components, a more suitable transformation kernel can be determined based on the information of the chroma block. This improves the accuracy of determining the transformation kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0367] Each module of the above-mentioned transformation device can also be used to perform other actions in the above-mentioned method embodiments. All relevant content of each step involved in the above-mentioned method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0368] In the case of using integrated units, FIG18 shows another possible structural schematic diagram of the conversion device involved in the above embodiments. As shown in FIG18, the conversion device 180 provided in this application embodiment may include: a processing module 181 and a communication module 182. The processing module 181 can be used to control and manage the operation of the conversion device. For example, the processing module 181 can be used to support the conversion device in executing steps 101, 101a, 102, 102a, 102b, 102a1, 102a2, 102a11 to 102a13, 103 and 104 in the above method embodiments, and / or other processes used in the technology described herein. The communication module 182 can be used to support the communication of the conversion device with other network entities, such as communication with other conversion devices. In one example, as shown in FIG18, the conversion device may also include a storage module 183 for storing the program code and data of the conversion device, such as storing images or image blocks reconstructed by the encoder.

[0369] The processing module 181 can be a processor, such as the encoder 112 in Figure 2. The communication module 182 can be a transceiver, transceiver circuit, or communication interface, such as the communication interface 113 in Figure 1. The storage module 183 can be a memory.

[0370] For more details on how the modules included in the above-mentioned conversion device implement the above functions, please refer to the descriptions in the previous method embodiments, which will not be repeated here.

[0371] Accordingly, this application also provides a conversion device, which can be any conversion device in the video decoding process, and can be a decoder or a device within a decoder. Based on the above method examples, the conversion device can be divided into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, other division methods may be used.

[0372] With each functional module divided according to its corresponding function, Figure 19 shows a possible structural schematic diagram of the transformation device involved in the above embodiments. As shown in Figure 19, the transformation device 190 may include a prediction module 191 and a transformation module 192.

[0373] The prediction module 191 can be used to perform intra-frame prediction on the chroma block to obtain the prediction block of the chroma block; and determine the transform kernel of the chroma block based on the prediction block; the transform module 192 can be used to perform inverse transform on the first coefficient based on the transform kernel determined by the prediction module 191; wherein the first coefficient is the transform coefficient of the chroma block; or, the first coefficient is obtained by inverse quantization of the quantization coefficient of the chroma block.

[0374] In one possible implementation, the prediction module 191 described above can be used to: determine the first intra-frame prediction mode corresponding to the prediction block based on the prediction block; and determine the transform kernel of the chroma block based on the first intra-frame prediction mode.

[0375] In one possible implementation, the prediction module 191 described above can be specifically used to: determine the HoG of the prediction block based on the gradient information of the prediction block, wherein the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude; and determine the prediction mode of the first frame based on the HoG of the prediction block.

[0376] In one possible implementation, the prediction module 191 can specifically be used to determine the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode. Here, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among at least one gradient magnitude.

[0377] In one possible implementation, the prediction module 191 can be specifically used to determine the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode when there is a non-zero gradient magnitude in at least one gradient magnitude.

[0378] In one possible implementation, the prediction module 191 can also be used to determine the preset intra-frame prediction mode as the first intra-frame prediction mode when at least one gradient magnitude is 0.

[0379] In one possible implementation, the chroma block may include a first chroma block and a second chroma block, and the prediction block may include a first prediction block of the first chroma block and a second prediction block of the second chroma block. Specifically, the prediction module 191 may be used to: determine a first HoG of the first prediction block based on the gradient information of the first prediction block; determine a second HoG of the second prediction block based on the gradient information of the second prediction block; and accumulate the gradient magnitudes in the first HoG and the second HoG according to the gradient direction to obtain the HoG of the prediction block.

[0380] In one possible implementation, the prediction module 191 described above can be specifically used to: determine a first transform set based on a first intra-frame prediction mode and a first mapping relationship; and determine the transform kernel indicated by the first index information in the first transform set as the transform kernel of the chroma block based on the first index information in the bitstream. The first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set.

[0381] In one possible implementation, the prediction module 191 can be used to perform intra-frame prediction of the chroma block based on the reconstruction value of the luminance block corresponding to the chroma block, so as to obtain the prediction block of the chroma block.

[0382] This application provides a transformation device that, in determining the transformation kernel of a chroma block, can combine it with the prediction block of the chroma block. That is, the transformation device can determine the transformation kernel of the chroma block by combining the information of the chroma block. Since the information of the chroma block better reflects the chroma components, a more suitable transformation kernel can be determined based on the information of the chroma block. This improves the accuracy of determining the transformation kernel of the chroma block, thereby improving the transformation effect of the chroma block.

[0383] Each module of the above-mentioned transformation device can also be used to perform other actions in the above-mentioned method embodiments. All relevant content of each step involved in the above-mentioned method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0384] In the case of using integrated units, FIG20 shows another possible structural schematic diagram of the conversion device involved in the above embodiments. As shown in FIG20, the conversion device provided in the embodiments of this application may include: a processing module 201 and a communication module 202. The processing module 201 can be used to control and manage the operation of the conversion device. For example, the processing module 201 can be used to support the conversion device in executing steps 201, 201a, 202, 202a, 202b, 202a1, 202a2, 202a11 to 202a13, 203 and 204 in the above method embodiments, and / or other processes used in the technology described herein. The communication module 202 can be used to support the communication of the conversion device with other network entities, such as communication with other conversion devices. In one example, as shown in FIG20, the conversion device may also include a storage module 203 for storing the program code and data of the conversion device, such as storing decoded images or image blocks.

[0385] The processing module 201 can be a processor, such as the decoder 122 in Figure 3. The communication module 202 can be a transceiver, transceiver circuit, or communication interface, such as the communication interface 121 in Figure 1. The storage module 203 can be a memory.

[0386] For more details on how the modules included in the above-mentioned conversion device implement the above functions, please refer to the descriptions in the previous method embodiments, which will not be repeated here.

[0387] This application embodiment also provides an encoder, which can be the encoder 112 in FIG2, or the conversion device shown in FIG17 or 18, or the processor in the conversion device shown in FIG18.

[0388] This application embodiment also provides a decoder, which can be the decoder 122 in FIG3, or the conversion device shown in FIG19 or 20, or the processor in the conversion device shown in FIG20.

[0389] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they can implement the various processes of the above-described transformation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0390] The processor can be the encoder or decoder in the above embodiments, or it can be the processor within the encoder or decoder in the above embodiments. The readable storage medium can include a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0391] This application embodiment also provides a readable storage medium storing a bitstream, which is a bitstream generated based on the above-described encoder-side transformation method.

[0392] This application also provides a chip that may include a processor and a communication interface. The communication interface is coupled to the processor. The processor can be used to run programs or instructions to implement the various processes of the above-described transformation method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0393] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0394] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described transformation method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0395] It should be noted that the scope of the methods and apparatus in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0396] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0397] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0398] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0399] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0400] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0401] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0402] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

A transformation method applied to an encoder, characterized in that, include: Intra-frame prediction is performed on the chroma block to obtain the prediction block of the chroma block; Based on the prediction block, the transform kernel of the chroma block is determined; Based on the transformation kernel, the residual block of the chroma block or the transformation block of the chroma block is transformed. The method according to claim 1, characterized in that, Determining the transform kernel of the chroma block based on the prediction block includes: Based on the prediction block, determine the first intra-frame prediction mode corresponding to the prediction block; Based on the first intra-frame prediction mode, the transform kernel of the chroma block is determined. The method according to claim 2, characterized in that, The step of determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block includes: Based on the gradient information of the prediction block, the gradient histogram HoG of the prediction block is determined, and the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude. Based on the HoG of the prediction block, the first intra-frame prediction mode is determined. The method according to claim 3, characterized in that, Determining the first intra-frame prediction mode based on the HoG of the prediction block includes: The intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. Wherein, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among the at least one gradient magnitude. The method according to claim 4, characterized in that, The step of determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode includes: If there is a non-zero gradient magnitude among the at least one gradient magnitude, the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. The method according to claim 5, characterized in that, The method further includes: When at least one gradient magnitude is 0, the preset intra-frame prediction mode is determined as the first intra-frame prediction mode. According to any one of claims 3 to 6, the chroma block includes a first chroma block and a second chroma block, and the prediction block includes a first prediction block of the first chroma block and a second prediction block of the second chroma block; Determining the HoG of the prediction block based on the gradient information of the prediction block includes: Based on the gradient information of the first prediction block, determine the first HoG of the first prediction block; Based on the gradient information of the second prediction block, the second HoG of the second prediction block is determined; The gradient magnitudes in the first HoG and the second HoG are summed according to the gradient direction to obtain the HoG of the prediction block. The method according to any one of claims 2 to 7, characterized in that, Determining the transform kernel of the chroma block based on the first intra-frame prediction mode includes: Based on the first intra-frame prediction mode and the first mapping relationship, a first transform set is determined, wherein the first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set; Determine the transform kernel of the chroma block from the first transform set; Wherein, the first mapping relationship is one of a plurality of mapping relationships, and each of the plurality of mapping relationships is a mapping relationship between an intra-frame prediction mode and a transform set. The method according to claim 1, characterized in that, The method further includes: The first index information is written into the bitstream, and the first index information is used to indicate the transform kernel of the chroma block. The method according to claim 1, characterized in that, The method further includes: The first coefficients are inversely transformed using the aforementioned transformation kernel; Wherein, the first coefficient is either the transformation coefficient of the chroma block or the inverse quantization coefficient of the chroma block. The method according to any one of claims 1 to 10, characterized in that, The step of performing intra-frame prediction on the chroma block to obtain the prediction block of the chroma block includes: Based on the reconstructed value of the luminance block corresponding to the chroma block, intra-frame prediction is performed on the chroma block to obtain the prediction block of the chroma block. A transformation method applied to a decoder, characterized in that, include: Intra-frame prediction is performed on the chroma block to obtain the prediction block of the chroma block; Based on the prediction block, the transform kernel of the chroma block is determined; Based on the transformation kernel, the first coefficients are subjected to an inverse transformation; Wherein, the first coefficient is the transformation coefficient of the chroma block; or, the first coefficient is obtained by inverse quantization of the quantization coefficient of the chroma block. The method according to claim 12, characterized in that, Determining the transform kernel of the chroma block based on the prediction block includes: Based on the prediction block, determine the first intra-frame prediction mode corresponding to the prediction block; Based on the first intra-frame prediction mode, the transform kernel of the chroma block is determined. The method according to claim 13, characterized in that, The step of determining the first intra-frame prediction mode corresponding to the prediction block based on the prediction block includes: Based on the gradient information of the prediction block, the gradient histogram HoG of the prediction block is determined, and the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude. Based on the HoG of the prediction block, the first intra-frame prediction mode is determined. The method according to claim 14, characterized in that, Determining the first intra-frame prediction mode based on the HoG of the prediction block includes: The intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. Wherein, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among the at least one gradient magnitude. The method according to claim 15, characterized in that, The step of determining the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-prediction mode includes: If there is a non-zero gradient magnitude among the at least one gradient magnitude, the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. The method according to claim 16, characterized in that, The method further includes: When at least one gradient magnitude is 0, the preset intra-frame prediction mode is determined as the first intra-frame prediction mode. The method according to any one of claims 14 to 17, wherein the chroma block comprises a first chroma block and a second chroma block, and the prediction block comprises a first prediction block of the first chroma block and a second prediction block of the second chroma block; Determining the HoG of the prediction block based on the gradient information of the prediction block includes: Based on the gradient information of the first prediction block, determine the first HoG of the first prediction block; Based on the gradient information of the second prediction block, the second HoG of the second prediction block is determined; The gradient magnitudes in the first HoG and the second HoG are summed according to the gradient direction to obtain the HoG of the prediction block. The method according to any one of claims 13 to 18, characterized in that, Determining the transform kernel of the chroma block based on the first intra-frame prediction mode includes: Based on the first intra-frame prediction mode and the first mapping relationship, a first transform set is determined, wherein the first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set; Based on the first index information in the bitstream, the transform kernel indicated by the first index information in the first transform set is determined as the transform kernel of the chroma block. The method according to any one of claims 12 to 19, characterized in that, The step of performing intra-frame prediction on the chroma block to obtain the prediction block of the chroma block includes: Based on the reconstructed value of the luminance block corresponding to the chroma block, intra-frame prediction is performed on the chroma block to obtain the prediction block of the chroma block. A conversion device, characterized in that, include: Prediction module and transformation module; The prediction module is used to perform intra-frame prediction on the chroma block to obtain the prediction block of the chroma block; The prediction module is further configured to determine the transformation kernel of the chroma block based on the prediction block; The transformation module is used to transform the residual block of the chroma block or the transformation block of the chroma block based on the transformation kernel. The apparatus according to claim 21 is characterized in that, The prediction module is specifically used for: Based on the prediction block, determine the first intra-frame prediction mode corresponding to the prediction block; Based on the first intra-frame prediction mode, the transform kernel of the chroma block is determined. The apparatus according to claim 22 is characterized in that, The prediction module is specifically used for: Based on the gradient information of the prediction block, the gradient histogram HoG of the prediction block is determined, and the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude. Based on the HoG of the prediction block, the first intra-frame prediction mode is determined. The apparatus according to claim 23 is characterized in that, The prediction module is specifically used for: The intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. Wherein, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among the at least one gradient magnitude. The apparatus according to claim 24 is characterized in that, The prediction module is specifically used for: If there is a non-zero gradient magnitude among the at least one gradient magnitude, the intra-prediction mode corresponding to the first gradient direction in the HoG of the prediction block is determined as the first intra-prediction mode. The apparatus according to claim 25 is characterized in that, The prediction module is further configured to determine the preset intra-frame prediction mode as the first intra-frame prediction mode when the at least one gradient magnitude is 0. The apparatus according to any one of claims 23 to 26, wherein the chroma block comprises a first chroma block and a second chroma block, and the prediction block comprises a first prediction block of the first chroma block and a second prediction block of the second chroma block; The prediction module is specifically used for: Based on the gradient information of the first prediction block, determine the first HoG of the first prediction block; Based on the gradient information of the second prediction block, the second HoG of the second prediction block is determined; The gradient magnitudes in the first HoG and the second HoG are summed according to the gradient direction to obtain the HoG of the prediction block. The apparatus according to any one of claims 22 to 27 is characterized in that, The prediction module is specifically used for: Based on the first intra-frame prediction mode and the first mapping relationship, a first transform set is determined, wherein the first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set; Determine the transform kernel of the chroma block from the first transform set; Wherein, the first mapping relationship is one of a plurality of mapping relationships, and each of the plurality of mapping relationships is a mapping relationship between an intra-frame prediction mode and a transform set. The apparatus according to claim 21 is characterized in that, The device further includes: an encoding module; The encoding module is used to write first index information into the bitstream, and the first index information is used to indicate the transform kernel of the chroma block. The apparatus according to claim 21 is characterized in that, The transformation module is further configured to perform an inverse transformation on the first coefficient using the transformation kernel; Wherein, the first coefficient is either the transformation coefficient of the chroma block or the inverse quantization coefficient of the chroma block. The apparatus according to any one of claims 21 to 30 is characterized in that, The prediction module is specifically used to perform intra-frame prediction on the chroma block based on the reconstruction value of the luminance block corresponding to the chroma block, so as to obtain the prediction block of the chroma block. A conversion device, characterized in that, include: Prediction module and transformation module; The prediction module is used to perform intra-frame prediction on the chroma block to obtain the prediction block of the chroma block; The prediction module is further configured to determine the transformation kernel of the chroma block based on the prediction block; The transformation module is used to perform an inverse transformation on the first coefficients based on the transformation kernel; Wherein, the first coefficient is the transformation coefficient of the chroma block; or, the first coefficient is obtained by inverse quantization of the quantization coefficient of the chroma block. The apparatus according to claim 32 is characterized in that, The prediction module is specifically used for: Based on the prediction block, determine the first intra-frame prediction mode corresponding to the prediction block; Based on the first intra-frame prediction mode, the transform kernel of the chroma block is determined. The apparatus according to claim 33 is characterized in that, The prediction module is specifically used for: Based on the gradient information of the prediction block, the gradient histogram HoG of the prediction block is determined, and the HoG of the prediction block includes at least one gradient direction and at least one gradient magnitude. Based on the HoG of the prediction block, the first intra-frame prediction mode is determined. The apparatus according to claim 34 is characterized in that, The prediction module is specifically used to determine the intra-frame prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-frame prediction mode. Wherein, the first gradient direction is the gradient direction corresponding to the first gradient magnitude, and the first gradient magnitude is the maximum gradient magnitude among the at least one gradient magnitude. The apparatus according to claim 35 is characterized in that, The prediction module is specifically used to determine the intra-frame prediction mode corresponding to the first gradient direction in the HoG of the prediction block as the first intra-frame prediction mode when there is a non-zero gradient magnitude in the at least one gradient magnitude. The apparatus according to claim 36 is characterized in that, The prediction module is further configured to determine the preset intra-frame prediction mode as the first intra-frame prediction mode when the at least one gradient magnitude is 0. The apparatus according to any one of claims 34 to 37, wherein the chroma block comprises a first chroma block and a second chroma block, and the prediction block comprises a first prediction block of the first chroma block and a second prediction block of the second chroma block; The prediction module is specifically used for: Based on the gradient information of the first prediction block, determine the first HoG of the first prediction block; Based on the gradient information of the second prediction block, the second HoG of the second prediction block is determined; The gradient magnitudes in the first HoG and the second HoG are summed according to the gradient direction to obtain the HoG of the prediction block. The apparatus according to any one of claims 33 to 38 is characterized in that, The prediction module is specifically used for: Based on the first intra-frame prediction mode and the first mapping relationship, a first transform set is determined, wherein the first mapping relationship is the mapping relationship between the first intra-frame prediction mode and the first transform set; Based on the first index information in the bitstream, the transform kernel indicated by the first index information in the first transform set is determined as the transform kernel of the chroma block. The apparatus according to any one of claims 32 to 39 is characterized in that, The prediction module is specifically used to perform intra-frame prediction on the chroma block based on the reconstruction value of the luminance block corresponding to the chroma block, so as to obtain the prediction block of the chroma block. An encoder, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the transformation method as described in any one of claims 1 to 11. A decoder, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the transformation method as described in any one of claims 12 to 20. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the transformation method as described in any one of claims 1 to 11, or implement the steps of the transformation method as described in any one of claims 12 to 20. A readable storage medium, characterized in that, The readable storage medium stores a bitstream, which is a bitstream generated based on the transformation method described in any one of claims 1 to 11. A computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the transformation method as claimed in any one of claims 1 to 11, or to implement the steps of the transformation method as claimed in any one of claims 12 to 20.

Images