A video encoding and decoding method, apparatus, electronic device, and storage medium

Abstract

This invention provides a video encoding and decoding method, apparatus, electronic device, and storage medium, relating to the field of video processing technology. Specifically, the technical solution involves obtaining secondary transform identifier information corresponding to video frames in a video to be encoded. This secondary transform identifier information indicates whether a secondary transform is needed for the encoded block of the video frame. If the secondary transform identifier information indicates that a secondary transform is not needed for the encoded block of the video frame, then the low-frequency inseparable transform for the encoded block is skipped, and subsequent encoding processes are performed. The video encoding and decoding method, apparatus, electronic device, and storage medium provided by this invention can reduce the complexity of video encoding.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a video encoding and decoding method, apparatus, electronic device, and storage medium. Background Technology

[0002] In video coding, transform is a crucial tool for removing spatial redundancy from prediction residuals. The spatial redundancy characteristics of prediction residuals are related to the prediction mode, especially in intra-frame coding modes where the prediction residuals exhibit directionality when using directional prediction modes. After the primary transform on the prediction residuals, some low-frequency coefficients may still exhibit predictable patterns. H.266 / VVC (a next-generation international video coding standard) introduces a secondary transform after the primary transform, called the Low-frequency Non-separable Transform (LFNST), to further process these low-frequency coefficients that still exhibit predictable patterns. The transform coefficients after the secondary transform are then quantized to achieve video coding. While the secondary transform further removes redundancy, it complicates the entire video coding process. Summary of the Invention

[0003] The purpose of this invention is to provide a video encoding and decoding method, apparatus, electronic device, and storage medium to reduce the complexity of video encoding. The specific technical solution is as follows:

[0004] In a first aspect of this invention, a video encoding method is provided, comprising:

[0005] Obtain secondary transformation identifier information corresponding to video frames in the video to be encoded. The secondary transformation identifier information is used to identify whether a secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined based on the image content of the video frame.

[0006] If the secondary transformation identifier indicates that a secondary transformation is not required for the coding block of the video frame, then the low-frequency indivisible transformation for the coding block is skipped and the subsequent coding process is performed.

[0007] Optionally, the method further includes:

[0008] Obtain video frames from the video to be encoded;

[0009] Calculate the image gradient of the video frame based on the pixel values ​​of the video frame;

[0010] If the image gradient is greater than a preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is not required for the coding block of the video frame.

[0011] If the image gradient is not greater than a preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation needs to be performed on the coded block of the video frame.

[0012] Optionally, calculating the image gradient of the video frame based on the pixel values ​​of the video frame includes:

[0013] For each row of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences corresponding to each row.

[0014] The horizontal gradient of the video frame is obtained by summing the differences between each row.

[0015] For each column of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences corresponding to each column.

[0016] The vertical gradient of the video frame is obtained by summing the differences in each column.

[0017] The image gradient of the video frame is calculated based on the horizontal gradient and the vertical gradient.

[0018] Optionally, after obtaining the quadratic transform identifier information corresponding to the video frame, the method further includes:

[0019] Obtain the main transform coefficients corresponding to the coded block, wherein the main transform coefficients are obtained by performing a main transform on the coded block;

[0020] If the secondary transform identifier indicates that a secondary transform is required for the coded block of the video frame, then the main transform coefficients are subjected to a low-frequency inseparable transform.

[0021] Optionally, the method further includes:

[0022] A flag bit is added to the coded block, which is used to identify whether a low-frequency indivisible transform was performed on the coded block during the video encoding process.

[0023] In a second aspect of the invention, a video decoding method is also provided, comprising:

[0024] Obtain the flag bits corresponding to the encoded block;

[0025] If the flag indicates that no low-frequency inseparable transform was performed on the coded block during video encoding, then the inverse low-frequency inseparable transform is skipped after dequantization of the encoding result of the coded block, and the subsequent decoding process is performed. The flag is added to the coded block after obtaining the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that a secondary transform is not required for the coded block of the video frame, the low-frequency inseparable transform for the coded block is skipped and the subsequent encoding process is performed. The secondary transform identifier information is used to indicate whether a secondary transform is required for the coded block of the video frame, and the secondary transform identifier information is determined based on the image content of the video frame.

[0026] In a third aspect of the invention, a video encoding apparatus is also provided, comprising:

[0027] The first acquisition module is used to acquire secondary transformation identifier information corresponding to video frames in the video to be encoded. The secondary transformation identifier information is used to identify whether secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined according to the image content of the video frame.

[0028] The skip module is used to skip the low-frequency indivisible transform for the coding block and proceed with the subsequent coding process if the secondary transform identifier information indicates that a secondary transform is not required for the coding block of the video frame.

[0029] Optionally, the device further includes:

[0030] The second acquisition module is used to acquire video frames in the video to be encoded;

[0031] The calculation module is used to calculate the image gradient of the video frame based on the pixel values ​​of the video frame;

[0032] The determining module is configured to determine that if the image gradient is greater than a preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is not required for the coding block of the video frame; and if the image gradient is not greater than the preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is required for the coding block of the video frame.

[0033] Optionally, the calculation module is specifically configured to: calculate the absolute value of the difference between each pair of adjacent pixel values ​​for each row of the video frame, and sum the absolute values ​​of the differences between each pair of adjacent pixel values ​​to obtain the sum of the differences for each row; sum the sums of the differences for each row to obtain the horizontal gradient of the video frame; calculate the absolute value of the difference between each pair of adjacent pixel values ​​for each column of the video frame, and sum the absolute values ​​of the differences between each pair of adjacent pixel values ​​to obtain the sum of the differences for each column; sum the sums of the differences for each column to obtain the vertical gradient of the video frame; and calculate the image gradient of the video frame based on the horizontal gradient and the vertical gradient.

[0034] Optionally, the device further includes:

[0035] The third acquisition module is used to acquire the main transform coefficients corresponding to the coding block after acquiring the secondary transform identifier information corresponding to the video frame, wherein the main transform coefficients are obtained by performing a main transform on the coding block;

[0036] The transformation module is used to perform a low-frequency indivisible transformation on the main transformation coefficients if the secondary transformation identification information indicates that a secondary transformation is required for the coding block of the video frame.

[0037] Optionally, the device further includes:

[0038] An addition module is used to add a flag bit to the coding block, the flag bit being used to identify whether a low-frequency indivisible transform was performed on the coding block during the video encoding process.

[0039] In a fourth aspect of the invention, a video decoding apparatus is also provided, comprising:

[0040] The acquisition module is used to acquire the flag bits corresponding to the encoded block;

[0041] The decoding module is configured to skip the inverse low-frequency inseparable transform and proceed with subsequent decoding after inverse quantization of the encoding result of the encoding block if the flag bit indicates that no low-frequency inseparable transform was performed on the encoding block during video encoding. The flag bit is added to the encoding block after obtaining the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that a secondary transform is not required for the encoding block of the video frame, the module skips the low-frequency inseparable transform and proceeds with subsequent encoding. The secondary transform identifier information is used to indicate whether a secondary transform is required for the encoding block of the video frame, and the secondary transform identifier information is determined based on the image content of the video frame.

[0042] In a fifth aspect of the present invention, an electronic device is also provided, characterized in that it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0043] Memory, used to store computer programs;

[0044] A processor, when executing a program stored in memory, implements the steps of the method described in either the first or second aspect.

[0045] In a sixth aspect of the present invention, a computer-readable storage medium is also provided, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in either the first or second aspect.

[0046] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of the method described in either the first or second aspect.

[0047] The video encoding and decoding method, apparatus, electronic device, and storage medium provided in this invention can skip the low-frequency indivisible transformation of the encoding block and proceed with the subsequent encoding process when the secondary transformation identification information indicates that a secondary transformation is not required for the encoding block of the video frame. This can reduce the complexity of video encoding. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0049] Figure 1 A flowchart of a video encoding method provided in an embodiment of the present invention;

[0050] Figure 2 This is a flowchart illustrating the determination of secondary transformation identifier information in an embodiment of the present invention;

[0051] Figure 3 A flowchart of a video decoding method provided in an embodiment of the present invention;

[0052] Figure 4 A schematic diagram illustrating the video encoding and video decoding methods provided in the embodiments of the present invention;

[0053] Figure 5 This is a schematic diagram of a specific embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the transformation and inverse transformation in an embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram of a video encoding device provided in an embodiment of the present invention;

[0056] Figure 8 This is another structural schematic diagram of the video encoding device provided in an embodiment of the present invention;

[0057] Figure 9 This is another structural schematic diagram of the video encoding device provided in an embodiment of the present invention;

[0058] Figure 10 This is another structural schematic diagram of the video encoding device provided in an embodiment of the present invention;

[0059] Figure 11 This is a schematic diagram of the structure of a video decoding device provided in an embodiment of the present invention;

[0060] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0062] To further remove redundancy in the transform domain, H.266 / VVC introduces a quadratic transform technique, which employs a low-frequency inseparable transform. The quadratic transform is only applied to intra-frame modes, and in the case of a Discrete Cosine Transform (DCT), such as a two-dimensional DCT (DCT2), only the low-frequency components of the main transform coefficients are processed. Versatile Video Coding (VVC) can save 30% of the bitrate compared to High Efficiency Video Coding (HEVC), which is currently one of the most efficient coding standards. However, its complexity is also 10 times that of HEVC. To reduce the complexity of the video coding process, this invention provides a video coding method.

[0063] The video encoding method provided in this invention can be applied to electronic devices, specifically servers, terminals, encoders, etc.

[0064] This invention provides a video encoding method, which may include:

[0065] Obtain the secondary transformation identifier information corresponding to the video frame in the video to be encoded. The secondary transformation identifier information is used to identify whether a secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined based on the image content of the video frame.

[0066] If the secondary transform identifier indicates that a secondary transform is not required for the coding block of the video frame, then the low-frequency indivisible transform for the coding block is skipped and the subsequent coding process is performed.

[0067] In this embodiment of the invention, if the secondary transformation identifier does not require secondary transformation of the coding block of the video frame, the low-frequency indivisible transformation of the coding block is skipped and the subsequent coding process is performed, which can reduce the complexity of video coding.

[0068] Figure 1 A flowchart illustrating a video encoding method provided in an embodiment of the present invention. (Refer to...) Figure 1 The video encoding method provided in this embodiment of the invention may include:

[0069] S101, Obtain the secondary transformation identifier information corresponding to the video frame in the video to be encoded. The secondary transformation identifier information is used to identify whether a secondary transformation is needed for the encoding block of the video frame. The secondary transformation identifier information is determined according to the image content of the video frame.

[0070] S102, if the secondary transformation identifier indicates that a secondary transformation is not required for the coding block of the video frame, then the low-frequency indivisible transformation for the coding block is skipped and the subsequent coding process is performed.

[0071] Simply put, it means determining whether a secondary transformation is needed for the encoded blocks of a video frame based on the image content of the video frame itself.

[0072] In one possible implementation, the secondary transformation identifier information can be determined based on the pixel values ​​of the video frame, etc.

[0073] In this embodiment of the invention, secondary transform identifier information can be predetermined. For example, before video encoding, the secondary transform identifier information of each video frame in the video to be encoded is determined, and the secondary transform identifier information corresponding to each video frame is saved. For example, the video to be encoded includes video frame 1, video frame 2, and video frame 3. After determining the secondary transform identifier information of each video frame, the secondary transform identifier information corresponding to each video frame is saved. For example, identifier information 1 indicates that a secondary transform is required for the coding block of the video frame, and identifier information 2 indicates that a secondary transform is not required for the coding block of the video frame. Video frame 1 corresponds to identifier information 1, video frame 2 corresponds to identifier information 2, and video frame 3 corresponds to identifier information 2. In this way, during the video encoding process, the predetermined secondary transform identifier information can be directly obtained, and it can be determined whether to skip the low-frequency inseparable transform for the coding block and proceed with the subsequent encoding process based on the secondary transform identifier information. If the secondary transform identifier information indicates that a secondary transform is not required for the coding block of the video frame, then the low-frequency inseparable transform for the coding block is skipped and the subsequent encoding process proceeds.

[0074] In this embodiment of the invention, the secondary transform identifier information corresponding to the video frame can also be determined in real time during the video encoding process. For example, after the main transform is performed on the coding block, the secondary transform identifier information corresponding to the video frame in the video to be encoded is first determined. Then, based on the secondary transform identifier information, it is determined whether to skip the low-frequency inseparable transform for the coding block and proceed with the subsequent encoding process. If the secondary transform identifier information indicates that the coding block of the video frame does not need to be subjected to secondary transform, then the low-frequency inseparable transform for the coding block is skipped and the subsequent encoding process is proceeded.

[0075] In addition, the high complexity of video encoding can lead to time-consuming video encoding, as well as high computational resources and costs. The video encoding method provided in this embodiment of the invention reduces the complexity of the video encoding process, thereby shortening the video encoding time, reducing the consumption of computational resources, and reducing computational costs.

[0076] In one implementation, the secondary transformation identifier information can be a condition indicating whether a secondary transformation is required for the coded blocks of a video frame. For example, the secondary transformation identifier information may include condition 1 and condition 2, where condition 1 indicates that a secondary transformation is not required for the coded blocks of the video frame, and condition 2 indicates that a secondary transformation is required for the coded blocks of the video frame. This form of secondary transformation identifier information can be used in the process of determining the secondary transformation identifier information corresponding to a video frame in real time during video encoding in this embodiment of the invention.

[0077] In another possible implementation, the secondary transformation identifier can be a value determined based on whether a secondary transformation is needed for the coded blocks of a video frame. For example, it can be text, numbers, letters, etc. Different values ​​can be used to distinguish whether a secondary transformation is needed for the coded blocks of a video frame. For example, an identifier value of "1" indicates that a secondary transformation is needed for the coded blocks of the video frame; an identifier value of "0" indicates that a secondary transformation is not needed. Similarly, an identifier value of "on" indicates that a secondary transformation is needed for the coded blocks of the video frame; an identifier value of "off" indicates that a secondary transformation is not needed. Simply put, "on" indicates that a secondary transformation is enabled for the coded blocks of the video frame, meaning that a secondary transformation is needed after the main transformation. "off" indicates that a secondary transformation is disabled for the coded blocks of the video frame, meaning that a secondary transformation is not needed after the main transformation.

[0078] In video encoding, video frames are typically divided into encoding blocks. Each video frame in the video to be encoded corresponds to a quadratic transform identifier. During the encoding of a transform block within a video frame, the quadratic transform identifier corresponding to that video frame is obtained. Based on this identifier, it is determined whether to skip the low-frequency inseparable transform for the encoding block. In other words, it is determined whether to perform the low-frequency inseparable transform for the encoding block based on this identifier.

[0079] It is understood that the secondary transform identifier information in this embodiment of the invention is determined for video frames. During video encoding, whether the coding blocks of the same video frame undergo low-frequency inseparable transform should be the same. For example, if video frame 1 includes coding blocks 10, 11, and 12, then coding blocks 10, 11, and 12 either all undergo low-frequency inseparable transform or all skip the low-frequency inseparable transform. Specifically, when encoding coding blocks of the same video frame, the secondary transform identifier information of that video frame is obtained. Simply put, this embodiment of the invention performs or skips low-frequency inseparable transform on a frame-by-frame basis; figuratively speaking, this embodiment of the invention enables or disables low-frequency inseparable transform on a frame-by-frame basis.

[0080] If the secondary transformation identifier indicates that a secondary transformation needs to be performed on the coded block of the video frame, then a low-frequency indivisible transformation is performed on the coded block.

[0081] In an optional embodiment, after obtaining the quadratic transform identifier information corresponding to the video frame, the method further includes:

[0082] Obtain the main transform coefficients corresponding to the coding block, where the main transform coefficients are obtained by performing the main transform on the coding block; if the second transform identification information indicates that a second transform needs to be performed on the coding block of the video frame, perform a low-frequency non-separable transform on the main transform coefficients.

[0083] Judge whether to perform a low-frequency non-separable transform on the coding block according to the second transform identification information. If the second transform identification information indicates that a second transform does not need to be performed on the coding block of the video frame, skip the low-frequency non-separable transform on the coding block and perform the subsequent coding process; if the second transform identification information indicates that a second transform needs to be performed on the coding block of the video frame, first perform a low-frequency non-separable transform on the coding block and then perform the subsequent coding process.

[0084] Among them, the transform method adopted for the main transform can be determined according to the actual transform requirements. For example, in the VVC coding process, the DCT performed after prediction is used.

[0085] The specific process of the low-frequency non-separable transform is as follows:

[0086] The low-frequency non-separable transform only processes the low-frequency part of the main transform coefficients, so the high-frequency coefficients in the remaining main transform coefficients are set to 0. In one implementable way, for the 4*4 type low-frequency non-separable transform, the values in the upper left 4*4 area of the main transform matrix are retained, and the remaining values are set to 0, and 16 main transform coefficients are input.

[0087] When performing the low-frequency non-separable transform, the coefficient matrix of the 4*4 type low-frequency non-separable transform is

[0088]

[0089] Expand the matrix X into a one-dimensional vector Get

[0090]

[0091] To obtain the transformed transform coefficient vector Perform a non-separable transform Among them, T is a transform matrix with a size of 16*16. In the subsequent coding process, entropy coding can Reorganize into 4*4 blocks. In order to reduce the computational complexity and the space required to store the transform matrix, the 8*8 type low-frequency non-separable transform adopts the reduced non-separable transform (RT). The core idea of the reduced non-separable transform is to map an N-dimensional vector to an R-dimensional vector in a different space, where R < N and R / N is the reduction factor. Therefore, the reduced non-separable transform uses a transform matrix with a size of R*N, and the form is as follows.

[0092]

[0093] For example, in an 8x8 RT, N=48, R=16, the transformation matrix T r×N The value in the matrix can be predetermined, and the transformation matrix can be directly obtained during the low-frequency inseparable transformation process.

[0094] In H.266 / VVC, the 4x4 type low-frequency inseparable transform does not use the simplified inseparable transform, therefore the transform matrix size is 16x16. For the 8x8 type low-frequency inseparable transform, a simplified inseparable transform technique with a reduction factor of 1 / 3 is used, reducing the transform matrix size from 48x48 to 16x48. For the 8x8 type low-frequency inseparable transform, 48 principal transform coefficients are input, resulting in 16 secondary transform coefficients after the simplified inseparable transform. These secondary transform coefficients are then used in subsequent encoding processes to encode the video to be encoded.

[0095] Unlike DCT and Discrete Sine Transform (DST), which are derived mathematically directly from expressions, the transform kernels of LFNST are obtained through offline training. Ultimately, a method of using four transform sets, each containing two transform kernels, was adopted.

[0096] In this embodiment of the invention, based on the secondary transform identifier information, the low-frequency inseparable transform is directly skipped for the encoded blocks of video frames that do not require secondary transform. Specifically, during the video encoding process, quantization, entropy coding, and other subsequent encoding processes are performed directly after the main transform. For encoded blocks of video frames that require secondary transform, the low-frequency inseparable transform is performed first after the main transform, followed by quantization, entropy coding, and other subsequent encoding processes. Thus, directly skipping the low-frequency inseparable transform for encoded blocks of video frames that do not require secondary transform reduces the complexity of video encoding, while performing the low-frequency inseparable transform on encoded blocks of video frames that require secondary transform reduces the impact on the video encoding effect.

[0097] In this embodiment of the invention, if the secondary transform identifier indicates that a secondary transform is not required for the coded block of the video frame, then skipping the low-frequency inseparable transform for the coded block and proceeding with the subsequent encoding process accelerates the transform process. This can also be understood as a method to accelerate the secondary transform process. Furthermore, in this embodiment of the invention, the secondary transform identifier information corresponding to the video frame is determined based on the image content of the video frame, and then the decision to skip the low-frequency inseparable transform is made based on the secondary transform identifier information. This can also be understood as a fast method for accelerating the secondary transform process based on video content.

[0098] The secondary transformation identifier information of a video frame can be determined based on the image gradient of the video frame. The secondary transformation identifier information is used to indicate whether the image gradient of the video frame is not greater than a preset gradient threshold. If the image gradient is greater than the preset gradient threshold, it is used to indicate that no secondary transformation is required for the coding block of the video frame. If the image gradient is not greater than the preset gradient threshold, it is used to indicate that a secondary transformation is required for the coding block of the video frame.

[0099] In one alternative embodiment, such as Figure 2 As shown, the video encoding method provided in this embodiment of the invention may further include:

[0100] S201, Obtain video frames from the video to be encoded;

[0101] S202, calculates the image gradient of the video frame based on the pixel values ​​of the video frame.

[0102] For each row of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences for each row. The sum of the differences for each row is then added together to obtain the horizontal gradient of the video frame. For each column of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences for each column. The sum of the differences for each column is then added together to obtain the vertical gradient of the video frame. Based on the horizontal and vertical gradients, the image gradient of the video frame is calculated.

[0103] The horizontal gradient and the vertical gradient can be summed, and the summed value can be divided by the number of pixel values ​​in the video frame to obtain the image gradient of that video frame.

[0104] For simplicity, we'll use an 8x8 video frame size as an example. In reality, video frame sizes are generally larger, such as 1920x1080, 1280x720, etc.

[0105] For example, the pixel values ​​of a video frame are as follows:

[0106] A00 A01 A02 A03 A04 A05 A06 A07;

[0107] A10 A11 A12 A13 A14 A15 A16 A17;

[0108] A20 A21 A22 A23 A24 A25 A26 A27;

[0109] A30 A31 A32 A33 A34 A35 A36 A37;

[0110] A40 A41 A42 A43 A44 A45 A46 A47;

[0111] A50 A51 A52 A53 A54 A55 A56 A57;

[0112] A60 A61 A62 A63 A64 A65 A66 A67;

[0113] A70 A71 A72 A73 A74 A75 A76 A77;

[0114] Calculate the horizontal gradient grad_hor based on the pixel values ​​of the video frames:

[0115] grad_hor=|A01-A00|+|A02-A01|+…+|A07-A06|+|A11-A10|+|A12-A11|+…+|A17-A16|+…|A71-A70|+|A72-A71|+…+|A77-A76|;

[0116] Calculate the vertical gradient grad_ver based on the pixel values ​​of the video frames:

[0117] grad_ver=|A10-A00|+|A11-A01|+…+|A17-A07|+|A20-A10|+|A21-A11|+…+|A27-A17|+…|A70-A60|+|A71-A61|+…+|A77-A67|;

[0118] After obtaining the horizontal and vertical gradients, the image gradient of the video frame is calculated based on these gradients.

[0119] Image gradient = (grad_hor + grad_ver) / 64;

[0120] Image gradient can also be understood as the average gradient value.

[0121] S203, if the image gradient is greater than the preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that the secondary transformation is not required for the coding block of the video frame; if the image gradient is not greater than the preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that the secondary transformation is required for the coding block of the video frame.

[0122] The preset gradient threshold can be determined based on actual needs or experience. In one possible implementation, the preset gradient threshold is 30.

[0123] In one possible implementation, the image gradient being greater than or less than a preset gradient threshold can be directly used as the secondary transformation identifier. That is, the secondary transformation identifier can be either an image gradient greater than or less than a preset gradient threshold. The identifier indicating an image gradient greater than the preset gradient threshold does not require secondary transformation for the coded block of the video frame, while the identifier indicating an image gradient greater than or less than the preset gradient threshold requires secondary transformation for the coded block of the video frame.

[0124] In another possible implementation, the secondary transformation identifier is determined to have different identifier values ​​based on whether the image gradient is greater than or less than a preset gradient threshold. These different identifier values ​​indicate whether a secondary transformation is needed for the coded blocks of the video frame. Specifically, the identifier values ​​can be text, numbers, letters, etc.

[0125] For example, if the image gradient is greater than the preset gradient threshold, the identifier value can be determined to be the number "1". If the identifier value is "1", then the identifier needs to be transformed twice for the coding block of the video frame. If the image gradient is not greater than the preset gradient threshold, the identifier value can be determined to be the number "0". If the identifier value is "0", then the identifier does not need to be transformed twice for the coding block of the video frame.

[0126] For example, if the image gradient is not greater than a preset gradient threshold, the identifier value can be determined to be the text "On". If the identifier value is "On", it indicates that a secondary transformation is needed for the coded block of the video frame. If the image gradient is greater than the preset gradient threshold, the identifier value can be determined to be the text "Off". If the identifier value is "Off", it indicates that a secondary transformation is not needed for the coded block of the video frame. Simply put, the text "On" indicates that a secondary transformation is enabled for the coded block of the video frame, meaning that a secondary transformation is needed after the primary transformation. The text "Off" indicates that a secondary transformation is disabled for the coded block of the video frame, meaning that a secondary transformation is not needed after the primary transformation.

[0127] Through extensive statistics and analysis, it was found that the LFNST tool can effectively remove redundancy from video frames with certain content to improve encoding efficiency, but it is not effective for video frames with certain content.

[0128] When an image has a large gradient value, its complexity is generally high. High image complexity indicates complex frequency domain information, limiting the information compression gained from transforming to the frequency domain. Low-frequency inseparable transform (LDT) essentially transforms the result of the main transform to the frequency domain. Therefore, performing LDT on images with large gradient values ​​is not effective for overall video encoding, as it fails to effectively remove redundancy. Based on this, this embodiment of the invention disables LDT for video frames with large gradient values, which can also be understood as not performing LDT on these frames during video encoding.

[0129] In this embodiment of the invention, an image gradient greater than a preset gradient threshold indicates a large gradient value for a video frame. Specifically, the image gradient of the video frame is determined based on its pixel values. Different secondary transform identifiers are determined based on whether the image gradient is greater than the preset gradient threshold. If the image gradient is greater than the preset gradient threshold, the secondary transform identifier for the corresponding video frame indicates that a secondary transform is not required for the encoded block of the video frame; if the image gradient is not greater than the preset gradient threshold, the secondary transform identifier for the corresponding video frame indicates that a secondary transform is required for the encoded block of the video frame. Thus, during video encoding, secondary transforms can be selectively performed on or skipped for video frames based on the secondary transform identifier for the corresponding image frame, i.e., low-frequency inseparable transforms.

[0130] In this embodiment of the invention, when encoding video frames for which low-frequency inseparable transform does not produce good results for the overall video encoding, the low-frequency inseparable transform for the encoding block of the video frame is skipped; while when encoding video frames for which low-frequency inseparable transform does produce good results for the overall video encoding, the main transform coefficients are then subjected to low-frequency inseparable transform. In this way, the complexity of video encoding can be reduced, and the impact on the video encoding effect can be reduced.

[0131] In an optional embodiment, the video encoding method provided by the present invention may further include:

[0132] Add a flag bit to the coded block. The flag bit is used to indicate whether a low-frequency indivisible transform was performed on the coded block during the video encoding process.

[0133] Specifically, after the secondary transformation identifier indicates that the coding block of the video frame does not need to undergo secondary transformation and the low-frequency inseparable transformation of the coding block is skipped and the subsequent coding process is carried out, a flag bit indicating that the low-frequency inseparable transformation has not been performed can be added to the coding block, for example, adding a flag bit "0".

[0134] Alternatively, after the secondary transform identifier indicates that a secondary transform needs to be performed on the coding block of the video frame, and after performing a low-frequency inseparable transform on the main transform coefficients, a flag bit for the coding block that has undergone a low-frequency inseparable transform can be added, for example, adding a flag bit "1".

[0135] To facilitate determining whether an inverse low-frequency inseparable transform is needed for the coding block during subsequent decoding, in this embodiment of the invention, after determining that the secondary transform identifier indicates that a secondary transform is not needed for the coding block of the video frame, and skipping the low-frequency inseparable transform for the coding block to proceed with the subsequent encoding process, a flag bit is added to the coding block.

[0136] It is understood that the secondary transform identifier information determined in this embodiment of the invention is for video frames. During the video encoding process, whether the coding blocks of the same video frame have undergone low-frequency inseparable transform should be the same. For example, if video frame 1 includes coding block 10, coding block 11, and coding block 12, then coding blocks 10, 11, and 12 either all have undergone low-frequency inseparable transform, or none of them have undergone low-frequency inseparable transform.

[0137] The video decoding method provided in this embodiment of the invention can be applied to electronic devices. These electronic devices can be the same as those in the video encoding method described above, or they can be different devices. Specifically, the electronic devices can be servers, terminals, decoders, etc.

[0138] This invention also provides a video decoding method, such as... Figure 3 As shown, it may include:

[0139] S301, Obtain the flag bit corresponding to the coded block;

[0140] S302, if the flag indicates that the low-frequency inseparable transform was not performed on the coding block during the video encoding process, then the inverse low-frequency inseparable transform is skipped after the encoding result of the coding block is dequantized and the subsequent decoding process is performed.

[0141] The flag bit is added to the encoding block after obtaining the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that the encoding block of the video frame does not need to be subjected to secondary transform, the low-frequency indivisible transform of the encoding block is skipped and the subsequent encoding process is carried out. The secondary transform identifier information is used to indicate whether the encoding block of the video frame needs to be subjected to secondary transform. The secondary transform identifier information is determined according to the image content of the video frame.

[0142] In this embodiment of the invention, when decoding encoded blocks that have not undergone low-frequency indivisible transformation during video encoding, there is no need to perform inverse low-frequency indivisible transformation during video decoding, thus reducing the complexity of video decoding.

[0143] In this embodiment of the invention, during the video encoding process, a flag bit indicating that no secondary transformation is required for the coding block of the video frame can be added to the coding block after the low-frequency indivisible transformation for the coding block is skipped and the subsequent encoding process is carried out.

[0144] Alternatively, after the secondary transform identifier indicates that a secondary transform needs to be performed on the coding block of the video frame, and after performing a low-frequency inseparable transform on the main transform coefficients, a flag bit indicating that a low-frequency inseparable transform has been performed is added to the coding block.

[0145] During video decoding, if the flag indicates that a low-frequency inseparable transform was performed on the coded block during video encoding, then after dequantizing the encoding result of the coded block, the inverse low-frequency inseparable transform will be performed first, and then the subsequent decoding process will proceed.

[0146] In general, different flag bits are added during the video encoding process to indicate whether a low-frequency inseparable transform was performed on the coded block. During the decoding process, after the inverse main transform is performed on the coded block, the flag bits are used to determine whether a low-frequency inseparable transform was performed. If the flag bits indicate that a low-frequency inseparable transform was not performed on the coded block during video encoding, the inverse low-frequency inseparable transform is skipped after dequantization of the encoded result and the subsequent decoding process proceeds. If the flag bits indicate that a low-frequency inseparable transform was performed on the coded block during video encoding, the inverse low-frequency inseparable transform is performed first after dequantization of the encoded result, and then subsequent decoding processes such as inverse main transform and reconstruction are performed.

[0147] Figure 4 This is a schematic diagram illustrating the video encoding and video decoding methods provided in the embodiments of the present invention. (Refer to...) Figure 4 Transform quantization is a very time-consuming module in the encoder, which includes the main transform, secondary transform, and quantization processes. In the video encoding process of this invention, prediction is performed first, such as intra-frame prediction and inter-frame prediction; then the main transform is performed, and after the main transform, it is determined whether a secondary transform is needed, that is, whether a low-frequency inseparable transform is needed.

[0148] Specifically, the image gradient (simply put, the average gradient of the current frame) of the video block to be encoded is calculated. Then, it is determined whether the image gradient is greater than a preset gradient threshold. If the image gradient is greater than the preset gradient threshold, the low-frequency inseparable transform is turned off, that is, after the main transform, quantization and other subsequent encoding processes are performed directly. If the image gradient is not greater than the preset gradient threshold, the low-frequency inseparable transform is turned on, that is, after the main transform, the low-frequency inseparable transform is performed first, and then quantization and other subsequent encoding processes are performed.

[0149] Specifically, the image gradient of the video frame corresponding to the current coding block in the video to be encoded can be pre-calculated. Then, the secondary transform identifier information of the video frame is determined based on whether the image gradient is greater than a preset gradient threshold. Thus, after performing the main transform on the coding block, the secondary transform identifier information is obtained, and then it is directly determined whether a low-frequency inseparable transform is needed based on the secondary transform identifier information. If the secondary transform identifier information indicates that a secondary transform is not needed for the coding block of the video frame, the low-frequency inseparable transform for the coding block is skipped and the subsequent encoding process proceeds; if the secondary transform identifier information indicates that a secondary transform is needed for the coding block of the video frame, the low-frequency inseparable transform is performed on the coding block.

[0150] For each coding block, after skipping the low-frequency inseparable transform for the coding block and proceeding with the subsequent coding process, a flag bit indicating that the low-frequency inseparable transform has not been performed is added to the coding block; after performing the low-frequency inseparable transform on the main transform coefficients, a flag bit indicating that the low-frequency inseparable transform has been performed is added to the coding block.

[0151] Reference Figure 4 During video decoding, the encoded result of the coded block is first dequantized. After dequantization, the flag bit corresponding to the coded block is used to determine whether to perform the inverse quadratic transform, i.e., the inverse low-frequency inseparable transform. If the flag bit indicates that the low-frequency inseparable transform was not performed on the coded block during video encoding, the inverse quadratic transform is skipped after dequantization of the coded block's encoded result, and subsequent decoding processes such as the inverse main transform and reconstruction are performed. If the flag bit indicates that the low-frequency inseparable transform was performed on the coded block during video encoding, the inverse low-frequency inseparable transform is performed first after dequantization of the coded block's encoded result, followed by subsequent decoding processes such as the inverse main transform and reconstruction.

[0152] Skipping the low-frequency inseparable transform directly for encoded blocks of video frames that do not require secondary transformation reduces the complexity of video encoding. Furthermore, performing the low-frequency inseparable transform on encoded blocks of video frames that require secondary transformation minimizes the impact on video encoding quality. This also shortens video encoding time, reduces computational resource consumption and costs, and saves computational and bandwidth costs. Correspondingly, in this embodiment of the invention, when decoding encoded blocks that did not undergo the low-frequency inseparable transform during video encoding, there is no need for an inverse low-frequency inseparable transform during video decoding, further reducing the complexity of video decoding.

[0153] In this embodiment of the invention, video encoding can be VVC encoding, and correspondingly, video decoding can be VVC decoding. The video encoding method provided in this embodiment of the invention can be applied to a VVC encoder, thus significantly reducing company bandwidth costs and solving the problem of its impracticality due to excessive complexity, making it more practical. Furthermore, it facilitates faster deployment of VVC encoders.

[0154] In this embodiment of the invention, the video being viewed by the user can be processed. For example, the video to be encoded is the video to be provided to the user for viewing. The method provided by this embodiment of the invention can reduce the complexity of video encoding and video decoding. In general, it reduces the complexity of video processing. Thus, in the process of providing the video to the user for viewing, the video can be provided to the user more smoothly, and the user can be given a more ultimate viewing experience, such as a movie-watching experience.

[0155] In one possible way, such as Figure 5 As shown, the encoding and decoding of the video to be processed can be achieved through the following steps.

[0156] S501, Input video.

[0157] S502, Transform / Quantization

[0158] S503, Intra prediction;

[0159] S504; Motion compensation, motion estimation (Motion est. / comp.);

[0160] S505, Entropy coding;

[0161] S506, inverse quantization, inverse transformation (inv.quant / inv.transform);

[0162] S507, In-loop filter;

[0163] S508, reference frame buffer (decoded picture buffer).

[0164] First, predictive coding is performed. Intra-frame predictive coding can use either intra-frame prediction or inter-frame prediction. Typically, inter-frame predictive coding includes motion compensation and motion estimation.

[0165] The prediction residuals obtained from the input video through intra-frame prediction or inter-frame prediction are transformed and quantized. Specifically, the prediction residuals are transformed to obtain transformation coefficients, and the transformation coefficients are quantized to obtain quantization coefficients.

[0166] After quantization, entropy encoding is performed to obtain the encoding result.

[0167] The transformation can include a primary transformation and a secondary transformation. The primary transformation can be a discrete cosine transform, and the secondary transformation can be a low-frequency inseparable transform.

[0168] Among them, such as Figure 6 As shown, taking the 8*8 type low-frequency non-separable transform as an example of the secondary transform, after S601: Forward primary transform, S602: Forward low-frequency non-separable transform is performed, followed by S603: quantization. Specifically, the forward low-frequency non-separable transform uses a simplified non-separable transform. After inputting 48 primary transform coefficients, 16 secondary transform coefficients are obtained, and then these 16 secondary transform coefficients are quantized.

[0169] The transform kernels for the low-frequency inseparable transform are obtained through offline training, using a scheme of four transform sets, each containing two transform kernels.

[0170] In related technologies, an image frame to be encoded is first input, and encoded one by one according to CTUs (e.g., 128x128 square blocks). Then, the CTUs are divided into blocks called CUs. Predictive coding is then performed (including intra-frame prediction and inter-frame prediction modes, one of which can be chosen), followed by transform, quantization, and finally entropy coding. To obtain a reference frame, the transformed and quantized data needs to be inversely transformed and dequantized, then the predicted blocks are added to obtain reconstructed blocks. These reconstructed blocks are then filtered to obtain the final reconstructed block, which is saved in the DPB (decoded picture buffer) for use in inter-frame prediction.

[0171] In this embodiment of the invention, during the S502 transformation and quantization process, secondary transformation identifier information corresponding to the video frames in the video to be encoded can be obtained. Based on the secondary transformation identifier information, it is determined whether to perform a secondary transformation, i.e., whether to perform a low-frequency inseparable transformation. The secondary transformation identifier information can be predetermined. For example, after obtaining the video, for each video frame, based on the pixel values ​​of the video frame, the image gradient of the video frame is calculated. If the image gradient is greater than a preset gradient threshold, the secondary transformation identifier information corresponding to the video frame is determined to be "off," indicating that a secondary transformation is not required for the encoded block of the video frame. If the image gradient is not greater than the preset gradient threshold, the secondary transformation identifier information corresponding to the video frame is determined to be "on," indicating that a secondary transformation is required for the encoded block of the video frame.

[0172] For example, after inputting a video frame, image analysis is performed on the frame to determine whether secondary transformation should be disabled, and this information is recorded. During video encoding, this recorded information is used to determine whether secondary transformation should be performed on the encoded blocks of the video frame. If the current frame is determined to require LFNST, it is marked, and subsequent transformations of all blocks in this frame will perform the primary transformation first, followed by the secondary transformation. If the current frame is determined not to require LFNST, it is also marked, and subsequent transformations of all blocks in this frame will only perform the primary transformation, without the secondary transformation.

[0173] The specific determination of the secondary transformation identifier information and the determination of whether to skip the secondary transformation or perform the secondary transformation based on the secondary transformation identifier information have been described in detail in the above embodiments, and will not be repeated here.

[0174] Entropy coding compresses all the obtained data, including motion vectors, residual quantization coefficients, and parameters of predictive coding. Entropy coding includes variable-length coding, arithmetic coding, and Huffman coding.

[0175] The encoded result can then be decoded. Video decoding can be seen as the inverse process of video encoding, corresponding to transform and quantization. The coefficients obtained after inverse quantization are inversely transformed to obtain the residual data.

[0176] During video decoding, the encoded data is expanded according to the type of entropy coding to obtain converted encoded data. Corresponding to the encoding process, the converted encoded data includes motion vectors, residual quantization coefficients, and some parameters of predictive coding.

[0177] Specifically, the encoded results can be dequantized, inverse transformed, etc.

[0178] A loop filter is a linear low-pass filter that can filter out high-frequency components and noise. Based on the prediction mode of inter-frame prediction or intra-frame prediction, motion vectors are processed to generate predicted frames from the reference frame buffer. Residual data is then superimposed on these predicted frames to obtain the current frame, thus achieving decoding of the coded video.

[0179] correspond Figure 6 The transform quantization method shown describes the process during video decoding. After obtaining the video encoding result (e.g., the result of entropy encoding after quantization), the following steps are performed: S604: Inverse quantization; S605: Inverse low-frequency non-separable transform; S606: Inverse primary transform. Specifically, inverse quantization and inverse transform can be performed after receiving the bitstream data. Then, corresponding to the forward low-frequency non-separable transform, an inverse low-frequency non-separable transform is performed. Specifically, during the inverse low-frequency non-separable transform, an inverse simplified non-separable transform is used, resulting in 48 transform coefficients. The transform matrix of the inverse simplified non-separable transform is the transpose of the forward simplified non-separable transform matrix. Finally, an inverse primary transform is performed on these 48 transform coefficients to complete the inverse transform process.

[0180] The transformation process can include a primary transformation and a secondary transformation. In video decoding, the inverse transformation can include an inverse primary transformation and an inverse secondary transformation. The inverse primary transformation can be understood as the inverse process of the primary transformation, and the inverse secondary transformation can be understood as the inverse process of the secondary transformation. In this embodiment of the invention, for the transformed block of a video frame, the flag bit corresponding to the coded block can be obtained. If the flag bit indicates that no low-frequency inseparable transformation was performed on the coded block during video encoding, then after inverse quantization of the encoding result of the coded block, the inverse low-frequency inseparable transformation is skipped, and the subsequent decoding process is performed.

[0181] This invention also provides a video encoding device, such as... Figure 7 As shown, it may include:

[0182] The first acquisition module 701 is used to acquire secondary transformation identifier information corresponding to video frames in the video to be encoded. The secondary transformation identifier information is used to identify whether secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined according to the image content of the video frame.

[0183] The skip module 702 is used to skip the low-frequency indivisible transform for the coding block and proceed with the subsequent coding process if the secondary transform identifier information indicates that a secondary transform is not required for the coding block of the video frame.

[0184] Optionally, such as Figure 8 As shown, the device also includes:

[0185] The second acquisition module 801 is used to acquire video frames in the video to be encoded.

[0186] The calculation module 802 is used to calculate the image gradient of a video frame based on the pixel values ​​of the video frame;

[0187] The determination module 803 is used to determine that if the image gradient is greater than a preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is not required for the coding block of the video frame; if the image gradient is not greater than the preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is required for the coding block of the video frame.

[0188] Optionally, the calculation module 802 is specifically used to calculate the absolute value of the difference between each pair of adjacent pixel values ​​for each row of the video frame, and sum the absolute values ​​of the differences between each pair of adjacent pixel values ​​to obtain the sum of the differences for each row; add the sum of the differences for each row to obtain the horizontal gradient of the video frame; calculate the absolute value of the difference between each pair of adjacent pixel values ​​for each column of the video frame, and sum the absolute values ​​of the differences between each pair of adjacent pixel values ​​to obtain the sum of the differences for each column; add the sum of the differences for each column to obtain the vertical gradient of the video frame; and calculate the image gradient of the video frame based on the horizontal gradient and the vertical gradient.

[0189] Optionally, such as Figure 9 As shown, the device also includes:

[0190] The third acquisition module 901 is used to acquire the main transform coefficients corresponding to the coding block after acquiring the secondary transform identifier information corresponding to the video frame. The main transform coefficients are obtained by performing a main transform on the coding block.

[0191] The transformation module 902 is used to perform a low-frequency indivisible transformation on the main transformation coefficients if the secondary transformation identification information indicates that a secondary transformation is required for the coding block of the video frame.

[0192] Optionally, such as Figure 10 As shown, the device also includes:

[0193] Add module 1001 to add flag bits to the coding block. The flag bits are used to indicate whether a low-frequency indivisible transform was performed on the coding block during the video encoding process.

[0194] This invention also provides a video decoding device, such as... Figure 11 As shown, it includes:

[0195] The acquisition module 1101 is used to acquire the flag bits corresponding to the coded block;

[0196] The decoding module 1102 is used to skip the inverse low-frequency inseparable transform and proceed with the subsequent decoding process after inverse quantization of the encoding result of the encoding block if a flag bit indicates that no low-frequency inseparable transform was performed on the encoding block during the video encoding process. The flag bit is added to the encoding block after obtaining the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that a secondary transform is not required for the encoding block of the video frame, the low-frequency inseparable transform of the encoding block is skipped and the subsequent encoding process is proceeded with. The secondary transform identifier information is used to indicate whether a secondary transform is required for the encoding block of the video frame. The secondary transform identifier information is determined according to the image content of the video frame.

[0197] This invention also provides an electronic device, such as... Figure 12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204. The processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0198] Memory 1203 is used to store computer programs;

[0199] The processor 1201 is used to execute the program stored in the memory 1203 to implement the video encoding method or video decoding method provided in the above embodiments.

[0200] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0201] The communication interface is used for communication between the aforementioned terminal and other devices.

[0202] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0203] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0204] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the video encoding methods or video decoding methods described in the above embodiments.

[0205] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the video encoding or video decoding methods described in the above embodiments.

[0206] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0207] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0209] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method of video coding, the method comprising: include: Obtain the secondary transformation identifier information corresponding to the video frame in the video to be encoded. The secondary transformation identifier information is used to identify whether a secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined according to the image content of the video frame. The secondary transformation identifier information of each video frame in the video to be encoded is determined before video encoding, and the secondary transformation identifier information corresponding to each video frame is saved. If the secondary transformation identifier indicates that a secondary transformation is not required for the coding block of the video frame, then the low-frequency indivisible transformation for the coding block is skipped and the subsequent coding process is performed. The method further includes: Obtain video frames from the video to be encoded; Calculate the image gradient of the video frame based on the pixel values ​​of the video frame; If the image gradient is greater than a preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is not required for the coding block of the video frame. If the image gradient is not greater than a preset gradient threshold, then the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation needs to be performed on the coding block of the video frame. Whether the coding block of the same video frame is subjected to a low-frequency indivisible transformation is the same.

2. The method of claim 1, wherein, Calculating the image gradient of the video frame based on the pixel values ​​of the video frame includes: For each row of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences corresponding to each row. The horizontal gradient of the video frame is obtained by summing the differences between each row. For each column of pixel values ​​in the video frame, the absolute value of the difference between each pair of adjacent pixel values ​​is calculated sequentially, and the absolute values ​​of the differences between each pair of adjacent pixel values ​​are summed to obtain the sum of the differences corresponding to each column. The vertical gradient of the video frame is obtained by summing the differences in each column. The image gradient of the video frame is calculated based on the horizontal gradient and the vertical gradient.

3. The method of claim 1, wherein, After obtaining the quadratic transform identifier information corresponding to the video frame, the method further includes: Obtain the main transform coefficients corresponding to the coding block, wherein the main transform coefficients are obtained by performing a main transform on the coding block; If the secondary transform identifier indicates that a secondary transform is required for the coded block of the video frame, then the main transform coefficients are subjected to a low-frequency inseparable transform.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A flag bit is added to the coded block, which is used to identify whether a low-frequency indivisible transform was performed on the coded block during the video encoding process.

5. A method of video decoding, the method comprising: include: Obtain the flag bits corresponding to the encoded block; If the flag indicates that no low-frequency inseparable transform was performed on the coded block during video encoding, then the inverse low-frequency inseparable transform is skipped after dequantization of the encoded result of the coded block, and the subsequent decoding process proceeds. The flag is added to the coded block after obtaining the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that a secondary transform is not required for the coded block of the video frame, the low-frequency inseparable transform for the coded block should be skipped before proceeding with the subsequent encoding process. The secondary transform identifier information is used to indicate whether a secondary transform is required for the coded block of the video frame. The secondary transform identifier information is determined based on the image content of the video frame and is used during video encoding. The secondary transform identifier information of each video frame in the video to be encoded is determined and saved. The secondary transform identifier information is obtained in the following way: the video frame in the video to be encoded is obtained; the image gradient of the video frame is calculated based on the pixel value of the video frame; if the image gradient is greater than a preset gradient threshold, it is determined that the secondary transform identifier information corresponding to the video frame does not need to be used for the encoding block of the video frame; if the image gradient is not greater than the preset gradient threshold, it is determined that the secondary transform identifier information corresponding to the video frame needs to be used for the encoding block of the video frame. The case of whether the encoding block of the same video frame is used for low-frequency inseparable transform is the same.

6. A video encoding apparatus, comprising: include: The first acquisition module is used to acquire secondary transformation identifier information corresponding to video frames in the video to be encoded. The secondary transformation identifier information is used to identify whether secondary transformation is required for the encoding block of the video frame. The secondary transformation identifier information is determined according to the image content of the video frame. The secondary transformation identifier information of each video frame in the video to be encoded is determined before video encoding, and the secondary transformation identifier information corresponding to each video frame is saved. The skip module is used to skip the low-frequency indivisible transform for the coding block and proceed with the subsequent coding process if the secondary transform identifier information indicates that a secondary transform is not required for the coding block of the video frame. The device further includes: The second acquisition module is used to acquire video frames in the video to be encoded; The calculation module is used to calculate the image gradient of the video frame based on the pixel values ​​of the video frame; The determination module is used to determine that if the image gradient is greater than a preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is not required for the coding block of the video frame; if the image gradient is not greater than the preset gradient threshold, the secondary transformation identifier information corresponding to the video frame indicates that a secondary transformation is required for the coding block of the video frame. The case of whether the coding block of the same video frame is subjected to low-frequency indivisible transformation is the same.

7. An apparatus for video decoding, the apparatus comprising: include: The acquisition module is used to acquire the flag bits corresponding to the encoded block; The decoding module is configured to skip the inverse low-frequency inseparable transform and proceed with subsequent decoding after inverse quantization of the encoded block if the flag bit indicates that no low-frequency inseparable transform was performed on the encoded block during video encoding. The flag bit is added to the encoded block after acquiring the secondary transform identifier information corresponding to the video frame in the video to be encoded. If the secondary transform identifier information indicates that a secondary transform is not required for the encoded block of the video frame, the module skips the low-frequency inseparable transform and proceeds with subsequent encoding. The secondary transform identifier information is used to indicate whether a secondary transform is required for the encoded block of the video frame. The secondary transform identifier information is determined based on the image content of the video frame. Before frequency encoding, the secondary transform identifier information of each video frame in the video to be encoded is determined and saved. The secondary transform identifier information corresponding to each video frame is obtained in the following way: the video frame in the video to be encoded is obtained; based on the pixel value of the video frame, the image gradient of the video frame is calculated; if the image gradient is greater than a preset gradient threshold, it is determined that the secondary transform identifier information corresponding to the video frame indicates that no secondary transform is required for the coding block of the video frame; if the image gradient is not greater than the preset gradient threshold, it is determined that the secondary transform identifier information corresponding to the video frame indicates that a secondary transform is required for the coding block of the video frame. The case of whether the coding block of the same video frame is subject to low-frequency inseparable transform is the same.

8. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.

Images