Video encoding processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a video coding processing method, apparatus, device and storage medium. The technical solution provided by the embodiments of the present application divides the unit to be coded into multiple sub-blocks to be coded, and determines the residual ratio information of each sub-block to be coded, determines whether to perform an intra-frame sub-block division check on the unit to be coded according to the residual ratio information, and when it is determined that the intra-frame sub-block division check is not to be performed on the unit to be coded, the current unit to be coded can be coded based on adjacent coded units, thereby reducing redundant intra-frame sub-block division check processes, effectively reducing the complexity of intra-frame prediction, and improving video coding efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of coding technology, and in particular to a video coding processing method, apparatus, device and storage medium. Background Art

[0002] When encoding video images, the international video coding standard H.266 / VVC often uses intra sub-block partitioning (ISP) technology for intra-frame prediction. Based on the intra-frame prediction of the coding unit of the original international video coding standard HEVC, it divides the current coding unit into multiple sub-blocks in the same direction horizontally or vertically, predicts each sub-block separately in turn, and the reconstructed value after each sub-block prediction can be used as a reference for the next sub-block, thereby reducing the residual after the prediction of each sub-block and achieving more efficient video compression.

[0003] In order to select an intra-frame sub-block division mode suitable for the current coding unit, each coding unit that performs intra-frame sub-block division check needs to perform horizontal and vertical division of intra-frame sub-blocks on several intra-frame angle modes with the highest probability in the angle mode candidate list, and select an intra-frame sub-block division mode with the lowest rate-distortion cost for the current coding unit through rate-distortion cost decision. This process includes a large number of redundant intra-frame sub-block division calculations, which greatly increases the complexity of intra-frame prediction and results in low video coding efficiency. Summary of the invention

[0004] The embodiments of the present application provide a video coding processing method, apparatus, device and storage medium to solve the technical problems in the related art that the calculation of sub-block division within a frame is relatively redundant, the complexity of intra-frame prediction is high, and the video coding efficiency is low, thereby reducing the complexity of intra-frame prediction and improving video coding efficiency.

[0005] In a first aspect, an embodiment of the present application provides a video encoding processing method, including:

[0006] Dividing the unit to be encoded into a plurality of sub-blocks to be encoded;

[0007] Determine the residual ratio information of the unit to be encoded according to the residual information of each sub-block to be encoded;

[0008] Determine whether to perform intra-frame sub-block division check on the unit to be encoded based on the residual ratio information;

[0009] In response to a determination result that the intra-frame sub-block division check is not performed on the unit to be encoded, encoding processing is performed on the unit to be encoded based on adjacent encoded units.

[0010] In a second aspect, an embodiment of the present application provides a video encoding processing device, including a unit division module, a residual determination module, a residual analysis module and an encoding processing module, wherein:

[0011] The unit division module is configured to divide the unit to be encoded into a plurality of sub-blocks to be encoded;

[0012] The residual determination module is configured to determine the residual ratio information of the to-be-encoded unit according to the residual information of each of the to-be-encoded sub-blocks;

[0013] The residual analysis module is configured to determine whether to perform intra-frame sub-block division check on the unit to be encoded based on the residual ratio information;

[0014] The encoding processing module is configured to perform encoding processing on the unit to be encoded based on adjacent encoded units in response to a judgment result that the intra-frame sub-block division check is not performed on the unit to be encoded.

[0015] In a third aspect, an embodiment of the present application provides a video encoding processing device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding processing method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the video encoding processing method as described in the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the video encoding processing method described in the first aspect.

[0020] The embodiment of the present application divides the unit to be encoded into multiple sub-blocks to be encoded, and determines the residual ratio information of each sub-block to be encoded, and determines whether to perform an intra-frame sub-block division check on the unit to be encoded according to the residual ratio information. When it is determined that the intra-frame sub-block division check is not to be performed on the unit to be encoded, the current unit to be encoded can be encoded based on adjacent encoded units, thereby reducing redundant intra-frame sub-block division check processes, effectively reducing the complexity of intra-frame prediction, and improving video coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a video encoding processing method provided by an embodiment of the present application;

[0022] Figure 2 is a flowchart of another video encoding processing method provided by an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of a residual information calculation method and a flow chart of a determination process provided in an embodiment of the present application;

[0024] Figure 4 This is a schematic diagram of a sub-block division check and judgment process within a frame provided by an embodiment of the present application;

[0025] Figure 5 is a structural diagram of a video encoding processing device provided in an embodiment of the present application;

[0026] Figure 6 It is a structural diagram of a video encoding processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0028] Figure 1 A flowchart of a video encoding processing method provided in an embodiment of the present application is given. The video encoding processing method provided in an embodiment of the present application can be executed by a video encoding processing device, which can be implemented by hardware and / or software and integrated in a video encoding processing device.

[0029] The following description is made by taking a video encoding processing device executing a video encoding processing method as an example. Figure 1 , the video encoding processing method includes:

[0030] S101: Divide a unit to be encoded into a plurality of sub-blocks to be encoded.

[0031] Among them, the unit to be encoded is a coding unit (CU, CodingUnit) on the video frame that needs to be encoded. It can be understood that there are multiple coding units on the video frame, and each coding unit is encoded in turn according to a set order (for example, from left to right, from top to bottom) based on a set video coding standard (for example, the international standard for video coding H.266 / VVC, the international standard for video coding H.265 / HEVC), until the encoding of the current video frame to be encoded is completed.

[0032] In the related art, in order to achieve more efficient image compression, the international video coding standard H.266 / VVC usually uses intra sub-block partitioning technology (ISP, Intra Sub-partition) to encode the coding unit. As an intra-frame prediction technology in the latest generation of international video coding standard H.266 / VVC, the intra-frame sub-block partitioning technology divides the current coding unit into multiple sub-blocks in the same direction horizontally or vertically on the basis of the original intra-frame prediction of the coding unit, and predicts each sub-block separately in turn to determine the reconstruction value. The reconstructed value after each sub-block prediction can be used as a reference for the next sub-block, so as to reduce the residual after each sub-block prediction, so as to achieve a more efficient compression effect.

[0033] The intra-frame sub-block division algorithm is implemented after completing the angle prediction of the unit to be coded, that is, after deciding on the angle mode candidate list for intra-frame prediction of the unit to be coded, the intra-frame sub-block division algorithm will divide the coding unit into 2 or 4 sub-blocks in the same direction, and select the front angle mode in the angle mode candidate list for prediction in units of sub-blocks. In the process of intra-frame sub-block division prediction, the first few modes in the angle mode candidate list of the current unit to be coded will be traversed. In the process of intra-frame sub-block division check, the intra-frame angle mode used by each sub-block is the same, and the reconstructed pixels of each sub-block can be used as the reference pixels of the next sub-block, which shortens the distance between each sub-block and its reference pixel. The pixels in the sub-block are more correlated with the reference pixels, thereby reducing the prediction residual and improving the compression efficiency. However, in order to select an intra-frame sub-block division mode suitable for the current unit to be encoded, each unit to be encoded that performs intra-frame sub-block division check needs to perform horizontal and vertical intra-frame sub-block divisions on several intra-frame angle modes with the highest probability in the angle mode candidate list, and select an intra-frame sub-block division mode with the lowest rate-distortion cost for the current unit to be encoded based on the rate-distortion cost decision. However, the above intra-frame sub-block division check process greatly increases the complexity of intra-frame prediction and limits the video coding efficiency. In order to solve the above technical problems, this solution divides the unit to be encoded into multiple sub-blocks to be encoded according to the possible division methods of the intra-frame sub-block division technology before performing an intra-frame sub-block division check on the unit to be encoded, and determines whether the current sub-block to be encoded needs to be checked for intra-frame sub-block division based on the residual ratio information of the sub-block to be encoded. It is not necessary to perform an intra-frame sub-block division check on all units to be encoded, thereby reducing redundant intra-frame sub-block division check processes and improving video coding efficiency. The distribution of the residual after angle prediction of the current unit to be encoded in the sub-block to be encoded is used to determine whether the current unit to be encoded is suitable for the intra-frame sub-block division check, thereby skipping redundant intra-frame sub-block division calculations and achieving video coding acceleration.

[0034] Exemplarily, for each unit to be encoded, after determining the candidate list of angle modes for intra-frame prediction of the unit to be encoded, the unit to be encoded will be divided into multiple sub-blocks to be encoded. For example, the unit to be encoded is divided into 2 or 4 sub-blocks to be encoded according to the possible division method of the intra-frame sub-block division technology.

[0035] The number of sub-blocks to be encoded that are divided can be determined according to the size of the unit to be encoded. For example, when the unit size of the unit to be encoded is within a first size range (e.g., 4*8 or 8*4), the unit to be encoded can be divided into a first number (e.g., 2) of sub-blocks to be encoded; when the unit size of the unit to be encoded is within a second size range (e.g., greater than 4*8 or greater than 8*4), the unit to be encoded can be divided into a second number (e.g., 4) of sub-blocks to be encoded, wherein the size corresponding to the second size range is greater than the size corresponding to the first size range; when the unit size of the unit to be encoded is within a third size range (e.g., 4*4), the unit to be encoded is not divided, wherein the size corresponding to the first size range is greater than the size corresponding to the third size range, wherein the size corresponding to the third size range may be the minimum encoding size for video encoding.

[0036] For example, for a unit to be encoded with a unit size of 4*4, since the minimum encoding size of the international video coding standard H.266 / VVC is 4*4, and the intra-frame sub-block division technology requires that the sub-block size must be at least 16 pixels, the 4*4 unit to be encoded needs to be predicted as a whole, for example, using adjacent encoded units for intra-frame prediction, and there is no need to perform subsequent steps to determine whether an intra-frame sub-block division check is required; for a unit to be encoded with a unit size of 4*8 or 8*4, it can be divided into 2 sub-blocks to be encoded; for a unit to be encoded with a unit size of more than 4*8 or more than 8*4, it can be divided into 4 sub-blocks to be encoded.

[0037] S102: Determine residual ratio information of the unit to be encoded according to the residual information of each sub-block to be encoded.

[0038] Exemplarily, after obtaining a plurality of sub-blocks to be encoded of a unit to be encoded, the residual information corresponding to each sub-block to be encoded is determined, and the residual ratio information of the unit to be encoded is determined according to the residual information of each sub-block to be encoded. The residual information may represent the residual between the predicted value of the sub-block to be encoded and the original pixel value, and the residual may represent the correlation between the sub-block to be encoded and the reference pixel, that is, the residual information may characterize the amount of information of the residual of the sub-block to be encoded. Generally, the stronger the correlation between the sub-block to be encoded and the reference pixel, the smaller the residual between the predicted value of the corresponding sub-block to be encoded and the original pixel value.

[0039] The residual information of the sub-block to be encoded can be represented by information that can characterize the degree of distortion after the prediction of the unit to be encoded, for example, the residual information of the sub-block to be encoded can be represented by a combination of one or more of Hadamard transform score (HAD), Sum of Absolute Difference (SAD), Mean Squared Error (MSE), Sum of Squared Difference (SSD) and Rate-Distortion-Cost (RD-Cost). The residual ratio information of the unit to be encoded can be represented based on the distribution of the residual information of the sub-block to be encoded in the unit to be encoded, for example, the residual ratio information can be the ratio of the largest residual information in the residual information of these sub-blocks to be encoded.

[0040] S103: Determine whether to perform intra-frame sub-block division check on the coding unit based on the residual ratio information.

[0041] Exemplarily, after determining the residual ratio information of the unit to be encoded, it is determined whether to perform an intra-frame sub-block division check on the current unit to be encoded based on the residual ratio information. For example, when the residual ratio information reflects that the correlation between a certain part of the pixels in the unit to be encoded and the reference pixels of the adjacent encoded unit does not reach the set strength, it can be determined that the unit to be encoded needs to be checked for intra-frame sub-block division, and the intra-frame prediction method for the unit to be encoded is determined based on the result of the intra-frame sub-block division check, so as to reduce the rate distortion cost in the encoding process and improve the video encoding compression rate. When the residual ratio information reflects that the correlation between the pixels in the unit to be encoded and the reference pixels of the adjacent encoded units reaches the set strength, the intra-frame sub-block division check of the current unit to be encoded can be skipped, reducing the redundant intra-frame sub-block division check process and improving the video encoding efficiency.

[0042] S104: In response to a determination result that the intra-frame sub-block division check is not performed on the unit to be coded, encoding is performed on the unit to be coded based on adjacent coded units.

[0043] Exemplarily, when determining that no intra-frame sub-block division check is performed on the unit to be encoded, it is considered that the pixels within the current unit to be encoded have a strong correlation with the reference pixels of the adjacent encoded units, and the intra-frame sub-block division check of the current unit to be encoded can be skipped, and the unit to be encoded can be directly encoded based on the adjacent encoded units, thereby greatly improving the encoding speed of intra-frame prediction of the unit to be encoded during the video encoding process.

[0044] In a possible embodiment, in the step of determining whether to perform an intra-frame sub-block division check on the coding unit based on the residual ratio information, when it is determined that the intra-frame sub-block division check needs to be performed on the coding unit, the coding process of the coding unit to be coded can be completed according to the complete intra-frame sub-block division technology. Based on this, after determining whether to perform an intra-frame sub-block division check on the coding unit based on the residual ratio information, this solution also includes: in response to the determination result of the intra-frame sub-block division check on the coding unit to be coded, coding the coding unit to be coded based on the intra-frame sub-block division technology.

[0045] Exemplarily, when determining the result of the intra-frame sub-block division check for the coding unit, it is considered that the correlation between the pixels in the current coding unit and the reference pixels of the adjacent coded unit is weak, and the coding unit needs to be coded based on the intra-frame sub-block division technology. That is, the intra-frame sub-block division check is performed on the coding unit horizontally and / or vertically, and the optimal intra-frame prediction method is selected for coding based on the rate-distortion cost decision.

[0046] In a possible embodiment, when encoding a coding unit based on an intra-frame sub-block division technology, it includes: determining an intra-frame sub-block division check result of the coding unit; determining an intra-frame prediction method for the coding unit based on the intra-frame sub-block division check result, and encoding the coding unit according to the intra-frame prediction method.

[0047] Exemplarily, when it is determined that the intra-frame sub-block division check needs to be performed on the coding unit, the results of the intra-frame sub-block division check of the coding unit in the horizontal and vertical directions are determined, and the rate-distortion cost of encoding based on the horizontal intra-frame sub-block division, the rate-distortion cost of encoding based on the vertical intra-frame sub-block division, and the rate-distortion cost of encoding the coding unit as the entire coding unit are calculated according to the results of the intra-frame sub-block division check, and the intra-frame prediction mode corresponding to the minimum rate-distortion cost is used as the intra-frame prediction mode of the current coding unit, and the coding unit is encoded according to the intra-frame prediction mode. That is, taking the coding sub-block as a unit, the front angle modes in the angle mode candidate list are selected in turn for prediction, and each coding unit to be encoded for the intra-frame sub-block division check needs to perform horizontal and vertical intra-frame sub-block division on several intra-frame angle modes with the highest probability in the angle mode candidate list, respectively, and based on the rate-distortion cost decision, from encoding based on the horizontal intra-frame sub-block division, encoding based on the vertical intra-frame sub-block division, and encoding as the entire coding unit, an intra-frame prediction mode with the minimum rate-distortion cost of the current coding unit is selected for encoding.

[0048] In a possible embodiment, when determining the intra-frame prediction mode for the coding unit based on the intra-frame sub-block division check result, the solution includes:

[0049] When the intra-frame sub-block division check result meets the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction method for the coding unit is to encode the coding unit based on the intra-frame sub-block division technology; when the intra-frame sub-block division check result does not meet the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction method for the coding unit is to encode the coding unit based on adjacent coded units.

[0050] Exemplarily, after determining the result of the intra-frame sub-block division check of the unit to be coded, it is determined whether the result of the intra-frame sub-block division check satisfies the intra-frame sub-block division processing condition. Whether the result of the intra-frame sub-block division check satisfies the intra-frame sub-block division processing condition may be calculated based on the result of the intra-frame sub-block division check to calculate a first rate-distortion cost for encoding based on horizontal intra-frame sub-block division, a second rate-distortion cost for encoding based on vertical intra-frame sub-block division, and a third rate-distortion cost for encoding the unit to be coded as the entire coding unit. If the first rate-distortion cost or the second rate-distortion cost is less than the third rate-distortion cost, it is considered that the result of the intra-frame sub-block division check satisfies the intra-frame sub-block division processing condition. If both the first rate-distortion cost and the second rate-distortion cost are greater than or equal to the third rate-distortion cost, it is considered that the result of the intra-frame sub-block division check does not satisfy the intra-frame sub-block division processing condition.

[0051] If the result of the intra-frame sub-block division check meets the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction method of the unit to be coded is to encode the unit to be coded based on the intra-frame sub-block division technology, that is, the unit to be coded is divided in the corresponding direction with the minimum rate distortion cost to obtain multiple sub-blocks (sub-blocks to be coded), and the sub-block is used as a unit and the adjacent coded sub-block or adjacent coded unit is used as a reference for encoding. If the result of the intra-frame sub-block division check does not meet the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction method of the unit to be coded is to encode the unit to be coded based on the adjacent coded unit, that is, the entire unit to be coded is used as a unit and the adjacent coded unit is used as a reference for encoding.

[0052] In the above, by dividing the unit to be encoded into a plurality of sub-blocks to be encoded, and determining the residual ratio information of each sub-block to be encoded, it is determined whether to perform an intra-frame sub-block division check on the unit to be encoded according to the residual ratio information, and when it is determined that the unit to be encoded is not to be subjected to an intra-frame sub-block division check, the current unit to be encoded can be encoded based on the adjacent encoded units, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving the video coding efficiency. And when it is determined that the unit to be encoded needs to be subjected to an intra-frame sub-block division check, the intra-frame prediction method for the unit to be encoded is determined according to the intra-frame sub-block division check result, effectively improving the video compression effect. In the process of an intra-frame sub-block division check, the intra-frame angle mode used by each sub-block is the same, and the reconstructed pixel of each sub-block can be used as the reference pixel of the next sub-block, shortening the distance between each sub-block and its reference pixel, and the pixel in the sub-block has a stronger correlation with the reference pixel, effectively reducing the residual of the intra-frame prediction, and improving the video coding compression efficiency.

[0053] Based on the above embodiments, Figure 2 A flowchart of another video encoding processing method provided in an embodiment of the present application is given, and the video encoding processing method is a specific embodiment of the above-mentioned video encoding processing method. Figure 2 , the video encoding processing method includes:

[0054] S201: Divide a unit to be encoded into a plurality of sub-blocks to be encoded.

[0055] S202: Determine a residual information calculation method according to the sub-block size of the sub-block to be encoded.

[0056] Exemplarily, after the unit to be encoded is divided into a plurality of sub-blocks to be encoded, the residual information calculation method of each sub-block to be encoded is determined according to the sub-block size of each sub-block to be encoded, and different sub-block sizes correspond to different residual calculation methods. The residual information calculation method may be a combination of one or more of the Hadamard transform score, the absolute error sum, the mean square error, the square error sum, and the rate distortion cost of the sub-block to be encoded. This embodiment uses the Hadamard transform score and the absolute error sum to represent the residual information as an example for description.

[0057] In a possible embodiment, since residual transformation is also required when finally encoding the to-be-encoded unit or the to-be-encoded sub-block, the residual information calculation method taking residual transformation into consideration can be preferentially used to calculate residual information, thereby improving video encoding efficiency. Figure 3 A schematic diagram of a flow chart for determining a residual information calculation method is provided. When determining the residual information calculation method according to the sub-block size of the sub-block to be encoded, the scheme includes:

[0058] S2021: Determine a size range corresponding to the sub-block size of the sub-block to be encoded.

[0059] S2022: when the sub-block size of the sub-block to be encoded is within the first set range, determining that the residual information calculation method of the sub-block to be encoded is the first set calculation method, and the values ​​of the side lengths of the sizes within the first set range are all greater than the first set value.

[0060] S2023: When the sub-block size of the sub-block to be encoded is within the second set range, determine that the residual information calculation method of the sub-block to be encoded is the second set calculation method, and there is a side length with a value of the first set value in the size within the second set range.

[0061] Exemplarily, after the unit to be encoded is divided into a plurality of sub-blocks to be encoded, the sub-block sizes corresponding to these sub-blocks to be encoded (these sub-blocks to be encoded have the same size) are determined, and the size range corresponding to the sub-block size of the sub-block to be encoded is determined. Among them, the size range provided by this scheme includes a first set range and a second set range, and the values ​​of the side lengths of the sizes within the first set range are all greater than the first set value, and the sizes within the second set range have side lengths whose values ​​are the first set value. For example, if the first set value is set to 1, then the size range corresponding to the first set range is not 1*N or N*1, where N is greater than or equal to 8, and the size range corresponding to the second set range is 1*N or N*1. That is, when the sub-block to be encoded has a side length of the first set value (for example, 1), the sub-block size of the sub-block to be encoded is within the second set range, and when the side lengths of the sub-block to be encoded are all greater than the first set value, the sub-block size of the sub-block to be encoded is within the first set range.

[0062] When it is determined that the sub-block size of the sub-block to be encoded is within the first set range, the residual information calculation method of the sub-block to be encoded is determined to be the first set calculation method. When the sub-block size of the sub-block to be encoded is within the second set range, the residual information calculation method of the sub-block to be encoded is determined to be the second set calculation method. Among them, the first set calculation method is to calculate the Hadamard transform score of the sub-block to be encoded, and the second set calculation method is to calculate the absolute error sum (or one of the mean square error, square error sum and rate distortion cost) of the sub-block to be encoded. It needs to be explained that, because the calculation process of the Hadamard transform score takes into account the influence of the residual transformation to a certain extent, it can better represent the size of the residual after transformation and quantization, which helps to more accurately determine whether to skip the intra-frame sub-block division check and improve the video coding efficiency.

[0063] S203: Calculate the residual information of each to-be-encoded sub-block according to the residual information calculation method, and determine the maximum residual ratio of the to-be-encoded unit according to the residual information.

[0064] Among them, this scheme represents the residual ratio information of the unit to be encoded by the maximum residual ratio corresponding to each residual information, which more intuitively and effectively reflects the correlation between the pixels in the current unit to be encoded and the adjacent encoded units.

[0065] Exemplarily, after determining the residual information calculation method of the sub-block to be encoded, the residual information of each sub-block to be encoded is calculated based on the determined residual information calculation method, and the maximum residual ratio of the current unit to be encoded is calculated based on the residual information of these sub-blocks to be encoded, that is, the largest residual information among the residual information of these sub-blocks to be encoded is calculated, and the ratio of the largest residual information in all the residual information is calculated, so as to obtain the maximum residual ratio of the unit to be encoded.

[0066] In one possible embodiment, when calculating the maximum residual ratio of the unit to be encoded, it can be calculated according to different residual information. Based on this, the sub-scheme calculates the residual information of each sub-block to be encoded according to the residual information calculation method, and determines the maximum residual ratio of the unit to be encoded according to the residual information, including: when the residual information calculation method is the first set calculation method, the Hadamard transform score of each sub-block to be encoded is calculated, and the maximum residual ratio of the sub-block to be encoded is determined based on the ratio of the maximum Hadamard transform score in the Hadamard transform score; when the residual information calculation method is the second set calculation method, the absolute error sum of each sub-block to be encoded is calculated, and the maximum residual ratio of the sub-block to be encoded is determined based on the ratio of the maximum absolute error sum in the absolute error sum.

[0067] Exemplarily, when the residual information calculation method is the first set calculation method (i.e., calculating the Hadamard transform score of the sub-block to be encoded), the Hadamard transform score of each sub-block to be encoded is calculated. The maximum Hadamard transform score among these Hadamard transform scores is determined, and the proportion of the maximum Hadamard transform score in these Hadamard transform scores is calculated, and the proportion of the maximum Hadamard transform score in these Hadamard transform scores is used as the maximum residual proportion of the current sub-block to be encoded.

[0068] When the residual information calculation method is the first set calculation method (at this time, the sub-block size of the sub-block to be encoded is within the first set range, for example, the sub-block size of the sub-block to be encoded is not 1*N or N*1), taking the unit to be encoded as divided into 4 sub-blocks to be encoded as an example, the maximum residual ratio of the sub-block to be encoded is:

[0069]

[0070] Among them, HAD0, ..., HAD3 are Hadamard transform scores corresponding to the four sub-blocks to be encoded respectively.

[0071] When the residual information calculation method is the second set calculation method (i.e., calculating the absolute error sum of the sub-block to be encoded), the absolute error sum of each sub-block to be encoded is calculated. The maximum absolute error sum among these absolute error sums is determined, and the proportion of the maximum absolute error sum in these absolute error sums is calculated, and the proportion of the maximum absolute error sum in these absolute error sums is used as the maximum residual proportion of the current sub-block to be encoded.

[0072] When the residual information calculation method is the second set calculation method (at this time, the sub-block size of the sub-block to be encoded is within the second set range, for example, the sub-block size of the sub-block to be encoded is 1*N or N*1), taking the unit to be encoded as divided into 4 sub-blocks to be encoded as an example, the maximum residual ratio of the sub-block to be encoded is:

[0073]

[0074] Among them, SAD0, ..., SAD3 are the absolute error sums corresponding to the four sub-blocks to be encoded respectively.

[0075] S204: Determine whether to perform intra-frame sub-block division check on the coding unit based on the residual ratio information.

[0076] Exemplarily, after determining the residual ratio information of the unit to be coded, further according to the residual ratio information, whether the intra-frame sub-block division check of the unit to be coded can be skipped, wherein whether the intra-frame sub-block division check can be skipped can be determined based on the comparison result of the residual ratio information and a preset threshold value. Based on this, if Figure 4 As shown in a schematic diagram of a sub-block division check and judgment process within a frame, this solution includes:

[0077] S2041: Determine whether the residual ratio information reaches a set threshold.

[0078] S2042: When the residual ratio information reaches a set threshold, determine to perform an intra-frame sub-block division check on the coding unit.

[0079] S2043: When the residual ratio information does not reach the set threshold, determine not to perform intra-frame sub-block division check on the coding unit.

[0080] Exemplarily, after determining the maximum residual ratio of the unit to be encoded, the maximum residual ratio is compared with a set threshold to determine whether the maximum residual ratio reaches the set threshold. If the maximum residual ratio of the current unit to be encoded reaches the set threshold, it is considered that the current unit to be encoded is likely to select the intra-frame sub-block division mode as the optimal mode for encoding after the intra-frame sub-block division check. If the maximum residual ratio of the current unit to be encoded does not reach the set threshold, it is considered that the probability of the current unit to be encoded selecting the intra-frame sub-block division mode as the optimal mode for encoding after the intra-frame sub-block division check is small, and it is determined not to perform the intra-frame sub-block division check on the current unit to be encoded, thereby reducing redundant intra-frame sub-block division check processes.

[0081] In a possible embodiment, the residual information provided by this scheme includes the first residual information in the first direction (e.g., horizontal direction) of the sub-block to be encoded and the second residual information in the second direction (e.g., vertical direction) of the sub-block to be encoded. Correspondingly, the residual ratio information (maximum residual ratio) provided by this scheme includes the first residual ratio information in the first direction of the sub-block to be encoded and the second residual ratio information in the second direction of the sub-block to be encoded. Among them, the calculation of the residual information and the residual ratio information in each direction is similar to the above-mentioned residual information calculation method, and this embodiment will not be repeated. Correspondingly, for the first direction and the second direction, the residual ratio information (maximum residual ratio) in the corresponding direction is calculated respectively, and the residual ratio information is compared with the set threshold value of the corresponding direction. When the residual ratio information in one direction reaches the set threshold, it is considered that the current unit to be encoded may select the intra-frame sub-block division mode as the best mode for encoding processing after the intra-frame sub-block division check in the corresponding direction, and it is determined to perform the intra-frame sub-block division check in the corresponding direction on the unit to be encoded. When the residual ratio information in both directions does not reach the set threshold, it is considered that the probability of the current unit to be encoded selecting the intra-frame sub-block division mode as the optimal mode for encoding processing after the intra-frame sub-block division check in both directions is small, and it is determined not to perform the intra-frame sub-block division check in the corresponding direction on the unit to be encoded.

[0082] S205: In response to the judgment result that the intra-frame sub-block division check is not performed on the unit to be encoded, encoding processing is performed on the unit to be encoded based on adjacent encoded units.

[0083] In the above, by dividing the unit to be encoded into a plurality of sub-blocks to be encoded, and determining the residual ratio information of each sub-block to be encoded, it is determined whether to perform an intra-frame sub-block division check on the unit to be encoded according to the residual ratio information, and when it is determined that the unit to be encoded is not to be subjected to an intra-frame sub-block division check, the current unit to be encoded can be encoded based on the adjacent encoded units, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving video encoding efficiency. At the same time, the Hadamard transform score that takes into account the influence of the transformation is used as a standard for measuring the residual distribution, which better represents the size of the residual after transformation and quantization, and realizes a more accurate intra-frame sub-block division skip judgment. The image quality of the encoded video is effectively maintained consistent, and the data used to calculate the residual information are all from the intra-frame calculation process of the original unit to be encoded, which occupies less memory and ensures efficient execution of video encoding. On the premise of reducing the impact on the objective performance of the encoder and maintaining the consistency of the subjective image quality, the encoding speed of the encoder is improved, thereby improving the computing resource utilization rate of the server side, saving server costs, and reducing the processor resource utilization rate of the client side, effectively improving the user experience.

[0084] Figure 5 is a structural diagram of a video encoding processing device provided by an embodiment of the present application. Figure 5 The video encoding processing device includes a unit division module 51, a residual determination module 52, a residual analysis module 53 and an encoding processing module 54.

[0085] Among them, the unit division module 51 is configured to divide the unit to be encoded into multiple sub-blocks to be encoded; the residual determination module 52 is configured to determine the residual ratio information of the unit to be encoded according to the residual information of each sub-block to be encoded; the residual analysis module 53 is configured to determine whether to perform an intra-frame sub-block division check on the unit to be encoded based on the residual ratio information; the encoding processing module 54 is configured to respond to the judgment result that the intra-frame sub-block division check is not performed on the unit to be encoded, and perform encoding processing on the unit to be encoded based on adjacent encoded units.

[0086] In the above, by dividing the unit to be encoded into multiple sub-blocks to be encoded, and determining the residual ratio information of each sub-block to be encoded, it is determined whether to perform an intra-frame sub-block division check on the unit to be encoded according to the residual ratio information, and when it is determined that the intra-frame sub-block division check is not to be performed on the unit to be encoded, the current unit to be encoded can be encoded based on adjacent encoded units, thereby reducing redundant intra-frame sub-block division check processes, effectively reducing the complexity of intra-frame prediction, and improving video coding efficiency.

[0087] On the basis of the above embodiment, the residual information includes first residual information in a first direction of the sub-block to be encoded and second residual information in a second direction of the sub-block to be encoded.

[0088] Based on the above embodiment, the residual determination module 52 is configured as follows:

[0089] Determining a residual information calculation method according to a sub-block size of a sub-block to be encoded;

[0090] The residual information of each to-be-encoded sub-block is calculated according to the residual information calculation method, and the maximum residual ratio of the to-be-encoded unit is determined according to the residual information.

[0091] On the basis of the above embodiment, when the residual information calculation method is determined according to the sub-block size of the sub-block to be encoded, the residual determination module 52 is configured as follows:

[0092] When the sub-block size of the sub-block to be encoded is within the first set range, determining that the residual information calculation method of the sub-block to be encoded is the first set calculation method, and the values ​​of the side lengths of the sizes within the first set range are all greater than the first set value;

[0093] When the subblock size of the subblock to be encoded is within the second set range, the residual information calculation method of the subblock to be encoded is determined to be the second set calculation method, and there is a side length with a value of the first set value in the size within the second set range.

[0094] On the basis of the above embodiment, when the residual information of each to-be-encoded sub-block is calculated according to the residual information calculation method and the maximum residual ratio of the to-be-encoded unit is determined according to the residual information, the residual determination module 52 is configured as follows:

[0095] When the residual information calculation method is the first set calculation method, the Hadamard transform score of each sub-block to be encoded is calculated, and the maximum residual proportion of the sub-block to be encoded is determined based on the proportion of the maximum Hadamard transform score in the Hadamard transform score;

[0096] When the residual information calculation method is the second set calculation method, the absolute error sum of each sub-block to be encoded is calculated, and the maximum residual ratio of the sub-block to be encoded is determined based on the ratio of the maximum absolute error sum in the absolute error sum.

[0097] Based on the above embodiment, the residual analysis module 53 is configured as follows:

[0098] When the residual error ratio information reaches a set threshold, determining to perform an intra-frame sub-block division check on the coding unit to be coded;

[0099] When the residual ratio information does not reach the set threshold, it is determined not to perform intra-frame sub-block division check on the coding unit.

[0100] Based on the above embodiment, the encoding processing module 54 is further configured to: in response to the judgment result of the intra-frame sub-block division check on the unit to be encoded, perform encoding processing on the unit to be encoded based on the intra-frame sub-block division technology.

[0101] On the basis of the above embodiment, when the encoding processing module 54 performs encoding processing on the encoding unit based on the intra-frame sub-block division technology, it is configured as follows:

[0102] Determine a sub-block division check result within the frame of the coding unit to be coded;

[0103] An intra-frame prediction mode for the coding unit to be coded is determined based on the intra-frame sub-block division check result, and the coding unit to be coded is performed according to the intra-frame prediction mode.

[0104] On the basis of the above embodiment, when the encoding processing module 54 determines the intra-frame prediction mode for the coding unit based on the intra-frame sub-block division check result, it is configured as follows:

[0105] When the intra-frame sub-block division check result meets the intra-frame sub-block division processing condition, determining that the intra-frame prediction mode for the coding unit to be coded is to perform coding processing on the coding unit based on the intra-frame sub-block division technology;

[0106] When the intra-frame sub-block division check result does not meet the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction mode of the to-be-coded unit is to perform coding processing on the to-be-coded unit based on adjacent coded units.

[0107] It is worth noting that in the embodiment of the above-mentioned video encoding processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0108] The embodiment of the present application also provides a video encoding processing device, which can integrate the video encoding processing device provided by the embodiment of the present application. Figure 6 is a structural diagram of a video encoding processing device provided in an embodiment of the present application. Figure 6 The video encoding processing device includes: an input device 63, an output device 64, a memory 62 and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the video encoding processing method provided in the above embodiment. The video encoding processing device, equipment and computer provided above can be used to execute the video encoding processing method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0109] The embodiment of the present application also provides a storage medium storing computer executable instructions, which are used to execute the video encoding processing method provided in the above embodiment when executed by a computer processor. Of course, the storage medium storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the video encoding processing method provided above, can also execute the related operations in the video encoding processing method provided in any embodiment of the present application. The video encoding processing device, equipment and storage medium provided in the above embodiment can execute the video encoding processing method provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, please refer to the video encoding processing method provided in any embodiment of the present application.

[0110] In some possible implementations, various aspects of the method provided by the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to perform the steps of the method according to various exemplary embodiments of the present disclosure described above in this specification. For example, the computer device may perform the video encoding processing method recorded in the embodiment of the present disclosure. The program product may adopt any combination of one or more readable media.

Claims

1. A video encoding processing method, characterized in that: include: Dividing the unit to be encoded into a plurality of sub-blocks to be encoded; Determine residual ratio information of the unit to be encoded according to the residual information of each of the sub-blocks to be encoded, wherein the residual ratio information is determined based on the distribution of the residual information of the sub-blocks to be encoded in the unit to be encoded; Determine whether to perform intra-frame sub-block division check on the unit to be encoded based on the residual ratio information; In response to a determination result that the intra-frame sub-block division check is not performed on the unit to be encoded, encoding processing is performed on the unit to be encoded based on adjacent encoded units.

2. The video encoding processing method according to claim 1, characterized in that: The residual information includes first residual information in a first direction of the sub-block to be encoded and second residual information in a second direction of the sub-block to be encoded.

3. The video encoding processing method according to claim 1, characterized in that: The determining the residual ratio information of the to-be-encoded unit according to the residual information of each to-be-encoded sub-block includes: Determining a residual information calculation method according to the sub-block size of the sub-block to be encoded; The residual information of each of the to-be-encoded sub-blocks is calculated according to the residual information calculation method, and the maximum residual ratio of the to-be-encoded unit is determined according to the residual information.

4. The video encoding processing method according to claim 3, characterized in that: The determining of the residual information calculation method according to the sub-block size of the sub-block to be encoded includes: When the sub-block size of the sub-block to be encoded is within a first set range, determining that the residual information calculation method of the sub-block to be encoded is a first set calculation method, and the values ​​of the side lengths of the sizes within the first set range are all greater than the first set value; When the subblock size of the subblock to be encoded is within the second set range, the residual information calculation method of the subblock to be encoded is determined to be the second set calculation method, and there is a side length with a value of the first set value in the size within the second set range.

5. The video encoding processing method according to claim 4, characterized in that: The calculating the residual information of each of the to-be-encoded sub-blocks according to the residual information calculation method, and determining the maximum residual ratio of the to-be-encoded unit according to the residual information, includes: When the residual information calculation method is the first set calculation method, a Hadamard transform score of each of the sub-blocks to be encoded is calculated, and a maximum residual proportion of the sub-block to be encoded is determined based on the proportion of the maximum Hadamard transform score in the Hadamard transform score; When the residual information calculation method is the second set calculation method, the absolute error sum of each sub-block to be encoded is calculated, and the maximum residual proportion of the sub-block to be encoded is determined based on the proportion of the maximum absolute error sum in the absolute error sum.

6. The video encoding processing method according to claim 1, characterized in that: The determining whether to perform an intra-frame sub-block division check on the unit to be encoded based on the residual ratio information includes: When the residual error ratio information reaches a set threshold, determining to perform an intra-frame sub-block division check on the unit to be encoded; When the residual error ratio information does not reach a set threshold, it is determined not to perform an intra-frame sub-block division check on the unit to be encoded.

7. The video encoding processing method according to any one of claims 1 to 6, characterized in that: The video encoding processing method further includes: In response to a judgment result of performing an intra-frame sub-block division check on the unit to be encoded, encoding processing is performed on the unit to be encoded based on an intra-frame sub-block division technology.

8. The video encoding processing method according to claim 7, characterized in that: The encoding process of the unit to be encoded based on the intra-frame sub-block division technology includes: Determine a sub-block division check result within the frame of the unit to be encoded; An intra-frame prediction mode for the unit to be encoded is determined based on the intra-frame sub-block division check result, and encoding processing is performed on the unit to be encoded according to the intra-frame prediction mode.

9. The video encoding processing method according to claim 8, characterized in that: The determining of the intra-frame prediction mode for the unit to be encoded based on the intra-frame sub-block division check result includes: When the intra-frame sub-block division check result satisfies the intra-frame sub-block division processing condition, determining that the intra-frame prediction mode of the unit to be encoded is to perform encoding processing on the unit to be encoded based on the intra-frame sub-block division technology; When the intra-frame sub-block division check result does not meet the intra-frame sub-block division processing condition, it is determined that the intra-frame prediction mode of the unit to be encoded is to encode the unit to be encoded based on adjacent encoded units.

10. A video encoding processing device, characterized in that: It includes a unit division module, a residual determination module, a residual analysis module and a coding processing module, wherein: The unit division module is configured to divide the unit to be encoded into a plurality of sub-blocks to be encoded; The residual determination module is configured to determine the residual ratio information of the unit to be encoded according to the residual information of each of the sub-blocks to be encoded, wherein the residual ratio information is determined based on the distribution of the residual information of the sub-blocks to be encoded in the unit to be encoded; The residual analysis module is configured to determine whether to perform intra-frame sub-block division check on the unit to be encoded based on the residual ratio information; The encoding processing module is configured to perform encoding processing on the unit to be encoded based on adjacent encoded units in response to a judgment result that the intra-frame sub-block division check is not performed on the unit to be encoded.

11. A video encoding processing device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding processing method as described in any one of claims 1-9.

12. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the video encoding processing method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video encoding processing method according to any one of claims 1 to 9 is implemented.

Images