Video encoding processing method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide a video encoding processing method, apparatus, device, and storage medium. The technical solution provided by the embodiments of the present application divides an encoding-waiting unit into a plurality of encoding-waiting sub-blocks, determines the residual occupancy rate information of each encoding-waiting sub-block, determines whether to perform an intra-frame sub-block division inspection on the encoding-waiting unit based on the residual occupancy rate information, and when it is determined not to perform an intra-frame sub-block division inspection on the encoding-waiting unit, encoding processing can be performed on the current encoding-waiting unit based on adjacent encoded units, thereby reducing the redundant intra-frame sub-block division inspection process, effectively reducing the complexity of intra-frame prediction, and improving the video encoding efficiency.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of encoding, and in particular to a video encoding processing method, apparatus, device and storage medium.

[0002] The present application claims priority to Chinese Patent Application No. 202210839092.7 filed with the China National Intellectual Property Administration on July 14, 2022, the entire content of which is incorporated herein by reference.

Background Art

[0003] When encoding video images, the international video coding standard H.266 / VVC often uses Intra Sub-partition (ISP) technology to perform intra prediction. Based on the intra prediction of coding units in the existing international video coding standard HEVC, the current coding unit is divided into a plurality of sub-blocks in the same direction in the horizontal or vertical direction, prediction is performed individually for each sub-block in sequence, and the reconstructed value after prediction of each sub-block can be used as a reference for the next sub-block, thereby reducing the residual after prediction of each sub-block and achieving more efficient video compression.

[0004] In order to select an intra sub-block partition mode suitable for the current coding unit, each coding unit subjected to intra sub-block partition checking needs to perform horizontal and vertical partitioning of intra sub-blocks respectively for several high-probability intra angular modes in the angular mode candidate list, and screen out one intra sub-block partition mode with the minimum rate-distortion cost of the current coding unit through rate-distortion cost determination. In this process, a large amount of redundant intra sub-block partition calculation is involved, which greatly increases the complexity of intra prediction and leads to low video encoding efficiency.

Summary of the Invention

Means for Solving the Problem

[0005] Embodiments of the present application provide a video encoding processing method, apparatus, device, and storage medium, thereby solving the technical problems in related technologies where the calculation of in-frame subblock division is redundant, the complexity of in-frame prediction is high, and the video encoding efficiency is low, thereby reducing the complexity of in-frame prediction and improving video encoding efficiency.

[0006] In the first embodiment, an embodiment of the present application provides a video encoding processing method, said video encoding processing method Dividing the coding waiting unit into multiple coding waiting subblocks, Based on the residual information of each of the coding waiting subblocks, the residual occupancy information of the coding waiting unit is determined. Based on the residual occupancy information, a determination is made as to whether or not to perform an in-frame subblock partitioning check on the encoding waiting unit. In response to a decision not to perform an in-frame subblock partitioning check on the aforementioned encoding-awaiting unit, encoding processing is performed on the aforementioned encoding-awaiting unit based on the adjacent encoded unit. Includes.

[0007] In a second embodiment of the present application, a video encoding apparatus is provided, the video encoding apparatus includes a unit division module, a residual determination module, a residual analysis module, and an encoding processing module. The unit division module is arranged to divide the coding waiting unit into multiple coding waiting subblocks. The residual determination module is configured to determine the residual occupancy information of the coding waiting unit based on the residual information of each coding waiting subblock. The residual analysis module is configured to determine whether or not to perform an in-frame subblock partitioning check on the coding waiting unit based on the residual occupancy information. The encoding processing module is configured to perform encoding on the waiting unit based on adjacent encoded units, in response to a decision not to perform an in-frame subblock division check on the waiting unit.

[0008] In a third aspect, embodiments of the present application provide a video encoding processing device, the video encoding processing device comprising memory and one or more processors, The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, they cause the one or more processors to execute the video encoding method described in the first embodiment.

[0009] In a fourth embodiment of the present invention, a storage medium for storing computer-executable instructions is provided, which is used to execute the video encoding method described in the first embodiment when the computer-executable instructions are executed by a computer processor.

[0010] In the fifth embodiment, an embodiment of the present application provides a computer program product, the computer program product comprising a computer program stored in a computer storage medium readable storage medium, and at least one processor of the device reads and executes the computer program from the computer storage medium readable storage medium so that the device performs the video encoding processing method described in the first embodiment.

[0011] The embodiment of the present invention divides a coding waiting unit into multiple coding waiting subblocks, determines the residual occupancy information of each coding waiting subblock, determines whether or not to perform an in-frame subblock division check on the coding waiting unit based on the residual occupancy information, and if it is determined not to perform an in-frame subblock division check on the coding waiting unit, performs coding on the current coding waiting unit based on the adjacent coded unit, thereby reducing the redundant in-frame subblock division check process, effectively reducing the complexity of in-frame prediction, and improving video coding efficiency. [Brief explanation of the drawing]

[0012] [Figure 1] This is a flowchart of the video encoding processing method provided in the embodiment of the present application. [Figure 2] This is a flowchart of another video encoding method provided in the embodiments of the present application. [Figure 3] This is a schematic diagram of the definitive flow of the residual information calculation method provided in the embodiment of the present application. [Figure 4] This is a schematic diagram of the decision flow for frame subblock division inspection provided in the embodiment of the present application. [Figure 5] This is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. [Figure 6] This is a schematic diagram of the structure of a video encoding processing device provided in an embodiment of the present application. [Modes for carrying out the invention]

[0013] To further clarify the purpose, technical proposal, and advantages of this application, specific embodiments of this application will be described in more detail below, accompanied by drawings. It should be understood that the specific embodiments described herein are not limiting to this application and are used solely for illustrative purposes. For the sake of clarity, the drawings show only the parts relevant to this application, not all of it. Before describing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods explained in flowcharts. In flowcharts, each operation (or step) is described as a sequential process, but many of these operations can occur in parallel, simultaneously, or be performed simultaneously. The order of the operations can be rearranged. Once the operations are completed, the process may be terminated, but additional steps not included in the drawings may be included. The above processes can correspond to methods, functions, recipes, subroutines, subprograms, etc.

[0014] Figure 1 shows a flowchart of a video encoding processing method provided in an embodiment of the present invention, which can be executed by a video encoding processing device, which can be implemented in hardware and / or software and integrated into a video encoding processing device.

[0015] The following explanation will describe a video encoding processing device executing a video encoding method as an example. Referring to Figure 1, the video encoding method includes the following:

[0016] S101: Divide the encoding waiting unit into multiple encoding waiting subblocks.

[0017] Here, the unit to be encoded is a coding unit (CU, Coding Unit) in a video frame that requires encoding processing. There are a plurality of coding units in a video frame, and it can be understood that before the encoding of the current to-be-encoded video frame is completed, encoding processing is performed sequentially for each coding unit according to a set order (for example, an order from left to right and from top to bottom) based on a set video coding standard (for example, the international video coding standard H.266 / VVC, the international video coding standard H.265 / HEVC).

[0018] In the related art, in order to achieve more efficient image compression, the international video coding standard H.266 / VVC usually performs encoding processing on coding units by using an intra sub-partition (ISP, Intra Sub-partition) technique. As a type of intra prediction technique in the latest generation of the international video coding standard H.266 / VVC, the intra sub-partition technique divides the current coding unit into a plurality of sub-blocks in the same direction, either horizontal or vertical, based on the intra prediction of the existing coding unit, performs prediction individually for each sub-block in sequence, and determines a reconstruction value. The reconstructed value of each sub-block after prediction can be used as a reference for the next sub-block, which reduces the residual after prediction of each sub-block and achieves a more efficient compression effect.

[0019] The in-frame subblock partitioning algorithm is performed after the angle prediction of the awaiting unit is complete. That is, after determining the candidate list of angle modes for the in-frame prediction performed by the awaiting unit, the in-frame subblock partitioning algorithm divides the coding unit into two or four subblocks in the same direction, and, using each subblock as a unit, selects and predicts the angle modes that are earlier in the candidate list of angle modes in order. In the process of in-frame subblock partitioning prediction, several modes can be traversed from the top of the candidate list of angle modes for the current awaiting unit. In the process of a single in-frame subblock partitioning check, each subblock uses the same in-frame angle mode, and the reconstructed pixels of each subblock can be used as the reference pixels of the next subblock, shortening the distance between each subblock and its reference pixels, increasing the correlation between pixels in the subblock and the reference pixels, thereby reducing the predicted residuals and improving the efficiency of compression. However, in order to select an in-frame subblock partitioning mode that fits the current coding waiting unit, each coding waiting unit performing an in-frame subblock partitioning check must perform horizontal and vertical in-frame subblock partitioning for several high-probability in-frame angular modes in the candidate list of angular modes, and then select the type of in-frame subblock partitioning mode that has the smallest rate distortion cost for the current coding waiting unit based on rate distortion cost determination. However, the above in-frame subblock partitioning check process significantly increases the complexity of in-frame prediction and limits video coding efficiency.To solve the above technical problem, the present solution divides a to-be-encoded unit into a plurality of to-be-encoded sub-blocks based on possible division manners of the intra sub-block partitioning technique in advance before performing intra sub-block partitioning check on the to-be-encoded unit, determines whether the current to-be-encoded sub-block needs to undergo intra sub-block partitioning check based on residual occupancy information of the to-be-encoded sub-blocks, avoids the need of performing intra sub-block partitioning check on all to-be-encoded units, reduces redundant intra sub-block partitioning check processes, improves video encoding efficiency, and determines whether the current to-be-encoded unit is suitable for intra sub-block partitioning check by using the distribution situation of residuals after angle prediction of the current to-be-encoded unit in the to-be-encoded sub-blocks, thereby skipping redundant intra sub-block partitioning calculation and realizing acceleration of video encoding.

[0020] For illustration, for each to-be-encoded unit that needs to be encoded, after determining a candidate list of angular modes for intra prediction performed by the to-be-encoded unit, the to-be-encoded unit is divided into a plurality of to-be-encoded sub-blocks. For example, the to-be-encoded unit is divided into two or four to-be-encoded sub-blocks based on a possible division manner provided by the intra sub-block partitioning technique.

[0021] Here, the number of subblocks to be divided into coding await can be determined based on the dimensions of the coding await unit. For example, if the unit dimensions of the coding await unit are within a first dimension range (e.g., 4*8 or 8*4), the coding await unit can be divided into a first number (e.g., 2) coding await subblocks. If the unit dimensions of the coding await unit are within a second dimension range (e.g., 4*8 or larger or 8*4 or larger), the coding await unit can be divided into a second number (e.g., 4) coding await subblocks, where the dimensions corresponding to the second dimension range are larger than the dimensions corresponding to the first dimension range. If the unit dimensions of the coding await unit are within a third dimension range (e.g., 4*4), no division is performed on the coding await unit, where the dimensions corresponding to the first dimension range are larger than the dimensions corresponding to the third dimension range, where the dimensions corresponding to the third dimension range may be the minimum coding dimensions for video coding.

[0022] For example, for a unit awaiting coding with unit dimensions of 4*4, the minimum coding dimension of the international video coding standard H.266 / VVC is 4*4, and the in-frame subblock division technique requires that each subblock have at least 16 pixels. Therefore, the 4*4 unit awaiting coding must be predicted as a single whole, eliminating the need for subsequent steps such as performing in-frame prediction using adjacent coded units to determine whether or not an in-frame subblock division check is necessary. For a unit awaiting coding with unit dimensions of 4*8 or 8*4, it can be divided into two subblocks. For a unit awaiting coding with unit dimensions of 4*8 or larger, or 8*4 or larger, it can be divided into four subblocks.

[0023] S102: Determine the residual occupancy information of each coding awaiting unit based on the residual information of each coding awaiting subblock.

[0024] For example, after obtaining multiple coding-awaiting subblocks of a coding-awaiting unit, residual information corresponding to each coding-awaiting subblock is determined, and residual occupancy information of the coding-awaiting unit is determined based on the residual information of each coding-awaiting subblock. Here, the residual information can represent the residual between the predicted value and the original pixel value of the coding-awaiting subblock, and this residual can represent the relationship between the coding-awaiting subblock and the reference pixel. In other words, the residual information can characterize the size of the information amount of the residual of the coding-awaiting subblock. Generally, the stronger the relationship between the coding-awaiting subblock and the reference pixel, the smaller the residual between the predicted value and the original pixel value of the corresponding coding-awaiting subblock.

[0025] Here, the residual information of a subblock awaiting coding can be represented by information that can characterize the degree of misjudgment after prediction of the unit awaiting coding, for example, by one or more combinations of the following: Hadamard transformation points (HAD), sum of absolute difference (SAD), mean squared error (MSE), sum of squared difference (SSD), and rate distortion cost (RD-Cost). Here, the residual occupancy information of the unit awaiting coding can be represented based on the situation in which the residual information of the subblocks awaiting coding is distributed to the unit awaiting coding, for example, the residual occupancy information may be the occupancy rate of the largest residual among the residual information of these subblocks awaiting coding.

[0026] S103: Based on the residual occupancy information, determine whether or not to perform an in-frame subblock partitioning check on the encoding waiting unit.

[0027] For example, after determining the residual occupancy information of a unit awaiting coding, a decision is made based on the residual occupancy information whether or not to perform an in-frame subblock division check on the current unit awaiting coding. For instance, if the residual occupancy information reflects that the relationship between a partial pixel in the unit awaiting coding and a reference pixel in an adjacent coded unit does not reach a set strength, it can be determined that an in-frame subblock division check is necessary for the unit awaiting coding. Furthermore, by determining the in-frame prediction method for the unit awaiting coding based on the in-frame subblock division check results, the rate distortion cost in the coding process is reduced, and the video coding compression ratio is increased. If the residual occupancy information reflects that the relationship between a pixel in the unit awaiting coding and a reference pixel in an adjacent coded unit has reached a set strength, the in-frame subblock division check on the current unit awaiting coding can be skipped, reducing the redundant in-frame subblock division check process and improving video coding efficiency.

[0028] S104: In response to the decision not to perform an in-frame subblock partitioning check on the unit awaiting encoding, encoding is performed on the unit awaiting encoding based on the adjacent encoded unit.

[0029] For example, if a decision is made not to perform an in-frame subblock partitioning check on a unit awaiting encoding, it is considered that there is a strong correlation between the pixels in the current unit awaiting encoding and the reference pixels of the adjacent encoded unit. In this case, the in-frame subblock partitioning check for the current unit awaiting encoding can be skipped, and the encoding process can be performed directly on the unit awaiting encoding based on the adjacent encoded unit, significantly improving the encoding speed of in-frame predictions for units awaiting encoding during the video encoding process.

[0030] In one possible embodiment, in the step of determining whether or not to perform an in-frame subblock partitioning check on a unit awaiting coding based on residual occupancy information, if it is determined that an in-frame subblock partitioning check should be performed on the unit awaiting coding, the coding process for the unit awaiting coding can be completed according to the full in-frame subblock partitioning technique. Based on this, the present solution further includes, after determining whether or not to perform an in-frame subblock partitioning check on a unit awaiting coding based on residual occupancy information, performing the coding process on the unit awaiting coding based on the in-frame subblock partitioning technique in response to the determination result that an in-frame subblock partitioning check should be performed on the unit awaiting coding.

[0031] For example, if the decision to perform an in-frame subblock partitioning check on a unit awaiting encoding is finalized, it is considered that the relationship between the pixels in the current unit awaiting encoding and the reference pixels of the adjacent encoded unit is weak, and it is necessary to perform encoding on the unit awaiting encoding based on the in-frame subblock partitioning technique. That is, an in-frame subblock partitioning check is performed on the unit awaiting encoding from the horizontal and / or vertical directions, and the optimal in-frame prediction method is selected based on the rate distortion cost determination, and the encoding process is performed.

[0032] In one possible embodiment, when encoding is performed on a unit awaiting encoding based on an in-frame subblock partitioning technique, the process includes determining the in-frame subblock partitioning check result for the unit awaiting encoding, determining an in-frame prediction scheme for the unit awaiting encoding based on the in-frame subblock partitioning check result, and performing encoding on the unit awaiting encoding according to the in-frame prediction scheme.

[0033] For example, if it is determined that an in-frame subblock partitioning check is necessary for a unit awaiting coding, the results of the horizontal and vertical in-frame subblock partitioning checks for the unit awaiting coding are determined. Based on the in-frame subblock partitioning check results, the rate distortion cost of coding based on horizontal in-frame subblock partitioning, the rate distortion cost of coding based on vertical in-frame subblock partitioning, and the rate distortion cost of coding the unit awaiting coding as a whole unit are calculated. The in-frame prediction method corresponding to the minimum rate distortion cost is set as the current in-frame prediction method for the unit awaiting coding, and coding is performed on the unit awaiting coding according to that in-frame prediction method. In other words, each coding-waiting unit, using coding-waiting subblocks as units, sequentially selects and predicts the angular modes that appear earlier in the candidate list of angular modes, and performs an in-frame subblock partitioning check. For several in-frame angular modes with high probabilities in the candidate list of angular modes, it is necessary to perform horizontal and vertical in-frame subblock partitioning, respectively. Based on the rate distortion cost determination, coding is performed by selecting an in-frame prediction method that minimizes the rate distortion cost of the current coding-waiting unit from among coding based on horizontal in-frame subblock partitioning, coding based on vertical in-frame subblock partitioning, and coding as a whole coding unit, and then performing the coding process.

[0034] In one possible embodiment, if the present solution determines the in-frame prediction method for the encoding waiting unit based on the in-frame subblock partitioning check result, The method includes the following: if the in-frame subblock partitioning check result satisfies the in-frame subblock partitioning processing conditions, the in-frame prediction method for the awaiting coding unit is determined to be a method that performs coding on the awaiting coding unit based on the in-frame subblock partitioning technique; and if the in-frame subblock partitioning check result does not satisfy the in-frame subblock partitioning processing conditions, the in-frame prediction method for the awaiting coding unit is determined to be a method that performs coding on the awaiting coding unit based on the adjacent coded unit.

[0035] For example, after determining the results of the in-frame subblock partitioning check for the unit awaiting encoding, it is determined whether the in-frame subblock partitioning check results satisfy the in-frame subblock partitioning processing conditions. Here, whether the in-frame subblock partitioning check results satisfy the in-frame subblock partitioning processing conditions may be determined by calculating a first rate distortion cost for encoding based on horizontal in-frame subblock partitioning, a second rate distortion cost for encoding based on vertical in-frame subblock partitioning, and a third rate distortion cost for encoding performed by the unit awaiting encoding as a whole encoding unit, based on the in-frame subblock partitioning check results. If the first rate distortion cost or the second rate distortion cost is smaller than the third rate distortion cost, it is considered that the in-frame subblock partitioning check results satisfy the in-frame subblock partitioning processing conditions. If both the first rate distortion cost and the second rate distortion cost are greater than or equal to the third rate distortion cost, it is considered that the in-frame subblock partitioning check results do not satisfy the in-frame subblock partitioning processing conditions.

[0036] If the in-frame subblock division check result satisfies the in-frame subblock division processing conditions, the in-frame prediction method for the awaiting coding unit is determined to be a method that performs coding on the awaiting coding unit based on the in-frame subblock division technique. That is, the awaiting coding unit is divided in the direction corresponding to the minimum rate distortion cost to obtain multiple subblocks (awaiting coding subblocks), and coding is performed on each subblock unit by referencing the adjacent coded subblock or adjacent coded unit. If the in-frame subblock division check result does not satisfy the in-frame subblock division processing conditions, the in-frame prediction method for the awaiting coding unit is determined to be a method that performs coding on each awaiting coding unit based on the adjacent coded unit. That is, coding is performed on the entire awaiting coding unit by referencing the adjacent coded unit.

[0037] In summary, by dividing a unit awaiting encoding into multiple subblocks, determining the residual occupancy information for each subblock, and determining whether or not to perform an in-frame subblock division check on the unit based on the residual occupancy information, if it is determined that an in-frame subblock division check should not be performed on the unit, the encoding process can be performed on the current unit based on the adjacent encoded unit, thereby reducing the redundant in-frame subblock division check process, effectively reducing the complexity of in-frame prediction, and improving video encoding efficiency. Furthermore, if it is determined that an in-frame subblock division check is necessary for the unit awaiting encoding, the in-frame prediction method for the unit is determined based on the in-frame subblock division check results, effectively improving the video compression effect. In the process of a single in-frame subblock division check, the in-frame angular mode used by each subblock is the same, and the reconstructed pixels of each subblock can be used as the reference pixels of the next subblock, shortening the distance between each subblock and its reference pixels, strengthening the correlation between pixels in the subblock and the reference pixels, effectively reducing the residual of in-frame prediction, and improving video encoding compression efficiency.

[0038] Based on the above embodiment, Figure 2 shows a flowchart of another video encoding method provided in the embodiment of the present application, which is an embodiment of the above video encoding method. Referring to Figure 2, the video encoding method includes the following:

[0039] S201: Divide the coding waiting unit into multiple coding waiting subblocks.

[0040] S202: Determine the method for calculating residual information based on the subblock dimensions of the subblock awaiting encoding.

[0041] As an example, after dividing a coding awaiting unit into multiple coding awaiting subblocks, the method for calculating residual information for each coding awaiting subblock is determined based on the subblock dimensions of each coding awaiting subblock, with different subblock dimensions corresponding to different residual calculation methods. Here, the method for calculating residual information may be one or more combinations of the number of Hadamard transform points, the sum of absolute differences, the mean squared error, the sum of squared errors, and the rate distortion cost of the coding awaiting subblock. In this embodiment, the example of showing residual information using the number of Hadamard transform points and the sum of absolute differences is explained.

[0042] In one possible embodiment, when encoding is finally performed on a waiting unit or waiting subblock, a residual transformation operation is also required. Therefore, a residual information calculation method that takes residual transformation into account can be preferentially used to calculate residual information, thereby improving video encoding efficiency. Based on this, as shown in the schematic diagram of the determination flow of the residual information calculation method provided in Figure 3, this solution, when determining the residual information calculation method based on the subblock dimensions of the waiting subblock, includes the following:

[0043] S2021: Determine the dimension range corresponding to the subblock dimensions of the subblock awaiting encoding.

[0044] S2022: If the subblock dimensions of a subblock awaiting encoding are within the first setting range, the calculation method for the residual information of the subblock awaiting encoding is determined as the first setting calculation method, and the side lengths of the dimensions within the first setting range are all greater than the first setting value.

[0045] S2023: If the subblock dimensions of a subblock awaiting encoding fall within the second setting range, the calculation method for the residual information of the subblock awaiting encoding is determined as the second setting calculation method, and dimensions within the second setting range have side lengths whose numerical value is the first setting value.

[0046] For example, after dividing a coding awaiting unit into multiple coding awaiting subblocks, the subblock dimensions corresponding to these coding awaiting subblocks (which have the same dimensions) are determined, and the dimensional range corresponding to the subblock dimensions of the coding awaiting subblocks is determined. Here, the dimensional range provided in this solution includes a first setting range and a second setting range, and the numerical values ​​of the side lengths of the dimensions within the first setting range are all greater than the first setting value, while the dimensions within the second setting range include side lengths whose numerical value is the first setting value. For example, if the first setting value is 1, the dimensional range corresponding to the first setting range is not 1*N or N*1, where N is 8 or greater, and the dimensional range corresponding to the second setting range is 1*N or N*1. That is, if a coding awaiting subblock has a side length of the first setting value (e.g., 1), the subblock dimensions of the coding awaiting subblock are within the second setting range, and if all the side lengths of the coding awaiting subblocks are greater than the first setting value, the subblock dimensions of the coding awaiting subblocks are within the first setting range.

[0047] If it is determined that the subblock dimensions of a subblock awaiting encoding fall within the first set range, the method for calculating the residual information of the subblock awaiting encoding is determined as the first set calculation method. If the subblock dimensions of a subblock awaiting encoding fall within the second set range, the method for calculating the residual information of the subblock awaiting encoding is determined as the second set calculation method. Here, the first set calculation method is a method for calculating the number of Hadamard transform points of a subblock awaiting encoding, and the second set calculation method is a method for calculating the sum of absolute differences (or one of the mean squared error, sum of squared errors, and rate distortion cost) of a subblock awaiting encoding. Note that the calculation process for the number of Hadamard transform points takes into account the effect of residual transformation to some extent, so it better represents the size of the residuals after transformation and quantization, helps to more accurately determine whether or not to skip the in-frame subblock division check, and can improve video encoding efficiency.

[0048] S203: Calculate the residual information for each coding await subblock according to the residual information calculation method, and determine the maximum residual occupancy rate of the coding await unit based on the residual information.

[0049] Here, this solution represents the residual occupancy information of a unit awaiting coding by the maximum residual occupancy corresponding to each piece of residual information, and more intuitively and effectively reflects the correlation between pixels in the current unit awaiting coding and adjacent coded units.

[0050] For example, after determining the method for calculating residual information for each coding-awaiting subblock, the residual information for each coding-awaiting subblock is calculated based on the determined residual information calculation method. Furthermore, the maximum residual occupancy rate of the current coding-awaiting unit is calculated based on the residual information of these coding-awaiting subblocks. That is, the maximum residual information among the residual information of these coding-awaiting subblocks is calculated, and the occupancy rate of the maximum residual information in all residual information is calculated to obtain the maximum residual occupancy rate of the coding-awaiting unit.

[0051] In one possible embodiment, when calculating the maximum residual occupancy of a coding awaiting unit, the calculation can be performed based on different residual information. Based on this, the present solution calculates the residual information of each coding awaiting subblock according to a residual information calculation method, and when determining the maximum residual occupancy of a coding awaiting unit based on the residual information, if the residual information calculation method is a first setting calculation method, it includes calculating the number of Hadamard transform points of each coding awaiting subblock and determining the maximum residual occupancy of the coding awaiting subblock based on the occupancy rate of the maximum number of Hadamard transform points; and if the residual information calculation method is a second setting calculation method, it includes calculating the sum of absolute differences of each coding awaiting subblock and determining the maximum residual occupancy of the coding awaiting subblock based on the occupancy rate of the maximum sum of absolute differences.

[0052] For example, if the residual information calculation method is the first setting calculation method (i.e., calculating the number of Hadamard transform points for each subblock awaiting coding), the number of Hadamard transform points for each subblock awaiting coding is calculated. The maximum number of Hadamard transform points among these is determined, the occupancy rate of the maximum number of Hadamard transform points among these Hadamard transform points is calculated, and this occupancy rate of the maximum number of Hadamard transform points among these Hadamard transform points is taken as the maximum residual occupancy rate for the current subblock awaiting coding.

[0053] If the residual information calculation method is the first setting calculation method (in this case, the subblock dimensions of the coding awaiting subblock are within the first setting range, for example, the subblock dimensions of the coding awaiting subblock are not 1*N or N*1), then, taking the example of dividing the coding awaiting unit into four coding awaiting subblocks, the maximum residual occupancy rate of the coding awaiting subblocks is [Equation 1].

[0054]

number

[0055] Here, [Equation 2] is the number of Hadamard transform points corresponding to four coding-awaiting subblocks.

[0056]

number

[0057] If the residual information calculation method is the second setting calculation method (i.e., calculating the sum of absolute differences of the subblocks awaiting coding), the sum of absolute differences of each subblock awaiting coding is calculated. The largest sum of absolute differences among these sums of absolute differences is determined, the occupancy rate of the largest sum of absolute differences among these sums of absolute differences is calculated, and this occupancy rate of the largest sum of absolute differences among these sums of absolute differences is taken as the maximum residual occupancy rate of the current subblock awaiting coding.

[0058] If the residual information calculation method is the second setting calculation method (in this case, the subblock dimensions of the coding awaiting subblock are within the second setting range, for example, the subblock dimensions of the coding awaiting subblock are 1*N or N*1), then, taking the example of dividing the coding awaiting unit into four coding awaiting subblocks, the maximum residual occupancy rate of the coding awaiting subblocks is [Equation 3].

[0059]

number

[0060] Here, [Equation 4] is the sum of the absolute differences corresponding to four coding-awaiting subblocks.

[0061]

number

[0062] S204: Based on residual occupancy information, determine whether or not to perform an in-frame subblock partitioning check on the encoding waiting unit.

[0063] For example, after determining the residual occupancy information of a unit awaiting coding, it is further determined whether or not to skip the in-frame subblock partitioning check for the unit awaiting coding based on the residual occupancy information. Here, the determination of whether or not to skip the in-frame subblock partitioning check can be determined based on the result of comparing the residual occupancy information with a pre-set threshold. Based on this, as shown in the schematic diagram of the in-frame subblock partitioning check determination flow provided in Figure 4, this solution includes the following when determining whether or not to perform an in-frame subblock partitioning check for a unit awaiting coding based on the residual occupancy information.

[0064] S2041: The residual occupancy information determines whether or not the set threshold has been reached.

[0065] S2042: If the residual occupancy information reaches the set threshold, it is confirmed to perform an in-frame subblock partitioning check on the encoding waiting unit.

[0066] S2043: If the residual occupancy information does not reach a certain threshold, it is determined that no in-frame subblock partitioning check will be performed on the encoding-awaiting unit.

[0067] For example, after determining the maximum residual occupancy of a unit awaiting encoding, the maximum residual occupancy is compared with a set threshold to determine whether the maximum residual occupancy has reached the set threshold. If the current unit awaiting encoding has reached the set threshold, it is considered possible that the unit will select the in-frame subblock division mode as the optimal mode after the in-frame subblock division check and perform the encoding process. If the current unit awaiting encoding has not reached the set threshold, it is considered unlikely that the unit will select the in-frame subblock division mode as the optimal mode after the in-frame subblock division check and perform the encoding process. Therefore, it is decided not to perform the in-frame subblock division check on the current unit awaiting encoding, thereby reducing the redundant in-frame subblock division check process.

[0068] In one possible embodiment, the residual information provided in this solution includes first residual information in the first direction of the coding awaiting subblock (e.g., horizontal direction) and second residual information in the second direction of the coding awaiting subblock (e.g., vertical direction). Accordingly, the residual occupancy information (maximum residual occupancy) provided in this solution includes first residual occupancy information in the first direction of the coding awaiting subblock and second residual occupancy information in the second direction of the coding awaiting subblock. Here, the calculation of residual information and residual occupancy information in each direction is similar to the residual information calculation method described above, and a detailed explanation is omitted in this embodiment. Accordingly, for the first and second directions respectively, the residual occupancy information (maximum residual occupancy) in the corresponding direction is calculated, and the residual occupancy information is compared with a set threshold for the corresponding direction. If the residual occupancy information for one direction has reached the set threshold, the current unit awaiting encoding may perform encoding with the selected in-frame subblock division mode as the optimal mode after undergoing in-frame subblock division check for the corresponding direction. Therefore, it is confirmed that the in-frame subblock division check for the corresponding direction will be performed on the unit awaiting encoding. If the residual occupancy information for both directions has not reached the set threshold, the probability that the current unit awaiting encoding will perform encoding with the selected in-frame subblock division mode as the optimal mode after undergoing in-frame subblock division check for both directions is considered small. Therefore, it is confirmed that the in-frame subblock division check for the corresponding direction will not be performed on the unit awaiting encoding.

[0069] S205: In response to the decision not to perform an in-frame subblock partitioning check on the unit awaiting encoding, encoding is performed on the unit awaiting encoding based on the adjacent encoded unit.

[0070] In summary, by dividing a unit awaiting encoding into multiple subblocks, determining the residual occupancy information for each subblock, and deciding whether or not to perform an in-frame subblock division check on the unit based on the residual occupancy information, and then deciding whether or not to perform an in-frame subblock division check on the unit, encoding can be performed on the current unit based on adjacent encoded units. This reduces the redundant in-frame subblock division check process, effectively reduces the complexity of in-frame prediction, and improves video encoding efficiency. At the same time, by using the number of Hadamard transform points, which takes the effects of transformation into account, as a standard for measuring the residual distribution, the size of the residuals after transformation and quantization is better represented, and a more accurate in-frame subblock division skip decision is achieved. This method effectively maintains consistency in video quality after encoding, and since the data used to calculate residual information is obtained from the in-frame calculation process of the original encoding waiting unit, it requires less memory, ensuring highly efficient video encoding, reducing the impact on the encoder's objective performance, and maintaining subjective consistency in image quality. By increasing the encoder's encoding speed, it increases the utilization of computation resources on the server side, saving server costs, and lowering the resource utilization of the client's processor, thereby effectively improving the user experience.

[0071] Figure 5 is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present invention. Referring to Figure 5, the video encoding device includes a unit division module 51, a residual determination module 52, a residual analysis module 53, and an encoding processing module 54.

[0072] Here, the unit division module 51 is configured to divide the coding-awaiting unit into multiple coding-awaiting subblocks. The residual determination module 52 is configured to determine the residual occupancy information of the coding-awaiting unit based on the residual information of each coding-awaiting subblock. The residual analysis module 53 is configured to determine whether or not to perform an in-frame subblock division check on the coding-awaiting unit based on the residual occupancy information. The coding processing module 54 is configured to respond to the decision not to perform an in-frame subblock division check on the coding-awaiting unit and to perform coding processing on the coding-awaiting unit based on the adjacent coded unit.

[0073] In summary, by dividing a coding awaiting unit into multiple coding awaiting subblocks, determining the residual occupancy information for each coding awaiting subblock, and determining whether or not to perform an in-frame subblock division check on the coding awaiting unit based on the residual occupancy information, and deciding whether or not to perform an in-frame subblock division check on the coding awaiting unit, the coding process can be performed on the current coding awaiting unit based on the adjacent coded unit, thereby reducing the redundant in-frame subblock division check process, effectively reducing the complexity of in-frame prediction, and improving video coding efficiency.

[0074] Based on the above embodiment, the residual information includes first residual information in the first direction of the encoding awaiting subblock and second residual information in the second direction of the encoding awaiting subblock.

[0075] Based on the above embodiment, the residual determination module 52 is: Based on the subblock dimensions of the subblock awaiting encoding, the method for calculating residual information is determined. The residual information for each coding await subblock is calculated according to the residual information calculation method, and the maximum residual occupancy rate of the coding await unit is determined based on the residual information. They are arranged in this manner.

[0076] Based on the above embodiment, the residual determination module 52 determines the method for calculating residual information based on the subblock dimensions of the encoding await subblock. If the subblock dimensions of a subblock awaiting encoding fall within the first setting range, the calculation method for the residual information of the subblock awaiting encoding is determined as the first setting calculation method, and the side lengths of the dimensions within the first setting range are all greater than the first setting value. If the subblock dimensions of a subblock awaiting encoding fall within the second setting range, the calculation method for the residual information of the subblock awaiting encoding is determined as the second setting calculation method, and dimensions within the second setting range have side lengths whose numerical value is the first setting value. They are arranged in this manner.

[0077] Based on the above embodiment, the residual determination module 52 calculates the residual information of each coding waiting subblock according to the residual information calculation method, and when determining the maximum residual occupancy rate of the coding waiting unit based on the residual information, If the residual information calculation method is the first setting calculation method, the number of Hadamard transform points for each coding await subblock is calculated, and the maximum residual occupancy rate of the coding await subblock is determined based on the occupancy rate of the maximum number of Hadamard transform points. If the residual information calculation method is the second setting calculation method, the sum of absolute differences of each coding await subblock is calculated, and the maximum residual occupancy rate of the coding await subblock is determined based on the occupancy rate of the maximum absolute difference sum in the sum of absolute differences. They are arranged in this manner.

[0078] Based on the above embodiment, the residual analysis module 53 is: When the residual occupancy information reaches the set threshold, it is confirmed that an in-frame subblock partitioning check will be performed on the encoding waiting unit. If the residual occupancy information does not reach a certain threshold, it is determined that no in-frame subblock partitioning check will be performed on the encoding-awaiting unit. They are arranged in this manner.

[0079] Based on the above embodiment, the encoding processing module 54 is further configured to perform encoding processing on the encoding waiting unit based on the in-frame subblock division technique in response to the determination result of performing an in-frame subblock division check on the encoding waiting unit.

[0080] Based on the above embodiment, when the encoding processing module 54 performs encoding processing on the encoding waiting unit based on the in-frame subblock division technique, Confirm the results of the in-frame subblock partitioning check for the encoding awaiting unit. Based on the results of the in-frame subblock partitioning check, the in-frame prediction method for the units awaiting coding is determined, and coding is performed on the units awaiting coding according to the in-frame prediction method. They are arranged in this manner.

[0081] Based on the above embodiment, when the encoding processing module 54 determines the in-frame prediction method for the encoding waiting unit based on the in-frame subblock division check result, If the in-frame subblock partitioning check result satisfies the in-frame subblock partitioning processing conditions, the in-frame prediction method for the encoding-awaiting unit is determined to be a method that performs encoding processing on the encoding-awaiting unit based on the in-frame subblock partitioning technique. If the in-frame subblock partitioning check result does not satisfy the in-frame subblock partitioning processing conditions, the in-frame prediction method for the unit awaiting encoding is determined to be a method that performs encoding on the unit awaiting encoding based on the adjacent encoded unit. They are arranged in this manner.

[0082] In the above embodiment of the video encoding processing device, each included unit and module is divided according to functional logic, but this division is not limited to the above; it is sufficient if the corresponding function can be realized. Furthermore, the specific names of each functional unit are set to be easily distinguishable from one another and do not limit the scope of protection of the embodiment of this application.

[0083] Embodiments of the present application also provide a video encoding processing device, which can integrate the video encoding processing device provided in the embodiments of the present application. Figure 6 is a schematic diagram of the structure of the video encoding processing device provided in the embodiments of the present application. Referring to Figure 6, the video encoding processing device includes an input device 63, an output device 64, a memory 62, and one or more processors 61, the memory 62 for storing one or more programs, and when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the video encoding processing method provided in the embodiments described above. The provided video encoding processing device, device, and computer are used to execute the video encoding processing method provided in any of the embodiments described above and have corresponding functions and beneficial effects.

[0084] Embodiments of the present application also provide a storage medium that stores computer-executable instructions, which, when executed by a computer processor, are used to execute the video encoding method provided in the above-described embodiment. Of course, with respect to the storage medium that stores the computer-executable instructions provided in the embodiment of the present application, the computer-executable instructions can not only execute the video encoding method provided as described above, but can also execute related operations in the video encoding method provided in any embodiment of the present application. The video encoding processing apparatus, device, and storage medium provided in the above-described embodiment can execute the video encoding method provided in any embodiment of the present application, and technical details not described in detail in the above-described embodiment can be referenced to the video encoding method provided in any embodiment of the present application.

[0085] In some possible embodiments, each mode of the method provided in this disclosure may be implemented as a program product, which includes program code, and when the program product is run on a computer device, the program code is used to cause the computer device to perform steps in the methods according to the various exemplary embodiments of the disclosure described herein, for example, the computer device may perform the video encoding processing method described in the embodiments of the disclosure. Here, the program product may employ one or more readable media in any combination.

Claims

1. Dividing the coding waiting unit into multiple coding waiting subblocks, Based on the residual information of each of the coding waiting subblocks, the residual occupancy information of the coding waiting unit is determined. Based on the residual occupancy information, a determination is made as to whether or not to perform an in-frame subblock partitioning check on the encoding waiting unit. In response to a decision not to perform an in-frame subblock partitioning check on the aforementioned encoding-awaiting unit, encoding processing is performed on the aforementioned encoding-awaiting unit based on the adjacent encoded unit. Includes, Based on the residual information of each of the aforementioned coding waiting subblocks, determining the residual occupancy information of the coding waiting unit is: Based on the subblock dimensions of the aforementioned encoding awaiting subblock, the method for calculating residual information is determined. The residual information of each coding waiting subblock is calculated according to the residual information calculation method, and the maximum residual occupancy rate of the coding waiting unit is determined based on the residual information. Includes, Determining the method for calculating residual information based on the subblock dimensions of the aforementioned encoding awaiting subblock is: If the subblock dimensions of the encoding-awaiting subblock are within the first setting range, the calculation method for the residual information of the encoding-awaiting subblock is determined as the first setting calculation method, and the numerical values ​​of the side lengths of the dimensions within the first setting range are all greater than the first setting value. If the subblock dimensions of the coding await subblock are within the second setting range, the calculation method for the residual information of the coding await subblock is determined as the second setting calculation method, and the dimensions within the second setting range include a side length whose numerical value is the first setting value. Includes, Calculating residual information for each of the coding waiting subblocks according to the residual information calculation method, and determining the maximum residual occupancy rate of the coding waiting unit based on the residual information, When the residual information calculation method is the first setting calculation method, the number of Hadamard transform points for each of the coding awaiting subblocks is calculated, and the maximum residual occupancy rate of the coding awaiting subblock is determined based on the occupancy rate of the maximum number of Hadamard transform points in the number of Hadamard transform points. When the residual information calculation method is the second setting calculation method, the sum of the absolute differences of each of the coding awaiting subblocks is calculated, and the maximum residual occupancy rate of the coding awaiting subblocks is determined based on the occupancy rate of the maximum absolute difference sum in the sum of absolute differences. including, Video encoding processing method.

2. The residual information includes a first residual information in the first direction of the coding awaiting subblock and a second residual information in the second direction of the coding awaiting subblock. The video encoding processing method according to claim 1.

3. Based on the residual occupancy information, determining whether or not to perform an in-frame subblock partitioning check on the encoding waiting unit is: When the residual occupancy information reaches a set threshold, it is determined that an in-frame subblock partitioning check will be performed on the encoding waiting unit. If the residual occupancy information does not reach the set threshold, it is determined that no in-frame subblock partitioning check will be performed on the encoding waiting unit. including, The video encoding processing method according to claim 1.

4. The aforementioned video encoding processing method further includes: The process includes performing encoding on the aforementioned encoding waiting unit based on an in-frame subblock division technique in response to a determination result of performing an in-frame subblock division check on the aforementioned encoding waiting unit. The video encoding processing method according to claim 1.

5. Performing encoding processing on the encoding waiting unit based on the aforementioned in-frame subblock division technique means that To determine the result of the in-frame subblock division check for the aforementioned encoding waiting unit, Based on the results of the in-frame subblock division check, the in-frame prediction method for the coding waiting unit is determined, and coding processing is performed on the coding waiting unit according to the in-frame prediction method. including, The video encoding processing method according to claim 4.

6. Determining the in-frame prediction method for the encoding waiting unit based on the results of the in-frame subblock division check is: If the in-frame subblock division check result satisfies the in-frame subblock division processing conditions, the in-frame prediction method for the coding waiting unit is determined to be a method that performs coding on the coding waiting unit based on the in-frame subblock division technique. If the result of the in-frame subblock division check does not satisfy the in-frame subblock division processing conditions, the in-frame prediction method for the waiting-to-code unit is determined to be a method that performs encoding on the waiting-to-code unit based on adjacent encoded units. including, The video encoding processing method according to claim 5.

7. It includes a unit division module, a residual determination module, a residual analysis module, and an encoding processing module. The unit division module is arranged to divide the coding waiting unit into multiple coding waiting subblocks. The residual determination module is configured to determine the residual occupancy information of the coding waiting unit based on the residual information of each coding waiting subblock. The residual analysis module is configured to determine whether or not to perform an in-frame subblock partitioning check on the coding waiting unit based on the residual occupancy information. The encoding processing module is configured to respond to a decision result indicating that no in-frame subblock partitioning check should be performed on the encoding-waiting unit, and to perform encoding processing on the encoding-waiting unit based on the adjacent encoded unit. Based on the residual information of each of the aforementioned coding waiting subblocks, determining the residual occupancy information of the coding waiting unit is: Based on the subblock dimensions of the aforementioned encoding awaiting subblock, the method for calculating residual information is determined. The residual information of each coding waiting subblock is calculated according to the residual information calculation method, and the maximum residual occupancy rate of the coding waiting unit is determined based on the residual information. Includes, Determining the method for calculating residual information based on the subblock dimensions of the aforementioned encoding awaiting subblock is: If the subblock dimensions of the encoding-awaiting subblock are within the first setting range, the calculation method for the residual information of the encoding-awaiting subblock is determined as the first setting calculation method, and the numerical values ​​of the side lengths of the dimensions within the first setting range are all greater than the first setting value. If the subblock dimensions of the coding await subblock are within the second setting range, the calculation method for the residual information of the coding await subblock is determined as the second setting calculation method, and the dimensions within the second setting range include a side length whose numerical value is the first setting value. Includes, Calculating residual information for each of the coding waiting subblocks according to the residual information calculation method, and determining the maximum residual occupancy rate of the coding waiting unit based on the residual information, When the residual information calculation method is the first setting calculation method, the number of Hadamard transform points for each of the coding awaiting subblocks is calculated, and the maximum residual occupancy rate of the coding awaiting subblock is determined based on the occupancy rate of the maximum number of Hadamard transform points in the number of Hadamard transform points. When the residual information calculation method is the second setting calculation method, the sum of the absolute differences of each of the coding awaiting subblocks is calculated, and the maximum residual occupancy rate of the coding awaiting subblocks is determined based on the occupancy rate of the maximum absolute difference sum in the sum of absolute differences. including, Video encoding and processing unit.

8. It includes memory and one or more processors, The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding processing method described in any one of claims 1 to 6. Video encoding processing device.

9. When executed by a computer processor, the computer-executable instructions are used to perform the video encoding processing method according to any one of claims 1 to 6. A storage medium that stores computer-executable commands.

10. When executed by a processor, the video encoding processing method according to any one of claims 1 to 6 is realized. Computer program.

Images