Video Encoding Processing Method, Apparatus, Device, and Storage Medium

Abstract

Embodiments of the present application provide a video encoding processing method, apparatus, device, and storage medium. The technical solution provided by the embodiments of the present application divides an encoding-waiting unit into a plurality of encoding-waiting sub-blocks, determines the residual occupancy rate information of each encoding-waiting sub-block, determines whether to perform an intra-frame sub-block division inspection on the encoding-waiting unit based on the residual occupancy rate information, and when it is determined not to perform an intra-frame sub-block division inspection on the encoding-waiting unit, encoding processing can be performed on the current encoding-waiting unit based on adjacent encoded units, thereby reducing the redundant intra-frame sub-block division inspection process, effectively reducing the complexity of intra-frame prediction, and improving the video encoding efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of encoding, and particularly to a video encoding processing method, apparatus, device, and storage medium.

[0002] This application claims the priority of the application with the application number 202210839092.7 filed with the China National Intellectual Property Administration on July 14, 2022, and all its contents are incorporated herein by reference.

Background Art

[0003] When encoding a video image, the international standard for video encoding, H.266 / VVC, often performs intra prediction by applying the intra sub-partition (ISP) technology. Based on the intra prediction of the existing video encoding international standard HEVC coding unit, the current coding unit is divided into multiple sub-blocks in the same direction horizontally or vertically, and individual predictions are made for each sub-block in sequence. Moreover, the reconstructed value after prediction of each sub-block can be used as a reference for the next sub-block, thereby reducing the residual after prediction of each sub-block and achieving more efficient video compression.

[0004] In order to select an intra sub-block partitioning mode that fits the current coding unit, each coding unit performing an intra sub-block partitioning check divides the intra sub-block horizontally and vertically for several intra angular modes with high probabilities in the candidate list of angular modes, and needs to select a kind of intra sub-block partitioning mode with the smallest rate-distortion cost of the current coding unit by rate-distortion cost determination. In this process, a lot of redundant intra sub-block partitioning calculations are included, greatly increasing the complexity of intra prediction and resulting in low video encoding efficiency.

Summary of the Invention

Means for Solving the Problems

[0005] In an embodiment of the present application, a video encoding processing method, apparatus, device, and storage medium are provided, so as to solve the technical problems in the related art that the intra-frame sub-block division calculation is redundant, the complexity of intra-frame prediction is high, and the video encoding efficiency is low, reduce the complexity of intra-frame prediction, and improve the video encoding efficiency.

[0006] In a first aspect, an embodiment of the present application provides a video encoding processing method, and the video encoding processing method includes: dividing an encoding-waiting unit into a plurality of encoding-waiting sub-blocks; determining residual occupancy information of the encoding-waiting unit based on residual information of each of the encoding-waiting sub-blocks; judging whether to perform an intra-frame sub-block division check on the encoding-waiting unit based on the residual occupancy information; responding to a determination result of not performing an intra-frame sub-block division check on the encoding-waiting unit, and performing an encoding process on the encoding-waiting unit based on an adjacent encoded unit; and the like.

[0007] In a second aspect, an embodiment of the present application provides a video encoding processing apparatus, and the video encoding processing apparatus includes a unit division module, a residual determination module, a residual analysis module, and an encoding processing module, wherein the unit division module is arranged to divide an encoding-waiting unit into a plurality of encoding-waiting sub-blocks, the residual determination module is arranged to determine residual occupancy information of the encoding-waiting unit based on residual information of each of the encoding-waiting sub-blocks, the residual analysis module is arranged to judge whether to perform an intra-frame sub-block division check on the encoding-waiting unit based on the residual occupancy information, The encoding processing module is arranged to perform an encoding process on the unit waiting for encoding based on an adjacent encoded unit in response to a determination result of not performing an in-frame sub-block division check on the unit waiting for encoding.

[0008] In a third aspect, an embodiment of the present application provides a video encoding processing device, which includes a memory and one or more processors. The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the video encoding processing method described in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer-executable instructions, which is used to execute the video encoding processing method described in the first aspect when the computer-executable instructions are executed by a computer processor.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium so that the device executes the video encoding processing method described in the first aspect.

[0011] Embodiments of the present application divide an encoding-waiting unit into a plurality of encoding-waiting sub-blocks, determine the residual occupancy information of each encoding-waiting sub-block, and determine whether to perform an intra-frame sub-block division check on the encoding-waiting unit based on the residual occupancy information. When it is determined not to perform an intra-frame sub-block division check on the encoding-waiting unit, encoding processing can be performed on the current encoding-waiting unit based on adjacent encoded units, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving video encoding efficiency.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following will further elaborate on specific embodiments of this application in conjunction with the drawings. It should be understood that the specific embodiments described herein do not limit this application and are only used to explain this application. For the convenience of explanation, only the parts related to this application are shown in the drawings, not all the content. Before elaborating on exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods explained by flowcharts. In a flowchart, each operation (or step) is explained as a sequential process, but many of these operations can be performed in parallel, simultaneously, or concurrently. Additionally, the order of each operation can be rearranged. When the operation is completed, the above process can be terminated, but it can also include additional steps not included in the drawings. The above process can correspond to methods, functions, recipes, subroutines, subprograms, etc.

[0014] FIG. 1 shows a flowchart of a video encoding processing method provided by an embodiment of this application. The video encoding processing method provided by an embodiment of this application can be executed by a video encoding processing device. The video encoding processing device can be implemented in a hardware and / or software manner and can be integrated into a video encoding processing device.

[0015] Hereinafter, an example will be used to explain the execution of the video encoding processing method by the video encoding processing device. Referring to FIG. 1, the video encoding processing method includes the following.

[0016] S101: Divide the encoding waiting unit into a plurality of encoding waiting sub-blocks.

[0017] Here, the encoding waiting unit is an encoding unit (CU, Coding Unit) in a video frame that requires encoding processing. There are multiple encoding units in a video frame, and until the encoding of the current encoding waiting video frame is completed, based on the set video encoding standard (for example, the international video encoding standard H.266 / VVC, the international video encoding standard H.265 / HEVC), it can be understood that the encoding process is performed on each encoding unit in sequence according to the set order (for example, from left to right, from top to bottom).

[0018] In related technologies, in order to achieve more efficient image compression, the international video encoding standard H.266 / VVC usually performs encoding processing on the encoding unit by using the intra sub-partition technology (ISP, Intra Sub-partition). The intra sub-partition technology, as a kind of intra prediction technology in the latest generation of international video encoding standard H.266 / VVC, is based on the intra prediction of the existing encoding unit, divides the current encoding unit into multiple sub-blocks in the same direction horizontally or vertically, performs individual predictions on each sub-block in sequence, and determines the reconstructed value. The reconstructed value after prediction of each sub-block can be used as a reference for the next sub-block, reducing the residual after prediction of each sub-block and achieving a more efficient compression effect.

[0019] The intra-frame sub-block division algorithm is performed after the angle prediction of the unit to be encoded is completed. That is, after determining the candidate list of the angle modes of the intra-frame prediction performed by the unit to be encoded, the intra-frame sub-block division algorithm divides the encoding unit into two or four sub-blocks in the same direction, and, taking the sub-blocks as units, sequentially selects the angle modes in the front of the candidate list of the angle modes to perform prediction. In the process of intra-frame sub-block division prediction, several modes from the top of the candidate list of the current unit-to-be-encoded angle modes can be traversed. In the process of one intra-frame sub-block division inspection, the intra-frame angle modes used by each sub-block are all the same, the reconstructed pixels of each sub-block can be used as the reference pixels of the next sub-block, the distance between each sub-block and its reference pixels is shortened, the correlation between the pixels and the reference pixels in the sub-block becomes stronger, thereby reducing the predicted residual and improving the compression efficiency. However, in order to select the intra-frame sub-block division mode that suits the current unit to be encoded, each unit to be encoded performing the intra-frame sub-block division inspection needs to perform the intra-frame sub-block division in the horizontal and vertical directions respectively for several intra-frame angle modes with high probabilities in the candidate list of the angle modes, and select the intra-frame sub-block division mode with the smallest rate-distortion cost of the current unit to be encoded based on the rate-distortion cost determination. However, the above intra-frame sub-block division inspection process greatly increases the complexity of the intra-frame prediction and limits the video encoding efficiency.To solve the above technical problem, before performing an intra-frame sub-block division check on an encoding pending unit, the present solution first divides the encoding pending unit into a plurality of encoding pending sub-blocks based on possible division methods of the intra-frame sub-block division technique, determines whether the current encoding pending sub-block needs to perform an intra-frame sub-block division check based on the residual occupancy information of the encoding pending sub-block, and does not need to perform an intra-frame sub-block division check on all encoding pending units, reducing the redundant intra-frame sub-block division check process, improving the video encoding efficiency, and utilizing the situation where the residual after angle prediction of the current encoding pending unit is distributed among the encoding pending sub-blocks to determine whether the current encoding pending unit is suitable for the intra-frame sub-block division check, thereby skipping the redundant intra-frame sub-block division calculation and realizing the acceleration of video encoding.

[0020] Exemplarily, for each encoding pending unit that needs to be encoded, after determining a candidate list of angle modes for intra-frame prediction performed by the encoding pending unit, the encoding pending unit is divided into a plurality of encoding pending sub-blocks. For example, the encoding pending unit is divided into two or four encoding pending sub-blocks based on possible division methods by the intra-frame sub-block division technique.

[0021] Here, the number of sub-blocks waiting for encoding after being split can be determined based on the dimension of the unit waiting for encoding. For example, when the unit dimension of the unit waiting for encoding is within the first dimension range (e.g., 4*8 or 8*4), the unit waiting for encoding can be split into the first number (e.g., 2) of sub-blocks waiting for encoding. When the unit dimension of the unit waiting for encoding is within the second dimension range (e.g., 4*8 or more or 8*4 or more), the unit waiting for encoding can be split into the second number (e.g., 4) of sub-blocks waiting for encoding, where the dimension corresponding to the second dimension range is larger than the dimension corresponding to the first dimension range. When the unit dimension of the unit waiting for encoding is within the third dimension range (e.g., 4*4), no splitting is performed on the unit waiting for encoding, where the dimension corresponding to the first dimension range is larger than the dimension corresponding to the third dimension range, and here, the dimension corresponding to the third dimension range may be the minimum encoding dimension for video encoding.

[0022] For example, for a unit waiting for encoding with a unit dimension of 4*4, the minimum encoding dimension of the video encoding international standard H.266 / VVC is 4*4, and since the intra-frame sub-block splitting technology requires that the sub-block dimension needs at least 16 pixels, it is necessary to perform prediction on the 4*4 unit waiting for encoding as a whole. For example, perform intra-frame prediction using adjacent encoded units, and it is necessary to perform subsequent steps to determine whether it is necessary to perform intra-frame sub-block splitting inspection. For a unit waiting for encoding with a unit dimension of 4*8 or 8*4, it can be split into two sub-blocks waiting for encoding. For a unit waiting for encoding with a unit dimension of 4*8 or more or 8*4 or more, it can be split into four sub-blocks waiting for encoding.

[0023] S102: Determine the residual occupancy information of the unit waiting for encoding based on the residual information of each sub-block waiting for encoding.

[0024] Exemplarily, after obtaining a plurality of sub-blocks to be encoded in the encoding waiting unit, residual information corresponding to each sub-block to be encoded is determined, and residual occupancy information of the encoding waiting unit is determined based on the residual information of each sub-block to be encoded. Here, the residual information can represent the residual between the predicted value and the original pixel value of the sub-block to be encoded, and the residual can represent the relevance between the sub-block to be encoded and the reference pixel, that is, the residual information can characterize the size of the information amount of the residual of the sub-block to be encoded. Generally, the stronger the relevance between the sub-block to be encoded and the reference pixel, the smaller the residual between the predicted value and the original pixel value of the corresponding sub-block to be encoded.

[0025] Here, the residual information of the sub-block to be encoded can be indicated by information that can characterize the degree of distortion after prediction of the encoding waiting unit. For example, it can be represented by one or a combination of multiple of the Hadamard transform points (HAD), sum of absolute differences (SAD), mean squared error (MSE), sum of squared differences (SSD), and rate-distortion cost (RD-Cost). Here, the residual occupancy information of the encoding waiting unit can be represented based on the situation where the residual information of the sub-block to be encoded is distributed in the encoding waiting unit. For example, the residual occupancy information may be the occupancy of the maximum residual information among these residual information of these sub-blocks to be encoded.

[0026] S103: Determine whether to perform an intra-frame sub-block division check on the encoding waiting unit based on the residual occupancy information.

[0027] Exemplarily, after determining the residual occupancy ratio information of the unit to be encoded, based on the residual occupancy ratio information, it is determined whether to perform an intra-frame sub-block division check on the current unit to be encoded. For example, when the residual occupancy ratio information reflects that the relevance between the partial pixels in the unit to be encoded and the reference pixels of the adjacent encoded units does not reach the set strength, it can be determined that it is necessary to perform an intra-frame sub-block division check on the unit to be encoded, and based on the intra-frame sub-block division check result, determine the intra-frame prediction method for the unit to be encoded, thereby reducing the rate-distortion cost in the encoding process and increasing the video encoding compression ratio. When the residual occupancy ratio information reflects that the relevance between the pixels in the unit to be encoded and the reference pixels of the adjacent encoded units all reach the set strength, the intra-frame sub-block division check for the current unit to be encoded can be skipped, reducing the redundant intra-frame sub-block division check process and improving the video encoding efficiency.

[0028] S104: In response to the determination result of not performing an intra-frame sub-block division check on the unit to be encoded, based on the adjacent encoded units, perform an encoding process on the unit to be encoded.

[0029] Exemplarily, when determining the result of not performing an intra-frame sub-block division check on the unit to be encoded, it is considered that the relevance between the pixels in the current unit to be encoded and the reference pixels of the adjacent encoded units is strong, skip the intra-frame sub-block division check for the current unit to be encoded, and directly perform an encoding process on the unit to be encoded based on the adjacent encoded units, thereby significantly improving the encoding speed of the intra-frame prediction of the unit to be encoded in the video encoding process.

[0030] In one possible embodiment, in the step of determining whether to perform an intra-frame sub-block division inspection on an encoding unit based on the residual occupancy information, if it is determined that it is necessary to perform an intra-frame sub-block division inspection on the encoding unit, the encoding process for the encoding unit can be completed according to the complete intra-frame sub-block division technique. Based on this, this solution further includes, after determining whether to perform an intra-frame sub-block division inspection on an encoding unit based on the residual occupancy information, responding to the determination result of performing an intra-frame sub-block division inspection on the encoding unit, and performing an encoding process on the encoding unit based on the intra-frame sub-block division technique.

[0031] Exemplarily, when the determination result of performing an intra-frame sub-block division inspection on the encoding unit is determined, it is considered that the correlation between the pixels in the current encoding unit and the reference pixels of the encoded unit adjacent thereto is weak, and it is necessary to perform an encoding process on the encoding unit based on the intra-frame sub-block division technique. That is, perform an intra-frame sub-block division inspection on the encoding unit from the horizontal direction and / or the vertical direction, and select the optimal intra-frame prediction method based on the rate-distortion cost determination to perform the encoding process.

[0032] In one possible embodiment, when performing an encoding process on an encoding unit based on the intra-frame sub-block division technique, it includes determining the intra-frame sub-block division inspection result for the encoding unit, determining the intra-frame prediction method for the encoding unit based on the intra-frame sub-block division inspection result, and performing an encoding process on the encoding unit according to the intra-frame prediction method.

[0033] Exemplarily, when it is determined that it is necessary to perform an intra-frame sub-block division inspection on the unit to be encoded, the horizontal and vertical intra-frame sub-block division inspection results for the unit to be encoded are determined, and based on the intra-frame sub-block division inspection results, the rate-distortion cost for encoding based on the horizontal intra-frame sub-block division, the rate-distortion cost for encoding based on the vertical intra-frame sub-block division, and the rate-distortion cost for encoding the unit to be encoded as the entire encoding unit are calculated, and the intra-frame prediction method corresponding to the minimum rate-distortion cost is set as the intra-frame prediction method of the current unit to be encoded, and encoding processing is performed on the unit to be encoded according to the intra-frame prediction method. That is, taking the sub-block to be encoded as a unit, the angle modes in the front of the candidate list of the angle mode are sequentially selected for prediction, and each unit to be encoded that performs the intra-frame sub-block division inspection needs to perform horizontal and vertical intra-frame sub-block divisions for several intra-frame angle modes with high probabilities in the candidate list of the angle mode. Based on the rate-distortion cost determination, from the options of encoding based on the horizontal intra-frame sub-block division, encoding based on the vertical intra-frame sub-block division, and encoding as the entire encoding unit, a kind of intra-frame prediction method with the minimum rate-distortion cost of the current unit to be encoded is selected for encoding processing.

[0034] In one possible embodiment, for this solution, when determining the intra-frame prediction method for the unit to be encoded based on the intra-frame sub-block division inspection result, When the intra-frame sub-block division inspection result satisfies the intra-frame sub-block division processing condition, the intra-frame prediction method for the unit waiting for encoding is determined to be a method of performing encoding processing on the unit waiting for encoding based on the intra-frame sub-block division technology. When the intra-frame sub-block division inspection result does not satisfy the intra-frame sub-block division processing condition, the intra-frame prediction method for the unit waiting for encoding is determined to be a method of performing encoding processing on the unit waiting for encoding based on adjacent encoded units.

[0035] Exemplarily, after determining the intra-frame sub-block division inspection result for the unit waiting for encoding, it is determined whether the intra-frame sub-block division inspection result satisfies the intra-frame sub-block division processing condition. Here, whether the intra-frame sub-block division inspection result satisfies the intra-frame sub-block division processing condition may be determined by calculating a first rate-distortion cost for encoding based on the horizontal intra-frame sub-block division, a second rate-distortion cost for encoding based on the vertical intra-frame sub-block division, and a third rate-distortion cost for encoding the unit waiting for encoding as the entire encoding unit according to the intra-frame sub-block division inspection result. When the first rate-distortion cost or the second rate-distortion cost is smaller than the third rate-distortion cost, the intra-frame sub-block division inspection result is considered to satisfy the intra-frame sub-block division processing condition. When both the first rate-distortion cost and the second rate-distortion cost are equal to or greater than the third rate-distortion cost, the intra-frame sub-block division inspection result is considered not to satisfy the intra-frame sub-block division processing condition.

[0036] When the in-frame sub-block division inspection result satisfies the in-frame sub-block division processing conditions, the in-frame prediction method for the unit waiting for encoding is determined to be a method of performing encoding processing on the unit waiting for encoding based on the in-frame sub-block division technology. That is, the unit waiting for encoding is divided into a plurality of sub-blocks (sub-blocks waiting for encoding) in the direction corresponding to the minimum rate-distortion cost, and encoding processing is performed with adjacent encoded sub-blocks or adjacent encoded units as references on a sub-block basis. When the in-frame sub-block division inspection result does not satisfy the in-frame sub-block division processing conditions, the in-frame prediction method for the unit waiting for encoding is determined to be a method of performing encoding processing on the unit waiting for encoding based on adjacent encoded units. That is, encoding processing is performed with adjacent encoded units as references on the entire unit waiting for encoding as a unit.

[0037] As described above, the unit to be coded is divided into a plurality of sub-blocks to be coded, and the residual occupancy information of each sub-block to be coded is determined. Based on the residual occupancy information, it is determined whether to perform an intra-frame sub-block division check on the unit to be coded. If it is determined not to perform the intra-frame sub-block division check on the unit to be coded, coding processing can be performed on the current unit to be coded based on adjacent coded units, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving the video coding efficiency. Further, if it is determined that an intra-frame sub-block division check needs to be performed on the unit to be coded, an intra-frame prediction method for the unit to be coded is determined based on the intra-frame sub-block division check result, effectively improving the video compression effect. In the process of one intra-frame sub-block division check, the intra-frame angular modes used by each sub-block are all the same, the reconstructed pixels of each sub-block can be used as the reference pixels of the next sub-block, the distance between each sub-block and its reference pixels is shortened, the correlation between the pixels and the reference pixels in the sub-block becomes stronger, the residual of the intra-frame prediction is effectively reduced, and the video coding compression efficiency is improved.

[0038] Based on the above embodiments, FIG. 2 shows a flowchart of another video coding processing method provided by an embodiment of the present application, and the video coding processing method is a specific implementation of the above video coding processing method. Referring to FIG. 2, the video coding processing method includes the following.

[0039] S201: Divide the unit to be coded into a plurality of sub-blocks to be coded.

[0040] S202: Determine a calculation method for residual information based on the sub-block dimensions of the sub-block to be coded.

[0041] Exemplarily, after dividing the unit to be encoded into a plurality of sub-blocks to be encoded, based on the sub-block size of each sub-block to be encoded, a calculation method for the residual information of each sub-block to be encoded is determined, and different sub-block sizes correspond to different residual calculation methods. Here, the calculation method of the residual information may be to calculate one or a combination of multiple types among the number of Hadamard transform points, sum of absolute differences, mean squared error, sum of squared errors, and rate-distortion cost of the sub-block to be encoded. In this embodiment, an example will be described by adopting the number of Hadamard transform points and the sum of absolute differences to represent the residual information.

[0042] In one possible embodiment, when finally performing an encoding process on the unit to be encoded or the sub-block to be encoded, since an operation of performing a residual transform is also required, a calculation method of residual information considering the residual transform can be preferentially used to calculate the residual information, thereby improving the video encoding efficiency. Based on this, as shown in the schematic diagram of the determination flow of the calculation method of the residual information provided in FIG. 3, when this solution determines the calculation method of the residual information based on the sub-block size of the sub-block to be encoded, it includes the following.

[0043] S2021: Determine the dimension range corresponding to the sub-block size of the sub-block to be encoded.

[0044] S2022: When the sub-block size of the sub-block to be encoded is within the first set range, determine the calculation method of the residual information of the sub-block to be encoded as the first set calculation method, and the numerical values of the side lengths of the dimensions within the first set range are all greater than the first set value.

[0045] S2023: When the sub-block size of the sub-block to be encoded is within the second set range, determine the calculation method of the residual information of the sub-block to be encoded as the second set calculation method, and among the dimensions within the second set range, there is a side length with a numerical value equal to the first set value.

[0046] Exemplarily, after dividing the unit to be encoded into a plurality of sub-blocks to be encoded, the sub-block dimensions corresponding to these sub-blocks to be encoded (the dimensions of these sub-blocks to be encoded are the same) are determined, and a dimension range corresponding to the sub-block dimension of the sub-block to be encoded is determined. Here, the dimension range provided in this solution includes a first setting range and a second setting range, and the numerical values of the side lengths of the dimensions within the first setting range are all greater than the first setting value, and there is a side length whose numerical value is the first setting value among the dimensions within the second setting range. For example, when the first setting value is 1, the dimension range corresponding to the first setting range is not 1*N or N*1, where N is 8 or more, and the dimension range corresponding to the second setting range is 1*N or N*1. That is, when there is a side length of the first setting value (for example, 1) in the sub-block to be encoded, the sub-block dimension of the sub-block to be encoded is within the second setting range, and when all the side lengths of the sub-block to be encoded are greater than the first setting value, the sub-block dimension of the sub-block to be encoded is within the first setting range.

[0047] When it is determined that the sub-block dimension of the sub-block to be encoded is within the first setting range, the calculation method of the residual information of the sub-block to be encoded is determined as the first setting calculation method. When the sub-block dimension of the sub-block to be encoded is within the second setting range, the calculation method of the residual information of the sub-block to be encoded is determined as the second setting calculation method. Here, the first setting calculation method is a method for calculating the number of Hadamard transform points of the sub-block to be encoded, and the second setting calculation method is a method for calculating the sum of absolute differences (or one of the mean squared error, sum of squared errors, and rate distortion cost) of the sub-block to be encoded. Note that in the process of calculating the number of Hadamard transform points, the influence of residual transform is considered to a certain extent, so it can better represent the size of the residual after transformation and quantization, and helps to more accurately determine whether to skip the in-frame sub-block division inspection, and can improve the video encoding efficiency.

[0048] S203: Calculate the residual information of each sub-block to be encoded according to the calculation method of the residual information, and determine the maximum residual occupancy rate of the unit to be encoded based on the residual information.

[0049] Here, this solution represents the residual occupancy information of the unit to be encoded by the maximum residual occupancy rate corresponding to each residual information, and more intuitively and effectively reflects the correlation between the pixels in the current unit to be encoded and the encoded units adjacent thereto.

[0050] Exemplarily, after determining the calculation method of the residual information of the sub-blocks to be encoded, calculate the residual information of each sub-block to be encoded based on the determined calculation method of the residual information respectively, and calculate the maximum residual occupancy rate of the current unit to be encoded based on the residual information of these sub-blocks to be encoded. That is, calculate the maximum residual information among the residual information of these sub-blocks to be encoded, and calculate the occupancy rate of the maximum residual information in all the residual information, so as to obtain the maximum residual occupancy rate of the unit to be encoded.

[0051] In one possible embodiment, when calculating the maximum residual occupancy rate of the unit to be encoded, the calculation can be performed based on different residual information. Based on this, when this solution calculates the residual information of each sub-block to be encoded according to the calculation method of the residual information and determines the maximum residual occupancy rate of the unit to be encoded based on the residual information, when the calculation method of the residual information is the first set calculation method, calculate the Hadamard transform points of each sub-block to be encoded, and determine the maximum residual occupancy rate of the sub-block to be encoded based on the occupancy rate of the maximum Hadamard transform points in the Hadamard transform points; when the calculation method of the residual information is the second set calculation method, calculate the sum of absolute differences of each sub-block to be encoded, and determine the maximum residual occupancy rate of the sub-block to be encoded based on the occupancy rate of the maximum sum of absolute differences in the sum of absolute differences.

[0052] Exemplarily, when the calculation method of the residual information is the first set calculation method (that is, calculating the number of Hadamard transform points of the sub-block waiting for encoding), calculate the number of Hadamard transform points of each sub-block waiting for encoding. Determine the maximum Hadamard transform point among these Hadamard transform points, calculate the occupancy rate of the maximum Hadamard transform point among these Hadamard transform points, and set the occupancy rate of the maximum Hadamard transform point among these Hadamard transform points as the maximum residual occupancy rate of the current sub-block waiting for encoding.

[0053] When the calculation method of the residual information is the first set calculation method (at this time, the sub-block size of the sub-block waiting for encoding is within the first set range. For example, the sub-block size of the sub-block waiting for encoding is not 1*N or N*1), taking the example of dividing the encoding waiting unit into four sub-blocks waiting for encoding, the maximum residual occupancy rate of the sub-block waiting for encoding is [Number 1].

[0054]

Number

[0055] Here, [Number 2] are the numbers of Hadamard transform points corresponding to the four sub-blocks waiting for encoding respectively.

[0056]

Number

[0057] When the calculation method of the residual information is the second set calculation method (that is, calculating the sum of absolute differences of the sub-block waiting for encoding), calculate the sum of absolute differences of each sub-block waiting for encoding. Determine the maximum sum of absolute differences among these sums of absolute differences, calculate the occupancy rate of the maximum sum of absolute differences among these sums of absolute differences, and set the occupancy rate of the maximum sum of absolute differences among these sums of absolute differences as the maximum residual occupancy rate of the current sub-block waiting for encoding.

[0058] When the calculation method of the residual information is the second set calculation method (at this time, the sub-block size of the sub-block to be coded is within the second set range. For example, the sub-block size of the sub-block to be coded is 1*N or N*1), taking the division of the unit to be coded into four sub-blocks to be coded as an example, the maximum residual occupancy rate of the sub-block to be coded is [Equation 3].

[0059]

Equation

[0060] Here, [Equation 4] is the sum of absolute differences corresponding to the four sub-blocks to be coded respectively.

[0061]

Equation

[0062] S204: Based on the residual occupancy rate information, determine whether to perform an in-frame sub-block division check on the unit to be coded.

[0063] Exemplarily, after determining the residual occupancy rate information of the unit to be coded, further determine whether it is possible to skip the in-frame sub-block division check on the unit to be coded based on the residual occupancy rate information. Here, whether it is possible to skip the in-frame sub-block division check can be determined based on the comparison result between the residual occupancy rate information and a preset set threshold. Based on this, as shown in the schematic diagram of the determination flow of the in-frame sub-block division check provided in FIG. 4, when this solution determines whether to perform an in-frame sub-block division check on the unit to be coded based on the residual occupancy rate information, it includes the following.

[0064] S2041: Determine whether the residual occupancy rate information has reached the set threshold.

[0065] S2042: When the residual occupancy information reaches the set threshold, it is determined to perform an intra-frame sub-block division check on the encoding-waiting unit.

[0066] S2043: When the residual occupancy information does not reach the set threshold, it is determined not to perform an intra-frame sub-block division check on the encoding-waiting unit.

[0067] Exemplarily, after determining the maximum residual occupancy of the encoding-waiting unit, the maximum residual occupancy is compared with the set threshold to determine whether the maximum residual occupancy has reached the set threshold. If the maximum residual occupancy of the current encoding-waiting unit reaches the set threshold, it is considered that the current encoding-waiting unit may select the intra-frame sub-block division mode as the optimal mode and perform encoding processing after the intra-frame sub-block division check. If the maximum residual occupancy of the current encoding-waiting unit does not reach the set threshold, since it is considered that the probability of the current encoding-waiting unit selecting the intra-frame sub-block division mode as the optimal mode and performing encoding processing after the intra-frame sub-block division check is small, it is determined not to perform an intra-frame sub-block division check on the current encoding-waiting unit, and the redundant intra-frame sub-block division check process is reduced.

[0068] In one possible embodiment, the residual information provided by the present solution includes first residual information in the first direction (e.g., the horizontal direction) of the sub-block to be encoded and second residual information in the second direction (e.g., the vertical direction) of the sub-block to be encoded. Accordingly, the residual occupancy information (maximum residual occupancy) provided by the present solution includes first residual occupancy information in the first direction of the sub-block to be encoded and second residual occupancy information in the second direction of the sub-block to be encoded. Here, the calculation of the residual information and the residual occupancy information in each direction is similar to the calculation method of the above residual information, and detailed description is omitted in this embodiment. Accordingly, for the first direction and the second direction respectively, the residual occupancy information (maximum residual occupancy) in the corresponding direction is calculated respectively, and the residual occupancy information is compared with the set threshold in the corresponding direction. If there is a case where the residual occupancy information in one direction reaches the set threshold, it is considered that after the in-frame sub-block division inspection in the corresponding direction, the current unit to be encoded may perform encoding processing with the in-frame sub-block division mode as the optimal mode. Therefore, it is determined to perform the in-frame sub-block division inspection in the corresponding direction on the unit to be encoded. If the residual occupancy information in both directions does not reach the set threshold, it is considered that the probability of selecting the in-frame sub-block division mode as the optimal mode and performing encoding processing after the in-frame sub-block division inspection in both directions is small for the current unit to be encoded. Therefore, it is determined not to perform the in-frame sub-block division inspection in the corresponding direction on the unit to be encoded.

[0069] S205: In response to the determination result of not performing the in-frame sub-block division inspection on the unit to be encoded, encoding processing is performed on the unit to be encoded based on the adjacent encoded units.

[0070] As described above, the unit to be coded is divided into a plurality of sub-blocks to be coded, and the residual occupancy information of each sub-block to be coded is determined. Based on the residual occupancy information, it is determined whether to perform an intra-frame sub-block division check on the unit to be coded. When it is determined not to perform an intra-frame sub-block division check on the unit to be coded, coding processing can be performed on the current unit to be coded based on adjacent coded units, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving the video coding efficiency. At the same time, the number of Hadamard transform points considering the influence of transformation is used as a standard for measuring the residual distribution, which better represents the size of the residual after transformation and quantization, and realizes a more accurate intra-frame sub-block division skip determination. The consistency of the coded video quality is effectively maintained, and the data used for calculating the residual information is all obtained from the intra-frame calculation process of the original unit to be coded. Therefore, the memory occupancy is small, ensuring the efficient execution of video coding, reducing the objective performance impact on the encoder, and maintaining the subjective picture quality consistency. On the premise of increasing the coding speed of the encoder, the calculation resource utilization rate at the server end is increased, the server cost is saved, the resource utilization rate of the client's processor is reduced, and the user experience is effectively improved.

[0071] FIG. 5 is a schematic structural diagram of a video coding processing apparatus provided by an embodiment of the present application. Referring to FIG. 5, the video coding processing apparatus includes a unit division module 51, a residual determination module 52, a residual analysis module 53, and a coding processing module 54.

[0072] Here, the unit division module 51 is arranged to divide the unit to be encoded into a plurality of sub-blocks to be encoded. The residual determination module 52 is arranged to determine the residual occupancy information of the unit to be encoded based on the residual information of each sub-block to be encoded. The residual analysis module 53 is arranged to determine whether to perform an intra-frame sub-block division check on the unit to be encoded based on the residual occupancy information. The encoding processing module 54 is arranged to perform an encoding process on the unit to be encoded based on the adjacent encoded unit in response to a determination result of not performing an intra-frame sub-block division check on the unit to be encoded.

[0073] As described above, by dividing the unit to be encoded into a plurality of sub-blocks to be encoded, determining the residual occupancy information of each sub-block to be encoded, determining whether to perform an intra-frame sub-block division check on the unit to be encoded based on the residual occupancy information, and when it is determined not to perform an intra-frame sub-block division check on the unit to be encoded, an encoding process can be performed on the current unit to be encoded based on the adjacent encoded unit, thereby reducing the redundant intra-frame sub-block division check process, effectively reducing the complexity of intra-frame prediction, and improving the video encoding efficiency.

[0074] Based on the above embodiments, the residual information includes first residual information in the first direction of the sub-block to be encoded and second residual information in the second direction of the sub-block to be encoded.

[0075] Based on the above embodiments, the residual determination module 52 determines a calculation method of the residual information based on the sub-block size of the sub-block to be encoded, and calculates the residual information of each sub-block to be encoded according to the calculation method of the residual information, and determines the maximum residual occupancy of the unit to be encoded based on the residual information. is arranged as such.

[0076] Based on the above embodiments, when the residual determination module 52 determines the calculation method of the residual information based on the sub-block size of the sub-block to be encoded, when the sub-block size of the sub-block to be encoded is within the first setting range, the calculation method of the residual information of the sub-block to be encoded is determined as the first setting calculation method, and the numerical values of the side lengths of the dimensions within the first setting range are all greater than the first setting value. when the sub-block size of the sub-block to be encoded is within the second setting range, the calculation method of the residual information of the sub-block to be encoded is determined as the second setting calculation method, and there is a side length with a numerical value of the first setting value among the dimensions within the second setting range. It is arranged as follows.

[0077] Based on the above embodiments, when the residual determination module 52 calculates the residual information of each sub-block to be encoded according to the calculation method of the residual information and determines the maximum residual occupancy rate of the unit to be encoded based on the residual information, when the calculation method of the residual information is the first setting calculation method, calculate the Hadamard transform points of each sub-block to be encoded, and determine the maximum residual occupancy rate of the sub-block to be encoded based on the occupancy rate of the maximum Hadamard transform points in the Hadamard transform points. when the calculation method of the residual information is the second setting calculation method, calculate the sum of absolute differences of each sub-block to be encoded, and determine the maximum residual occupancy rate of the sub-block to be encoded based on the occupancy rate of the maximum sum of absolute differences in the sum of absolute differences. It is arranged as follows.

[0078] Based on the above embodiments, the residual analysis module 53 when the residual occupancy rate information reaches the set threshold, it is determined to perform an in-frame sub-block division check on the unit to be encoded. when the residual occupancy rate information does not reach the set threshold, it is determined not to perform an in-frame sub-block division check on the unit to be encoded. It is arranged as follows.

[0079] Based on the above embodiments, the encoding processing module 54 is further arranged to perform an encoding process on the units waiting for encoding in response to the determination result of performing an intra-frame sub-block division inspection on the units waiting for encoding and based on the intra-frame sub-block division technique.

[0080] Based on the above embodiments, when the encoding processing module 54 performs an encoding process on the units waiting for encoding based on the intra-frame sub-block division technique, it determines the intra-frame sub-block division inspection result for the units waiting for encoding, determines the intra-frame prediction method for the units waiting for encoding based on the intra-frame sub-block division inspection result, and performs an encoding process on the units waiting for encoding according to the intra-frame prediction method. It is arranged in this way.

[0081] Based on the above embodiments, when the encoding processing module 54 determines the intra-frame prediction method for the units waiting for encoding based on the intra-frame sub-block division inspection result, when the intra-frame sub-block division inspection result meets the intra-frame sub-block division processing conditions, the intra-frame prediction method for the units waiting for encoding is determined as the method of performing an encoding process on the units waiting for encoding based on the intra-frame sub-block division technique, when the intra-frame sub-block division inspection result does not meet the intra-frame sub-block division processing conditions, the intra-frame prediction method for the units waiting for encoding is determined as the method of performing an encoding process on the units waiting for encoding based on the adjacent encoded units. It is arranged in this way.

[0082] It should be noted that in the above embodiments of the video encoding processing apparatus, each unit and module included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized. Also, the specific names of each functional unit are set to be easily distinguishable from each other and do not limit the protection scope of the embodiments of the present application.

[0083] Embodiments of the present application also provide a video encoding processing device, which can integrate the video encoding processing apparatus provided in the embodiments of the present application. FIG. 6 is a schematic structural diagram of the video encoding processing device provided in the embodiments of the present application. Referring to FIG. 6, the video encoding processing device includes an input device 63, an output device 64, a memory 62, and one or more processors 61. The memory 62 is for storing one or more programs. When the one or more programs are executed by the one or more processors 61, the one or more processors 61 are caused to implement the video encoding processing method provided in the above embodiments. The above-provided video encoding processing apparatus, device, and computer are used to execute the video encoding processing method provided in any of the above embodiments and have corresponding functions and beneficial effects.

[0084] Embodiments of the present application also provide a storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor of a computer, they are used to execute the video encoding processing method provided in the above embodiments. Of course, for the storage medium storing the computer-executable instructions provided in the embodiments of the present application, the computer-executable instructions can not only execute the provided video encoding processing method as described above, but also execute related operations in the video encoding processing method provided in any of the embodiments of the present application. The video encoding processing apparatus, device, and storage medium provided in the above embodiments can execute the video encoding processing method provided in any of the embodiments of the present application. For technical details not described in detail in the above embodiments, reference can be made to the video encoding processing method provided in any of the embodiments of the present application.

[0085] In some possible embodiments, each aspect of the method provided by the present disclosure can be realized in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to execute the steps in the method according to various exemplary embodiments of the present disclosure described herein. For example, the computer device can execute the video encoding processing method described in the embodiments of the present disclosure. Here, the program product can adopt any combination of one or more readable media.

Claims

1. Dividing the unit to be encoded into a plurality of sub-blocks to be encoded; Determining residual occupancy information of the unit to be encoded based on residual information of each of the sub-blocks to be encoded; Judging whether to perform in-frame sub-block division inspection on the unit to be encoded based on the residual occupancy information; In response to a determination result of not performing in-frame sub-block division inspection on the unit to be encoded, performing an encoding process on the unit to be encoded based on an adjacent encoded unit; comprising A video encoding processing method.

2. The residual information includes first residual information in a first direction of the sub-block to be encoded and second residual information in a second direction of the sub-block to be encoded. The video encoding processing method according to Claim 1.

3. Determining the residual occupancy information of the unit to be encoded based on the residual information of each of the sub-blocks to be encoded includes: Determining a calculation method of residual information based on a sub-block size of the sub-block to be encoded; Calculating residual information of each of the sub-blocks to be encoded according to the calculation method of the residual information, and determining a maximum residual occupancy rate of the unit to be encoded based on the residual information; comprising The video encoding processing method according to Claim 1.

4. Determining a calculation method of residual information based on a sub-block size of the sub-block to be encoded includes: When the sub-block size of the sub-block to be encoded is within a first set range, determining the calculation method of the residual information of the sub-block to be encoded as a first set calculation method, and numerical values of side lengths of dimensions within the first set range are all greater than a first set value; When the sub-block size of the sub-block to be encoded is within a second set range, determining the calculation method of the residual information of the sub-block to be encoded as a second set calculation method, and there is a side length with a numerical value of the first set value among the dimensions within the second set range; comprising The video encoding processing method according to Claim 3.

5. Calculating residual information of each of the sub-blocks to be encoded according to the calculation method of the residual information, and determining a maximum residual occupancy rate of the unit to be encoded based on the residual information includes: When the calculation method of the residual information is the first set calculation method, calculate the Hadamard transform points of each sub-block waiting for encoding, and determine the maximum residual occupancy rate of the sub-blocks waiting for encoding based on the occupancy rate of the maximum Hadamard transform points in the Hadamard transform points. When the calculation method of the residual information is the second set calculation method, calculate the sum of absolute differences of each sub-block waiting for encoding, and determine the maximum residual occupancy rate of the sub-blocks waiting for encoding based on the occupancy rate of the maximum sum of absolute differences in the sum of absolute differences. including The video encoding processing method according to claim 4.

6. Based on the residual occupancy rate information, determining whether to perform an intra-frame sub-block division inspection on the unit waiting for encoding is When the residual occupancy rate information reaches the set threshold, determining to perform an intra-frame sub-block division inspection on the unit waiting for encoding. When the residual occupancy rate information does not reach the set threshold, determining not to perform an intra-frame sub-block division inspection on the unit waiting for encoding. including The video encoding processing method according to claim 1.

7. The video encoding processing method further includes responding to the determination result of performing an intra-frame sub-block division inspection on the unit waiting for encoding, and performing an encoding process on the unit waiting for encoding based on the intra-frame sub-block division technology. The video encoding processing method according to any one of claims 1 to 6.

8. Performing an encoding process on the unit waiting for encoding based on the intra-frame sub-block division technology is determining the intra-frame sub-block division inspection result for the unit waiting for encoding. determining an intra-frame prediction method for the unit waiting for encoding based on the intra-frame sub-block division inspection result, and performing an encoding process on the unit waiting for encoding according to the intra-frame prediction method. including The video encoding processing method according to claim 7.

9. Determining an intra-frame prediction method for the unit waiting for encoding based on the intra-frame sub-block division inspection result is When the in-frame sub-block division inspection result satisfies the in-frame sub-block division processing condition, the in-frame prediction method for the unit to be encoded is determined to be a method of performing encoding processing on the unit to be encoded based on the in-frame sub-block division technology, When the in-frame sub-block division inspection result does not satisfy the in-frame sub-block division processing condition, the in-frame prediction method for the unit to be encoded is determined to be a method of performing encoding processing on the unit to be encoded based on adjacent encoded units, including The video encoding processing method according to claim 8.

10. including a unit division module, a residual determination module, a residual analysis module, and an encoding processing module, The unit division module is arranged to divide the unit to be encoded into a plurality of sub-blocks to be encoded, The residual determination module is arranged to determine the residual occupancy information of the unit to be encoded based on the residual information of each sub-block to be encoded, The residual analysis module is arranged to determine whether to perform an in-frame sub-block division inspection on the unit to be encoded based on the residual occupancy information, The encoding processing module is arranged to perform encoding processing on the unit to be encoded based on adjacent encoded units in response to a determination result of not performing an in-frame sub-block division inspection on the unit to be encoded, Video encoding processing device.

11. including a memory and one or more processors, The memory is used to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding processing method according to any one of claims 1 to 9, Video encoding processing device.

12. The computer-executable instructions are used to execute the video encoding processing method according to any one of claims 1 to 9 when executed by a computer processor, A storage medium storing computer-executable instructions.

13. A computer program product, including a computer program which, when executed by a processor, implements the video encoding processing method according to any one of claims 1 to 9. A computer program product.

Images