Video coding block division method and device for hardware encoder and electronic equipment

Abstract

The embodiment of the invention discloses a video coding block division method and device for a hardware encoder and electronic equipment. A specific embodiment of the method comprises the following steps: in response to detection that the encoding speed of a hardware encoder is less than a preset encoding threshold value, performing basic encoding block division on a to-be-encoded video frame to obtain a basic encoding block set; executing the following steps on each basic coding block in the basic coding block set: determining each coding sub-block group corresponding to the basic coding block to obtain a coding sub-block group set; based on a parallel execution unit, determining the rate distortion cost corresponding to each coding sub-block group in the coding sub-block group set to obtain a rate distortion cost set; determining an optimal coding sub-block group based on the coding sub-block group set, the rate distortion cost set and a preset rate distortion cost condition; and determining a coding block division result corresponding to the to-be-coded video frame according to each determined optimal coding sub-block group. According to the embodiment, the code rate can be reduced while the image quality is ensured.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of video coding, and in particular, to a video coding block partitioning method, device and electronic equipment for hardware encoder. BACKGROUND

[0002] In the video coding process, coding block partitioning is one of the key steps for processing video frames. Unlike the partitioning method in software coding, hardware encoder has higher requirements for real-time performance. At present, when partitioning video coding blocks, the commonly used method is as follows: first, the video frame is divided into regular square coding blocks, and then the rate-distortion cost of each coding block with different granularity is determined through fixed pipelining, and the coding block with the lowest rate-distortion cost is taken as the final video coding block partitioning method.

[0003] However, when the above method is used to partition the video coding block, the following technical problems often exist: Only coding blocks with specific shapes are supported, which is difficult to adapt to dynamically changing images in video frames. There are fewer coding blocks in image regions with rich texture details, which can easily lead to a decrease in coding performance. There are too many coding blocks in regions with less texture details, which can easily lead to a waste of coding resources, so as to reduce the size of the code rate while ensuring image quality.

[0004] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present disclosure and, as such, can include information that does not form the prior art that is already known to those of ordinary skill in the art. SUMMARY

[0005] The summary of the present disclosure is used to introduce the concepts in a brief manner, which will be described in detail in the following detailed description. The summary of the present disclosure is not intended to identify key or essential features of the claimed technology solution, nor is it intended to be used to limit the scope of the claimed technology solution.

[0006] Some embodiments of the present disclosure propose a video coding block partitioning method, device and electronic equipment for hardware encoder to solve one or more of the technical problems mentioned in the above BACKGROUND section.

[0007] In a first aspect, some embodiments of the present disclosure provide a video coding block partitioning method for a hardware encoder, the method comprising: in response to detecting that a coding speed of the hardware encoder is less than a preset coding threshold, performing basic coding block partitioning on a video frame to be coded to obtain a basic coding block set; for each basic coding block in the basic coding block set, performing the following steps: determining, based on a plurality of preset partitioning manners, a plurality of coding sub-block groups corresponding to the basic coding block to obtain a coding sub-block group set; determining, based on a parallel execution unit, a rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set to obtain a rate-distortion cost set; determining, based on the coding sub-block group set, the rate-distortion cost set, and a preset rate-distortion cost condition, a preferred coding sub-block group; and determining, according to the determined preferred coding sub-block groups, a coding block partitioning result corresponding to the video frame to be coded.

[0008] In a second aspect, some embodiments of the present disclosure provide a video coding block partitioning apparatus for a hardware encoder, the apparatus comprising: a basic coding block partitioning unit configured to, in response to detecting that a coding speed of the hardware encoder is less than a preset coding threshold, perform basic coding block partitioning on a video frame to be coded to obtain a basic coding block set; an execution unit configured to, for each basic coding block in the basic coding block set, perform the following steps: determine, based on a plurality of preset partitioning manners, a plurality of coding sub-block groups corresponding to the basic coding block to obtain a coding sub-block group set; determine, based on a parallel execution unit, a rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set to obtain a rate-distortion cost set; determine, based on the coding sub-block group set, the rate-distortion cost set, and a preset rate-distortion cost condition, a preferred coding sub-block group; and a determination unit configured to determine, according to the determined preferred coding sub-block groups, a coding block partitioning result corresponding to the video frame to be coded.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a storage storing one or more programs configured to, when executed by the one or more processors, cause the one or more processors to implement the method described in any implementation manner of the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method described in any implementation manner of the first aspect.

[0011] The above various embodiments of the present disclosure have the following beneficial effects: through the video coding block division method for a hardware encoder of some embodiments of the present disclosure, the size of the code rate can be reduced while ensuring the image quality. Specifically, the reason why the size of the code rate cannot be reduced while ensuring the image quality is that only coding blocks of a specific shape are supported, it is difficult to adapt to images that dynamically change in a video frame, there are fewer coding blocks in image regions with rich texture details, which easily leads to a reduction in coding performance, there are too many coding blocks in regions with less texture details, which easily leads to a waste of coding resources, and thus the size of the code rate cannot be reduced while ensuring the image quality. Based on this, the video coding block division method for a hardware encoder of some embodiments of the present disclosure first performs basic coding block division on a to-be-coded video frame in response to detecting that the coding speed of the hardware encoder is less than a preset coding threshold, to obtain a basic coding block set. Then, for each basic coding block in the basic coding block set, the following steps are performed: first, based on preset division manners, each coding sub-block group corresponding to the basic coding block is determined, to obtain a coding sub-block group set. By supporting different division manners, the encoder can dynamically adjust the division of the coding blocks according to the content of different regions in the video frame, avoiding the disadvantages brought by a single shape of division. Second, based on a parallel execution unit, the rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set is determined, to obtain a rate-distortion cost set. In this way, the calculation delay can be reduced, and the real-time performance of the hardware encoder coding can be improved. Then, based on the coding sub-block group set, the rate-distortion cost set, and a preset rate-distortion cost condition, an optimal coding sub-block group is determined. By selecting the optimal coding sub-block group according to the preset rate-distortion cost condition, the coding quality and the code rate of each region can be accurately controlled, and unnecessary code rate waste can be avoided. Finally, according to the determined optimal coding sub-block groups, a coding block division result corresponding to the to-be-coded video frame is determined. In this way, the optimization of the image quality and the minimization of the code rate can be ensured, the coding performance is effectively improved, and unnecessary bit streams are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals can refer to the same or similar elements. It should be understood that the drawings are schematic, and elements and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of a video coding block division method for a hardware encoder according to the present disclosure; Figure 2 is a structural schematic diagram of some embodiments of a video coding block division apparatus for a hardware encoder according to the present disclosure; Figure 3This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure; Figure 4 This is an example diagram illustrating the partitioning method of some embodiments of the video coding block partitioning method for hardware encoders according to the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Figure 1 A flow 100 of some embodiments of a video coding block partitioning method for a hardware encoder according to the present disclosure is shown. The video coding block partitioning method for a hardware encoder includes the following steps: Step 101: In response to the detection that the encoding speed of the hardware encoder is less than the preset encoding threshold, the video frame to be encoded is divided into basic coding blocks to obtain a basic coding block set.

[0021] In some embodiments, the execution entity of the video coding block partitioning method for a hardware encoder may, in response to detecting that the encoding speed of the hardware encoder is less than a preset encoding threshold, partition the video frame to be encoded into basic coding blocks to obtain a basic coding block set. The encoding speed can be a value representing the number of video frames encoded by the hardware encoder per unit time. The preset encoding threshold can be a pre-set value. Here, the specific setting of the preset encoding threshold is not limited. The execution entity can be a server or a video encoding chip. The video encoding chip can be a chip capable of encoding video. For example, the video encoding chip can be an H.264 chip. The video encoding chip can include various pipelines. Each pipeline can be a module for executing instruction tasks. The hardware encoder can be a module capable of video encoding. For example, the hardware encoder can be an NVENC encoder. The video frame to be encoded can be a frame image in the video to be encoded. The video to be encoded can be video data that needs to be encoded. Each basic coding block in the basic coding block set can be a square image block of the same size. The size of the basic coding block is not specifically limited.

[0022] In practice, upon receiving the partitioning instruction from the target terminal, it can be determined that the encoding speed of the hardware encoder is less than the preset encoding threshold. The target terminal can be the terminal used by a technician. The partitioning instruction can be an instruction indicating the start of video coding block partitioning.

[0023] In some optional implementations of certain embodiments, the execution entity divides the video frame to be encoded into basic coding blocks to obtain a set of basic coding blocks, which may include the following steps: The first step is to preprocess the aforementioned video frames to be encoded to obtain preprocessed video frames. These preprocessed video frames can be the preprocessed video frames to be encoded.

[0024] In practice, the aforementioned video frames to be encoded can be adjusted to a preset image size using image processing functions, resulting in adjusted video frames as preprocessed video frames. The preset image size can be a pre-defined image size with the same width and height. For example, the preset image size could be 256×256. The image processing function can be any function capable of adjusting the image size. For example, the image processing function could be the `cv2.resize()` function.

[0025] The second step is to divide the preprocessed video frames into blocks to obtain the basic coded block set.

[0026] In practice, the aforementioned execution entity can use a block-splitting function to divide the preprocessed video frames into square image blocks of the same size, serving as a basic set of coded blocks. This block-splitting function can be any function capable of dividing video frames into blocks. For example, the block-splitting function could be the blkproc function.

[0027] As an example, the preprocessed video frames described above can be divided into individual 32×32 image blocks.

[0028] Step 102: For each basic coding block in the basic coding block set, perform the following steps: Step 1021: Based on the preset partitioning methods, determine the corresponding coding sub-block groups of the basic coding block to obtain the coding sub-block group set.

[0029] In some embodiments, the execution entity can determine the corresponding coding sub-block groups based on preset partitioning methods to obtain a coding sub-block set. Each coding sub-block in the coding sub-block set can be an image block obtained after processing the basic coding block. Each of the partitioning methods can be a way of partitioning the basic coding block. The partitioning methods can include uniform 4-partitioning, vertical 2-partitioning, horizontal 2-partitioning, horizontal asymmetric 4-partitioning, and vertical asymmetric 4-partitioning, etc. Example diagrams of the partitioning methods are shown below. Figure 4 As shown.

[0030] The above-mentioned uniform 4-partition is implemented as follows: First, the execution entity can obtain the image width and height of the basic coded block using a size acquisition function. This size acquisition function can be any function capable of obtaining the dimensions of an image block. For example, it could be the `image.shape()` function from OpenCV.

[0031] Secondly, in response to determining that the image height is an odd number, the difference between the image height and a first preset value can be determined as the height data. Similarly, in response to determining that the image width is an odd number, the difference between the image width and a first preset value can be determined as the width data. The first preset value can be a pre-set value. For example, the first preset value can be 1.

[0032] Then, in response to determining that the image height and the image width are both even numbers, the image height and the image width can be determined as height data and width data, respectively.

[0033] Next, the ratio of the determined height data to the second preset value can be defined as the height ratio. Similarly, the ratio of the determined width data to the aforementioned second preset value can be defined as the width ratio. The aforementioned second preset value can be a pre-set value. For example, the aforementioned second preset value can be 2.

[0034] Then, 0, the aforementioned height ratio, and 0, the aforementioned width ratio can be combined to form the top-left slice range. For example, when the height ratio is 8 and the width ratio is 6, the resulting top-left slice range can be [0:8, 0:6]. 0, the aforementioned height ratio, the aforementioned width ratio, and the aforementioned width data can be combined to form the top-right slice range. The aforementioned height ratio, the aforementioned height data, 0, and the aforementioned width ratio can be combined to form the bottom-left slice range. The aforementioned height ratio, the aforementioned height data, the aforementioned width ratio, and the aforementioned width data can be combined to form the bottom-right slice range.

[0035] Then, the basic coding block can be sliced ​​according to the slice ranges of the top left, top right, bottom left, and bottom right blocks to obtain four image blocks corresponding to the basic coding block, thus uniformly dividing the basic coding block into four parts. The slicing operation can be any operation capable of slicing an image. For example, the slicing operation can be array slicing in NumPy.

[0036] As an example: After reading the above basic coding block and naming it "image", the image block in the upper left corner of the above basic coding block can be sliced ​​according to the upper left slice range using the "image[0:8,0:6]" command.

[0037] The above vertical 2-division is implemented as follows: First, the width, height, height ratio, and width ratio of the aforementioned basic coding block can be determined. The method for determining these parameters can be referenced from the implementation of the uniform 4-partition method described above, and will not be repeated here.

[0038] Secondly, 0, the aforementioned height data, 0, and the aforementioned width ratio can be combined to form the left block slice range. The aforementioned width ratio, the aforementioned height data, the aforementioned width ratio, and the aforementioned width data can be combined to form the right block slice range.

[0039] Then, the basic coding block can be sliced ​​according to the left and right slice ranges through the above slicing operation to obtain two image blocks corresponding to the basic coding block, so as to divide the basic coding block vertically by 2.

[0040] The implementation method of the above level 2 partitioning is as follows: First, the width, height, height ratio, and width ratio of the aforementioned basic coding block can be determined. The method for determining these parameters can be referenced from the implementation of the uniform 4-partition method described above, and will not be repeated here.

[0041] Secondly, 0, the aforementioned height ratio, 0, and the aforementioned width data can be combined to form the upper slice range. The aforementioned height ratio, the aforementioned height data, 0, and the aforementioned width data can be combined to form the lower slice range.

[0042] Then, the basic coding block can be sliced ​​according to the above upper block slicing range and the above lower block slicing range through the above slicing operation to obtain two image blocks corresponding to the above basic coding block, so as to perform horizontal 2-division of the above basic coding block.

[0043] The above-mentioned horizontal asymmetric 4-partition is implemented as follows: First, the width, height, height ratio, and width ratio of the aforementioned basic coding block can be determined. The method for determining these parameters can be referenced from the implementation of the uniform 4-partition method described above, and will not be repeated here.

[0044] Secondly, one-quarter of the aforementioned height data can be designated as the first quarter of the height data. Three-quarters of the aforementioned height data can be designated as the second quarter of the height data.

[0045] Then, in response to determining that the aforementioned first altitude quarter data is not an integer, the largest positive integer less than the aforementioned first altitude quarter data can be determined as the first altitude range data. In response to determining that the aforementioned second altitude quarter data is not an integer, the largest positive integer less than the aforementioned second altitude quarter data can be determined as the second altitude range data.

[0046] Then, 0, the aforementioned first height range data, 0, and the aforementioned width data can be combined to form the upper block segmentation range. For example, when the first height range data is 4 and the aforementioned width data is 10, the corresponding upper block segmentation range can be [0:4,0:10].

[0047] The above-mentioned first height range data, the above-mentioned second height range data, 0, and the above-mentioned width ratio can be combined to form the middle left block segmentation range.

[0048] The aforementioned first height range data, the aforementioned second height range data, the aforementioned width ratio, and the aforementioned width data can be combined to form the middle right block segmentation range.

[0049] The above-mentioned second height range data, the above-mentioned height data, 0, and the above-mentioned width data can be combined into the next block segment range.

[0050] Finally, the basic coding block can be divided into four image blocks according to the above-mentioned upper block segmentation range, the above-mentioned middle left block segmentation range, the above-mentioned middle right block segmentation range, and the above-mentioned lower block segmentation range through the above-mentioned slicing operation, so as to obtain the above-mentioned basic coding block into four horizontal asymmetric partitions.

[0051] The above vertical asymmetric 4-partitioning is implemented as follows: First, the width, height, height ratio, and width ratio of the aforementioned basic coding block can be determined. The method for determining these parameters can be referenced from the implementation of the uniform 4-partition method described above, and will not be repeated here.

[0052] Secondly, one-quarter of the aforementioned width data can be defined as the first width quarter data. Three-quarters of the aforementioned width data can be defined as the second width quarter data.

[0053] Then, in response to determining that the first width quarter data is not an integer, the largest positive integer less than the first width quarter data can be determined as the first width range data. In response to determining that the second width quarter data is not an integer, the largest positive integer less than the second width quarter data can be determined as the second width range data.

[0054] Then, 0, the aforementioned first width range data, 0, and the aforementioned height data can be combined to form the left block segmentation range. 0, the aforementioned height ratio, the aforementioned first width range data, and the aforementioned second width range data can be combined to form the upper-middle block segmentation range. The aforementioned height ratio, the aforementioned height data, the aforementioned first width range data, and the aforementioned second width range data can be combined to form the lower-middle block segmentation range. 0, the aforementioned height data, the aforementioned second width range data, and the aforementioned width data can be combined to form the right block segmentation range.

[0055] Finally, the basic coding block can be divided into four image blocks according to the above-mentioned left block segmentation range, the above-mentioned upper-middle block segmentation range, the above-mentioned lower-middle block segmentation range, and the above-mentioned right block segmentation range through the above-mentioned slicing operation, so as to obtain the four image blocks corresponding to the above-mentioned basic coding block, and to perform vertical asymmetric 4-division of the above-mentioned basic coding block.

[0056] In some optional implementations of certain embodiments, the execution entity determines each group of coded sub-blocks corresponding to the basic coded block based on preset partitioning methods to obtain a set of coded sub-block groups, which may include the following steps: The first step involves dividing the basic coding block into sub-blocks based on each of the preset partitioning methods to generate a first group of coding sub-blocks, thus obtaining a first set of coding sub-blocks. Each first coding sub-block in the first set of coding sub-blocks can be an image block obtained by partitioning the basic coding block. Each first group of coding sub-blocks in the first set of coding sub-blocks corresponds to one partitioning method. The number of first groups of coding sub-blocks in the first set of coding sub-blocks is the same as the number of partitioning methods. In practice, for each of the partitioning methods, the executing entity can partition the basic coding block according to the partitioning method to obtain the corresponding image blocks as the first group of coding sub-blocks. The specific implementation of partitioning the basic coding block using the partitioning methods can be found in the implementation described in step 1021, and will not be repeated here. Then, the obtained first groups of coding sub-blocks can be determined as the first set of coding sub-blocks.

[0057] The second step is to perform the following steps for each of the first coded sub-block groups in the above-mentioned first coded sub-block group set: The first sub-step involves performing the following steps for each first coded sub-block in the aforementioned first coded sub-block group: In sub-step one, based on each of the preset partitioning methods, the first coded sub-block is further partitioned to generate a second coded sub-block group, resulting in a second coded sub-block set. Each second coded sub-block in the second coded sub-block set can be an image block obtained after partitioning the first coded sub-block. The second coded sub-blocks in each second coded sub-block group can be combined to form a corresponding first coded sub-block. The number of second coded sub-block groups in the second coded sub-block set is the same as the number of partitioning methods mentioned above. Each second coded sub-block group in the second coded sub-block set corresponds to one partitioning method. In practice, for each of the aforementioned partitioning methods, the first coded sub-block can be partitioned to generate a second coded sub-block group. Then, the resulting second coded sub-block groups can be defined as the second coded sub-block set. The specific implementation of partitioning the first coded sub-block using the aforementioned partitioning methods can be found in the implementation described in step 1021, and will not be repeated here.

[0058] The second sub-step involves determining the initial coded sub-block set corresponding to the first coded sub-block set based on the obtained sets of second coded sub-blocks. Each initial coded sub-block in the initial coded sub-block set can be an image block obtained by combining the second coded sub-blocks from the second coded sub-block sets. In practice, for each second coded sub-block set in the aforementioned sets of second coded sub-blocks, firstly, the partitioning method corresponding to the second coded sub-block set can be determined as the target partitioning method. Secondly, the second coded sub-block set and the target partitioning method can be sent to a target server. The target server can be a server capable of recombinizing image blocks into a single image block based on the partitioning method of each image block. Then, the image block sent by the target server is received as the initial coded sub-block. Next, the obtained initial coded sub-blocks can be determined as the initial coded sub-block set. Finally, the obtained initial coded sub-block sets can be determined as the initial coded sub-block set.

[0059] As an example, assume the first coded sub-block group mentioned above contains two first coded sub-blocks. One of the first coded sub-blocks can be vertically divided by 2 into two second coded sub-blocks, resulting in a corresponding second coded sub-block group. The other first coded sub-block can be horizontally divided by 2 into two second coded sub-blocks, resulting in a corresponding second coded sub-block group. The combination of each second coded sub-block contained in the two second coded sub-block groups can be the initial coded sub-block corresponding to the first coded sub-block group. The partitioning method corresponding to the initial coded sub-block can be a combination of vertical 2-division and horizontal 2-division. The partitioning method can also be a combination of vertical 2-division and uniform 4-division, or a combination of horizontal asymmetric 4-division and vertical asymmetric 4-division.

[0060] The third step involves determining the corresponding set of coded sub-blocks for the aforementioned basic coded blocks based on the determined initial sets of coded sub-blocks. Each coded sub-block in the aforementioned set can be an image block obtained by combining the corresponding initial sets of coded sub-blocks. In practice, for each initial set of coded sub-blocks, firstly, the partitioning methods corresponding to the initial sets of coded sub-blocks can be determined as initial target partitioning methods. Secondly, the initial sets of coded sub-blocks and the initial target partitioning methods can be sent to the target server. Then, the image blocks sent by the target server are received as coded sub-blocks. Next, the resulting coded sub-blocks can be determined as coded sub-block sets. Finally, the resulting sets of coded sub-blocks can be determined as coded sub-block sets.

[0061] In some alternative implementations of certain embodiments, the execution entity determines each group of coded sub-blocks corresponding to the basic coded block based on preset partitioning methods to obtain a set of coded sub-block groups, which may include the following steps: The first step involves determining the pre-selected coding sub-block groups corresponding to the aforementioned basic coding block based on preset partitioning methods and pre-processing algorithms, thus obtaining a pre-selected coding sub-block set. Each pre-selected coding sub-block in this set can be an image block obtained by partitioning the aforementioned basic coding block. Each pre-selected coding sub-block group in this set corresponds to a probability value. This probability value characterizes the quality of the corresponding pre-selected coding sub-block group.

[0062] The aforementioned pre-processing algorithm determines the corresponding coding unit (CU) partitioning method based on the content of the video frame to be encoded. The CU partitioning method determined by the pre-processing algorithm is also based on a preset partitioning method. This pre-processing algorithm can be deployed in the inter-frame motion estimation (IME) module of the video encoding chip. Alternatively, it can be deployed before the inter-frame motion estimation module of the video encoding chip.

[0063] The second step involves identifying each pre-selected coding sub-block group that meets a preset parallelism condition within the aforementioned pre-selected coding sub-block group set as a coding sub-block group, thus obtaining a coding sub-block group set. The preset parallelism condition can be that the probability value corresponding to the pre-selected coding sub-block group is a target probability value. The target probability value can be the probability value that ranks first in parallel quantity in the probability value sequence. The probability value sequence can be a sequence composed of the probability values ​​corresponding to the aforementioned pre-selected coding sub-block group set arranged from smallest to largest. The number of parallel processes can be the number of parallel pipelines in the aforementioned video encoding chip. For example, the number of parallel processes can be 2.

[0064] In practice, firstly, the number of parallel pipelines in the aforementioned video encoding chip can be determined as the number of parallel lines. Secondly, the top number of parallel probability values ​​with the smallest probability values ​​in the aforementioned pre-selected coding sub-block sets can be determined as each target probability value. Then, each pre-selected coding sub-block set corresponding to each of the aforementioned target probability values ​​can be determined as a coding sub-block set.

[0065] In addressing the aforementioned technical problems in the application scenario—real-time scenarios such as cloud gaming or cloud applications—the following technical issues often arise: When determining the video frame segmentation method, all possible segmentation methods need to be used to segment the video frames, followed by encoding of the segmented frames to determine the target segmentation method. This can easily lead to high computational resource consumption and low efficiency during segmentation. Considering the specific requirements of this application scenario—cloud gaming or cloud applications need to encode game visuals or interfaces in real-time, requiring high encoding efficiency—we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity determines each pre-selected coding sub-block group corresponding to the basic coding block based on preset partitioning methods and pre-processing algorithms, thereby obtaining a pre-selected coding sub-block group set, which may include the following steps: The first step involves image scaling of the aforementioned basic coded block to obtain an image block to be processed. This image block can be obtained by reducing the basic coded block to a preset size. The preset size can be a pre-defined size, such as 16×16. In practice, the executing entity can use an image processing function to reduce the basic coded block to the preset size, obtaining the reduced-size basic coded block as the image block to be processed. This image processing function can be any function capable of reducing the size of the basic coded block. For example, the image processing function could be the `imresize` function.

[0066] The second step involves dividing the image block to be processed into a group of sub-image blocks for each of the above-mentioned division methods. Each sub-image block in this group can be an image block obtained by dividing the image block to be processed. Each group of sub-image blocks corresponds to one division method. In practice, the specific implementation of dividing the image block to be processed using the above-mentioned division methods can include the implementation described in step 1021, and will not be repeated here.

[0067] The third step involves performing the following steps for each of the obtained sub-image block groups to be processed: The first sub-step involves encoding each sub-image block in the aforementioned group of sub-image blocks to obtain its rate-distortion cost. This rate-distortion cost can be the Rate-Distortion Cost (RDCost). In practice, for each sub-image block in the aforementioned group of sub-image blocks, an encoding library function can be used to encode the sub-image block and obtain its corresponding rate-distortion cost. This encoding library function can be any library function capable of encoding image blocks and obtaining the rate-distortion cost of the encoded image block. For example, the encoding library function could be x264.

[0068] The second sub-step involves summing the individual rate distortion costs mentioned above to determine the total rate distortion cost.

[0069] The fourth step involves normalizing the determined total rate distortion costs to obtain normalized total rate distortion costs as probability values. Each probability value can be a total rate distortion cost scaled to between 0 and 1. Each probability value corresponds to a partitioning method. In practice, the executing entity can use normalization techniques to normalize each total rate distortion cost to between 0 and 1, obtaining normalized total rate distortion costs as probability values. The normalization technique can be any technique capable of normalizing a set of data to between 0 and 1. For example, the normalization technique could be Min-Max normalization.

[0070] Fifth, based on the above probability values, arrange the above partitioning methods to obtain a partitioning method sequence. In practice, the above partitioning methods can be arranged into a partitioning method sequence in ascending order of probability values. Each partitioning method in the partitioning method sequence corresponds to one probability value.

[0071] Step 6: Based on the above-described partitioning sequence, the basic coding block is partitioned to obtain pre-selected coding sub-block groups. In practice, firstly, the sub-sequences corresponding to the first preset number of partitioning methods in the above-described partitioning sequence can be determined as the target partitioning sequence. The preset number can be a pre-defined quantity. For example, the preset number can be 2. Secondly, for each target partitioning method in the above-described target partitioning sequence, the basic coding block can be partitioned according to the target partitioning method to obtain each image block of the basic coding block as a pre-selected coding sub-block group. Thus, each pre-selected coding sub-block group can be determined. The specific implementation of each target partitioning method in the above-described target partitioning sequence can be found in the specific implementation method described in step 1021, and will not be repeated here.

[0072] Step 7: Determine each of the above pre-selected coding sub-block groups as a pre-selected coding sub-block group set.

[0073] The above technical solution and its related content, combined with step 103, serve as an inventive point of this disclosure, solving the problem of "low efficiency during partitioning." Factors leading to high computational resource consumption and low efficiency during partitioning often include: when determining the partitioning method for video frames, all partitioning methods need to be used to partition the video frames, and then the partitioned video frames are encoded to determine the target partitioning method for the video frames. This easily leads to high computational resource consumption and low efficiency during partitioning. Solving these factors can improve the efficiency of partitioning. To achieve this effect, this disclosure first performs image scaling on the basic coding block to obtain the image block to be processed. This allows for the reduction of the basic coding block, thereby reducing the computational resources consumed when processing the basic coding block. Second, for each of the above partitioning methods, the image block to be processed is partitioned based on the above partitioning method to obtain a group of sub-image blocks to be processed. Thus, the image block to be processed can be partitioned according to the above partitioning methods. Then, for each of the obtained sub-image block groups to be processed, the following steps are performed: Encoding is then performed on each sub-image block in the aforementioned sub-image block group to obtain individual rate-distortion costs. Thus, the rate-distortion cost of each sub-image block to be processed can be obtained. Next, the sum of the aforementioned rate-distortion costs is determined as the total rate-distortion cost. Thus, the sum of the rate-distortion costs corresponding to the sub-image block group to be processed can be obtained. Then, the determined total rate-distortion costs are normalized to obtain normalized total rate-distortion costs as probability values, where each probability value corresponds to a partitioning method. Next, based on the aforementioned probability values, the aforementioned partitioning methods are arranged to obtain a partitioning method sequence. Thus, based on the obtained total rate-distortion costs, the aforementioned partitioning methods are sorted to determine their priority. Then, based on the aforementioned partitioning method sequence, the aforementioned basic coding block is partitioned to obtain various pre-selected coding sub-block groups. Therefore, the basic coding block can be divided according to the determined partitioning sequence. Finally, the aforementioned pre-selected coding sub-block groups are determined as the pre-selected coding sub-block set. Because the basic coding block can be scaled down to a smaller size to obtain the image block to be processed, and then the image block to be processed can be divided using the aforementioned partitioning methods, and the priority of each partitioning method is determined based on the rate-distortion cost of each partitioned image block, only the partitioning methods with higher priority are used to partition the basic coding block, instead of using all partitioning methods, thus reducing the computational resources consumed during partitioning and improving partitioning efficiency.

[0074] Step 1022: Based on the parallel execution unit, determine the rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set to obtain the rate-distortion cost set.

[0075] In some embodiments, the execution entity can determine the rate-distortion cost corresponding to each coded sub-block group in the coded sub-block group set based on the parallel execution unit, thereby obtaining a rate-distortion cost set. The rate-distortion cost can be the rate-distortion cost (RDCost). The parallel execution unit can be a module capable of encoding image blocks and obtaining the rate-distortion cost of the image blocks. For example, the parallel execution unit can be an H.266 / VVC encoder. In practice, for each coded sub-block group in the coded sub-block group set, and for each coded sub-block in the coded sub-block group, firstly, the execution entity can generate the rate-distortion cost of the coded sub-block as the sub-block rate-distortion cost using the parallel execution unit. Secondly, the sum of the generated rate-distortion costs of each sub-block can be determined as the rate-distortion cost corresponding to the coded sub-block group. Finally, the determined rate-distortion costs can be determined as the rate-distortion cost set corresponding to the coded sub-block group set.

[0076] Step 1023: Determine the preferred coding sub-block set based on the coding sub-block set, the rate-distortion cost set, and the preset rate-distortion cost conditions.

[0077] In some embodiments, the execution entity may determine a preferred set of encoded sub-blocks based on the aforementioned set of encoded sub-blocks, the aforementioned rate-distortion cost set, and preset rate-distortion cost conditions. Each preferred encoded sub-block in the preferred set of encoded sub-blocks may be a filtered encoded sub-block.

[0078] In addressing the aforementioned technical problems in the application scenario of live streaming on mobile devices, the following technical issues arise: During encoding partitioning, the rate-distortion cost of all encoded sub-blocks needs to be generated, leading to significant computational resource consumption. Considering the specific requirements of this application scenario—limited computing resources on mobile devices and high demands for real-time performance and image quality—we have decided to minimize the computational resources required for encoding partitioning to ensure smooth live streaming. In some optional implementations of certain embodiments, the execution entity, based on parallel execution units, determines the rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set, thereby obtaining a rate-distortion cost set; and determines the preferred coding sub-block group based on the coding sub-block group set, the rate-distortion cost set, and preset rate-distortion cost conditions, which may include the following steps: The first step involves identifying each coding sub-block group in the aforementioned coding sub-block set that satisfies a preset first coding sub-block condition as a first pipeline coding sub-block group, thus obtaining a first pipeline coding sub-block set. The preset first coding sub-block condition can be that each coding sub-block in the coding sub-block group is a first target sub-block. The first target sub-block can be an image block whose rate-distortion cost is less than a preset rate-distortion threshold. The preset rate-distortion threshold can be a pre-set value. The specific setting of the preset rate-distortion threshold is not limited here.

[0079] In practice, three pipelines can be set up in the aforementioned video encoding chip to determine the rate-distortion cost of the coded subblock group. These three pipelines can include a level 0 pipeline, a level 1 pipeline, and a level 2 pipeline. The level 0 pipeline can be a hardware module capable of performing intra-frame prediction on the basic coded blocks and determining the rate-distortion cost of the basic coded blocks. This hardware module can be a module pre-set by technicians in the aforementioned hardware encoder or the aforementioned video encoding chip to implement fixed functions. The level 1 pipeline can be a hardware module capable of performing inter-frame prediction on the aforementioned first pipeline coded subblock group and determining the rate-distortion cost of each first pipeline coded subblock in the aforementioned first pipeline coded subblock group. The level 2 pipeline can be a hardware module capable of performing inter-frame prediction on the second pipeline coded subblock group and determining the rate-distortion cost of each second pipeline coded subblock in the second pipeline coded subblock group. The aforementioned second pipeline coding sub-block group can be an image block composed of each second coding sub-block corresponding to each first pipeline coding sub-block in the aforementioned first pipeline coding sub-block group.

[0080] The second step is to perform the following steps for each of the first pipelined coding sub-blocks in the above-mentioned first pipelined coding sub-block set: The first sub-step involves performing the following steps for each first pipeline coding sub-block in the first pipeline coding sub-block group: Sub-step one: Determine the set of second coding sub-blocks corresponding to the first pipeline coding sub-block. Each second coding sub-block in the set can be an image block obtained by dividing the first pipeline coding sub-block. In practice, firstly, for each of the above division methods, the executing entity can divide the first pipeline coding sub-block using the above division method to obtain each image block corresponding to the first pipeline coding sub-block as a second coding sub-block set. The method of dividing the first pipeline coding sub-block using the above division methods can be referred to the specific implementation described in step 1021, and will not be repeated here. Then, the obtained sets of second coding sub-blocks can be determined as the second coding sub-block set.

[0081] Sub-step two involves determining the first pipeline rate-distortion cost corresponding to the first pipelined coding sub-block. This first pipeline rate-distortion cost can be the rate-distortion cost of the first pipelined coding sub-block. In practice, the executing entity can perform inter-frame prediction on the first pipelined coding sub-block using the first-level pipeline and obtain the rate-distortion cost corresponding to the first pipelined coding sub-block as the first pipeline rate-distortion cost.

[0082] Sub-step three involves determining the second pipeline rate-distortion cost corresponding to each second coding sub-block group in the aforementioned second coding sub-block group set, thus obtaining a second pipeline rate-distortion cost set. The second pipeline rate-distortion cost set can be the sum of the rate-distortion costs corresponding to each second coding sub-block group. In practice, for each second coding sub-block group in the aforementioned second coding sub-block group set, and for each second coding sub-block within that group, inter-frame prediction can be performed on the second coding sub-block using the aforementioned two-stage pipeline, generating the rate-distortion cost corresponding to the inter-frame predicted second coding sub-block. Secondly, the sum of the obtained rate-distortion costs can be determined as the second pipeline rate-distortion cost corresponding to the aforementioned second coding sub-block group. Finally, the determined second pipeline rate-distortion costs can be combined into a second pipeline rate-distortion cost set.

[0083] Sub-step four involves determining the target coding sub-block group based on the aforementioned first pipeline rate distortion cost, the aforementioned second coding sub-block set, and the aforementioned second pipeline rate distortion cost set. The target coding sub-block group can be a filtered set of second coding sub-block groups. In practice, the executing entity can determine the second pipeline rate distortion cost with the smallest value in the aforementioned second pipeline rate distortion cost set as the target second rate distortion cost. Furthermore, the second coding sub-block group corresponding to the target second rate distortion cost in the aforementioned second coding sub-block set can be determined as the target coding sub-block group.

[0084] The second sub-step involves identifying each target coded sub-block contained in the determined target coded sub-block groups as a candidate coded sub-block, thus obtaining a candidate coded sub-block group.

[0085] The third step is to determine the rate-distortion cost of each candidate coding sub-block group in the obtained candidate coding sub-block groups, thus obtaining a set of rate-distortion costs for candidate coding sub-block groups. The rate-distortion cost of each candidate coding sub-block group in this set can be the sum of the rate-distortion costs corresponding to each candidate coding sub-block group. In practice, for each candidate coding sub-block group and each candidate coding sub-block in the above candidate coding sub-block groups, firstly, the execution entity can encode the candidate coding sub-block using an encoder to obtain the rate-distortion cost of the encoded candidate coding sub-block. The encoder can be an encoder capable of encoding image blocks and obtaining the rate-distortion cost of the encoded image blocks. For example, the encoder can be an NVENC encoder. Secondly, the sum of the obtained rate-distortion costs can be determined as the rate-distortion cost of the candidate coding sub-block group corresponding to the above candidate coding sub-block group. Finally, the rate-distortion cost of each candidate coding sub-block group can be determined as the rate-distortion cost set of the candidate coding sub-block group corresponding to each of the above candidate coding sub-block groups.

[0086] The fourth step is to determine the candidate coding sub-block groups whose rate-distortion costs satisfy the preset rate-distortion cost conditions as the preferred coding sub-block groups. The preset rate-distortion cost condition can be defined as having the minimum rate-distortion cost value among the candidate coding sub-block groups.

[0087] The above technical solution and its related content, combined with step 103 above, serve as an inventive point of this disclosure, solving the problem of "excessive consumption of computing resources." Factors leading to excessive computing resource consumption often include: during encoding partitioning, the rate-distortion cost of all encoded sub-blocks needs to be generated, easily resulting in excessive computing resources consumed during encoding partitioning. Solving these factors can reduce the amount of computing resources consumed. To achieve this effect, this disclosure firstly identifies each encoded sub-block group in the above-mentioned encoded sub-block group set that satisfies the preset first encoded sub-block conditions as a first pipelined encoded sub-block group, obtaining a first pipelined encoded sub-block set. This allows for the selection of the first pipelined encoded sub-block set to be processed. Secondly, for each first pipelined encoded sub-block group in the above-mentioned first pipelined encoded sub-block set, the following steps are performed: Then, for each first pipelined encoded sub-block in the above-mentioned first pipelined encoded sub-block group, the following steps are performed: Firstly, the second encoded sub-block set corresponding to the above-mentioned first pipelined encoded sub-block is determined. This allows for the determination of the second encoded sub-block set. Next, the first pipeline rate-distortion cost corresponding to the first pipelined coding sub-block is determined. This generates the first pipeline rate-distortion cost. Then, the second pipeline rate-distortion cost corresponding to each second coding sub-block group in the second coding sub-block group set is determined, resulting in a second pipeline rate-distortion cost set. This generates the rate-distortion cost corresponding to each second coding sub-block in the second coding sub-block group set. Then, based on the first pipeline rate-distortion cost, the second coding sub-block group set, and the second pipeline rate-distortion cost set, target coding sub-block groups are determined. This determines the target coding sub-block groups. Next, each target coding sub-block contained in each determined target coding sub-block group is identified as a candidate coding sub-block, resulting in a candidate coding sub-block group. Then, the rate-distortion cost of each candidate coding sub-block group in the obtained candidate coding sub-block groups is determined, resulting in a candidate coding sub-block group rate-distortion cost set. This yields the rate-distortion cost of each candidate coding sub-block. Finally, the candidate coding sub-block groups whose rate-distortion costs satisfy the preset rate-distortion cost conditions are identified as preferred coding sub-block groups. Thus, preferred coding sub-block groups can be selected based on the aforementioned set of rate-distortion costs for candidate coding sub-block groups. Furthermore, because each coding sub-block group satisfying the preset first coding sub-block condition can be selected first when determining the preferred coding sub-block groups, and only the selected coding sub-block groups are processed, instead of processing all coding sub-block groups, the computational resources consumed during partitioning can be reduced.

[0088] In addressing the technical problems mentioned above, the application scenario—real-time scenarios such as game streaming—often presents the following challenges: a fixed pipeline design struggles to dynamically adjust to the real-time state of the hardware encoder. Regardless of whether the encoder is idle or under high load, it consumes the same amount of computing resources, leading to wasted resources and hindering further performance improvements in video encoding. Considering the specific requirements of this application scenario—the varying complexity of each game frame and the corresponding real-time load on the hardware encoder—we need to dynamically adjust the encoding quality based on the complexity of the game frame to maximize performance. Therefore, we have decided to adopt the following solution: In some alternative implementations of certain embodiments, after determining the preferred coding sub-block set based on the aforementioned coding sub-block set, the aforementioned rate-distortion cost set, and the preset rate-distortion cost condition, the execution entity may further perform the following steps: The first step, in response to determining that the computational load of the hardware encoder is below a preset threshold, is to perform the following steps: The first sub-step involves identifying each pre-selected coding sub-block group in the aforementioned pre-selected coding sub-block group set that satisfies the second preset parallel condition as the target coding sub-block group, thus obtaining the target coding sub-block group set. Here, the computational load can be the GPU utilization rate of the hardware encoder. The second preset parallel condition can be the top number of pre-selected coding sub-block groups with the smallest corresponding probability value from the aforementioned pre-selected coding sub-block group set, excluding the coding sub-block groups determined in other optional steps of step 1021. The preset threshold can be any value and is not specifically limited here.

[0089] In practice, in some alternative implementations of step 1021 above, the set of coded sub-blocks is determined from the set of pre-selected coded sub-blocks. First, each pre-selected coded sub-block group other than the one mentioned above can be identified as a candidate coded sub-block group, resulting in a candidate coded sub-block set. Second, the top few parallel candidate coded sub-block groups with the smallest corresponding probability values ​​in the candidate coded sub-block set can be identified as the target coded sub-block groups, resulting in the target coded sub-block set.

[0090] The second sub-step involves determining the target rate-distortion cost corresponding to each target coded sub-block group in the aforementioned target coded sub-block group set, thus obtaining a target rate-distortion cost set. Each target rate-distortion cost in this set can be the sum of the rate-distortion costs corresponding to the target coded sub-block group. In practice, for each target coded sub-block group in the aforementioned target coded sub-block group set, and for each target coded sub-block within that group, the encoder can be used to encode the target coded sub-block, obtaining its corresponding rate-distortion cost. Then, the sum of the obtained rate-distortion costs can be determined as the target rate-distortion cost corresponding to the aforementioned target coded sub-block group. Finally, the obtained target rate-distortion costs can be combined into a target rate-distortion cost set.

[0091] The third sub-step involves determining the target coding sub-block group corresponding to the target rate distortion cost that satisfies the preset target rate distortion cost condition as the candidate coding sub-block group. The preset target rate distortion cost condition can be that the value corresponding to the target rate distortion cost is minimized.

[0092] The fourth sub-step involves determining, in response to the determination that the target rate-distortion cost corresponding to the candidate coding sub-block group is lower than the rate-distortion cost corresponding to the preferred coding sub-block group, that the candidate coding sub-block group is designated as the preferred coding sub-block group. The rate-distortion cost corresponding to the preferred coding sub-block group can be the rate-distortion cost of the candidate coding sub-block group corresponding to the preferred coding sub-block group.

[0093] As an example, assume the hardware encoder described above has two parallel pipelines. In the above steps, the first and second pre-selected coding sub-block groups with the lowest probability values ​​are selected from the pre-selected coding sub-block group set and determined as the coding sub-block group set. Then, the coding sub-block groups with lower rate-distortion costs in the above coding sub-block group set are determined as the preferred coding sub-block groups. When the hardware encoder's computing resources are detected to be idle, firstly, the third and fourth pre-selected coding sub-block groups with the lowest probability values ​​can be selected from the above pre-selected coding sub-block group set as the target coding sub-block group set. Then, the target coding sub-block groups with lower rate-distortion costs in the above target coding sub-block group set can be determined as candidate coding sub-block groups. Finally, the rate-distortion costs between the candidate coding sub-block groups and the preferred coding sub-block groups can be compared, and the coding sub-block groups with lower rate-distortion costs are determined as the preferred coding sub-block groups.

[0094] In practice, firstly, preset computational load acquisition information can be sent to the target terminal. This target terminal can be the terminal corresponding to a technician. The computational load acquisition information can be information used to obtain the current computational load of the hardware encoder. For example, the computational load acquisition information could be "Please return the current computational load of the hardware encoder." Secondly, the computational load sent by the target terminal can be received.

[0095] The second step, in response to determining that the computational load of the hardware encoder is higher than the preset threshold, is to perform the following steps: The first sub-step involves obtaining the process IDs. Each process ID can be a process control identifier. Each process ID corresponds to a process load and a process priority value. The process load can be the GPU utilization of the process. The process priority value can be a numerical value that represents the priority of the process. For example, the process priority value can be the process's NI value (Nice Value).

[0096] In practice, the aforementioned executing entity can obtain the process IDs and the process load corresponding to each process ID using process information viewing commands. These process information viewing commands can be any commands capable of displaying process information. For example, the process information viewing command could be "nvidia-smi". Then, for each process ID, the process priority value can be viewed using process priority viewing commands. These process priority viewing commands can be any commands capable of displaying the process priority value. For example, the process priority viewing command could be the `ps` command. As an example, when the process ID is 123, it can be viewed using "ps -p". <123> The command "-o pid,ni,cmd" displays the process priority value of process number 123.

[0097] The second sub-step involves determining at least one process ID to be terminated based on the process priority values ​​included in the aforementioned process IDs. Each of these at least one process ID to be terminated can be a process control character of the process that needs to be terminated.

[0098] In practice, firstly, the aforementioned execution entity can sort the process numbers according to their priority values ​​from smallest to largest to obtain a process number sequence. Secondly, the difference between the computational load of the hardware encoder and the preset threshold can be determined as the load difference. Then, based on the process number sequence, the following cyclic steps are executed: First, the loop count is updated based on the sum of the loop count and the preset update value. Second, the first few loop counts of process numbers in the process number sequence are determined as at least one target process number. Then, the sum of the loads of at least one process corresponding to the at least one target process number is determined as the total load value. In response to determining that the total load value is less than the load difference, the cyclic steps are executed again using the updated loop count. In response to determining that the total load value is greater than or equal to the load difference, the first few loop counts of process numbers in the process number sequence are determined as at least one process number to be terminated. The loop count can be a numerical value representing the number of times the cyclic steps are executed. The initial loop count can be 0. The preset update value can be 1.

[0099] The third sub-step involves terminating at least one process corresponding to the at least one process number to be terminated. In practice, for each of the at least one process numbers to be terminated, the executing entity can terminate the process corresponding to that process number using a process termination instruction. This process termination instruction can be any instruction capable of terminating a process. For example, the process termination instruction can be the `kill` instruction. As an example, when the process number to be terminated is 123, it can be terminated using the `kill` instruction. <123> The command terminates the process with PID 123.

[0100] The above technical solution and its related content, combined with step 103 above, serve as an inventive point of this disclosure, solving the problem of "idle and wasted computing resources." Factors leading to idle and wasted computing resources often include: fixed pipeline designs are difficult to dynamically adjust according to the real-time state of the hardware encoder; regardless of whether the hardware encoder is idle or under high load, it consumes the same amount of computing resources, leading to idle and wasted computing resources and hindering further improvement in video encoding performance. Solving these factors can reduce idle and wasted computing resources. To achieve this effect, this disclosure, firstly, in response to determining that the computing load of the hardware encoder is below a preset threshold, performs the following steps: secondly, each pre-selected coding sub-block group in the above-mentioned pre-selected coding sub-block group set that satisfies the second preset parallel condition is determined as a target coding sub-block group, resulting in a target coding sub-block group set. Therefore, when the computing load of the hardware encoder is below the preset threshold, the above-mentioned pre-selected coding sub-block groups can be filtered based on the probability value of each pre-selected coding sub-block group in the pre-selected coding sub-block group set, instead of processing all pre-selected coding sub-block groups. Then, the target rate-distortion cost corresponding to each target coding sub-block group in the above target coding sub-block group set is determined, resulting in a target rate-distortion cost set. This allows the generation of the rate-distortion cost for each target coding sub-block group. Then, the target coding sub-block groups corresponding to the target rate-distortion costs in the above target rate-distortion cost set that satisfy the preset target rate-distortion cost conditions are determined as candidate coding sub-block groups. This allows the target coding sub-block group with the minimum target rate-distortion cost to be obtained. Then, in response to determining that the target rate-distortion cost corresponding to the above candidate coding sub-block group is lower than the rate-distortion cost corresponding to the preferred coding sub-block group, the above candidate coding sub-block group is determined as the preferred coding sub-block group. This allows the comparison of the rate-distortion costs of the candidate coding sub-block group and the preferred coding sub-block group, and the update of the preferred coding sub-block group. Then, in response to determining that the computational load of the above hardware encoder is higher than the above preset threshold, the following steps are performed: Then, each process number is obtained, wherein each process number corresponds to a process load and a process priority value. This allows the obtaining of the process load and process priority values ​​corresponding to each process number. Then, based on the process priority values ​​included in each of the aforementioned process numbers, at least one process number to be terminated is determined. This yields the process numbers that need to be terminated. Then, at least one process corresponding to the aforementioned at least one process number to be terminated is terminated. This reduces the computational load on the hardware encoder. Also, because the computational load on the hardware encoder is detected to be below a preset threshold, the rate-distortion cost of the candidate coding sub-block group and the preferred coding sub-block group can be compared to further determine the coding sub-blocks with better coding quality. This allows idle computing resources to be used to further improve coding quality, enabling dynamic adjustment of idle computing resources and thus reducing idle and wasted computing resources.

[0101] Step 103: Based on the determined preferred coding sub-block groups, determine the coding block division result corresponding to the video frame to be encoded.

[0102] In some embodiments, the execution entity can determine the coding block partitioning result corresponding to the video frame to be encoded based on the determined preferred coding sub-block groups. In practice, the preferred coding sub-blocks contained in the preferred coding sub-block groups can be combined into a preferred coding sub-block set. The preferred coding sub-block set is then determined as the coding block partitioning result corresponding to the video frame to be encoded.

[0103] The above embodiments of this disclosure have the following beneficial effects: the video coding block partitioning method for hardware encoders according to some embodiments of this disclosure can reduce the bitrate while ensuring image quality. Specifically, the reason why it is impossible to reduce the bitrate while ensuring image quality is that: only supporting coding blocks of specific shapes is difficult to adapt to dynamically changing images in video frames; fewer coding blocks in image regions with rich texture details can easily lead to a decrease in coding performance; and too many coding blocks in regions with less texture details can easily lead to a waste of coding resources, thus making it impossible to reduce the bitrate while ensuring image quality. Based on this, the video coding block partitioning method for hardware encoders according to some embodiments of this disclosure firstly, in response to detecting that the encoding speed of the hardware encoder is less than a preset encoding threshold, performs basic coding block partitioning on the video frame to be encoded to obtain a basic coding block set. Then, for each basic coding block in the above basic coding block set, the following steps are performed: firstly, based on preset partitioning methods, each coding sub-block group corresponding to the above basic coding block is determined to obtain a coding sub-block group set. By supporting different partitioning methods, the encoder can dynamically adjust the partitioning of coding blocks according to the content of different regions in the video frame, avoiding the drawbacks of partitioning with a single shape. Secondly, based on the parallel execution unit, the rate-distortion cost corresponding to each coding sub-block group in the above-mentioned coding sub-block group set is determined, resulting in a rate-distortion cost set. This reduces computational latency and improves the real-time performance of the hardware encoder. Next, based on the above-mentioned coding sub-block group set, the above-mentioned rate-distortion cost set, and preset rate-distortion cost conditions, optimal coding sub-block groups are determined. By selecting the optimal coding sub-block groups according to the preset rate-distortion cost conditions, the coding quality and bitrate of each region can be precisely controlled, avoiding unnecessary bitrate waste. Finally, based on the determined optimal coding sub-block groups, the coding block partitioning result corresponding to the above-mentioned video frame to be encoded is determined. This ensures the optimization of image quality and the minimization of bitrate, effectively improving coding performance and reducing unnecessary bitstreams.

[0104] Further reference Figure 2As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a video coding block partitioning apparatus for a hardware encoder, these apparatus embodiments being similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0105] like Figure 2 As shown, a video coding block partitioning apparatus 200 for a hardware encoder in some embodiments includes: a basic coding block partitioning unit 201, an execution unit 202, and a determination unit 203. The basic coding block partitioning unit 201 is configured to partition the video frame to be encoded into basic coding blocks in response to detecting that the encoding speed of the hardware encoder is less than a preset encoding threshold, thereby obtaining a basic coding block set. The execution unit 202 is configured to perform the following steps on each basic coding block in the basic coding block set: determining each coding sub-block group corresponding to the basic coding block based on preset partitioning methods, thereby obtaining a coding sub-block group set; determining the rate-distortion cost corresponding to each coding sub-block group in the coding sub-block group set based on a parallel execution unit, thereby obtaining a rate-distortion cost set; determining a preferred coding sub-block group based on the coding sub-block group set, the rate-distortion cost set, and a preset rate-distortion cost condition; and the determination unit 203 is configured to determine the coding block partitioning result corresponding to the video frame to be encoded based on the determined preferred coding sub-block groups.

[0106] It is understandable that the units described in the device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here.

[0107] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0108] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. The random access memory 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, the read-only memory 302, and the random access memory 303 are interconnected via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0109] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0110] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a read-only memory 302. When the computer program is executed by the processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0111] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0112] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0113] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: in response to detecting that the encoding speed of the hardware encoder is less than a preset encoding threshold, divide the video frame to be encoded into basic coding blocks to obtain a set of basic coding blocks; for each basic coding block in the set of basic coding blocks, perform the following steps: based on preset partitioning methods, determine each coding sub-block group corresponding to the basic coding block to obtain a set of coding sub-block groups; based on parallel execution units, determine the rate-distortion cost corresponding to each coding sub-block group in the set of coding sub-block groups to obtain a set of rate-distortion costs; based on the set of coding sub-block groups, the set of rate-distortion costs, and preset rate-distortion cost conditions, determine a preferred coding sub-block group; and based on the determined preferred coding sub-block groups, determine the coding block partitioning result corresponding to the video frame to be encoded.

[0114] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a basic coding block partitioning unit, an execution unit, and a determination unit. The names of these units do not necessarily limit the unit itself; for example, the basic coding block partitioning unit may also be described as "a unit that performs basic coding block partitioning on the video frame to be encoded."

[0117] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0118] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for video coding block partitioning for a hardware encoder, comprising: performing a base coding block partitioning on a video frame to be coded to obtain a base coding block set in response to detecting that a coding speed of the hardware encoder is less than a preset coding threshold; performing the following steps on each base coding block in the base coding block set: determining a coding sub-block group set corresponding to the base coding block based on preset partitioning manners; determining a rate-distortion cost set corresponding to each coding sub-block group in the coding sub-block group set based on a parallel execution unit; determining a preferred coding sub-block group based on the coding sub-block group set, the rate-distortion cost set and a preset rate-distortion cost condition; and determining a coding block partitioning result corresponding to the video frame to be coded according to the determined preferred coding sub-block group. The base coding block partitioning on the video frame to be coded to obtain the base coding block set comprises: pre-processing the video frame to be coded to obtain a pre-processed video frame; and performing a block processing on the pre-processed video frame to obtain the base coding block set. The determining the coding sub-block group set corresponding to the base coding block based on the preset partitioning manners comprises: determining a pre-selected coding sub-block group set corresponding to the base coding block based on the preset partitioning manners and a pre-processing algorithm; and determining each pre-selected coding sub-block group in the pre-selected coding sub-block group set that satisfies a preset parallel condition as a coding sub-block group to obtain the coding sub-block group set. The determining the coding sub-block group set corresponding to the base coding block based on the preset partitioning manners further comprises: performing a coding sub-block partitioning on the base coding block to generate a first coding sub-block group according to each partitioning manner in the preset partitioning manners to obtain a first coding sub-block group set; performing the following steps on each first coding sub-block group in the first coding sub-block group set: performing the following steps on each first coding sub-block in the first coding sub-block group: performing a coding sub-block partitioning on the first coding sub-block to generate a second coding sub-block group according to each partitioning manner in the preset partitioning manners to obtain a second coding sub-block group set; determining an initial coding sub-block group set corresponding to the first coding sub-block group according to the obtained second coding sub-block group set; and determining the coding sub-block group set corresponding to the base coding block according to the determined initial coding sub-block group set. 5.An apparatus for video coding block partitioning for a hardware encoder, comprising: a base coding block partitioning unit configured to perform a base coding block partitioning on a video frame to be coded to obtain a base coding block set in response to detecting that a coding speed of the hardware encoder is less than a preset coding threshold; and an execution unit configured to perform the following steps on each base coding block in the base coding block set: determining a coding sub-block group set corresponding to the base coding block based on preset partitioning manners; and determining a rate-distortion cost set corresponding to each coding sub-block group in the coding sub-block group set based on a parallel execution unit. ​ ​ 2. The method of claim 1, wherein, ​ ​ ​ 3. The method of claim 1, wherein, ​ ​ ​ 4. The method of claim 1, wherein, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ determine a preferred coding sub-block group based on the set of coding sub-block groups, the set of rate-distortion costs, and a preset rate-distortion cost condition; determine, according to the determined respective preferred coding sub-block group, a coding block partition result corresponding to the video frame to be coded. 6.An electronic device, comprising: one or more processors; a memory device having stored thereon one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-4.

7. A computer readable medium having stored thereon a computer program, wherein, The program is executed by the processor to implement the method according to any one of claims 1-4.

Images