Video encoder, video encoding method, electronic equipment, storage medium and computer program product

Abstract

The present disclosure relates to a video encoder, a video encoding method, an electronic device, a storage medium and a computer program product, the video encoder comprising: a rate distortion optimization module for receiving an ith probability model sent by an entropy encoding module when an (N + i) th coding tree unit CTU in a to-be-encoded video frame needs to be processed, where i is a positive integer greater than or equal to 1, and N is a positive integer greater than or equal to 1; n is a positive integer greater than 1; and the rate distortion optimization module is used for carrying out rate distortion optimization on the (N + i) th CTU by utilizing the ith probability model, and determining and sending coding information of the (N + i) th CTU to the entropy coding module. According to the embodiment of the invention, the calculation time for executing the code rate estimation in the rate distortion optimization process can be effectively reduced, and the overall calculation performance of the video encoder is effectively improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a video encoder, a video encoding method, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] In order to improve the encoding quality of a video encoder, in the rate-distortion optimization process, in order to obtain more accurate code rate information, a code rate estimation method of Context-Adaptive Binary Arithmetic Coding (CABAC) needs to be used. In the entropy coding scheme of a High Efficiency Video Coding (HEVC) encoder, the CABAC code rate estimation method can obtain higher compression efficiency.

[0003] In the CABAC code rate estimation process, the probability model of a CABAC syntax element is updated in real time with the code rate estimation calculation. In the prior art, in order to perform code rate estimation on a current Coding Unit (CU), the probability model updated after the code rate estimation performed on a previous CU needs to be used. Such a code rate estimation method has a pipeline calculation dependency relationship in hardware implementation, which causes a significant performance decline of a hardware video encoder. SUMMARY

[0004] Therefore, the present disclosure provides a video encoder, a video encoding method, an electronic device, a storage medium and a computer program product.

[0005] According to an aspect of the present disclosure, a video encoder is provided, which comprises a rate-distortion optimization module and an entropy coding module. The rate-distortion optimization module is configured to receive an i-th probability model sent by the entropy coding module when a (N+i)-th Coding Tree Unit (CTU) in a video frame to be encoded needs to be processed, wherein i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1. The rate-distortion optimization module is configured to perform rate-distortion optimization on the (N+i)-th CTU by using the i-th probability model, and determine and send encoding information of the (N+i)-th CTU to the entropy coding module.

[0006] In a possible implementation, the rate-distortion optimization module is configured to receive an initial probability model sent by the entropy coding module. The rate-distortion optimization module is configured to perform rate-distortion optimization on a j-th CTU in the video frame to be encoded by using the initial probability model, and determine and send encoding information of the j-th CTU to the entropy coding module, wherein j is a positive integer greater than or equal to 1 and less than or equal to N.

[0007] In a possible implementation, the entropy coding module comprises: an entropy coding sub-module; the entropy coding sub-module is configured to use the (k-1)th probability model to perform entropy coding on the coding information of the kth CTU to obtain the entropy coding result of the kth CTU, where the (k-1)th probability model is the initial probability model, and k is a positive integer greater than or equal to 1.

[0008] In a possible implementation, the entropy coding module comprises: a probability model updating sub-module; the probability model updating sub-module is configured to update the (k-1)th probability model according to the entropy coding result of the kth CTU, determine the kth probability model, and store the kth probability model.

[0009] In a possible implementation, the entropy coding module comprises: a probability model sending sub-module; the rate-distortion optimization module comprises: a probability model receiving sub-module; the probability model sending sub-module is configured to send the target probability model to the probability model receiving sub-module, where the target probability model is the initial probability model or the ith probability model.

[0010] In a possible implementation, each CTU in the to-be-coded video frame corresponds to a different coding unit (CU) division manner; the rate-distortion optimization module comprises: a code rate estimation sub-module; the code rate estimation sub-module is configured to, for any one CTU, use the probability model corresponding to the CTU to perform code rate estimation on each CU in each CU division manner corresponding to the CTU, to obtain a code rate estimation value of the CTU in each CU division manner, where the initial probability model corresponds to the CTU when the CTU is the jth CTU, and the ith probability model corresponds to the CTU when the CTU is the (N+i)th CTU.

[0011] In a possible implementation, the rate-distortion optimization module comprises: a rate-distortion optimization sub-module; the rate-distortion optimization sub-module is configured to, for any one CTU, perform rate-distortion optimization on the CTU according to the code rate estimation value of the CTU in each CU division manner, to determine the coding information of the CTU, where the coding information of the CTU comprises a target CU division manner corresponding to the CTU.

[0012] According to another aspect of the present disclosure, a video coding method is provided, comprising: when a (N+i)th coding tree unit (CTU) in a to-be-coded video frame needs to be processed, using a rate-distortion optimization module to receive an ith probability model sent by an entropy coding module, where i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1; using the rate-distortion optimization module and the ith probability model to perform rate-distortion optimization on the (N+i)th CTU, to determine the coding information of the (N+i)th CTU, and send the coding information of the (N+i)th CTU to the entropy coding module.

[0013] According to another aspect of the present disclosure, there is provided an electronic device comprising a memory, a processor, and a computer program stored on the memory, the processor executing the computer program to implement the steps of the above method.

[0014] According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above method.

[0015] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, or a non-transitory computer readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of the above method.

[0016] According to the video encoder of the embodiment of the present disclosure, when the N+i-th coding tree unit (CTU) in the video frame to be encoded needs to be processed, the rate-distortion optimization module receives the i-th probability model sent by the entropy encoding module, and uses the i-th probability model to perform rate-distortion optimization on the N+i-th CTU, and determines and sends the encoding information of the N+i-th CTU to the entropy encoding module. Compared with the calculation time consumption caused by the need to update the probability model for each CU to perform code rate estimation in the rate-distortion optimization process in the prior art, the video encoder of the embodiment of the present disclosure uses the i-th probability model delayed for N CTUs in the entropy encoding module to perform code rate estimation on the N+i-th CTU, thereby effectively reducing the calculation time consumption of performing code rate estimation in the rate-distortion optimization process, and further effectively improving the overall calculation performance of the video encoder.

[0017] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0019] Figure 1 A schematic diagram of a rate-distortion optimization module in the prior art is shown.

[0020] Figure 2 A block diagram of a video encoder according to an embodiment of the present disclosure is shown.

[0021] Figure 3 A schematic diagram of a video encoder according to an embodiment of the present disclosure is shown.

[0022] Figure 4 A flowchart of a video encoding method according to an embodiment of the present disclosure is shown.

[0023] Figure 5 A block diagram of an electronic device according to an embodiment of the disclosure is shown. DETAILED DESCRIPTION

[0024] Various exemplary embodiments, features, and aspects of the present disclosure will be described below in detail with reference to the accompanying drawings. The same reference numbers in the drawings represent the same elements or features having the same or similar functions. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically noted.

[0025] As used herein, the terms "include," "comprise," "have," or their variants are open-ended, and include one or more stated features, integers, elements, steps, components or functions but do not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof.

[0026] When an element is referred to as being "connected," "coupled," "responsive," or "related" to another element, it can be directly connected, coupled, responsive, or related to the other element, or intervening elements can be present.

[0027] Although the terms first, second, third, and the like can be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Thus, a first element / operation in some embodiments could be termed a second element / operation in other embodiments without departing from the teachings of the present inventive concept.

[0028] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0029] In addition, for the purpose of convenience and brevity, detailed descriptions of well-known functions and structures incorporated herein can not be described in detail. It should be appreciated that the present disclosure can be practiced with the exact details as other implementations can omit, add, or modify the procedures and / or components described herein.

[0030] The CABAC rate estimation method needs to go through the calculation process of binarization, context modeling, and binary arithmetic coding. Binarization refers to converting non-binary syntax elements into a binary string, and each binary bit is called a bin. According to the position and context of each bin, a probability model is selected for the current bin, which can give the probability estimation of the current bin being 0 or 1. Finally, according to the probability provided by the probability model of each bin, each bin in the binary string is gradually arithmetic coded to compress the data into a shorter bit stream.

[0031] The CABAC code rate estimation method has high computation complexity, slow encoding speed, great difficulty in hardware implementation, and high power consumption. However, the code rate estimation of the CABAC code rate estimation method is more accurate, can improve the mode decision accuracy of rate-distortion optimization, and further improve the encoding quality of the video encoder.

[0032] In the CABAC code rate estimation process, the probability model of the CABAC syntax element is updated in real time with the code rate estimation calculation. In the prior art, to perform code rate estimation on the current CU, the probability model updated after the code rate estimation performed on the previous CU is used. Such a code rate estimation method has a pipeline calculation dependency relationship in hardware implementation, which greatly reduces the performance of the hardware video encoder.

[0033] A coding tree unit (CTU) is the most basic and core processing unit in the HEVC video coding standard. A video frame to be encoded can be divided into multiple CTUs, and each CTU is an image block. The size of the CTU can be flexibly set according to the actual application scenario, for example, 64x64 CTU. A CTU can be kept complete, or can be divided into smaller CUs based on various CU division manners to form a tree structure. For example, a 64x64 CTU can correspond to 32x32, 16x16, 8x8, and 4x4 CU division manners, and therefore, a 64x64 CTU can be divided into 4 32x32 CUs based on the 32x32 CU division manner, or can be divided into 16 16x16 CUs based on the 16x16 division manner, or can be divided into 64 8x8 CUs based on the 8x8 CU division manner, or can be divided into 256 4x4 CUs based on the 4x4 CU division manner.

[0034] In the rate-distortion optimization process of the prior art, different CUs under different CU division manners use different probability models for code rate estimation. Figure 1 An example of a rate-distortion optimization module in the prior art is shown in the schematic diagram. Figure 1 The rate-distortion optimization module shown in the schematic diagram needs to use the probability model management and update module and the rate-distortion decision module to perform the processes of code rate estimation (4x4 code rate estimation, 8x8 code rate estimation, 16x16 code rate estimation, and 32x32 code rate estimation) and probability model update for the CUs under the four different CU division manners corresponding to the 64x64 CTU in the rate-distortion optimization process of the 64x64 CTU. The calculation process is very complex, and a large amount of hardware resources are occupied.

[0035] For any 64x64 CTU, code rate estimation is performed in parallel on the CUs under each CU division manner corresponding to the 64x64 CTU.

[0036] For example, the first 4x4 CU is rate estimated using the initial first probability model, and a rate estimate of the first 4x4 CU is obtained. The first probability model is updated according to the rate estimate of the first 4x4 CU. The second 4x4 CU is rate estimated using the updated first probability model, and a rate estimate of the second 4x4 CU is obtained. The first probability model is updated according to the rate estimate of the second 4x4 CU. This process is repeated for each 4x4 CU. The rate estimation of each 4x4 CU is started after the first probability model is updated according to the rate estimate of the previous 4x4 CU.

[0037] The first 8x8 CU is rate estimated using the initial second probability model, and a rate estimate of the first 8x8 CU is obtained. The first 8x8 CU is rate distortion optimized according to the rate estimate of the first 8x8 CU and the rate estimates of the four 4x4 CUs included in the first 8x8 CU, and a rate distortion optimization result of the first 8x8 CU is obtained. The second probability model is updated according to the rate distortion optimization result of the first 8x8 CU. The second 8x8 CU is rate estimated using the updated second probability model, and a rate estimate of the second 8x8 CU is obtained. The second 8x8 CU is rate distortion optimized according to the rate estimate of the second 8x8 CU and the rate estimates of the four 4x4 CUs included in the second 8x8 CU, and a rate distortion optimization result of the second 8x8 CU is obtained. The second probability model is updated according to the rate distortion optimization result of the second 8x8 CU. This process is repeated for each 8x8 CU. The rate estimation of each 8x8 CU is started after the second probability model is updated according to the rate distortion optimization of the previous 8x8 CU and the rate estimates of the four 4x4 CUs included in the previous 8x8 CU.

[0038] The first 16x16 CU is rate-distortion optimized according to the rate-distortion optimization result, and the fourth probability model is updated according to the rate-distortion optimization result; the second 32x32 CU is rate-distinctively estimated by using the updated fourth probability model, a rate-distinctively estimated value of the second 32x32 CU is obtained, the second 32x32 CU is rate-distortion optimized according to the rate-distortion optimization result, a rate-distortion optimization result of the second 32x32 CU is obtained, and the fourth probability model is updated according to the rate-distortion optimization result; and the process is repeated, and details are not repeated. The rate-distinctively estimated value of each 32x32 CU needs to wait for the rate-distortion optimization of the previous 32x32 CU and the four 16x16 CUs included in the previous 32x32 CU according to the rate-distinctively estimated values of the previous 32x32 CU and the four 16x16 CUs, and the fourth probability model is updated based on the rate-distortion optimization result.

[0039] The first 16x16 CU is rate-distortion optimized according to the rate-distortion optimization result, and the fourth probability model is updated according to the rate-distortion optimization result; the second 32x32 CU is rate-distinctively estimated by using the updated fourth probability model, a rate-distinctively estimated value of the second 32x32 CU is obtained, the second 32x32 CU is rate-distortion optimized according to the rate-distortion optimization result, a rate-distortion optimization result of the second 32x32 CU is obtained, and the fourth probability model is updated according to the rate-distortion optimization result; and the process is repeated, and details are not repeated. The rate-distinctively estimated value of each 32x32 CU needs to wait for the rate-distortion optimization of the previous 32x32 CU and the four 16x16 CUs included in the previous 32x32 CU according to the rate-distinctively estimated values of the previous 32x32 CU and the four 16x16 CUs, and the fourth probability model is updated based on the rate-distortion optimization result.

[0040] In the whole hardware implementation process of the prior art video encoder, the rate-distinctively estimated and probability model updating processes are complex, occupy a long time, and have low calculation efficiency, which leads to a decline in the calculation performance of the whole video encoder.

[0041] To solve the above technical problems, the video encoder provided by the embodiment of the present disclosure can reduce the calculation time of performing code rate estimation in the rate-distortion optimization process, and effectively improve the overall calculation performance of the video encoder. The video encoder of the embodiment of the present disclosure is described in detail below.

[0042] Figure 2 A block diagram of a video encoder according to an embodiment of the present disclosure is shown. As shown in the figure, the video encoder 20 comprises a rate-distortion optimization module 21, an entropy encoding module 22. The rate-distortion optimization module 21 is configured to receive the i-th probability model sent by the entropy encoding module 22 when the (N+i)-th CTU in the video frame to be encoded needs to be processed, wherein i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1; the rate-distortion optimization module 21 is configured to perform rate-distortion optimization on the (N+i)-th CTU using the i-th probability model, and determine and send the encoding information of the (N+i)-th CTU to the entropy encoding module 22. Figure 2

[0043] Compared with the calculation time caused by the need to update the probability model when different CUs perform code rate estimation in the rate-distortion optimization process in the prior art, the video encoder of the embodiment of the present disclosure uses the i-th probability model delayed by N CTUs in the entropy encoding module to perform code rate estimation on the (N+i)-th CTU, thereby effectively reducing the calculation time of performing code rate estimation in the rate-distortion optimization process, and further effectively improving the overall calculation performance of the video encoder.

[0044] Wherein, the specific value of N can be flexibly set according to the actual application scene, and the present disclosure does not make specific limitation hereon.

[0045] In a hardware video encoder, the video encoding process is designed as a long encoding pipeline. Among them, the CTU division, intra / inter prediction, transform quantization, rate-distortion optimization and other modules are in the front end and middle end of the pipeline, and the CABAC entropy encoding module is in the last stage of the entire pipeline. In this pipeline design, when the module at the front end of the pipeline is processing the K-th CTU, the CABAC entropy encoding module may still be processing the K-N-th CTU. If the rate-distortion optimization module performs code rate estimation and needs the CABAC information of the current CTU, it must wait for a long time, which will seriously slow down the throughput of the entire encoding pipeline and form a performance bottleneck. In addition, since the complexity of the code stream of different CTUs is different, the waiting time of the current CTU is not necessarily, that is, the encoding speed is not stable. During the entropy encoding process, the CABAC entropy encoding module may need to access the external DDR to read the probability model, which will also bring delay instability factors to the encoding process.

[0046] ​Since the video content is continuous in space and time, the probability models of adjacent CTUs are highly correlated. The probability models of the previous N CTUs, although "outdated", are still a very good estimate of the current CTU. Therefore, the video encoder of the embodiments of the present disclosure uses the probability models of the N delayed CTUs in the entropy coding module to perform rate estimation on the current CTU.

[0047] In a possible implementation, the rate-distortion optimization module 21 is configured to receive the initial probability model sent by the entropy coding module 22, and to perform rate-distortion optimization on the jth CTU in the video frame to be encoded using the initial probability model, and to determine and send the encoding information of the jth CTU to the entropy coding module 22, where j is a positive integer greater than or equal to 1 and less than or equal to N.

[0048] In the initialization stage, the entropy coding module sends a preset initial probability model to the rate-distortion optimization module, and the first N CTUs in the video frame to be encoded share the initial probability model, i.e., the rate-distortion optimization module performs rate-distortion optimization on the first N CTUs using the initial probability model. The initial probability model can be flexibly set according to actual application scenarios, which is not limited in the present disclosure.

[0049] In a possible implementation, the entropy coding module 22 includes a probability model sending sub-module, and the rate-distortion optimization module 21 includes a probability model receiving sub-module. The probability model sending sub-module is configured to send a target probability model to the probability model receiving sub-module, where the target probability model is the initial probability model or the ith probability model.

[0050] Before performing rate estimation on the first CTU in the video frame to be encoded, the probability model sending sub-module in the entropy coding module sends the initial probability model to the probability model receiving sub-module in the rate-distortion optimization module.

[0051] Starting from the (N+1)th CTU in the video frame to be encoded, before performing rate estimation on each CTU, the probability model sending sub-module in the entropy coding module sends the probability model of the N delayed CTUs updated according to the entropy coding result stored in the entropy coding module to the probability model receiving sub-module in the rate-distortion optimization module.

[0052] Since the probability model of the N delayed CTUs can be used starting from the (N+1)th CTU in the video frame to be encoded, and is directly obtained from the entropy coding module without waiting and accessing the external DDR, the encoding pipeline bottleneck is broken, and the unstable delay caused by accessing the external DDR is avoided, effectively improving the performance of the video encoder. In addition, since the obtained probability model is historical data that has been stably stored in the entropy coding module, the problem of unstable encoding speed is solved.

[0053] Because the rate distortion optimization module reuses the probability model in the entropy coding module, it can effectively reduce the hardware resource consumption of the rate distortion process.

[0054] In one possible implementation, the entropy coding module 22 includes: an entropy coding submodule; the entropy coding submodule is used to entropy code the coding information of the kth CTU using the (k-1)th probability model to obtain the entropy coding result of the kth CTU, wherein the (k-1)th probability model is the initial probability model, and k is a positive integer greater than or equal to 1.

[0055] For the first CTU in the video frame to be encoded, the entropy coding submodule in the entropy coding module performs entropy coding on the first CTU according to the initial probability model, and obtains the entropy coding result of the first CTU.

[0056] Starting from the second CTU in the video frame to be encoded, the entropy coding submodule in the entropy coding module performs entropy coding on the current CTU based on the probability model updated after the previous CTU has completed entropy coding, and obtains the entropy coding result of the current CTU.

[0057] In one possible implementation, the entropy coding module 22 includes: a probability model update submodule; the probability model update submodule is used to update the (k-1)th probability model according to the entropy coding result of the kth CTU, and to determine and store the kth probability model.

[0058] For the first CTU in the video frame to be encoded, the probability model update submodule in the entropy coding module updates the initial probability model based on the entropy coding result of the first CTU, and determines and stores the updated first probability model.

[0059] Starting from the second CTU in the video frame to be encoded, the probability model update submodule updates the probability model obtained after the entropy encoding of the previous CTU based on the entropy encoding result of the current CTU, and stores the probability model obtained after the second update.

[0060] Compared to existing technologies where each CU performs a code rate estimation and then updates the model, this embodiment only updates the model after the entire CTU has completed entropy coding, effectively reducing the time spent on model updates.

[0061] Figure 3 A schematic diagram of a video encoder according to an embodiment of the present disclosure is shown. Figure 3 As shown, the video encoder includes an entropy coding module and a rate-distortion optimization module. The entropy coding module includes a probability model sending submodule, an entropy coding submodule, and a probability model updating submodule; the rate-distortion optimization module includes a probability model receiving submodule.

[0062] Before rate estimation is performed on the first CTU in the to-be-encoded video frame, the probability model sending sub-module in the entropy encoding module sends an initial probability model to the probability model receiving sub-module in the rate-distortion optimization module. Assuming that the specific value of N is set to 4, the processing of the first N = 4 CTUs in the to-be-encoded video frame is as follows.

[0063] The rate-distortion optimization module performs rate-distortion optimization on the first CTU using the initial probability model, determines and sends the encoding information of the first CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the first CTU using the initial probability model, to obtain the entropy encoding result of the first CTU. The probability model updating sub-module updates the initial probability model according to the entropy encoding result of the first CTU, determines and stores the updated first probability model.

[0064] The rate-distortion optimization module performs rate-distortion optimization on the second CTU using the initial probability model, determines and sends the encoding information of the second CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the second CTU using the first probability model, to obtain the entropy encoding result of the second CTU. The probability model updating sub-module updates the first probability model according to the entropy encoding result of the second CTU, determines and stores the updated second probability model.

[0065] The rate-distortion optimization module performs rate-distortion optimization on the third CTU using the initial probability model, determines and sends the encoding information of the third CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the third CTU using the second probability model, to obtain the entropy encoding result of the third CTU. The probability model updating sub-module updates the second probability model according to the entropy encoding result of the third CTU, determines and stores the updated third probability model.

[0066] The rate-distortion optimization module performs rate-distortion optimization on the fourth CTU using the initial probability model, determines and sends the encoding information of the fourth CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the fourth CTU using the third probability model, to obtain the entropy encoding result of the fourth CTU. The probability model updating sub-module updates the third probability model according to the entropy encoding result of the fourth CTU, determines and stores the updated fourth probability model.

[0067] Starting from the N + 1 = 5th CTU, the rate-distortion optimization module starts to use the probability model of the N = 4 CTUs that are updated based on entropy encoding in the entropy encoding module.

[0068] Before rate estimation is performed on the 5th CTU in the to-be-encoded video frame, the probability model sending sub-module in the entropy encoding module sends the 1st probability model to the probability model receiving sub-module in the rate-distortion optimization module. The rate-distortion optimization module performs rate-distortion optimization on the 5th CTU by using the 1st probability model, determines and sends the encoding information of the 5th CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the 5th CTU by using the 4th probability model, and obtains the entropy encoding result of the 5th CTU. The probability model updating sub-module updates the 4th probability model according to the entropy encoding result of the 5th CTU, and determines and stores the updated 5th probability model.

[0069] Before rate estimation is performed on the 6th CTU in the to-be-encoded video frame, the probability model sending sub-module in the entropy encoding module sends the 2nd probability model to the probability model receiving sub-module in the rate-distortion optimization module. The rate-distortion optimization module performs rate-distortion optimization on the 6th CTU by using the 2nd probability model, determines and sends the encoding information of the 6th CTU to the entropy encoding sub-module. The entropy encoding sub-module performs entropy encoding on the encoding information of the 6th CTU by using the 5th probability model, and obtains the entropy encoding result of the 6th CTU. The probability model updating sub-module updates the 5th probability model according to the entropy encoding result of the 6th CTU, and determines and stores the updated 6th probability model.

[0070] In this way, the entropy encoding of all the CTUs in the to-be-encoded video frame is completed.

[0071] In a possible implementation, each CTU in the to-be-encoded video frame corresponds to a different CU partitioning manner; the rate-distortion optimization module 21 comprises: a rate estimation sub-module; the rate estimation sub-module is configured to, for any one CTU, perform rate estimation on each CU in each CU partitioning manner corresponding to the CTU by using the probability model corresponding to the CTU, to obtain the rate estimation value of the CTU in each CU partitioning manner, wherein the CTU corresponds to the initial probability model when the CTU is the jth CTU, and the CTU corresponds to the ith probability model when the CTU is the N+i CTU.

[0072] Compared with the time-consuming calculation caused by the fact that the probability model needs to be updated in the rate-distortion optimization process in the prior art, in the embodiment of the present disclosure, the rate estimation sub-module in the rate-distortion optimization module performs rate estimation on each CU in each CU partitioning manner corresponding to any one CTU by using the same probability model, to obtain the rate estimation value of the CTU in each CU partitioning manner, thereby effectively improving the calculation speed of performing rate estimation in the rate-distortion optimization process and reducing the time-consuming calculation of rate estimation.

[0073] For the j-th CTU among the first N CTUs in the video frame to be encoded, the bitrate estimation submodule in the rate-distortion optimization module uses the initial probability model to estimate the bitrate for each CU under each CU partitioning method corresponding to the j-th CTU, and obtains the bitrate estimate value of the j-th CTU under each CU partitioning method.

[0074] Starting from the (N+1)th CTU in the video frame to be encoded, for the (N+i)th CTU, the bitrate estimation submodule in the rate-distortion optimization module uses the i-th probability model that delays N CTUs to estimate the bitrate for each CU under each CU partitioning method corresponding to the (N+i)th CTU, and obtains the bitrate estimate value of the (N+i)th CTU under each CU partitioning method.

[0075] Taking a 64×64 CTU as an example, such as Figure 3 As shown, for any 64×64 CTU, the rate estimation submodule uses the same probability model: 4×4 rate estimation is performed on each 4×4 CU partitioned by the 4×4 CU partitioning method to obtain the rate estimate value for each 4×4 CU; 8×8 rate estimation is performed on each 8×8 CU partitioned by the 8×8 CU partitioning method to obtain the rate estimate value for each 8×8 CU; 16×16 rate estimation is performed on each 16×16 CU partitioned by the 16×16 CU partitioning method to obtain the rate estimate value for each 16×16 CU; and 32×32 rate estimation is performed on each 32×32 CU partitioned by the 32×32 CU partitioning method to obtain the rate estimate value for each 32×32 CU.

[0076] The first N CTUs in the video frame to be encoded perform entropy coding and probability model updates sequentially. Starting from the (N+1)th 64×64 CTU, for the (N+i)th CTU, the updated probability model after delaying N CTUs (the probability model updated after entropy coding of the (i-1)th CTU) is used to perform bitrate estimation (4×4 bitrate estimation, 8×8 bitrate estimation, 16×16 bitrate estimation, 32×32 bitrate estimation) on the CUs corresponding to the (N+i)th 64×64 CTU under the four CU partitioning methods in parallel. A model update is performed only after each CTU has completed entropy coding.

[0077] Compared to existing technologies where each different CU needs to update its model after performing entropy coding during the bitrate estimation process, the embodiments disclosed in this paper can effectively reduce the model update time, thereby effectively reducing computational complexity.

[0078] In a possible implementation, the rate-distortion optimization module 21 comprises a rate-distortion optimization submodule; and the rate-distortion optimization submodule is configured to, for any one CTU, perform rate-distortion optimization on the CTU according to the code rate estimation value of the CTU under each CTU partitioning manner, to determine the encoding information of the CTU, wherein the encoding information of the CTU comprises a target CU partitioning manner corresponding to the CTU.

[0079] For any one CTU, the rate-distortion optimization is performed on the CTU according to the code rate estimation value of the CTU under each CTU partitioning manner, to determine the encoding information of the CTU. The specific process of the rate-distortion optimization can refer to related technologies, and the present disclosure does not make a specific limitation thereon.

[0080] As shown in Figure 3 , the rate-distortion optimization module comprises a rate-distortion optimization submodule.

[0081] The encoding information of each CTU comprises a target CU partitioning manner corresponding to the CTU and other related information that needs to be entropy encoded, and the present disclosure does not make a specific limitation thereon.

[0082] The video encoder according to the embodiments of the present disclosure, when needing to process an N+i-th coding tree unit (CTU) in a to-be-encoded video frame, receives an i-th probability model sent by the entropy encoding module, and performs rate-distortion optimization on the N+i-th CTU by using the i-th probability model, to determine and send the encoding information of the N+i-th CTU to the entropy encoding module. Compared with the calculation time consumption caused by the fact that, in the prior art, the probability model needs to be updated every time the code rate estimation is performed on different CUs in the rate-distortion optimization process, the video encoder according to the embodiments of the present disclosure uses the i-th probability model delayed by N CTUs in the entropy encoding module to perform the code rate estimation on the N+i-th CTU, so as to effectively reduce the calculation time consumption of the code rate estimation in the rate-distortion optimization process, and further effectively improve the overall calculation performance of the video encoder.

[0083] It can be understood that, the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited by the length, the present disclosure will not make a further description. It can be understood by those skilled in the art that, in the above-mentioned method of the specific implementation, the specific execution order of each step should be determined according to its function and possible internal logic.

[0084] In addition, the present disclosure also provides a video encoding method, an electronic device, a computer readable storage medium, and a program.

[0085] Figure 4 A flowchart of a video encoding method according to an embodiment of the present disclosure is shown. The method can be applied to a video encoder as shown in Figure 2 , Figure 3 As shown inFigure 4 The method can include:

[0086] In step S41, when the N+i-th CTU in the video frame to be encoded needs to be processed, the rate-distortion optimization module receives the i-th probability model sent by the entropy encoding module, where i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1.

[0087] In step S42, the rate-distortion optimization module and the i-th probability model perform rate-distortion optimization on the N+i-th CTU, determine the encoding information of the N+i-th CTU, and send the encoding information to the entropy encoding module.

[0088] In a possible implementation, the method further includes:

[0089] The rate-distortion optimization module receives the initial probability model sent by the entropy encoding module.

[0090] The rate-distortion optimization module and the initial probability model perform rate-distortion optimization on the j-th CTU in the video frame to be encoded, determine the encoding information of the j-th CTU, and send the encoding information to the entropy encoding module, where j is a positive integer greater than or equal to 1 and less than or equal to N.

[0091] In a possible implementation, the entropy encoding module includes an entropy encoding submodule.

[0092] The method further includes:

[0093] The entropy encoding submodule and the k-1-th probability model perform entropy encoding on the encoding information of the k-th CTU to obtain the entropy encoding result of the k-th CTU, where the k-1-th probability model is the initial probability model, and k is a positive integer greater than or equal to 1.

[0094] In a possible implementation, the entropy encoding module includes a probability model updating submodule.

[0095] The method further includes:

[0096] The probability model updating submodule and the entropy encoding result of the k-th CTU update the k-1-th probability model to determine and store the k-th probability model.

[0097] In a possible implementation, the entropy encoding module includes a probability model sending submodule, and the rate-distortion optimization module includes a probability model receiving submodule.

[0098] The method further includes:

[0099] The probability model sending submodule sends the target probability model to the probability model receiving submodule, where the target probability model is the initial probability model or the i-th probability model.

[0100] In a possible implementation, each CTU in the video frame to be encoded corresponds to a different CU partitioning manner;

[0101] The method further includes:

[0102] For any one CTU, the rate of each CU under each CU partitioning manner corresponding to the CTU is estimated by using the rate estimation submodule and the probability model corresponding to the CTU, to obtain the rate estimation value of the CTU under each CU partitioning manner, wherein the CTU corresponds to the initial probability model when the CTU is the jth CTU, and the CTU corresponds to the ith probability model when the CTU is the N+i CTU.

[0103] In a possible implementation, the rate-distortion optimization module includes a rate-distortion optimization submodule.

[0104] For any one CTU, the rate-distortion optimization submodule and the rate estimation value of the CTU under each CU partitioning manner are used to perform rate-distortion optimization on the CTU, to determine the encoding information of the CTU, wherein the encoding information of the CTU includes the target CU partitioning manner corresponding to the CTU.

[0105] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0106] The embodiments of the present disclosure also provide an electronic device, including a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the above method.

[0107] The embodiments of the present disclosure also provide a non-volatile computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the above method.

[0108] The embodiments of the present disclosure also provide a computer program product, including a computer program or a non-volatile computer readable storage medium carrying the computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0109] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 can be provided as a server or a terminal device. Referring to Figure 5The electronic device 1900 includes a processing component 1922, which is further composed of one or more processors, and a memory resource represented by the memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0110] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0111] In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932 including computer program instructions, is also provided, which can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0112] The computer readable storage medium can be a tangible device that can retain and store programs for use by an instruction execution device. The computer readable storage medium, for example, can be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanism that can store and read instructions such as one or more of the following: a punch card, an eddy current card, a magnetic strip on a card, as well as any suitable combination of the foregoing. The computer readable storage medium used here is not to be interpreted as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or an electrical signal passing through a wire.

[0113] The computer program (or computer readable program instructions) described herein can be downloaded from a computer readable storage medium to various computing / processing devices by way of a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0114] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing / processing device, partly on the user's computing / processing device, as a stand-alone software package, partly on the user's computing / processing device and partly on a remote computing / processing device or entirely on the remote computing / processing device or server. In the latter scenario, the remote computing / processing device can be connected to the user's computing / processing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing / processing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0115] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operations to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the operations specified in the flow diagrams and / or block diagrams.

[0116] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, to cause a series of operational steps to be performed on the computer to produce a computer-implemented process. The instructions can also cause one or more processors of a computer or other programmable data processing apparatus to

[0117] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0118] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0119] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive of the disclosure. Many modifications and variations of the described embodiments are possible in light of this disclosure. It is intended that the scope of the disclosure be limited not by this detailed description, but rather by the claims appended hereto. The use of the terms "may" and "can" in the description is intended to convey that various embodiments of the present disclosure include, among other things, these and other possible features.

Claims

1. A video encoder, comprising: The video encoder comprises a rate-distortion optimization module and an entropy encoding module. The rate-distortion optimization module is configured to receive the i-th probability model sent by the entropy encoding module when a (N+i)-th coding tree unit (CTU) in the to-be-encoded video frame needs to be processed, wherein i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1. The rate-distortion optimization module is configured to perform rate-distortion optimization on the (N+i)-th CTU by using the i-th probability model, and determine and send the encoding information of the (N+i)-th CTU to the entropy encoding module.

2. The video encoder of claim 1, wherein The rate-distortion optimization module is configured to receive the initial probability model sent by the entropy encoding module. The rate-distortion optimization module is configured to perform rate-distortion optimization on a j-th CTU in the to-be-encoded video frame by using the initial probability model, and determine and send the encoding information of the j-th CTU to the entropy encoding module, wherein j is a positive integer greater than or equal to 1 and less than or equal to N. The entropy encoding module comprises an entropy encoding submodule.

3. The video encoder of claim 2, wherein, The entropy encoding submodule is configured to perform entropy encoding on the encoding information of the k-th CTU by using the (k-1)-th probability model to obtain the entropy encoding result of the k-th CTU, wherein the (k-1)-th probability model is the initial probability model, and k is a positive integer greater than or equal to 1. The entropy encoding module comprises a probability model updating submodule.

4. The video encoder of claim 3, wherein, The probability model updating submodule is configured to update the (k-1)-th probability model according to the entropy encoding result of the k-th CTU, and determine and store the k-th probability model. The entropy encoding module comprises a probability model sending submodule, and the rate-distortion optimization module comprises a probability model receiving submodule.

5. The video encoder of claim 2, wherein, The probability model sending submodule is configured to send a target probability model to the probability model receiving submodule, wherein the target probability model is the initial probability model or the i-th probability model. Each CTU in the to-be-encoded video frame corresponds to a different coding unit (CU) division manner.

6. The video encoder of claim 2, wherein, The rate-distortion optimization module comprises a code rate estimation submodule. The code rate estimation submodule is configured to, for any one CTU, perform code rate estimation on each CU in each CU division manner corresponding to the CTU by using the probability model corresponding to the CTU, to obtain a code rate estimation value of the CTU in each CU division manner, wherein the CTU corresponds to the initial probability model when the CTU is the j-th CTU, and the CTU corresponds to the i-th probability model when the CTU is the (N+i)-th CTU. The rate-distortion optimization module comprises a rate-distortion optimization submodule.

7. The video encoder of claim 6, wherein, The rate-distortion optimization submodule is configured to, for any one CTU, perform rate-distortion optimization on the CTU according to the code rate estimation value of the CTU in each CU division manner, to determine the encoding information of the CTU, wherein the encoding information of the CTU comprises a target CU division manner corresponding to the CTU. The video encoder comprises a rate-distortion optimization module and an entropy encoding module.

8. A method of video encoding, characterized by, ​ When a (N+I)-th coding tree unit (CTU) in the video frame to be encoded needs to be processed, the rate-distortion optimization module receives an i-th probability model sent by the entropy encoding module, wherein i is a positive integer greater than or equal to 1, and N is a positive integer greater than 1; The rate-distortion optimization module and the i-th probability model are used to perform rate-distortion optimization on the (N+I)-th CTU, and encoding information of the (N+I)-th CTU is determined and sent to the entropy encoding module.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 8. The processor executes the computer program to implement the steps of the method of claim 8.

10. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of claim 8.

11. A computer program product comprising a computer program or a non-transitory computer-readable storage medium bearing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of claim 8.

Images