Video encoding method, electronic device and storage medium

Abstract

The present application provides a video encoding method, an electronic device, and a storage medium, the method comprising: sending a first parameter of an M-1th encoding unit to an M-th encoding unit, wherein the first parameter of the M-1th encoding unit is determined by the distortion of the M-1th encoding unit, the M-1th encoding unit is the previous encoding unit of the M-th encoding unit, the M is a positive integer, and the M is greater than 1; calculating the distortion of the M-th encoding unit based on the first parameter of the M-1th encoding unit; determining the first parameter of the M-th encoding unit based on the distortion of the M-th encoding unit; sending the first parameter of the M-th encoding unit to the M+1th encoding unit, wherein the M+1th encoding unit is the next encoding unit of the M-th encoding unit. The method provided by the present application helps to improve the efficiency of video encoding.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a video encoding method, an electronic device and a storage medium. Background Art

[0002] A video encoder usually includes coding units such as macroblocks (MB). In order to ensure the coding efficiency of the video encoder, the video encoder uses the rate-distortion optimization (RDO) algorithm to select a set of coding modes with the best rate-distortion performance for the coding unit when encoding the above coding units. The optimal coding mode can be used to encode the current coding unit and as a reference for mode selection for the next coding unit.

[0003] However, when the video encoder selects the optimal coding mode for the coding unit using the rate-distortion optimization algorithm, it will traverse all optional coding modes and select the optimal coding mode from them, thereby reducing the coding efficiency of the entire video encoder. Summary of the invention

[0004] The present application provides a video encoding method, an electronic device and a storage medium, which are helpful to improve the efficiency of video encoding.

[0005] In a first aspect, the present application provides a video encoding method, comprising:

[0006] Sending a first parameter of an M-1th coding unit to an M-th coding unit, wherein the first parameter of the M-1th coding unit is determined by a distortion of the M-1th coding unit, the M-1th coding unit is a previous coding unit of the M-th coding unit, and M is a positive integer greater than 1;

[0007] Calculating a distortion of the M-th coding unit based on a first parameter of the M-1-th coding unit;

[0008] Determining a first parameter of the Mth coding unit based on the distortion of the Mth coding unit;

[0009] The first parameter of the Mth coding unit is sent to the M+1th coding unit, wherein the M+1th coding unit is a next coding unit of the Mth coding unit.

[0010] In the present application, by calculating the target coding parameters from the distortion in the current coding unit and sending the target coding parameters to the next coding unit, the target coding parameters can be calculated without waiting for the bit rate result, thereby improving the coding efficiency of the subsequent coding units.

[0011] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target intra-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target intra-frame prediction mode of the Mth coding unit.

[0012] In one possible implementation manner, determining the first parameter of the Mth coding unit based on the distortion of the Mth coding unit includes:

[0013] Comparing the distortions of the optional intra prediction modes in the Mth coding unit;

[0014] The optional intra-frame prediction mode corresponding to the minimum distortion is determined as the first parameter of the Mth coding unit.

[0015] In one possible implementation, the coding unit is a second type macroblock, the first type macroblock has inter-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target inter-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target inter-frame prediction mode of the Mth coding unit.

[0016] In one possible implementation manner, determining the first parameter of the Mth coding unit based on the distortion of the Mth coding unit includes:

[0017] Comparing the distortions of the optional inter-frame prediction modes in the M-th coding unit;

[0018] An optional inter-frame prediction mode corresponding to the minimum distortion is determined as a first parameter of the Mth coding unit.

[0019] In one possible implementation, the method further includes:

[0020] Determine a second parameter of the M-th coding unit based on the first parameter of the M-1-th coding unit;

[0021] The Mth coding unit is encoded based on a second parameter of the Mth coding unit.

[0022] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the second parameter of the Mth coding unit includes a second target intra-frame prediction mode of the Mth coding unit, and the second parameter of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0023] In one possible implementation, the coding unit is a second type macroblock, the second type macroblock has inter-frame prediction capability, the second parameter of the Mth coding unit includes a second target inter-frame prediction mode and motion information of the Mth coding unit, and the second target inter-frame prediction mode of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0024] In a second aspect, the present application provides a video encoding device, including:

[0025] A receiving module sends a first parameter of an M-1th coding unit to an M-th coding unit, wherein the first parameter of the M-1th coding unit is determined by a distortion of the M-1th coding unit, the M-1th coding unit is a previous coding unit of the M-th coding unit, and M is a positive integer greater than 1;

[0026] A calculation module, configured to calculate the distortion of the M-th coding unit based on the first parameter of the M-1-th coding unit;

[0027] A determination module, configured to determine a first parameter of the Mth coding unit based on the distortion of the Mth coding unit;

[0028] A sending module is used to send the first parameter of the Mth coding unit to the M+1th coding unit, wherein the M+1th coding unit is the next coding unit of the Mth coding unit.

[0029] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target intra-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target intra-frame prediction mode of the Mth coding unit.

[0030] In one possible implementation, the determination module is used to compare the distortion of each optional intra-frame prediction mode in the Mth coding unit;

[0031] The optional intra-frame prediction mode corresponding to the minimum distortion is determined as the first parameter of the Mth coding unit.

[0032] In one possible implementation, the coding unit is a second type macroblock, the first type macroblock has inter-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target inter-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target inter-frame prediction mode of the Mth coding unit.

[0033] In one possible implementation, the determination module is used to compare the distortion of each optional inter-frame prediction mode in the Mth coding unit;

[0034] An optional inter-frame prediction mode corresponding to the minimum distortion is determined as a first parameter of the Mth coding unit.

[0035] In one possible implementation, the device further includes:

[0036] A coding module, configured to determine a second parameter of the M-th coding unit based on a first parameter of the M-1-th coding unit;

[0037] The Mth coding unit is encoded based on a second parameter of the Mth coding unit.

[0038] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the second parameter of the Mth coding unit includes a second target intra-frame prediction mode of the Mth coding unit, and the second parameter of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0039] In one possible implementation, the coding unit is a second type macroblock, the first type macroblock has inter-frame prediction capability, the second parameter of the Mth coding unit includes a second target inter-frame prediction mode and motion information of the Mth coding unit, and the second target inter-frame prediction mode of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0040] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store a computer program; the processor is used to run the computer program to implement the video encoding method as described in the first aspect.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is executed on a computer, the computer implements the video encoding method as described in the first aspect.

[0042] In a fifth aspect, the present application provides a computer program. When the above-mentioned computer program runs on a processor of an electronic device, the electronic device executes the video encoding method described in the first aspect.

[0043] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the system architecture provided for an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a flow chart of an embodiment of a video encoding method provided by the present application;

[0046] Figure 3 A schematic diagram of a flow chart of another embodiment of the video encoding method provided by the present application;

[0047] Figure 4 A schematic diagram of a flow chart of another embodiment of the video encoding method provided by the present application;

[0048] Figure 5 A schematic diagram of a flow chart of another embodiment of the video encoding method provided by the present application;

[0049] Figure 6 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In the embodiments of the present application, unless otherwise specified, the character " / " indicates that the objects before and after the association are in an or relationship. For example, A / B can represent A or B. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0052] It should be pointed out that the words "first", "second", etc. involved in the embodiments of the present application are only used to distinguish the description purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.

[0053] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C may represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C may be an element itself, or a set containing one or more elements.

[0054] In the embodiments of the present application, "exemplary", "in some embodiments", "in another embodiment", etc. are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0055] In the embodiments of the present application, "of", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. In the embodiments of the present application, communication and transmission can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. For example, transmission can include sending and / or receiving, which can be a noun or a verb.

[0056] The equal to involved in the embodiments of the present application can be used in conjunction with greater than, and is applicable to the technical solution adopted when greater than, and can also be used in conjunction with less than, and is applicable to the technical solution adopted when less than. It should be noted that when equal to is used in conjunction with greater than, it cannot be used in conjunction with less than; when equal to is used in conjunction with less than, it cannot be used in conjunction with greater than.

[0057] When the video encoder uses the rate-distortion optimization algorithm to select the optimal coding mode for the coding unit, it will traverse all optional coding modes, calculate the rate-distortion cost corresponding to the optional coding mode, and take the optional coding mode with the smallest rate-distortion cost as the optimal coding mode. Among them, the rate-distortion cost is mainly composed of distortion and bit rate. The distortion can be the distortion between the original pixel and the reconstructed pixel of the coding unit. For example, the distortion can be obtained after the coding unit is predicted, transformed, quantized, dequantized and inversely transformed; the bit rate can be the bit rate required by the coding mode and the quantized residual coefficient. For example, the bit rate can be obtained after entropy coding the coding mode and the quantized residual coefficient. Entropy coding can include methods such as context-adaptive variable-length coding (CAVLC) and context-adaptive binary arithmetic coding (CABAC).

[0058] Taking CABAC entropy coding as an example, when CABAC entropy coding is used to calculate the bit rate of the coding mode, CABAC needs to first binarize the non-binary syntax element (Syntax Element, SE), and then encode the binary symbol according to the corresponding context model. In the coding process, the encoding of the current binary symbol depends on the result of the previous binary symbol encoding. Therefore, for the case where the quantization parameter (Quantization Parameter, QP) is small, the CABAC bit rate calculation model usually consumes a lot of time due to the large number of non-zero coefficients after quantization. However, the time consumed by the distortion calculation of the coding unit is much less than the time consumed by the bit rate calculation. Therefore, after obtaining the distortion of the coding unit, the video encoder still needs to wait for a long time to obtain the result of the bit rate calculation. Since the distortion and bit rate calculation of the subsequent coding unit are dependent on the coding mode of the previous coding unit, the excessive time to obtain the coding mode of the previous coding unit means that the processing efficiency of the next coding unit is reduced, which will seriously affect the processing speed of the entire video encoder.

[0059] Based on the above problems, an embodiment of the present application proposes a video encoding method, which is applied to an electronic device, which may be a terminal device with functions such as shooting, and / or recording, and / or screen recording and / or screen projection. The electronic device may be a fixed terminal, such as a desktop computer, a monitoring device, a smart screen, a large screen, a laptop computer, etc., and the electronic device may also be a mobile terminal, such as a mobile phone, a tablet, a vehicle-mounted terminal, a driving recorder, etc. The mobile terminal may also be referred to as a user equipment (UE), an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device. The mobile terminal can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set top box (STB), a customer premise equipment (CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, for example, a mobile terminal in a 5G network or a mobile terminal in a future Public Land Mobile Network (PLMN) network. The mobile terminal can also be a wearable device. Wearable devices can also be called wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, watches, etc.

[0060] Figure 1A schematic diagram of a coding unit provided for an embodiment of the present application, wherein the coding unit may include multiple types of macroblocks, for example, a first type macroblock and a second type macroblock, a macroblock may be a block composed of multiple pixels, and taking the H264 video encoder as an example, a macroblock may be a block of 16*16 pixels. It is understandable that a macroblock may also be divided into multiple sub-blocks, for example, the size of each sub-block may be 4*4 or 8*8, and the sub-block may also be referred to as a prediction unit (PU). The first type macroblock may use the decoded pixels in the current slice as a reference for intra-frame prediction, and therefore, the first type macroblock may have intra-frame prediction capability, that is, an intra-frame prediction coding mode may be set for the first type macroblock; the second type macroblock may use the previously decoded image as a reference image for inter-frame prediction, and therefore, the second type macroblock may have inter-frame prediction capability, that is, an inter-frame prediction coding mode may be set for the second type macroblock.

[0061] It can be understood that the encoding unit may further include more types of macroblocks, and the above-mentioned first type macroblocks and second type macroblocks are merely exemplary descriptions and do not constitute a limitation on the embodiments of the present application.

[0062] Now combined Figure 2-Figure 5 The video encoding method provided in the embodiment of the present application is exemplified. The video encoding method provided in the embodiment of the present application can be applied to various video encoders, such as H264, H265, H266, etc., but it is not enough to limit the embodiment of the present application. In some embodiments, it can also be applied to other types of video encoders. The following is an exemplary description using the H264 encoder as an example.

[0063] Figure 2 A schematic diagram of a flow chart of an embodiment of a video encoding method provided by the present application. Figure 2 The illustrated embodiment can be applied to the selection of an intra-frame prediction mode, and specifically includes the following steps:

[0064] Step 201: The video encoder sends the first target intra prediction mode of the previous coding unit to the current coding unit.

[0065] Specifically, the previous coding unit can obtain two target intra-frame prediction modes, for example, a first target intra-frame prediction mode and a second target intra-frame prediction mode, wherein the first target intra-frame prediction mode of the previous coding unit is used to send to the current coding unit, so that the current coding unit selects the target intra-frame prediction mode based on the first target intra-frame prediction mode sent by the previous coding unit. The second target intra-frame prediction mode of the previous coding unit is used to perform real-time encoding on the previous coding unit. The manner in which the previous coding unit obtains the first target intra-frame prediction mode and the second target intra-frame prediction mode is described below and will not be repeated here.

[0066] After the video encoder determines the first target intra prediction mode for the previous coding unit, the first target intra prediction mode may be sent to the current coding unit. Accordingly, the current coding unit may receive the first target intra prediction mode sent by the previous coding unit.

[0067] It can be understood that after the previous coding unit determines the second target intra-frame prediction mode, the video encoder can use the second target intra-frame prediction mode to perform real-time encoding on the previous coding unit, while the first target intra-frame prediction mode of the previous coding unit is not used for real-time encoding.

[0068] Taking the previous coding unit as coding unit A and the current coding unit as coding unit B as an example, coding unit A can obtain the first target intra-frame prediction mode and the second target intra-frame prediction mode by calculation. After coding unit A obtains the first target intra-frame prediction mode, the first target intra-frame prediction mode can be sent to coding unit B, so that coding unit B calculates the first target intra-frame prediction mode of coding unit B based on the first target intra-frame prediction mode of coding unit A; in addition, after coding unit A obtains the second target intra-frame prediction mode, the second target intra-frame prediction mode can be used for real-time encoding.

[0069] Step 202: The video encoder calculates the distortion of each optional intra-frame prediction mode in the current coding unit based on the first target intra-frame prediction mode of the previous coding unit, and determines the first target intra-frame prediction mode of the current coding unit based on the distortion of each optional intra-frame prediction mode in the current coding unit.

[0070] Specifically, when the current coding unit receives the first target intra prediction mode of the previous coding unit, the video encoder may calculate the distortion of each optional intra prediction mode in the current coding unit based on the first target intra prediction mode of the previous coding unit.

[0071] The distortion of each optional intra-frame prediction mode in the current coding unit can be calculated by the original pixel and the reconstructed pixel of the current coding unit, and the reconstructed pixel can be determined by the current coding unit based on the first target intra-frame prediction mode of the previous coding unit. The specific method of obtaining the reconstructed pixel can refer to the existing related technology, which will not be repeated here.

[0072] In a specific implementation, the distortion of each optional intra-frame prediction mode in the current coding unit can be obtained by calculating the sum of squared errors (Sum of Squared Error, SSE), where the calculation formula of the sum of squared errors can be as follows:

[0073]

[0074] sse is the sum of square errors of the optional intra prediction modes in the current coding unit, s i is the i-th original pixel, r mode,i is the i-th reconstructed pixel of each optional intra-frame prediction mode in the current coding unit, mode is the optional intra-frame prediction mode in the current coding unit, and N is the total number of pixels included in the current coding unit.

[0075] Through the above-mentioned square error sum calculation formula, the square error sum corresponding to each optional intra-frame prediction mode in the current coding unit can be calculated, and the optional intra-frame prediction mode corresponding to the minimum square error sum can be determined as the first target intra-frame prediction mode of the current coding unit. The first target intra-frame prediction mode of the current coding unit can be used to send to the next coding unit, and the first target intra-frame prediction mode of the current coding unit can be used to calculate the first target intra-frame prediction mode of the next coding unit.

[0076] It can be understood that the above-mentioned method of determining the distortion by calculating the sum of square errors is only an exemplary description, but does not constitute a limitation on the embodiments of the present application. In some embodiments, the distortion can also be determined by other mathematical methods.

[0077] Step 203: The video encoder sends the first target intra prediction mode of the current coding unit to the next coding unit.

[0078] Specifically, after the current coding unit determines the first target intra-frame prediction mode of the current coding unit, the video encoder may send the first target intra-frame prediction mode of the current coding unit to the next coding unit, so that the next coding unit may select the first target intra-frame prediction mode of the next coding unit based on the first target intra-frame prediction mode of the current coding unit. The way in which the next coding unit determines the first target intra-frame prediction mode of the next coding unit can refer to the way in which the current coding unit determines the first target intra-frame prediction mode of the current coding unit in the above embodiment, which will not be repeated here.

[0079] Step 204: The video encoder calculates the rate-distortion cost of each optional intra-frame prediction mode in the current coding unit based on the first target intra-frame prediction mode of the previous coding unit, and determines the second target intra-frame prediction mode of the current coding unit based on the rate-distortion cost of each optional intra-frame prediction mode in the current coding unit.

[0080] Specifically, the speed at which the video encoder calculates the bit rate is slower than the speed at which the distortion is calculated. Therefore, after calculating the distortion of each optional intra-frame prediction mode in the current coding unit, the video encoder can wait for the calculation results of the bit rates of each optional intra-frame prediction mode in the current coding unit. When the current coding unit obtains the calculation results of the bit rates of each optional intra-frame prediction mode in the current coding unit, the video encoder can determine the rate-distortion cost based on the calculation results of the distortion. It can be understood that the bit rate of the current coding unit can also be determined by the first target intra-frame prediction mode of the previous coding unit.

[0081] In a specific implementation, the calculation method of the above rate-distortion cost can be shown by the following formula:

[0082] J(s,r,mode|λ mode )=sse(s,r,mode)+λ mode ×R(s,r,mode);

[0083] Where J is the rate-distortion cost of each optional intra-frame prediction mode in the current coding unit, λ mode is the Lagrangian factor of each optional intra-frame prediction mode in the current coding unit, and R is the bit rate of each optional intra-frame prediction mode in the current coding unit.

[0084] It can be understood that through the above-mentioned rate-distortion cost calculation formula, the rate-distortion cost corresponding to each optional intra-frame prediction mode in the current coding unit can be calculated, and the optional intra-frame prediction mode corresponding to the minimum rate-distortion cost can be determined as the second target intra-frame prediction mode of the current coding unit, and the second target intra-frame prediction mode can be used to perform real-time encoding of the current coding unit.

[0085] Now take the coding unit as the first type of macroblock as an example, combined with Figure 3 The above video encoding method is exemplarily described. Figure 3 , the first type macroblock A is the previous first type macroblock, the first type macroblock B is the current first type macroblock, and the first type macroblock C is the next first type macroblock. After calculating and obtaining the first target intra-frame prediction mode of the first type macroblock A, the first type macroblock A sends the first target intra-frame prediction mode of the first type macroblock A to the first type macroblock B. After receiving the first target intra-frame prediction mode of the first type macroblock A, the first type macroblock B can calculate the distortion of the first type macroblock B based on the first target intra-frame prediction mode of the first type macroblock A, determine the first target intra-frame prediction mode of the first type macroblock B according to the distortion of the first type macroblock B, and can send the first target intra-frame prediction mode of the first type macroblock B to the first type macroblock C. In addition, the first type macroblock B can also calculate the distortion and bit rate of the first type macroblock B based on the first target intra-frame prediction mode of the first type macroblock A, determine the rate-distortion cost of the first type macroblock B by the distortion and bit rate of the first type macroblock B, determine the second target intra-frame prediction mode by the rate-distortion cost of the first type macroblock B, and encode the first type macroblock B using the second target intra-frame prediction mode of the first type macroblock B.

[0086] In an embodiment of the present application, the current coding unit calculates the distortion through the target intra-frame prediction mode of the previous coding unit, determines the target intra-frame prediction mode of the current coding unit by the distortion, and sends the target intra-frame prediction mode of the current coding unit to the next coding unit, thereby obtaining the target intra-frame prediction mode without waiting for the calculation result of the bit rate, so that the subsequent coding unit can calculate the distortion and bit rate as soon as possible according to the target intra-frame prediction mode of the previous coding unit, thereby improving the coding efficiency.

[0087] The above text exemplifies the video encoding method by selecting an intra-frame prediction mode. Next, the following text exemplifies the video encoding method by selecting an inter-frame prediction mode.

[0088] Figure 4 A schematic diagram of another embodiment of the video encoding method provided in the present application specifically includes the following steps:

[0089] Step 401: The video encoder sends the first target inter-frame prediction mode of the previous coding unit to the current coding unit.

[0090] Specifically, the previous coding unit can obtain two target inter-frame prediction modes, for example, a first target inter-frame prediction mode and a second target inter-frame prediction mode, wherein the first target inter-frame prediction mode of the previous coding unit is used to send to the current coding unit, so that the current coding unit selects the target inter-frame prediction mode based on the first target inter-frame prediction mode sent by the previous coding unit. The second target inter-frame prediction mode of the previous coding unit is used to encode the previous coding unit. The manner in which the previous coding unit obtains the first target inter-frame prediction mode and the second target intra-frame prediction mode is described below and will not be repeated here.

[0091] After the video encoder determines the first target inter-frame prediction mode for the previous coding unit, the first target inter-frame prediction mode may be sent to the current coding unit. Accordingly, the current coding unit may receive the first target inter-frame prediction mode sent by the previous coding unit.

[0092] It can be understood that after the previous coding unit determines the second target inter-frame prediction mode, the video encoder can use the second target inter-frame prediction mode to perform real-time encoding on the previous coding unit, while the first target inter-frame prediction mode of the previous coding unit is not used for real-time encoding.

[0093] Taking the previous coding unit as coding unit A and the current coding unit as coding unit B as an example, coding unit A can obtain the first target inter-frame prediction mode and the second target inter-frame prediction mode by calculation. After coding unit A obtains the first target inter-frame prediction mode, the first target inter-frame prediction mode can be sent to coding unit B, so that coding unit B calculates the first target inter-frame prediction mode of coding unit B based on the first target inter-frame prediction mode of coding unit A; in addition, after coding unit A obtains the second target inter-frame prediction mode, the second target inter-frame prediction mode can be used for real-time encoding.

[0094] Step 402: The video encoder calculates the distortion of each optional inter-frame prediction mode in the current coding unit based on the first target inter-frame prediction mode of the previous coding unit, and determines the first target inter-frame prediction mode of the current coding unit based on the distortion of each optional inter-frame prediction mode in the current coding unit.

[0095] Specifically, when the current coding unit receives the first target inter-frame prediction mode of the previous coding unit, the video encoder may calculate the distortion of each optional inter-frame prediction mode in the current coding unit based on the first target inter-frame prediction mode of the previous coding unit.

[0096] The distortion of each optional inter-frame prediction mode in the current coding unit can be calculated by the original pixel and the reconstructed pixel of the current coding unit, and the reconstructed pixel can be determined by the current coding unit based on the first target inter-frame prediction mode of the previous coding unit. The calculation method of the distortion of each optional inter-frame prediction mode in the current coding unit can specifically refer to the calculation method of the distortion of each optional intra-frame prediction mode in the above embodiment, which will not be repeated here.

[0097] It can be understood that, with reference to the above-mentioned calculation formula for the sum of square errors, the sum of square errors corresponding to each optional inter-frame prediction mode in the current coding unit can be calculated, and the minimum square error and corresponding optional inter-frame prediction mode can be determined as the first target inter-frame prediction mode of the current coding unit, and the first target inter-frame prediction mode of the current coding unit can be used to send to the next coding unit, and the first target inter-frame prediction mode of the current coding unit can be used to calculate the first target inter-frame prediction mode of the next coding unit.

[0098] Step 403: The video encoder performs motion estimation based on the first target inter-frame prediction mode of the previous coding unit to obtain motion information of the current coding unit.

[0099] Specifically, after the current coding unit receives the first target inter-frame prediction mode of the previous coding unit, the video encoder can obtain the corresponding motion information based on the first target inter-frame prediction mode of the previous coding unit, wherein the motion information may include information such as motion vector (Motion Estimation, MV). It can be understood that the above motion information may also include other information in the motion estimation process, and the embodiment of the present application does not specifically limit this. Then, the current coding unit can perform motion vector prediction (Motion Vector Prediction, MVP) based on the motion information and use the motion information as a starting point candidate for motion search (Motion Search, MS) of the current coding unit.

[0100] Step 404: the video encoder sends the first target inter-frame prediction mode of the current coding unit to the next coding unit.

[0101] Specifically, after the current coding unit determines the first target inter-frame prediction mode of the current coding unit, the video encoder can send the first target inter-frame prediction mode of the current coding unit to the next coding unit, so that the next coding unit can select the first target inter-frame prediction mode of the next coding unit based on the first target inter-frame prediction mode of the current coding unit. The way in which the next coding unit determines the first target inter-frame prediction mode of the next coding unit can refer to the way in which the current coding unit determines the first target inter-frame prediction mode of the current coding unit in the above embodiment, which will not be repeated here.

[0102] Step 405: The video encoder determines the rate-distortion cost of each optional inter-frame prediction mode in the current coding unit based on the first target inter-frame prediction mode of the previous coding unit, and determines the second target inter-frame prediction mode of the current coding unit based on the rate-distortion cost of each optional inter-frame prediction mode in the current coding unit.

[0103] Specifically, the rate-distortion cost of each optional inter-frame prediction mode in the current coding unit can be determined by the distortion and the corresponding bit rate corresponding to each optional inter-frame prediction mode in the current coding unit, wherein the calculation method of the rate-distortion cost of each optional inter-frame prediction mode in the current coding unit can specifically refer to the calculation method of the rate-distortion cost of each optional intra-frame prediction mode in the current coding unit in the above-mentioned embodiment, which will not be repeated here.

[0104] By calculating the rate-distortion cost of each optional inter-frame prediction mode in the current coding unit, the rate-distortion cost corresponding to each optional inter-frame prediction mode in the current coding unit can be calculated, and the optional inter-frame prediction mode corresponding to the minimum rate-distortion cost can be determined as the second target inter-frame prediction mode of the current coding unit, and the second target inter-frame prediction mode can be used together with motion information to perform real-time encoding on the current coding unit.

[0105] Now take the coding unit as the second type macroblock as an example, combined with Figure 5 The above video encoding method is exemplarily described. Figure 5 , the second type macroblock A is the previous second type macroblock, the second type macroblock B is the current second type macroblock, and the second type macroblock C is the next second type macroblock. After calculating and obtaining the first target inter-frame prediction mode of the second type macroblock A, the second type macroblock A sends the first target inter-frame prediction mode of the second type macroblock A to the second type macroblock B. After receiving the first target inter-frame prediction mode of the second type macroblock A, the second type macroblock B can calculate the distortion of the second type macroblock B based on the first target inter-frame mode of the second type macroblock A, determine the first target inter-frame prediction mode of the second type macroblock B according to the distortion of the second type macroblock B, and can send the first target inter-frame prediction mode of the second type macroblock B to the second type macroblock C. In addition, the second type macroblock B can also calculate the distortion and bit rate of the second type macroblock B based on the first target inter-frame prediction mode of the second type macroblock A, determine the rate-distortion cost of the second type macroblock B by the distortion and bit rate of the second type macroblock B, and determine the second target inter-frame prediction mode by the rate-distortion cost of the second type macroblock B; and motion estimation can be performed based on the first target inter-frame prediction mode of the second type macroblock A, thereby obtaining the motion information of the second type macroblock B, and the second type macroblock B can be encoded using the second target inter-frame prediction mode and motion information of the second type macroblock B.

[0106] In an embodiment of the present application, the current coding unit calculates the distortion through the target inter-frame prediction mode of the previous coding unit, determines the target inter-frame prediction mode of the current coding unit by the distortion, and sends the target inter-frame prediction mode of the current coding unit to the next coding unit, thereby obtaining the target inter-frame prediction mode without waiting for the calculation result of the bit rate, so that the subsequent coding unit can calculate the distortion and bit rate as soon as possible according to the target inter-frame prediction mode of the previous coding unit, thereby improving the coding efficiency.

[0107] Figure 6 This is a schematic diagram of the structure of an embodiment of the video encoding device of the present application. Figure 6 As shown, the video encoding device 60 may include: a receiving module 61, a calculating module 62, a determining module 63 and a sending module 64; wherein,

[0108] A receiving module 61 sends a first parameter of an M-1th coding unit to an M-th coding unit, wherein the first parameter of the M-1th coding unit is determined by the distortion of the M-1th coding unit, the M-1th coding unit is a previous coding unit of the M-th coding unit, and M is a positive integer, and M is greater than 1;

[0109] A calculation module 62, configured to calculate the distortion of the M-th coding unit based on the first parameter of the M-1-th coding unit;

[0110] A determination module 63, configured to determine a first parameter of the Mth coding unit based on the distortion of the Mth coding unit;

[0111] The sending module 64 is used to send the first parameter of the Mth coding unit to the M+1th coding unit, wherein the M+1th coding unit is the next coding unit of the Mth coding unit.

[0112] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target intra-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target intra-frame prediction mode of the Mth coding unit.

[0113] In one possible implementation, the determination module 63 is used to compare the distortion of each optional intra-frame prediction mode in the Mth coding unit;

[0114] The optional intra-frame prediction mode corresponding to the minimum distortion is determined as the first parameter of the Mth coding unit.

[0115] In one possible implementation, the coding unit is a second type macroblock, the first type macroblock has inter-frame prediction capability, the first parameter of the M-1th coding unit is used to characterize the first target inter-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target inter-frame prediction mode of the Mth coding unit.

[0116] In one possible implementation, the determination module 63 is used to compare the distortions of the optional inter-frame prediction modes in the Mth coding unit;

[0117] An optional inter-frame prediction mode corresponding to the minimum distortion is determined as a first parameter of the Mth coding unit.

[0118] In one possible implementation, the device 60 further includes:

[0119] A coding module, configured to determine a second parameter of the M-th coding unit based on a first parameter of the M-1-th coding unit;

[0120] The Mth coding unit is encoded based on a second parameter of the Mth coding unit.

[0121] In one possible implementation, the coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the second parameter of the Mth coding unit includes a second target intra-frame prediction mode of the Mth coding unit, and the second parameter of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0122] In one possible implementation, the coding unit is a second type macroblock, the first type macroblock has inter-frame prediction capability, the second parameter of the Mth coding unit includes a second target inter-frame prediction mode and motion information of the Mth coding unit, and the second target inter-frame prediction mode of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

[0123] Figure 6 The video encoding device 60 provided in the illustrated embodiment may be used to execute the technical solution of the method embodiment illustrated in the present application, and its implementation principle and technical effects may be further referred to the relevant description in the method embodiment.

[0124] It should be understood that the division of the various modules of the above video encoding device 60 is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in a processor element or instructions in the form of software.

[0125] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0126] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0127] Combine the following Figure 7 The exemplary electronic device provided in the embodiments of the present application is further introduced. Figure 7 A schematic structural diagram of an electronic device 700 is shown.

[0128] The electronic device 700 may include: at least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the present application. Figure 1-Figure 5 The illustrated embodiment provides a video encoding method.

[0129] Figure 7 A block diagram of an exemplary electronic device 700 suitable for implementing the embodiments of the present application is shown. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0130] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: one or more processors 710, a memory 720, a communication bus 740 connecting different system components (including the memory 720 and the processor 710), and a communication interface 730.

[0131] The communication bus 740 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0132] The electronic device 700 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0133] The memory 720 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Figure 7Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 740 via one or more data medium interfaces. The memory 720 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0134] A program / utility having a set (at least one) of program modules may be stored in memory 720, such program modules including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules typically perform the functions and / or methods of the embodiments described herein.

[0135] The electronic device 700 may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via the communication interface 730. Furthermore, the electronic device 700 may also communicate with the network adapter ( Figure 7 The network adapter can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the communication bus 740. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Drives; hereinafter referred to as: RAID) systems, tape drives, and data backup storage systems.

[0136] The processor 710 executes various functional applications and data processing by running the programs stored in the memory 720, such as implementing the remote access method provided in the embodiment of the present application.

[0137] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 700. In other embodiments of the present application, the electronic device 700 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0138] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0139] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method provided by the embodiment shown in the present application.

[0140] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is run on a computer, it enables the computer to execute the method provided by the embodiment shown in the present application.

[0141] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0142] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0145] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video encoding method, characterized in that: The method comprises: Sending a first parameter of an M-1th coding unit to an M-th coding unit, wherein the first parameter of the M-1th coding unit is determined by a distortion of the M-1th coding unit, the M-1th coding unit is a previous coding unit of the M-th coding unit, M is a positive integer, and M is greater than 1; wherein the first parameter includes an intra-frame prediction mode or an inter-frame prediction mode; Calculating a distortion of the M-th coding unit based on a first parameter of the M-1-th coding unit; Determining a first parameter of the Mth coding unit based on the distortion of the Mth coding unit; Sending the first parameter of the Mth coding unit to the M+1th coding unit, wherein the M+1th coding unit is a next coding unit of the Mth coding unit; The calculating the distortion of the M-th coding unit based on the first parameter of the M-1-th coding unit comprises: Determine a reconstructed pixel based on a first parameter of the M-1th coding unit; A distortion of the Mth coding unit is calculated based on the reconstructed pixels and original pixels of the Mth coding unit.

2. The method according to claim 1, characterized in that The coding unit is a first type macroblock, which has intra-frame prediction capability. The first parameter of the M-1th coding unit is used to characterize the first target intra-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target intra-frame prediction mode of the Mth coding unit.

3. The method according to claim 2, characterized in that The determining a first parameter of the Mth coding unit based on the distortion of the Mth coding unit comprises: Comparing the distortions of the optional intra prediction modes in the Mth coding unit; The optional intra-frame prediction mode corresponding to the minimum distortion is determined as the first parameter of the Mth coding unit.

4. The method according to claim 1, characterized in that: The coding unit is a second type macroblock, which has inter-frame prediction capability. The first parameter of the M-1th coding unit is used to characterize the first target inter-frame prediction mode of the M-1th coding unit, and the first parameter of the Mth coding unit is used to characterize the first target inter-frame prediction mode of the Mth coding unit.

5. The method according to claim 4, characterized in that The determining a first parameter of the Mth coding unit based on the distortion of the Mth coding unit comprises: Comparing the distortions of the optional inter-frame prediction modes in the M-th coding unit; An optional inter-frame prediction mode corresponding to the minimum distortion is determined as a first parameter of the Mth coding unit.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Determine a second parameter of the M-th coding unit based on the first parameter of the M-1-th coding unit; wherein the second parameter includes an intra-frame prediction mode or an inter-frame prediction mode; Encoding the M-th coding unit based on a second parameter of the M-th coding unit; The determining the second parameter of the Mth coding unit based on the first parameter of the M-1th coding unit comprises: Determine a bit rate based on the first parameter of the M-1th coding unit; A second parameter of the Mth coding unit is determined based on the code rate and the distortion of the Mth coding unit.

7. The method according to claim 6, characterized in that The coding unit is a first type macroblock, the first type macroblock has intra-frame prediction capability, the second parameter of the Mth coding unit includes a second target intra-frame prediction mode of the Mth coding unit, and the second parameter of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

8. The method according to claim 6, characterized in that The coding unit is a second type macroblock, which has inter-frame prediction capability. The second parameter of the Mth coding unit includes a second target inter-frame prediction mode and motion information of the Mth coding unit. The second target inter-frame prediction mode of the Mth coding unit is determined by the rate-distortion cost of the Mth coding unit.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the video encoding method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the video encoding method according to any one of claims 1 to 8 is implemented.

Images