Encoding control method based on video encoding load

Abstract

This invention relates to an encoding control method based on video encoding load, characterized by: pre-setting a blacklist according to the processing performance of the device, and then including: setting Fg and BRg based on business requirements; checking whether the current device is in the blacklist; if so, encoding the video data using a first encoding method; if not, encoding the video data using a second encoding method, and outputting the current encoding duration t, the current frame size s, and the current frame quantization precision qp; calculating the normalized encoding duration T based on t, s, and qp; and calculating the encoding processing gap time GapT based on T and the target-specified maximum encoding time per frame Tg. i If GapT i If the value is greater than ThreshT, then the first encoding method is used to encode the video data; otherwise, the second encoding method is used. This invention can dynamically adjust different encoding methods based on business needs and the processing performance of specific devices, making it highly adaptable.

Description

Encoding control method based on video encoding load Technical Field

[0001] This invention relates to the field of video compression, and in particular to a coding control method based on video coding load. Background Technology

[0002] Modern video encoding and decoding technologies such as AV1 and HEVC offer high compression efficiency but also high complexity; while classic video coding technologies like H.264, although less complex, have lower compression efficiency. The compression efficiency difference between the two can be more than double, and the higher complexity comes at the cost of potentially increasing computational resources by 3 to 10 times.

[0003] In existing technologies, devices processing video data vary significantly in performance, with some boasting powerful processing capabilities while others are less capable. Given the varying compression efficiencies and computational resource requirements of different video codec technologies, there is a desire to utilize more complex modern video codec technologies (such as AV1, HEVC, VVC, or other modern technologies) on high-performance machines, while using classic encoding methods (such as H.264 or VP8) on less powerful machines. Simultaneously, there is a desire to prioritize the use of modern video codec technologies on any system platform, based on business requirements (encoding size, frame rate), dynamically switching to classic encoding methods when performance is insufficient. However, current technologies lack a single encoding control method that is applicable to diverse hardware processing devices and adaptable to different application scenarios based on business needs; therefore, further improvements are needed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a video coding load-based coding control method that can dynamically adjust different coding methods based on business needs and the processing performance of specific devices, in contrast to the above-mentioned prior art.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is as follows: a coding control method based on video coding load, characterized in that: a blacklist is pre-set according to the processing performance of the device, and then coding control is performed in the following manner:

[0006] Step 1: Set the target frame rate Fg and target bitrate BRg based on business requirements;

[0007] Step 2: Check if the current device is on the blacklist. If so, directly encode the video data using the first encoding method; otherwise, proceed to Step 3.

[0008] Step 3: Encode the video data using the second encoding method and output the current encoding duration t, current frame size s, and current frame quantization precision qp required for one frame of video encoding data; let the node of the current encoding output be i, the node of the previous frame output be i-1, the first node be 1, and i be a natural number greater than or equal to 2; where the encoding efficiency of the second encoding method is greater than that of the first encoding method.

[0009] The average quantization precision (QP) value of the encoded code is updated using a moving average method. i :

[0010] QP i =a×QP i-1 +b×qp;

[0011] QP1 = a × QP0 + b × qp;

[0012] The value of QP0 is a constant; a and b are preset constants;

[0013] Step 4: Based on the current coding duration t, current frame size s, and current frame quantization precision qp obtained in Step 3, calculate the normalized coding duration T using the following formula:

[0014] T = alpha × t; where alpha is the normalization coefficient, and the formula for calculating alpha can be any one of the following three formulas:

[0015] Formula 1:

[0016]

[0017] Formula 2:

[0018] alpha = (QP) i / qp) 2

[0019] Formula 3:

[0020]

[0021] And it satisfies alpha = min(max(alpha, 0.1), 3);

[0022] Step 5: Based on the normalized coding duration T obtained in Step 4 and the target maximum coding time Tg per frame, calculate the coding processing gap time GapT. i :

[0023] The target specifies a maximum coding time per frame, Tg = 1 / Fg;

[0024] GapT i=GapT i-1 +(T-Tg)

[0025] If GapT i If the value is less than 0, set GapT = 0;

[0026] Step 6: Calculate the encoding processing gap time (GapT) obtained in Step 5. i Compare with the preset time gap threshold ThreshT, if GapT i If the value is greater than ThreshT, then use the first encoding method to encode the video data; otherwise, return to step 3.

[0027] Preferably, when the second encoding method is AV1 encoding, the value of QP0 is 30; when the second encoding method is H.264 encoding, the value of QP0 is 23.

[0028] Preferably, the quantization precision qp of the current frame is in the range of 20 to 50. When the qp value is less than 20, qp is set to 20, and when the qp value is greater than 50, qp is set to 50.

[0029] As a preferred option, the preset time difference threshold ThreshT is set to a value between 800ms and 2000ms.

[0030] Preferably, the unit of the current encoding duration t is ms, the unit of the normalized encoding duration T is ms, the unit of the current frame size s is kbit, and the unit of the target maximum encoding time Tg per frame is ms.

[0031] As a preferred option, the target frame rate Fg is set to 30fps and the target bit rate BRg is set to 1200bbps.

[0032] Preferably, a = 0.995; b = 0.005.

[0033] As preferred options, the following encoding methods are arranged in descending order of encoding efficiency:

[0034] H.266 encoding method, AV1 encoding method, H.265 encoding method, VP9 encoding method, H.264 encoding method, VP8 encoding method, H.263 encoding method, MJPEG encoding method;

[0035] The first encoding method and the second encoding method can be any one of the above encoding methods, and it is only necessary to satisfy that the encoding efficiency of the second encoding method is greater than that of the first encoding method, and the difference between the encoding efficiency of the second encoding method and the encoding efficiency of the first encoding method exceeds a preset threshold, such as preferably 30%.

[0036] As a preferred option, if the device's CPU uses an ARM series chip, devices with processing performance less than ARM V8 will be blacklisted; if the device's CPU uses an x86 architecture chip, devices with processing performance less than G5 will be blacklisted.

[0037] As an improvement, in step 3, a separate processing thread is established. This processing thread starts a timer. If the timer exceeds the preset time difference threshold ThreshT and no frame of video encoded data is output, the first encoding method is used to encode the video data directly. Otherwise, the original processing method of step 3 is continued.

[0038] Compared with existing technologies, the advantages of this invention are as follows: by establishing a blacklist mechanism, the processing performance of obviously unsuitable devices is avoided, thus ensuring the user experience of blacklisted devices; by setting target bitrate and target frame rate to establish a benchmark for encoding processing performance indicators, it can adapt to different application scenarios; by estimating the current video complexity through the size of the encoded output data and the size of the quantization parameters, the processing time is normalized; based on the normalized processing time, it is compared with the processing performance benchmark, and the difference is accumulated to filter noise and make the encoding fallback decision more stable. Attached Figure Description

[0039] Figure 1 is a flowchart of the encoding control method based on video encoding load in an embodiment of the present invention. Detailed Implementation

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0041] This embodiment provides an encoding control method based on video encoding load, applicable to video conferencing. Each device participating in the video conferencing has different video data encoding and decoding capabilities. A blacklist is pre-defined based on the device's processing performance. The blacklist and whitelist can be simply built into the source code. For example, for mobile phone or tablet systems, ARM series chips are the mainstream processors. After obtaining the device's CPU model, it can be agreed that devices with processing performance greater than or equal to ARM v8 are whitelisted, and devices with processing performance less than ARM v8 are blacklisted, such as armv6, armv7, and armv7s. However, sometimes the supported versions cannot be determined solely from the CPU model name. It is advisable to focus on deducing and classifying the naming rules of CPUs from major chip manufacturers such as Apple, Qualcomm, Samsung, MTK, and Huawei. For x86 architecture CPUs, it is also necessary to study the naming rules of mainstream manufacturers (Intel, AMD, Huawei, and other domestic chip manufacturers) to set up blacklists and whitelists. For example, for Intel chips, devices with processing performance less than G5 are blacklisted, and so on. Encoding control is then performed in the following manner:

[0042] Step 1: Set the target frame rate Fg and target bitrate BRg based on business requirements;

[0043] Common settings include Fg = 30fps; BRg = 1200kbps;

[0044] Step 2: Check if the current device is on the blacklist. If so, directly encode the video data using the first encoding method; otherwise, proceed to Step 3.

[0045] Step 3: Encode the video data using the second encoding method, and output the current encoding duration t, current frame size s, and current frame quantization precision qp required for one frame of video encoding data; the unit of the current encoding duration t is ms, and the unit of the current frame size s is kbit;

[0046] Let the node currently encoded and output be i, the node output in the previous frame be i-1, and the first node be 1, where i is a natural number greater than or equal to 2; the coding efficiency of the second coding method is greater than that of the first coding method.

[0047] The current frame quantization precision qp ranges from 20 to 50. When the qp value is less than 20, qp is set to 20. When the qp value is greater than 50, qp is set to 50.

[0048] The average quantization precision (QP) value of the encoded code is updated using a moving average method. i :

[0049] QP i =a×QPi-1 +b×qp;

[0050] QP1 = a × QP0 + b × qp;

[0051] The value of QP0 is a constant; a and b are preset constants; the preferred value of a is 0.995; the preferred value of b is 0.005;

[0052] When the second encoding method is AV1 encoding, the value of QP0 is 30; when the second encoding method is H.264 encoding, the value of QP0 is 23.

[0053] Step 4: Based on the current coding duration t, current frame size s, and current frame quantization precision qp obtained in Step 3, calculate the normalized coding duration T using the following formula:

[0054] T = alpha × t; where alpha is the normalization coefficient, and the formula for calculating alpha can be any one of the following three formulas:

[0055] Formula 1:

[0056]

[0057] Formula 2:

[0058] alpha = (QP) i / qp) 2

[0059] Formula 3:

[0060]

[0061] And it satisfies alpha = min(max(alpha, 0.1), 3);

[0062] When the encoder of the device has relatively accurate bitrate control, such as the AV1 encoding method, Formula 1 can be used preferentially; when the encoder of the device has insufficient bitrate control and the bitrate control deviation is known to be large, Formula 2 can be used preferentially; if the encoder of the device is set to fixed QP (which is usually not done in RTC systems), then the higher the video complexity, the larger the size of the encoded content, and Formula 3 can be used preferentially in this case.

[0063] Step 5: Based on the normalized coding duration T obtained in Step 4 and the target maximum coding time Tg per frame, calculate the coding processing gap time GapT. i Normalized coding duration T, target maximum coding time per frame Tg, and coding processing gap time GapT i The units are all in milliseconds (ms);

[0064] The target specifies a maximum coding time per frame, Tg = 1 / Fg;

[0065] GapT i =GapT i-1 +(T-Tg)

[0066] If GapT i If the value is less than 0, set GapT = 0;

[0067] Step 6: Calculate the encoding processing gap time (GapT) obtained in Step 5. i Compare with the preset time gap threshold ThreshT, if GapT i If the value is greater than ThreshT, then the first encoding method is used to encode the video data; otherwise, return to step 3. The preset time difference threshold ThreshT ranges from 800ms to 2000ms, with a preferred value of 1000ms.

[0068] Considering that the above control method encodes each frame, if the encoding is extremely slow, a timer needs to be started in parallel to check. In step 3, a separate processing thread is established, and this processing thread starts a timer. When the timer time exceeds the preset time difference threshold ThreshT and it is found that no frame of video encoded data has been output, the first encoding method is used directly to encode the video data. Otherwise, the original processing method of step 3 is continued.

[0069] Arrange the different encoding methods in descending order of encoding efficiency:

[0070] H.266 encoding method, AV1 encoding method, H.265 encoding method, VP9 encoding method, H.264 encoding method, VP8 encoding method, H.263 encoding method, MJPEG encoding method;

[0071] The first encoding method and the second encoding method can be any one of the above encoding methods, and it is only necessary to satisfy that the encoding efficiency of the second encoding method is greater than that of the first encoding method, and the difference between the encoding efficiency of the second encoding method and the encoding efficiency of the first encoding method exceeds a preset threshold, such as preferably 30%. For example, the second encoding method adopts the AV1 encoding method, while the first encoding method adopts the H.264 encoding method.

Claims

1. A coding control method based on video coding load, characterized in that: A blacklist is pre-defined based on the device's processing performance, and then encoding control is performed as follows: Step 1: Set the target frame rate Fg and target bit rate BRg based on business requirements; Step 2: Check if the current device is in the blacklist. If so, directly use the first encoding method to encode the video data; otherwise, proceed to Step 3; Step 3: Use the second encoding method to encode the video data and output the current encoding duration t, current frame size s, and current frame quantization precision qp required for one frame of video encoding data; Let the node of the current encoding output be i, the node of the previous frame output be i-1, the first node be 1, and i be a natural number greater than or equal to 2; The second encoding method has a higher encoding efficiency than the first encoding method; the moving average method is used to update the average quantization precision value QP. i QP i =a×QP i-1 +b×qp; QP1 = a × QP0 + b × qp; The value of QP0 is a constant; a and b are preset constants; Step 4: Based on the current coding duration t, current frame size s, and current frame quantization precision qp obtained in Step 3, calculate the normalized coding duration T using the following formula: T = alpha × t; where alpha is the normalization coefficient, and the formula for calculating alpha is any one of the following three formulas: Formula 1: Formula 2: alpha = (QP) i / qp) 2 Formula 3: And satisfy alpha = min(max(alpha, 0.1), 3); Step 5: Based on the normalized coding duration T obtained in Step 4 and the target specified maximum coding time Tg per frame, calculate the coding processing gap time GapT. i The target specifies a maximum encoding time per frame, Tg = 1 / Fg; GapT i =GapT i-1 +(T-Tg) if GapT i If the value is less than 0, set GapT = 0; Step 6: Set the encoding processing gap time GapT calculated in Step 5 to 0. i Compare with the preset time gap threshold ThreshT, if GapT i If the value is greater than ThreshT, then use the first encoding method to encode the video data; otherwise, return to step 3.

2. The encoding control method based on video encoding load according to claim 1, characterized in that: When the second encoding method is AV1 encoding, the value of QP0 is 30; when the second encoding method is H.264 encoding, the value of QP0 is 23.

3. The encoding control method based on video encoding load according to claim 1, characterized in that: The current frame quantization precision qp ranges from 20 to 50. When the qp value is less than 20, qp is set to 20. When the qp value is greater than 50, qp is set to 50.

4. The encoding control method based on video encoding load according to claim 1, characterized in that: The preset time difference threshold ThreshT ranges from 800ms to 2000ms.

5. The encoding control method based on video encoding load according to claim 1, characterized in that: The unit of the current encoding duration t is ms, the unit of the normalized encoding duration T is ms, the unit of the current frame size s is kbit, and the unit of the target maximum encoding time Tg per frame is ms.

6. The encoding control method based on video encoding load according to claim 1, characterized in that: Set the target frame rate Fg to 30fps and the target bit rate BRg to 1200bbps.

7. The encoding control method based on video encoding load according to claim 1, characterized in that: a = 0.995; b = 0.

005.

8. The encoding control method based on video encoding load according to claim 1, characterized in that: The following encoding methods are arranged in descending order of encoding efficiency: H.266 encoding method, AV1 encoding method, H.265 encoding method, VP9 encoding method, H.264 encoding method, VP8 encoding method, H.263 encoding method, and MJPEG encoding method. The first encoding method and the second encoding method can be any one of the above encoding methods, and it is only necessary to satisfy that the encoding efficiency of the second encoding method is greater than that of the first encoding method, and the difference between the encoding efficiency of the second encoding method and the encoding efficiency of the first encoding method exceeds a preset threshold.

9. The encoding control method based on video encoding load according to claim 1, characterized in that: If a device's CPU uses an ARM series chip, devices with processing performance less than ARM V8 will be blacklisted; if a device's CPU uses an x86 architecture chip, devices with processing performance less than G5 will be blacklisted.

10. The encoding control method based on video encoding load according to claim 1, characterized in that: In step 3, a separate processing thread is established. This processing thread starts a timer. If the timer exceeds the preset time difference threshold ThreshT and no video encoded data is output, the first encoding method is used to encode the video data directly. Otherwise, the original processing method of step 3 is continued.

Images