Video coding control method, device and equipment

Abstract

The embodiments of this specification disclose a video encoding control method, apparatus, and device. The method includes: obtaining a video frame to be encoded, sending the video frame to a first encoder for encoding; determining whether the first encoder uses hardware encoding; if so, sending the video frame to be encoded to a second encoder for encoding; detecting the encoding performance of the second encoder for the video frame to be encoded, and obtaining corresponding encoding performance data; and determining, based on the encoding performance data, whether to use the second encoder for encoding, over the first encoder and the second encoder, preferentially.

Description

Technical Field

[0001] This specification relates to the field of image processing technology, and in particular to a video encoding control method, apparatus, and device. Background Art

[0002] Video encoding converts one video format into another, primarily for compression and data reduction. Common applications include online video, such as video calls and live webcasts. In these scenarios, video encoding with high compression ratios while maintaining high definition offers unique advantages, minimizing network bandwidth while delivering more video information and achieving high-definition video over narrowband.

[0003] In practical applications, there are a large number of optional video encoders. However, currently users often manually set an encoder based on their own experience or use the default encoder to encode the video. The obtained effect may not be very good, and it is difficult for users to detect and further identify it. Based on this, there is a need for a solution that can more effectively identify the encoder that is more suitable for the current video encoding task. Summary of the Invention

[0004] One or more embodiments of this specification provide a video encoding control method, apparatus, device, and storage medium to solve the following technical problem: a solution is needed that can more effectively identify an encoder that is more suitable for the current video encoding task.

[0005] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0006] One or more embodiments of this specification provide a video encoding control method, including:

[0007] Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

[0008] Determining whether the first encoder adopts a hardware encoding method;

[0009] If yes, the video frame to be encoded is also sent to the second encoder for encoding;

[0010] detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data;

[0011] According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0012] One or more embodiments of this specification provide a video encoding control device, including:

[0013] The video frame main encoding module obtains the video frame to be encoded and sends the video frame to be encoded to the first encoder for encoding;

[0014] An encoding mode determination module is configured to determine whether the first encoder adopts a hardware encoding mode;

[0015] The video frame bypass encoding module sends the to-be-encoded video frame to the second encoder for encoding if yes;

[0016] a bypass performance detection module, detecting the encoding performance of the second encoder on the video frame to be encoded, and obtaining corresponding encoding performance data;

[0017] The encoder switching selection module determines, based on the encoding performance data, whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0018] One or more embodiments of this specification provide a video encoding control device, including:

[0019] at least one processor; and,

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0022] Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

[0023] Determining whether the first encoder adopts a hardware encoding method;

[0024] If yes, the video frame to be encoded is also sent to the second encoder for encoding;

[0025] detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data;

[0026] According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0027] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0028] Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

[0029] Determining whether the first encoder adopts a hardware encoding method;

[0030] If yes, the video frame to be encoded is also sent to the second encoder for encoding;

[0031] detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data;

[0032] According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0033] At least one of the above-mentioned technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: based on the filtering of the hardware encoding mode, it can ensure that it is in a suitable detection state that does not affect the current main encoder in performing the video encoding task as much as possible. In this suitable detection state, the encoding performance of other encoders under the specified specifications is bypassed and detected, and the encoding performance data is accumulated, so that the encoder whose current performance meets the requirements can be dynamically identified in a timely manner, and then the encoder that may be better and more suitable for the next video encoding task can be enabled in a timely manner as the main encoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 A flowchart of a video encoding control method provided in one or more embodiments of this specification;

[0036] Figure 2 In an application scenario provided by one or more embodiments of this specification, Figure 1 A schematic flow chart of a specific implementation scheme of the method;

[0037] Figure 3 A schematic flow chart of a coding mode branch control solution provided in one or more embodiments of this specification;

[0038] Figure 4 A schematic diagram of a flow chart of a joint encoding scheme of a main encoder and a bypass encoder provided in one or more embodiments of this specification;

[0039] Figure 5 A schematic diagram of the structure of a video encoding control device provided in one or more embodiments of this specification;

[0040] Figure 6 A schematic diagram of the structure of a video encoding control device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0041] The embodiments of this specification provide a video encoding control method, apparatus, device, and storage medium.

[0042] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0043] As mentioned in the background, there are a large number of video encoders available for practical use. For example, H.264, H.265, AV1, and H.266 offer significantly improved compression rates compared to H.264. However, these encoders lack hardware encoding support. For example, most machines do not support hardware encoding for AV1 and require software encoding. Consequently, encoding times for the same specifications on different machines vary. In the field of real-time audio and video communication (RTC), excessively long encoding times can result in frames per second (FPS) not meeting design specifications, thus failing to meet product requirements. However, some machines offer superior performance, enabling these encoders to achieve good performance, leveraging their advantages to maintain high compression rates while also maintaining high definition.

[0044] In view of this, this application provides a targeted solution to identify whether the target machine can support the target encoder to achieve good performance on the machine, accumulate more performance data to maximize the utilization coverage of the encoder, and overall help to perform video encoding tasks on the machine more efficiently and with higher quality. The solution of this application is further described in detail below.

[0045] Figure 1This is a flowchart of a video encoding control method provided in one or more embodiments of this specification. This process can be executed on machines with video encoding requirements, such as smartphones, tablets, and surveillance cameras, and is particularly suitable for execution on high-performance machines. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0046] Figure 1 The process in includes the following steps:

[0047] S102: Obtain a video frame to be encoded, and send the video frame to be encoded to a first encoder for encoding.

[0048] There are currently multiple video frames that need to be encoded, and this demand is used as the video encoding task this time. These video frames are usually ordered continuous frames or sample frames in the same video. The video frames to be encoded in step S102 belong to these video frames, for example, the first frame of these video frames or multiple continuous frames at the front of the time sequence. Of course, after this step, there may be other video frames to be encoded.

[0049] In one or more embodiments of the present specification, for a video encoding task, the first encoder is the default encoder or the encoder currently in use. When the video frame to be encoded is sent to the first encoder for encoding, it means that the first encoder is currently responsible for executing the video encoding task. During the period when the first encoder maintains its priority, the encoding result of the first encoder will be used as the official result. During this period, the encoding performance of the first encoder actually determines the corresponding business effect and user experience.

[0050] S104: Determine whether the first encoder adopts hardware encoding.

[0051] In one or more embodiments of this specification, whether the first encoder adopts the hardware encoding method is actively determined. There are two main considerations:

[0052] First, while the first encoder maintains its priority, a bypass encoder is introduced at the same time, and the current video encoding task is also used to detect the encoding performance of the bypass encoder. In this case, on the machine where the first encoder is located, there may be two or more encoders including the first encoder that repeatedly perform the same video encoding task. When the first encoder adopts hardware encoding, the introduction of other encoders for bypass detection has little impact on the performance of each other. However, if the first encoder adopts software encoding, since other encoders introduced for bypass detection may also adopt software encoding, they may occupy CPU resources, which may not be conducive to task completion.

[0053] Secondly, for machines with better performance, especially those with powerful CPUs, it is difficult to fully utilize the CPU capabilities when the first encoder uses hardware encoding. In this case, other encoders introduced for bypass detection may use software encoding, which may show better performance results. Therefore, it is more worth trying. This is also the reason why the solution mentioned above is particularly suitable for execution on machines with better performance.

[0054] S106: If yes, the video frame to be encoded is also sent to the second encoder for encoding.

[0055] In one or more embodiments of this specification, as previously described, when a first encoder employs hardware encoding, a second encoder is introduced as a bypass encoder, allowing the second encoder to perform the same encoding tasks as the first encoder. The first encoder can be pre-configured as the primary encoder, and the second encoder can be pre-configured as a bypass encoder whose performance is to be tested, to clearly define the respective roles of the two encoders in performing their tasks.

[0056] Before encoding, it is necessary to set corresponding encoding specifications for the corresponding encoder. Encoding specifications include, for example, encoding resolution (for example, 480P, 720P, 1080P, 2K, 4K, etc.), encoding bit rate (for example, 1Mbps, 10M Mbps, 40MMbps, etc.). Since bypass detection is performed to discover encoders with higher encoding performance, the requirements and expectations for the second encoder can be increased, and higher encoding specifications can be set for the second encoder. If the encoding specifications set for the first encoder are used as a reference, higher encoding specifications can be set for the second encoder. For example, the same encoding resolution as the first encoder and a higher encoding bit rate can be set for the second encoder. For example, this can help to more efficiently discover other encoders with significantly better performance than the first encoder, and also help to find the upper limit of encoding specifications suitable for the current encoding task.

[0057] S108: Detect the encoding performance of the second encoder on the video frame to be encoded, and obtain corresponding encoding performance data.

[0058] In one or more embodiments of this specification, encoding performance data includes at least encoding duration and may also include information such as memory usage, CPU usage, and bit error rate of the corresponding process. This encoding performance data can directly impact the performance of subsequent applications corresponding to the video frames, such as decoding and the smoothness of the decoded video presentation. For example, even with sufficient network bandwidth, video call between users may still experience video freezes and frame drops, which can also affect other concurrently running applications.

[0059] S110: Determine, based on the encoding performance data, whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0060] In one or more embodiments of this specification, a desired encoding performance threshold is pre-set, and then the encoding performance data of the second encoder is compared with the encoding performance threshold. If the desired performance is achieved, consideration may be given to switching from the first encoder to the second encoder (for example, in the next frame or during the next encoding task switch), with the second encoder serving as the official executor of the encoding task. After the switch, the first encoder loses its priority. Alternatively, the relevant data may be recorded and a decision made later. This approach of setting a threshold based on the desired performance has the additional advantage of eliminating the need to collect the actual encoding performance data of the first encoder.

[0061] If you want to switch encoders dynamically and timely, you can also dynamically compare the real-time encoding performance of the first encoder and the second encoder, and use the encoder with better performance as the main encoder in a timely manner, so that you can switch between the first encoder and the second encoder once or multiple times.

[0062] In one or more embodiments of this specification, considering that in actual applications, different video frames have different amounts of information and the encoding environment at different encoding times may also vary, the encoding performance of the same encoder for encoding different video frames may also fluctuate. Therefore, it is necessary to accumulate a certain amount of encoding performance data of the second encoder before making a more accurate decision. Based on this, since the video frame to be encoded is only one frame or a few frames at the beginning of the time sequence, after detecting the encoding performance of the second encoder for the video frame to be encoded, other video frames to be encoded subsequent to the video frame to be encoded can also be used to continue detecting the encoding performance of the second encoder to obtain corresponding encoding performance data until the specified detection task is completed. For example, after accumulating a specified total number of video frames or completing the encoding of a specified video, the specified detection task is considered to be completed.

[0063] pass Figure 1 The method, based on filtering of the hardware encoding mode, can ensure that it is in a suitable detection state that minimizes the impact on the current main encoder's execution of video encoding tasks. In this suitable detection state, the encoding performance of other encoders under specified specifications is bypassed and detected, and encoding performance data is accumulated, so that the encoder whose current performance meets the requirements can be dynamically identified in a timely manner, and then the encoder that may be better and more suitable for the next video encoding task can be promptly enabled as the main encoder.

[0064] based on Figure 1 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0065] More intuitively, in an application scenario provided by one or more embodiments of this specification, Figure 1 A flow chart of a specific implementation scheme of the method is as follows: Figure 2 As shown. Figure 2 The process involves a video input module, a video encoding control module, a first encoder, a second encoder, an encoding switching module, and an encoding performance statistics platform.

[0066] Figure 2 The process includes the following steps:

[0067] By operating the video encoding control module, the encoding control is initialized. The first encoder is currently selected as the formal encoder, also as the main encoder, and the target detects the second encoder. The second encoder is used as a bypass encoder. The second encoder can especially be an encoder using software encoding.

[0068] The video input module triggers the video encoding control module to start executing control actions for a specific frame through the first video frame to be encoded V1.

[0069] The video encoding control module initializes the first encoder and sets corresponding encoding specifications, such as encoding resolution, encoding bit rate, etc.

[0070] The first encoder may use either hardware or software encoding. The video encoding control module determines the encoding method used by the first encoder. If the encoding method is hardware encoding, and the current encoding does not place much pressure on the CPU of the host machine, the second encoder may be enabled for bypass detection. Specifically, the second encoder is created and initialized, and the corresponding encoding specifications are set, for example, to be no less than those of the first encoder. The total number of expected detection encoding frames, referred to as the detection target number, may also be set.

[0071] After the first encoder is ready (exists and has been initialized), the video encoding control module sends the video frame V1 to be encoded to the first encoder for encoding. If the second encoder is also ready, the video encoding control module sends the video frame V1 to be encoded to the second encoder for encoding, and adds 1 to the detection encoding frame count, indicating that the video frame V1 to be encoded has been newly detected.

[0072] Other video frames to be encoded after the video frame to be encoded V1 can also be sent to the first encoder for encoding and the second encoder for encoding detection. For example, for the Nth video frame to be encoded Vn, the video frame to be encoded Vn is sent to the first encoder for encoding. If the second encoder is ready and the detection task has not yet ended, the video frame to be encoded Vn is also sent to the second encoder for encoding, and the detection encoding frame count is increased by 1. During the detection process, the encoding performance data of the second encoder is recorded. When the detection encoding frame count reaches the detection target number, the detection can be terminated, and the performance detection result is generated based on the continuously recorded encoding performance data.

[0073] Assuming that encoding performance is primarily measured from the perspective of encoding time, the average encoding time of the video frames corresponding to the second encoder can be determined based on the encoding time of the video frames encoded in the detection task. If the average encoding time of the video frames is less than a set threshold (representing the expected performance), it can be determined that the second encoder should be used for encoding next time. Then, for all detected video frames starting from the to-be-encoded video frame V1 until the end of the detection period, if the performance of the second encoder meets the expected performance, for example, if the average encoding time of the video frames is less than the set threshold, the encoding switching module can immediately switch the main encoder from the first encoder to the second encoder. In addition, the performance data can be reported to the encoding performance statistics platform for subsequent decision-making on whether to officially use the second encoder as the default encoder for the first time when executing encoding tasks in the future. If the second encoder is a new type of encoder, it may have performance advantages. In this case, based on the performance data accumulated by the platform, the second encoder's coverage of the entire platform's encoding tasks can be gradually expanded, which helps to more stably and reliably popularize the use of the second encoder in actual services (such as live game broadcasts, video conferencing, etc.).

[0074] In one or more embodiments of this specification, Figure 1 In the method, the main consideration is the case where the first encoder adopts the hardware encoding method. As mentioned above, the hardware encoding support of some encoders is insufficient. In practical applications, it is also common for the first encoder to adopt software encoding. In this case, consider dynamically dividing the current frame into intra-frame areas, allowing the second encoder to also try to participate in the formal encoding of small areas, and perform bypass performance detection based on small areas. Based on this, one or more embodiments of this specification also provide a flow chart of a coding mode branch control scheme, such as Figure 3 shown.

[0075] Figure 3 The process in includes the following steps:

[0076] S302: Obtain a video frame to be encoded, and send the video frame to be encoded to a first encoder for encoding.

[0077] S304: Determine whether the first encoder adopts hardware encoding.

[0078] S306: If yes, the video frame to be encoded is also sent to the second encoder for full-area detection encoding to obtain corresponding encoding performance data.

[0079] Full-area detection coding means that for any video frame to be encoded involved in the detection task, the second encoder will encode the entire area within the frame. However, the obtained encoding result is only used to detect the encoding performance and will not be used as the official encoding result of the frame.

[0080] S308: Otherwise, the video frame to be encoded is also sent to the second encoder for local area joint formal encoding to obtain corresponding encoding performance data.

[0081] If hardware encoding is not used, it is considered that software encoding is used, and the second encoder is likely to use the same software encoding method as the first encoder. In this case, the first encoder and the second encoder mainly rely on the computing resources of the CPU on the same machine for encoding, and there may even be a resource competition relationship between the two. In this case, consider letting the first encoder and the second encoder jointly perform formal encoding. For any video frame to be encoded, the two are responsible for encoding part of the area respectively, and the encoding results obtained by the two will be used as the formal encoding results of the frame.

[0082] Furthermore, based on the idea of local area joint formal coding, one or more embodiments of this specification also provide a flowchart of a joint coding scheme of a main encoder and a bypass encoder, such as Figure 4 shown.

[0083] Figure 4 The process in includes the following steps:

[0084] S402: Obtain a video frame to be encoded, and send the video frame to be encoded to a first encoder for encoding.

[0085] S404: Determine whether the first encoder adopts hardware encoding.

[0086] S406: If not, and it is determined that the first encoder uses software encoding, then execute: dividing the first area and the second area in the video frame to be encoded (other video frames to be encoded can also be divided in the same way in sequence); using the first encoder to encode the first area, and using the second encoder to encode the second area.

[0087] For the second encoder, since the performance of the initial joint encoding is not clear, the second encoder can first try to encode a smaller area. Initially, the second area can be smaller than or even much smaller than the first area. When dividing, for example, you can directly use horizontal or vertical lines to divide the frames. This method can conveniently divide the area ratio according to the length of each side of the video frame to be encoded. If image recognition and segmentation capabilities are supported, you can also consider dynamically identifying the foreground and background in the image, and encode them by the first encoder and the second encoder respectively. The advantage of the latter solution is that the background is relatively unimportant and can be encoded by the second encoder, which is more fault-tolerant.

[0088] The encoding results of the first encoder and the second encoder for the same frame are merged to obtain the overall encoding result of the frame. During subsequent decoding, it is also necessary to distinguish the first area and the second area, use different decoders for decoding, and then merge the decoding results.

[0089] S408: Detecting encoding performance of the second encoder on the second region, and obtaining corresponding encoding performance data;

[0090] S410: Determine whether to update the ratio of the first area to the second area based on the encoding performance data, so as to encode subsequent video frames according to the updated ratio.

[0091] If the corresponding encoding performance data of the second encoder continues to meet the set performance requirements, the proportion of the second area relative to the first area can be gradually expanded accordingly until it is completely switched to encoding by the second encoder alone.

[0092] Furthermore, during joint encoding, it is difficult to accurately set the area ratio so that the first encoder and the second encoder complete the encoding of the corresponding area of the same frame at the same time. The encoder that completes first may need to wait for the other encoder to complete the current frame. If the waiting time is long, the encoding time of the entire frame will be dragged down. In order to reduce the waiting time, an additional area is divided out in the video frame to be encoded, called the third area, as the advantage preemptive encoding area. In this case, the first area, the second area and the third area in the same frame together constitute the complete frame. For the same frame, either the first encoder or the second encoder may complete the encoding of the corresponding area of the frame first. The encoder that completes first is called the current frame advantage encoder. The current frame advantage encoder encodes the third area of the frame. The third area can be set to a smaller area. In this way, the moment when the lagging encoder completes the encoding of the corresponding area may be closer to the moment when the current frame advantage encoder completes the encoding of the third area, which helps to reduce the waiting time.

[0093] In one or more embodiments of this specification, the proportion of the second area is smaller than that of the first area, and the proportion of the third area is smaller than that of the second area. For example, the first area is set to a proportion of 5 / 10, the second area is set to a proportion of 4 / 10, and the third area is set to a proportion of 1 / 10.

[0094] In one or more embodiments of the present specification, since the encoder that completes first is uncertain, in order to avoid conflicts or repeated encoding of the third area as much as possible, a status indicator bit is set for the third area. Taking the case where the status indicator bit is a binary bit (for example, a value of 0 or 1), after the first encoder or the second encoder completes the encoding of the corresponding area, it queries whether the status indicator bit is in the first value (indicating that the third area of the current frame is idle and encoding has not yet begun); if so, the status indicator bit is set to the second value (indicating that the encoding of the third area will begin), and it is determined to be the dominant encoder of the current frame, encode the third area, and after the encoding of the current frame is completed, the status indicator bit is set to the first value (indicating that the third area of the next frame is idle and encoding has not yet begun); otherwise, if the status indicator bit is in the second value, it waits for encoding of the next frame.

[0095] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 5 、 Figure 6 The apparatus and device can accordingly execute the above method and related optional solutions.

[0096] Figure 5 A schematic structural diagram of a video encoding control device provided in one or more embodiments of this specification, the device comprising:

[0097] The video frame main encoding module 502 obtains the video frame to be encoded and sends the video frame to be encoded to the first encoder for encoding;

[0098] The encoding mode determination module 504 determines whether the first encoder adopts a hardware encoding mode;

[0099] The video frame bypass encoding module 506 sends the to-be-encoded video frame to the second encoder for encoding if yes;

[0100] A bypass performance detection module 508 detects the encoding performance of the second encoder on the video frame to be encoded and obtains corresponding encoding performance data;

[0101] The encoder switching selection module 510 determines, based on the encoding performance data, whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0102] Optionally, the video frame main channel encoding module 502 sets the first encoder as the main channel encoder, initializes the first encoder, and sets corresponding encoding specifications before sending the video frame to be encoded to the first encoder for encoding;

[0103] The video frame bypass encoding module 506 creates and initializes a second encoder as a bypass encoder whose performance is to be detected, and sets corresponding encoding specifications;

[0104] The video frame to be encoded is also sent to the prepared second encoder for encoding.

[0105] Optionally, the encoding specifications include encoding resolution and encoding bit rate;

[0106] The encoding specification correspondingly set for the bypass encoder is higher than the encoding specification correspondingly set for the main encoder.

[0107] Optionally, the encoder switching selection module 510 continues to detect the encoding performance of the second encoder using video frames subsequent to the video frame to be encoded, and obtains corresponding encoding performance data until a specified detection task is completed;

[0108] According to the encoding performance data corresponding to the video frame encoded in the detection task, it is determined whether the second encoder should be used for encoding next time.

[0109] Optionally, the encoder switching selection module 510 determines an average encoding duration of the video frames corresponding to the second encoder according to the encoding durations of the video frames encoded in the detection task;

[0110] If the average encoding time of the video frame is less than the set threshold, it is determined that the second encoder will be used for encoding next time.

[0111] Optionally, it also includes:

[0112] The single-frame joint encoding module 512, after determining whether the first encoder adopts the hardware encoding method, if not, and it is determined that the first encoder adopts the software encoding method, executes:

[0113] Dividing the to-be-encoded video frame into a first region and a second region;

[0114] The first region is encoded using the first encoder, and the second region is encoded using the second encoder.

[0115] Optionally, the bypass performance detection module 508 detects encoding performance of the second encoder on the second region after encoding the second region using the second encoder to obtain corresponding encoding performance data;

[0116] The encoder switching selection module 510 determines whether to update the ratio of the first area to the second area according to the encoding performance data, so as to encode subsequent video frames according to the updated ratio.

[0117] Optionally, the encoder switching selection module 510 gradually expands the proportion of the second area relative to the first area if the encoding performance data corresponding to the second encoder continues to meet the set performance requirements, until it is completely switched to encoding by the second encoder alone.

[0118] Optionally, the single-frame joint encoding module 512 divides a third region from the to-be-encoded video frame and sets a status indication bit for the third region;

[0119] After encoding the first region using the first encoder and encoding the second region using the second encoder, after the first encoder or the second encoder completes encoding the corresponding region, querying whether the status indicator bit is at a first value;

[0120] If so, the state indicator is set to the second value, and the state indicator is determined to be the dominant encoder of the current frame, and the third region is encoded. After the encoding of the current frame is completed, the state indicator is set to the first value;

[0121] Otherwise, if the status indicator is at the second value, waiting to encode the next frame.

[0122] Optionally, the proportion of the second area is smaller than that of the first area, and the proportion of the third area is smaller than that of the second area.

[0123] Figure 6 A schematic diagram of the structure of a video encoding control device provided in one or more embodiments of this specification, the device comprising:

[0124] at least one processor; and,

[0125] a memory communicatively connected to the at least one processor; wherein,

[0126] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0127] Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

[0128] Determining whether the first encoder adopts a hardware encoding method;

[0129] If yes, the video frame to be encoded is also sent to the second encoder for encoding;

[0130] detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data;

[0131] According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0132] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0133] Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding;

[0134] Determining whether the first encoder adopts a hardware encoding method;

[0135] If yes, the video frame to be encoded is also sent to the second encoder for encoding;

[0136] detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data;

[0137] According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

[0138] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0139] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0140] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0141] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0142] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0145] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0146] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0147] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A video encoding control method, comprising: Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding; Determining whether the first encoder adopts a hardware encoding method; If yes, the video frame to be encoded is also sent to the second encoder for encoding; detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data; According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

2. The method according to claim 1, before sending the to-be-encoded video frame to the first encoder for encoding, the method further comprises: Setting the first encoder as the main encoder, initializing the first encoder, and setting corresponding encoding specifications; The step of sending the to-be-encoded video frame to the second encoder for encoding specifically includes: Create and initialize a second encoder as a bypass encoder for the performance to be detected, and set the corresponding encoding specifications; The video frame to be encoded is also sent to the prepared second encoder for encoding.

3. The method according to claim 2, wherein the encoding specifications include encoding resolution and encoding bit rate; The encoding specification correspondingly set for the bypass encoder is higher than the encoding specification correspondingly set for the main encoder.

4. The method according to claim 1, wherein determining, based on the encoding performance data, whether to preferentially use the second encoder for encoding between the first encoder and the second encoder comprises: Using video frames subsequent to the video frame to be encoded, continuing to detect the encoding performance of the second encoder to obtain corresponding encoding performance data until the specified detection task is completed; According to the encoding performance data corresponding to the video frame encoded in the detection task, it is determined whether the second encoder should be used for encoding next time.

5. The method according to claim 4, wherein determining whether to preferentially use the second encoder for encoding next time based on encoding performance data corresponding to the video frame encoded in the detection task specifically comprises: Determining an average encoding duration of video frames corresponding to the second encoder according to corresponding encoding durations of video frames encoded in the detection task; If the average encoding time of the video frame is less than the set threshold, it is determined that the second encoder will be used for encoding next time.

6. The method according to claim 1, after determining whether the first encoder adopts a hardware encoding method, the method further comprises: If not, and it is determined that the first encoder adopts software encoding, then execute: Dividing the to-be-encoded video frame into a first region and a second region; The first region is encoded using the first encoder, and the second region is encoded using the second encoder.

7. The method according to claim 6, further comprising: after encoding the second region using the second encoder; detecting encoding performance of the second encoder on the second region to obtain corresponding encoding performance data; According to the encoding performance data, it is determined whether to update the ratio of the first area to the second area, so as to encode subsequent video frames according to the updated ratio.

8. The method according to claim 7, wherein determining whether to update the ratio of the first area to the second area based on the encoding performance data specifically comprises: If the encoding performance data corresponding to the second encoder continues to meet the set performance requirements, the proportion of the second area relative to the first area is gradually expanded accordingly until it is completely switched to encoding by the second encoder alone.

9. The method according to claim 6, wherein dividing the first area and the second area in the video frame to be encoded further comprises: Dividing a third area in the to-be-encoded video frame, and setting a status indication bit for the third area; After encoding the first region using the first encoder and encoding the second region using the second encoder, the method further includes: After the first encoder or the second encoder completes encoding of the corresponding area, querying whether the status indicator bit is at a first value; If so, the state indicator is set to the second value, and the state indicator is determined to be the dominant encoder of the current frame, and the third region is encoded. After the encoding of the current frame is completed, the state indicator is set to the first value; Otherwise, if the status indicator is at the second value, waiting to encode the next frame. 10 . The method according to claim 9 , wherein the proportion of the second area is smaller than that of the first area, and the proportion of the third area is smaller than that of the second area.

11. A video encoding control device, comprising: The video frame main encoding module obtains the video frame to be encoded and sends the video frame to be encoded to the first encoder for encoding; An encoding mode determination module is configured to determine whether the first encoder adopts a hardware encoding mode; The video frame bypass encoding module sends the to-be-encoded video frame to the second encoder for encoding if yes; a bypass performance detection module, detecting the encoding performance of the second encoder on the video frame to be encoded, and obtaining corresponding encoding performance data; The encoder switching selection module determines, based on the encoding performance data, whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

12. The apparatus according to claim 11, wherein the video frame main path encoding module sets the first encoder as the main path encoder, initializes the first encoder, and sets corresponding encoding specifications before sending the video frame to be encoded to the first encoder for encoding; The video frame bypass encoding module creates and initializes a second encoder as a bypass encoder whose performance is to be detected, and sets corresponding encoding specifications; The video frame to be encoded is also sent to the prepared second encoder for encoding.

13. The apparatus according to claim 12, wherein the encoding specifications include encoding resolution and encoding bit rate; The encoding specification correspondingly set for the bypass encoder is higher than the encoding specification correspondingly set for the main encoder.

14. The apparatus of claim 11, wherein the encoder switching selection module continues to detect the encoding performance of the second encoder using video frames subsequent to the video frame to be encoded, and obtains corresponding encoding performance data until a specified detection task is completed; According to the encoding performance data corresponding to the video frame encoded in the detection task, it is determined whether the second encoder should be used for encoding next time.

15. The apparatus of claim 14, wherein the encoder switching selection module determines an average encoding duration of video frames corresponding to the second encoder based on encoding durations of video frames encoded in the detection task; If the average encoding time of the video frame is less than the set threshold, it is determined that the second encoder will be used for encoding next time.

16. The apparatus of claim 11, further comprising: The single-frame joint encoding module, after determining whether the first encoder adopts the hardware encoding method, if not, and determining that the first encoder adopts the software encoding method, executes: Dividing the to-be-encoded video frame into a first region and a second region; The first region is encoded using the first encoder, and the second region is encoded using the second encoder.

17. The apparatus according to claim 16, wherein the bypass performance detection module detects encoding performance of the second encoder on the second region after encoding the second region using the second encoder to obtain corresponding encoding performance data; The encoder switching selection module determines whether to update the ratio of the first area to the second area based on the encoding performance data, so as to encode subsequent video frames according to the updated ratio.

18. The device as claimed in claim 17, wherein the encoder switching selection module gradually expands the proportion of the second area relative to the first area if the encoding performance data corresponding to the second encoder continues to meet the set performance requirements, until it is completely switched to encoding by the second encoder alone.

19. The apparatus according to claim 16, wherein the single-frame joint encoding module divides a third region in the to-be-encoded video frame and sets a status indicator bit for the third region; After encoding the first region using the first encoder and encoding the second region using the second encoder, after the first encoder or the second encoder completes encoding the corresponding region, querying whether the status indicator bit is at a first value; If so, the state indicator is set to the second value, and the state indicator is determined to be the dominant encoder of the current frame, and the third region is encoded. After the encoding of the current frame is completed, the state indicator is set to the first value; Otherwise, if the status indicator is at the second value, waiting to encode the next frame. 20 . The device of claim 19 , wherein the proportion of the second area is smaller than that of the first area, and the proportion of the third area is smaller than that of the second area.

21. A video encoding control device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform: Obtaining a video frame to be encoded, and sending the video frame to be encoded to a first encoder for encoding; Determining whether the first encoder adopts a hardware encoding method; If yes, the video frame to be encoded is also sent to the second encoder for encoding; detecting encoding performance of the second encoder on the to-be-encoded video frame to obtain corresponding encoding performance data; According to the encoding performance data, it is determined whether to use the second encoder for encoding in priority between the first encoder and the second encoder.

Images