Method and device for adjusting video coding thread load, equipment and storage medium

Abstract

The invention provides a method and device for adjusting a video coding thread load, equipment and a storage medium. The method comprises the following steps: acquiring a tth real-time parameter of each video coding channel; if the t-th real-time parameter of at least one video coding channel is not matched with the (t-1)-th real-time parameter, acquiring the t-th channel load of each video coding channel; for a video coding thread, taking the sum of the tth channel loads of the k video coding channels as the tth thread load of the video coding thread; if the tth thread load of at least one video coding thread is not matched with the (t-1) th thread load, regrouping the plurality of video coding channels based on the tth total channel load to obtain a plurality of channel groups; and each channel group is matched with one video coding thread. According to the technical scheme, the thread load of the video coding thread can be dynamically adjusted, and then the problem that the thread load of the video coding thread is unbalanced is solved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, equipment and storage medium for adjusting a video encoding thread load. Background Art

[0002] Currently, video compression protocols are mainly used for video encoding, and a dedicated SoC (System on Chip) is designed for video compression acceleration processing. Before starting the SoC hardware processing, each channel video needs to undergo business logic control and data preparation before hardware compression processing, such as frame rate control, GOP (Group of Pictures) level bit rate control, frame level bit rate control, motion intensity detection, etc. In security monitoring systems, multi-channel video encoding is a key link to ensure smooth transmission and storage of monitoring images.

[0003] In the related art, multi-channel video encoding processes the pre-processing work of each video encoding channel by creating a video encoding thread. However, since the mapping relationship between the video encoding thread and the video encoding channel is static, and the real-time parameters of different video encoding channels will change, the thread load of the video encoding thread will also change accordingly, which in turn leads to an unbalanced thread load of the video encoding thread. Summary of the invention

[0004] The embodiment of the present application provides a method, apparatus, device and storage medium for adjusting the thread load of video encoding, which can dynamically adjust the thread load of video encoding thread, thereby solving the problem of unbalanced thread load of video encoding thread. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present application, a method for adjusting a video encoding thread load is provided, the method comprising:

[0006] Obtaining a t-th real-time parameter of each video encoding channel; the real-time parameter indicates a video stream feature;

[0007] If the t-th real-time parameter of at least one of the video encoding channels does not match the t-1-th real-time parameter, obtaining the t-th channel load of each of the video encoding channels; the t-th channel load is obtained based on the t-th real-time parameter;

[0008] For one of the video encoding threads, taking the sum of the t-th channel loads of the k video encoding channels as the t-th thread load of the video encoding thread;

[0009] If the t-th thread load of at least one of the video encoding threads does not match the t-1-th thread load, the multiple video encoding channels are regrouped based on the t-th total channel load to obtain multiple channel groups; each channel group matches one video encoding thread.

[0010] In a possible implementation, the real-time parameters include the height of the image, the width of the image, and an output frame rate.

[0011] In a possible implementation manner, obtaining the t-th channel load of each of the video encoding channels includes:

[0012] For one of the video encoding channels, the product of the width of the image, the height of the image and the output frame rate in the t-th real-time parameter is used as the t-th channel load.

[0013] In a possible implementation manner, the video encoding thread matches k of the video encoding channels.

[0014] In a possible implementation, if the t-th thread load of at least one of the video encoding threads does not match the t-1-th thread load, the step includes:

[0015] For one of the video encoding threads, if the difference between the load of the tth thread and the load of the t-1th thread is greater than a preset threshold, the load of the tth thread does not match the load of the t-1th thread.

[0016] In a possible implementation manner, the t-th total channel load is the sum of the t-th channel loads of all the video encoding channels.

[0017] In a possible implementation, the regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups includes:

[0018] For the first video encoding thread, based on the t-th total channel load and x, obtain the t-th thread load average; x is the total number of the video encoding threads, and x is an integer greater than or equal to 2;

[0019] Arrange the t-th channel load of each video encoding channel in reverse order to obtain a channel load queue;

[0020] If the load of the t-th channel at the top of the channel load queue is greater than or equal to the average load of the t-th thread, the video encoding channel corresponding to the t-th channel load at the top is used as a channel group; the channel group is matched with the first video encoding thread; or

[0021] If the first channel load in the channel load queue is less than the average load of the t-th thread, traverse the remaining t-th channel loads to obtain a channel group; the channel group includes the video encoding channel corresponding to the n-th t-th channel load and the video encoding channel corresponding to the t-th channel load at the first position; the n-th t-th channel load satisfies the first preset condition; n is an integer greater than or equal to 2 and less than or equal to x.

[0022] In a possible implementation, traversing the remaining t-th channel loads to obtain a channel group includes:

[0023] Obtain a first difference between the first t-th channel load and the t-th thread load average;

[0024] Obtaining a second difference between each of the remaining t-th channel loads and the first difference;

[0025] Obtaining the nth t-th channel load; the second difference corresponding to the nth t-th channel load satisfies the first preset condition;

[0026] The video encoding channel corresponding to the t-th channel load at the nth position and the video encoding channel corresponding to the t-th channel load at the first position are taken as one of the channel groups.

[0027] In a possible implementation manner, the regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups further includes:

[0028] For the q+1th video encoding thread, after determining the qth channel group, the tth remaining total channel load is obtained; the qth channel load sum is the sum of the channel loads of the qth channel group; the tth remaining total channel load is the difference between the tth total channel load and the qth channel load sum; q is an integer greater than or equal to 1 and less than or equal to x;

[0029] Based on the t-th remaining total channel load and the remaining number, the t-th update thread load mean is obtained; the remaining number is the difference between x and q;

[0030] Arrange the t-th channel load of each remaining video encoding channel in reverse order to obtain an updated channel load queue;

[0031] If the load of the t-th channel at the top of the update channel load queue is greater than or equal to the average load of the t-th update thread, obtain a channel group; the channel group is the video encoding channel corresponding to the t-th channel load at the top of the update channel load queue; or

[0032] If the t-th channel load at the first position in the update channel load queue is less than the average load of the t-th update thread, traverse the remaining t-th channel loads in the update channel load queue to obtain a channel group; the channel group includes the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue and the video encoding channel corresponding to the p-th t-th channel load; the difference between the p-th t-th channel load and the third difference satisfies the second preset condition; the third difference is the difference between the t-th channel load at the first position and the average load of the t-th update thread; p is an integer greater than or equal to 2.

[0033] In a possible implementation manner, the regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups further includes:

[0034] After (x-1) channel groups are determined, the remaining video encoding channels are used as the xth channel group; the xth channel group is matched with the xth video thread encoding.

[0035] According to a second aspect of an embodiment of the present application, a device for adjusting a video encoding thread load is provided, comprising:

[0036] A first acquisition module is used to acquire the tth real-time parameter of each video encoding channel; the real-time parameter indicates a video stream feature;

[0037] A second acquisition module is used to acquire a t-th channel load of each video encoding channel if the t-th real-time parameter of at least one video encoding channel does not match the t-1-th real-time parameter; the t-th channel load is obtained based on the t-th real-time parameter;

[0038] A determination module, configured to use, for one of the video encoding threads, a sum of the t-th channel loads of the k video encoding channels as a t-th thread load of the video encoding thread;

[0039] An adjustment module is used for regrouping the plurality of video encoding channels based on the tth total channel load to obtain a plurality of channel groups if the tth thread load of at least one of the video encoding threads does not match the t-1th thread load; each channel group matches one of the video encoding threads.

[0040] According to a third aspect of an embodiment of the present application, a computer device is provided, characterized in that the computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executes a method for adjusting the video encoding thread load.

[0041] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that at least one program is stored in the computer-readable storage medium, and the at least one program is loaded by a processor and executes a method for adjusting the video encoding thread load.

[0042] In an embodiment of the present application, an embodiment of the present application provides a method for adjusting the video encoding thread load, and the real-time parameters of each video encoding channel are detected in real time. When the real-time parameters of at least one video encoding channel change, the channel load of each video encoding channel is recalculated, and the thread load of each video encoding thread is recalculated based on each video encoding channel load. If at least one thread load does not match the thread load calculated last time, the video encoding channel that matches each video encoding thread is reallocated. Thereby, the thread load of each video encoding thread is adjusted, and then the balance between the thread loads of each video encoding thread is re-achieved. Through the above technical solution, the embodiment of the present application avoids the problem of a long delay in a certain video encoding channel, thereby improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0045] Figure 2 It is a first mapping relationship diagram between a video encoding thread and a video encoding channel provided by the related technology;

[0046] Figure 3 It is a second mapping relationship diagram between a video encoding thread and a video encoding channel provided by the related technology;

[0047] Figure 4 It is a flowchart of a method for adjusting a video encoding thread load provided in an embodiment of the present application;

[0048] Figure 5is a first flow chart of step 404 provided according to an embodiment of the present application;

[0049] Figure 6 is a second flow chart of step 404 provided according to an embodiment of the present application;

[0050] Figure 7 It is a structural schematic diagram of a device for adjusting a video encoding thread load provided in accordance with an embodiment of the present application;

[0051] Figure 8 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0052] Fig. 9 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0054] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0055] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0056] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0057] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0058] Figure 1It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0059] In the terminal 101, a device for adjusting the video encoding thread load and a SoC are provided.

[0060] The terminal 101 may be a smartphone having a device for adjusting the video encoding thread load and a SoC, a wearable device, a personal computer, a laptop, a tablet computer, a smart TV, a car terminal, etc.

[0061] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0062] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0063] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.

[0064] Figure 2 It is a first mapping relationship diagram between a video encoding thread and a video encoding channel provided by the related technology.

[0065] Combine the following Figure 2 The first mapping relationship diagram is exemplified.

[0066] In the first mapping relationship diagram, the video encoding thread corresponds to the video encoding channel one by one. That is, a video encoding thread needs to be created for each video encoding channel to facilitate the pre-processing of each video encoding channel. For example: the first video encoding thread corresponds to the first video encoding channel. The second video encoding thread corresponds to the second video encoding channel, etc. However, due to the limited embedded CPU resources and memory resources. Therefore, creating a video encoding thread for each video encoding channel will increase the memory overhead and increase the CPU load. In addition, frequent context switching of video encoding threads will reduce the performance of the system. For example: the hardware encoding of a SoC (System on Chip) chip supports a maximum of 20 1080P@24fps (frames per second) h.264 / h.265 (a coding standard) video encoding channels, and 30 1080p@30fps Jpeg / Mjpeg (a coding standard) video encoding channels, so the total number of video encoding threads reaches 50. If the resolution and frame rate are lower, the number of channels supported is greater, and the number of video encoding threads required also increases synchronously.

[0067] Figure 3 It is a second mapping relationship diagram between a video encoding thread and a video encoding channel provided by the related technology.

[0068] Combine the following Figure 3 The second mapping relationship diagram is exemplified.

[0069] In the second mapping relationship diagram, multiple video encoding channels share one video encoding thread. That is, one video encoding thread handles the pre-processing work of multiple video encoding channels. For example: the first video encoding thread corresponds to the first video encoding channel and the second video encoding channel. The second video encoding thread corresponds to the fourth video encoding channel, etc. Compared with the first mapping relationship, the second mapping relationship reduces the number of video encoding threads, and only twenty video encoding threads are required to handle the pre-processing work of fifty video encoding channels. However, since one video encoding thread handles the pre-processing work of multiple video encoding channels, and the mapping relationship between video encoding channels and video encoding threads is static. In the field of intelligent security multi-channel, video encoding channels will be dynamically destroyed and created, and video encoding channels will also cause channel load changes due to changes in parameters such as image resolution, output frame rate, and image compression rate. It may also cause some video encoding threads to idle, which in turn leads to unbalanced load of video encoding threads. The processing delay of different video encoding channels increases, and the performance of the system is significantly reduced.

[0070] In order to solve the technical problems existing in the related art, an embodiment of the present application provides a method for adjusting the thread load of a video encoding thread, which is used to dynamically adjust the thread load of a video encoding thread.

[0071] Figure 4 is a flow chart of a method for adjusting a video encoding thread load provided in accordance with an embodiment of the present application, such as Figure 4 As shown, in the embodiment of the present application, the method is described by taking the application on a device having a video encoding thread load adjustment and a SoC terminal as an example. The method comprises the following steps:

[0072] In step 401, the terminal obtains the tth real-time parameter of each video encoding channel; the real-time parameter indicates the characteristics of the video stream.

[0073] Wherein, t is an integer greater than or equal to 2. t can be understood as the t-th detection of the real-time parameters of the video encoding channel by the terminal.

[0074] In some embodiments, since the real-time parameters are dynamically changing, and the change of the real-time parameters will affect the channel load of the video encoding channel. Therefore, the terminal needs the real-time parameters to detect in order to determine the channel load of the video encoding channel in real time. Optionally, the terminal obtains the t-th real-time parameter of each video encoding channel based on user operation. For example: the user modifies the real-time parameters through the SDK (Software Development Kit) interface. Specifically, the user can modify the real-time parameters of at least one video encoding channel.

[0075] In one example, when the real-time parameters of any video encoding channel change, it is considered that the properties of the video encoding channel have changed, and the channel load of the video encoding channel will also change immediately. Therefore, it is also necessary to determine whether the thread load of the video encoding thread remains balanced after the change based on the change in the channel load of the video encoding channel. If the thread load of the video encoding thread remains balanced, there is no need to readjust the thread load of the video encoding thread. If the thread load of the video encoding thread is no longer balanced, it is necessary to readjust the thread load of each encoding thread.

[0076] In some embodiments, some parameters are selected as real-time parameters because these real-time parameters have a greater impact on the channel load of the video encoding channel, while the remaining parameters have a smaller impact on the channel load of the video encoding channel. Optionally, the real-time parameters include the height of the image, the width of the image, and the output frame rate. The output frame rate is the number of frames of the image output by the terminal per second.

[0077] In step 402, if the t-th real-time parameter of at least one video encoding channel does not match the t-1-th real-time parameter, the t-th channel load of each video encoding channel is obtained; the t-th channel load is obtained based on the t-th real-time parameter.

[0078] In some embodiments, obtaining the t-th channel load of each video encoding channel includes: for a video encoding channel, taking the product of the image width, the image height, and the output frame rate in the t-th real-time parameter as the t-th channel load. That is, as long as at least one of the image width, the image height, and the output frame rate changes, the channel load will change. That is, the t-th channel load will be inconsistent or mismatched with the t-1-th channel load.

[0079] For example, the load of the tth channel can be calculated by the following formula:

[0080] The tth channel load = image width × image height × output frame rate.

[0081] In step 403, the terminal uses the sum of the t-th channel loads of k video encoding channels as the t-th thread load of the video encoding thread for one video encoding thread.

[0082] In some embodiments, a video encoding thread is matched with k video encoding channels; k is an integer greater than or equal to 1.

[0083] In some embodiments, the terminal presets an initial mapping table, which is used to store the mapping relationship between the video encoding thread and the video encoding channel. Each video encoding thread matches k video encoding channels. That is, each video encoding thread processes the pre-processing work of the matched k video encoding channels. Then the sum of the t-th channel load of these k video encoding channels is the sum of the t-th thread load of the video encoding thread.

[0084] In step 404, if the t-th thread load of at least one video encoding thread does not match the t-1-th thread load, the plurality of video encoding channels are regrouped based on the t-th total channel load to obtain a plurality of channel groups.

[0085] In some embodiments, each channel group is matched with one video encoding thread; the tth total channel load is the sum of the tth channel loads of all video encoding channels.

[0086] In some embodiments, if the t-th thread load of at least one video encoding thread does not match the t-1-th thread load, it means that the t-th thread load of at least one video encoding thread is less than or greater than the t-1-th thread load. That is, the thread loads of the video encoding threads are no longer balanced, which will cause the performance of the system to degrade. Therefore, it is necessary to re-determine the thread load of each video encoding thread. Optionally, for a video encoding thread, if the difference between the t-th thread load and the t-1-th thread load is greater than a preset threshold, the t-th thread load does not match the t-1-th thread load.

[0087] In one example, when determining whether the thread load of the video encoding thread is balanced, the t-th thread load of each video encoding thread is compared with the t-1-th thread load to obtain the difference between the t-th thread load and the t-1-th thread load. Optionally, if the absolute value of the difference is greater than a preset threshold, it indicates that the t-th thread load has a large change relative to the t-1-th thread load, that is, the t-th thread load does not match the t-1-th thread load. For example: the preset threshold is 5% of the t-1-th thread load.

[0088] From the above analysis, it can be seen that the embodiment of the present application reduces the number of video encoding threads compared to the technical solution of creating one video encoding thread for each video encoding channel, thereby reducing CPU memory and relatively reducing the frequency of switching thread contexts.

[0089] Figure 5 It is a first flowchart diagram of step 404 provided according to an embodiment of the present application.

[0090] Combine the following Figure 5 Step 404 is exemplarily described.

[0091] In some embodiments, the above step 404 includes the following steps 4041 to 4043:

[0092] In step 4041, the terminal obtains the tth thread load average for the first video encoding thread based on the tth total channel load and x, where x is the total number of video encoding threads and x is an integer greater than or equal to 2.

[0093] In some embodiments, the total number of video encoding threads, ie, x, remains unchanged.

[0094] For example, the average load of the tth thread can be calculated by the following formula:

[0095] The average load of the tth thread = the tth total channel load / x.

[0096] In step 4042, the terminal arranges the t-th channel load of each video encoding channel in reverse order to obtain a channel load queue.

[0097] In some embodiments, in order to find the largest t-th channel load corresponding to the video encoding channel, each t-th channel load is arranged in reverse order according to the size of the t-th channel load, thereby obtaining a channel load queue. Then, the t-th channel load at the first position in the channel load queue is the largest t-th channel load. Being at the first position can also be expressed as the first position. Optionally, each video encoding thread is numbered, and each video encoding channel is also numbered. For example: the video encoding channels are numbered "001", "002", and "003", etc. The video encoding threads are numbered "01", "02", and "03", etc. For another example: the channel load queues are: the t-th channel load of "007", the t-th channel load of "003", and the t-th channel load of "001", etc.

[0098] In step 4043, if the load of the first t-th channel in the channel load queue is greater than or equal to the average load of the t-th thread, the video encoding channel corresponding to the first t-th channel load is taken as a channel group, wherein the channel group matches the first video encoding thread.

[0099] In some embodiments, the mapping relationship between the video encoding thread and the video encoding channel in the initial mapping table is changed to obtain an updated mapping table. Optionally, in the updated mapping table, there is a mapping relationship between the first video encoding thread and the video encoding channel corresponding to the t-th channel load in the first position. That is, the first video encoding thread matches the video encoding channel corresponding to the t-th channel load in the first position. For example: in the updated mapping table, "01" and "007" are stored as a row. That is, there is a mapping relationship between "01" and "007".

[0100] The above step 4043 is an implementation of obtaining a channel group. In another implementation, if the t-th channel load at the top of the channel load queue is less than the average of the t-th thread load, the remaining t-th channel loads are traversed to obtain a channel group; the channel group includes the video encoding channel corresponding to the n-th channel load and the video encoding channel corresponding to the t-th channel load at the top; the n-th channel load satisfies the first preset condition; n is an integer greater than or equal to 2 and less than or equal to x.

[0101] In some embodiments, not every t-th channel load corresponding to each video encoding channel is greater than or equal to the t-th thread load average. It is possible that each t-th channel load is less than the t-th thread load average. That is, the t-th channel load at the first position is less than the t-th thread load average. For example, the t-th channel load of "007" is less than the t-th thread load average. Then the video encoding channel numbered "007" cannot be used as a channel group alone. It is necessary to traverse the t-th channel loads corresponding to the remaining video encoding channels, and then find one or more video encoding channels to be used as a channel group together with the t-th channel load at the first position. Therefore, the sum of the t-th channel loads corresponding to each channel group is as close as possible to the t-th thread load average. The adjusted thread load of each video encoding thread is also as close as possible, thereby achieving thread load balancing of the video encoding thread.

[0102] In some embodiments, traversing the remaining t-th channel loads to obtain a channel group can be implemented as follows:

[0103] Obtain a first difference between the first t-th channel load and the t-th thread load average. Obtain a second difference between each remaining t-th channel load and the first difference. Obtain the n-th t-th channel load; the second difference corresponding to the n-th t-th channel load satisfies the first preset condition. The video encoding channel corresponding to the n-th t-th channel load and the video encoding channel corresponding to the first t-th channel load are taken as a channel group.

[0104] In one example, the first preset condition can be understood as determining the minimum second difference. The video encoding channel corresponding to the minimum second difference and the video encoding channel corresponding to the t-th channel load in the first place are taken as a channel group. For example: the t-th channel load corresponding to the minimum second difference is ranked nth in the channel load queue. Then the video encoding channel corresponding to the n-th t-th channel load and the video encoding channel corresponding to the t-th channel load in the first place are taken as a channel group. It should be noted that the minimum second difference means that the absolute value of the second difference is the smallest. That is, the second difference is closest in size to the first difference.

[0105] Figure 6 This is a second flow chart of step 404 provided according to an embodiment of the present application.

[0106] Combine the following Figure 6 Step 404 is exemplarily described.

[0107] In some embodiments, the above step 404 further includes the following steps 4044 to 4046:

[0108] In step 4044, the terminal obtains the tth remaining total channel load after determining the qth channel group for the q+1th video encoding thread.

[0109] In some embodiments, the qth channel load sum is the sum of the channel loads of the qth channel group; the tth remaining total channel load is the difference between the tth total channel load and the qth channel load sum; q is an integer greater than or equal to 1 and less than or equal to x.

[0110] In step 4045, the terminal obtains the average load of the tth update thread based on the tth remaining total channel load and the remaining number; the remaining number is the difference between x and q.

[0111] In step 4046, the terminal arranges the t-th channel load of each remaining video encoding channel in reverse order to obtain an updated channel load queue.

[0112] In some embodiments, after a channel group is determined, the t-th channel load corresponding to the video encoding channel included in the channel group is removed from the channel load queue, and the remaining t-th channel load is rearranged in reverse order to obtain an updated channel load queue. That is, based on the sum of the t-th channel loads of each remaining video encoding channel, the remaining total channel load is obtained. At the same time, a video encoding thread matching the channel group is also removed to obtain the remaining number of video encoding threads. For example: the t-th update thread load mean can be calculated by the following formula: t-th update thread load mean = remaining total channel load / remaining number.

[0113] In step 4047, if the load of the t-th channel at the first position in the update channel load queue is greater than or equal to the average load of the t-th update thread, a channel group is obtained.

[0114] In some embodiments, the channel group is the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue.

[0115] The above step 4047 is an implementation method for obtaining a channel group. In another implementation method, if the t-th channel load at the first position in the update channel load queue is less than the average load of the t-th update thread, the remaining t-th channel loads in the update channel load queue are traversed to obtain a channel group; the channel group includes the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue and the video encoding channel corresponding to the p-th t-th channel load; the difference between the p-th t-th channel load and the third difference satisfies the second preset condition; the third difference is the difference between the t-th channel load at the first position and the average load of the t-th update thread; p is an integer greater than or equal to 2. Among them, the second preset condition is that the absolute value of the difference between the p-th t-th channel load and the third difference and the difference between the third difference is the smallest. That is, the p-th t-th channel load is closest to the third difference.

[0116] In some embodiments, the above step 404 further includes:

[0117] After the (x-1) channel groups are determined, the remaining video encoding channels are taken as the xth channel group; the xth channel group is matched with the xth video thread encoding.

[0118] In one example, after the terminal reallocates the video encoding channel to each video encoding thread, it updates the initial mapping table to obtain an updated mapping table. The updated mapping table is used to store the reallocated video encoding channel to each video encoding thread.

[0119] In one example, each video encoding thread calls the hardware video encoder based on the updated mapping table and starts the encoding task, so as to process the encoding task of each video encoding channel. For example, each video encoding thread calls the SoC based on the updated mapping table and starts the encoding task.

[0120] In an embodiment of the present application, the real-time parameters of each video encoding channel are detected in real time. When the real-time parameters of at least one video encoding channel change, the channel load of each video encoding channel is recalculated, and the thread load of each video encoding thread is recalculated based on each video encoding channel load. If at least one thread load does not match the thread load calculated last time, the video encoding channel that matches each video encoding thread is reallocated. Thereby, the thread load of each video encoding thread is adjusted, and then the balance between the thread loads of each video encoding thread is re-achieved. Through the above technical solution, the embodiment of the present application avoids the problem of a long delay in a certain video encoding channel, thereby improving system performance.

[0121] Figure 7 700 is a structural diagram of a device for adjusting a video encoding thread load provided in accordance with an embodiment of the present application, the device comprising:

[0122] The first acquisition module 701 is used to acquire the tth real-time parameter of each video encoding channel; the real-time parameter indicates the characteristics of the video stream;

[0123] A second acquisition module 702 is used to acquire a t-th channel load of each video encoding channel if the t-th real-time parameter of at least one video encoding channel does not match the t-1-th real-time parameter; the t-th channel load is obtained based on the t-th real-time parameter;

[0124] A determination module 703 is used for taking, for a video encoding thread, the sum of the t-th channel loads of k video encoding channels as the t-th thread load of the video encoding thread;

[0125] The adjustment module 704 is used to regroup multiple video encoding channels based on the tth total channel load to obtain multiple channel groups if the tth thread load of at least one video encoding thread does not match the t-1th thread load; each channel group matches one video encoding thread.

[0126] In a possible implementation, the real-time parameters include image height, image width, and output frame rate.

[0127] In some embodiments, the apparatus is further configured to:

[0128] For a video encoding channel, the product of the image width, the image height and the output frame rate in the t-th real-time parameter is used as the t-th channel load.

[0129] In some embodiments, the video encoding thread is matched with k of the video encoding channels.

[0130] In some embodiments, the apparatus is further configured to:

[0131] For a video encoding thread, if the difference between the t-th thread load and the t-1-th thread load is greater than a preset threshold, the t-th thread load does not match the t-1-th thread load.

[0132] In some embodiments, the t-th said total channel load is the sum of the t-th said channel loads of all said video encoding channels.

[0133] In some embodiments, the apparatus is further configured to:

[0134] For the first video encoding thread, based on the t-th total channel load and x, obtain the t-th thread load mean; x is the total number of video encoding threads, and x is an integer greater than or equal to 2;

[0135] Arrange the t-th channel load of each video encoding channel in reverse order to obtain a channel load queue;

[0136] If the t-th channel load at the first position in the channel load queue is greater than or equal to the t-th thread load average, the video encoding channel corresponding to the t-th channel load at the first position is taken as a channel group; the channel group is matched with the first video encoding thread; or

[0137] If the first channel load in the channel load queue is less than the average load of the t-th thread, traverse the remaining t-th channel loads to obtain a channel group; the channel group includes the video encoding channel corresponding to the n-th t-th channel load and the video encoding channel corresponding to the t-th channel load in the first place; the n-th t-th channel load satisfies the first preset condition; n is an integer greater than or equal to 2 and less than or equal to x.

[0138] In some embodiments, the apparatus is further configured to:

[0139] Obtain the first difference between the load of the first t-th channel and the average load of the t-th thread;

[0140] Obtain a second difference between the remaining load of each t-th channel and the first difference;

[0141] Obtaining the nth t-th channel load; the second difference corresponding to the nth t-th channel load satisfies the first preset condition;

[0142] The video coding channel corresponding to the t-th channel load at the nth position and the video coding channel corresponding to the t-th channel load at the first position are taken as a channel group.

[0143] In some embodiments, the apparatus is further configured to:

[0144] For the q+1th video encoding thread, after determining the qth channel group, the tth remaining total channel load is obtained; the qth channel load sum is the sum of the channel loads of the qth channel group; the tth remaining total channel load is the difference between the tth total channel load and the qth channel load sum; q is an integer greater than or equal to 1 and less than or equal to x;

[0145] Based on the tth remaining total channel load and the remaining number, the tth update thread load mean is obtained; the remaining number is the difference between x and q;

[0146] Arrange the t-th channel load of each remaining video encoding channel in reverse order to obtain an updated channel load queue;

[0147] If the t-th channel load at the first position in the update channel load queue is greater than or equal to the t-th update thread load average, obtain a channel group; the channel group is the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue; or

[0148] If the t-th channel load at the first position in the update channel load queue is less than the average load of the t-th update thread, traverse the remaining t-th channel loads in the update channel load queue to obtain a channel group; the channel group includes the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue and the video encoding channel corresponding to the p-th t-th channel load; the difference between the p-th t-th channel load and the third difference satisfies the second preset condition; the third difference is the difference between the t-th channel load at the first position and the average load of the t-th update thread; p is an integer greater than or equal to 2.

[0149] In some embodiments, the apparatus is further configured to:

[0150] After the (x-1) channel groups are determined, the remaining video encoding channels are taken as the xth channel group; the xth channel group is matched with the xth video thread encoding.

[0151] It should be noted that: the device for adjusting the video encoding thread load provided in the above embodiment only uses the division of the above functional modules as an example when executing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for adjusting the video encoding thread load provided in the above embodiment and the method embodiment for adjusting the video encoding thread load belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0152] In an embodiment of the present application, the real-time parameters of each video encoding channel are detected in real time. When the real-time parameters of at least one video encoding channel change, the channel load of each video encoding channel is recalculated, and the thread load of each video encoding thread is recalculated based on each video encoding channel load. If at least one thread load does not match the thread load calculated last time, the video encoding channel that matches each video encoding thread is reallocated. Thereby, the thread load of each video encoding thread is adjusted, and then the balance between the thread loads of each video encoding thread is re-achieved. Through the above technical solution, the embodiment of the present application avoids the problem of a long delay in a certain video encoding channel, thereby improving system performance.

[0153] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.

[0154] Taking computer equipment as the terminal as an example, Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Figure 8The terminal 800 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0155] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0156] The processor 801 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0157] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is used to be executed by the processor 801 to implement the process provided by the method embodiment of the present application for the terminal execution in the above method.

[0158] In some embodiments, the terminal 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 804, a camera assembly 805, an audio circuit 806 and a power supply 807.

[0159] The peripheral device interface 803 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0160] The display screen 804 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 804 is a touch display screen, the display screen 804 also has the ability to collect touch signals on the surface or above the surface of the display screen 804. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 804 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 804 can be one, set on the front panel of the terminal 800; in other embodiments, the display screen 804 can be at least two, respectively set on different surfaces of the terminal 800 or in a folding design; in other embodiments, the display screen 804 can be a flexible display screen, set on the curved surface or folding surface of the terminal 800. Even, the display screen 804 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 804 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0161] The camera assembly 805 is used to capture images or videos. In some embodiments, the camera assembly 805 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 805 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0162] The audio circuit 806 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 806 may also include a headphone jack.

[0163] The power supply 807 is used to power various components in the terminal 800. The power supply 807 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 807 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0164] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0165] Take the computer device as a server as an example. Fig. 9It is a structural diagram of a server provided by an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, wherein at least one computer program is stored in the one or more memories 902, and the at least one computer program is loaded and executed by the one or more processors 901 to implement the above-mentioned method of adjusting the video encoding thread load. Of course, the server 900 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 900 may also include other components for implementing device functions, which will not be repeated here.

[0166] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above method. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0167] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0168] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for adjusting a video encoding thread load, characterized in that: include: Obtaining a t-th real-time parameter of each video encoding channel; the real-time parameter indicates a video stream feature; If the t-th real-time parameter of at least one of the video encoding channels does not match the t-1-th real-time parameter, obtaining the t-th channel load of each of the video encoding channels; the t-th channel load is obtained based on the t-th real-time parameter; For one of the video encoding threads, taking the sum of the t-th channel loads of the k video encoding channels as the t-th thread load of the video encoding thread; If the t-th thread load of at least one of the video encoding threads does not match the t-1-th thread load, the multiple video encoding channels are regrouped based on the t-th total channel load to obtain multiple channel groups; each channel group matches one video encoding thread.

2. The method according to claim 1, characterized in that The real-time parameters include the height of the image, the width of the image, and the output frame rate.

3. The method according to claim 2, characterized in that The obtaining of the t-th channel load of each of the video encoding channels comprises: For one of the video encoding channels, the product of the width of the image, the height of the image and the output frame rate in the t-th real-time parameter is used as the t-th channel load.

4. The method according to claim 1, characterized in that: The video encoding thread is matched with k video encoding channels.

5. The method according to claim 1, characterized in that: If the t-th thread load of at least one of the video encoding threads does not match the t-1-th thread load, the method comprises: For one of the video encoding threads, if the difference between the load of the tth thread and the load of the t-1th thread is greater than a preset threshold, the load of the tth thread does not match the load of the t-1th thread.

6. The method according to claim 1, characterized in that The t-th total channel load is the sum of the t-th channel loads of all the video encoding channels.

7. The method according to claim 1, characterized in that The regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups comprises: For the first video encoding thread, based on the t-th total channel load and x, obtain the t-th thread load average; x is the total number of the video encoding threads, and x is an integer greater than or equal to 2; Arrange the t-th channel load of each video encoding channel in reverse order to obtain a channel load queue; If the load of the t-th channel at the top of the channel load queue is greater than or equal to the average load of the t-th thread, the video encoding channel corresponding to the t-th channel load at the top is used as a channel group; the channel group is matched with the first video encoding thread; or If the first channel load in the channel load queue is less than the average load of the t-th thread, traverse the remaining t-th channel loads to obtain a channel group; the channel group includes the video encoding channel corresponding to the n-th t-th channel load and the video encoding channel corresponding to the t-th channel load at the first position; the n-th t-th channel load satisfies the first preset condition; n is an integer greater than or equal to 2 and less than or equal to x.

8. The method according to claim 7, characterized in that The traversing the remaining t-th channel loads to obtain a channel group includes: Obtain a first difference between the first t-th channel load and the t-th thread load average; Obtaining a second difference between each of the remaining t-th channel loads and the first difference; Obtaining the nth t-th channel load; the second difference corresponding to the nth t-th channel load satisfies the preset condition; The video encoding channel corresponding to the t-th channel load at the nth position and the video encoding channel corresponding to the t-th channel load at the first position are taken as one of the channel groups.

9. The method according to claim 7, characterized in that: The regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups further includes: For the q+1th video encoding thread, after determining the qth channel group, the tth remaining total channel load is obtained; the qth channel load sum is the sum of the channel loads of the qth channel group; the tth remaining total channel load is the difference between the tth total channel load and the qth channel load sum; q is an integer greater than or equal to 1 and less than or equal to x; Based on the t-th remaining total channel load and the remaining number, the t-th update thread load mean is obtained; the remaining number is the difference between x and q; Arrange the t-th channel load of each remaining video encoding channel in reverse order to obtain an updated channel load queue; If the load of the t-th channel at the top of the update channel load queue is greater than or equal to the average load of the t-th update thread, obtain a channel group; the channel group is the video encoding channel corresponding to the t-th channel load at the top of the update channel load queue; or If the t-th channel load at the first position in the update channel load queue is less than the average load of the t-th update thread, traverse the remaining t-th channel loads in the update channel load queue to obtain a channel group; the channel group includes the video encoding channel corresponding to the t-th channel load at the first position in the update channel load queue and the video encoding channel corresponding to the p-th t-th channel load; the difference between the p-th t-th channel load and the third difference satisfies the second preset condition; the third difference is the difference between the t-th channel load at the first position and the average load of the t-th update thread; p is an integer greater than or equal to 2.

10. The method according to claim 1, characterized in that The regrouping of the plurality of video encoding channels based on the t-th total channel load to obtain a plurality of channel groups further includes: After (x-1) channel groups are determined, the remaining video encoding channels are used as the xth channel group; the xth channel group is matched with the xth video thread encoding.

11. A device for adjusting a video encoding thread load, characterized in that: include: A first acquisition module is used to obtain the tth real-time parameter of each video encoding channel; The real-time parameters indicate video stream characteristics; A second acquisition module is used to acquire a t-th channel load of each video encoding channel if the t-th real-time parameter of at least one video encoding channel does not match the t-1-th real-time parameter; the t-th channel load is obtained based on the t-th real-time parameter; A determination module, configured to use, for one of the video encoding threads, a sum of the t-th channel loads of the k video encoding channels as a t-th thread load of the video encoding thread; An adjustment module is used for regrouping the plurality of video encoding channels based on the tth total channel load to obtain a plurality of channel groups if the tth thread load of at least one of the video encoding threads does not match the t-1th thread load; each channel group matches one of the video encoding threads.

12. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the method for adjusting the video encoding thread load according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by a processor to implement the method for adjusting the video encoding thread load according to any one of claims 1 to 10.

Images