Video analysis method, device and video processing equipment

Abstract

The embodiment of the present application provides a video analysis method, apparatus and video processing equipment. The method includes: after receiving an analysis task to be executed for a video, determining whether all first-class analysis engines are in a fully loaded state, wherein the first-class analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed; if all first-class analysis engines are in a fully loaded state, selecting a second-class analysis engine in an idle state as the target analysis engine, wherein the second-class analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm; switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm; and executing the analysis task to be executed through the target analysis engine. The analysis algorithm loaded by the analysis engine can be automatically configured according to the load status of each analysis engine, thereby effectively improving the real-time performance when executing the analysis task.

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a video analysis method, apparatus, and video processing equipment. Background Art

[0002] In some application scenarios, specific analysis tasks need to be performed on the video due to practical needs. For example, in order to facilitate the management of vehicles on the road, a camera can be used to shoot a video of the road, and a license plate number analysis task can be performed on the video to determine the license plate number of each vehicle appearing in the video.

[0003] In this article, electronic devices capable of performing video analysis tasks are referred to as video processing devices. These devices include an analysis engine that can analyze videos according to pre-defined methods. For example, an analysis engine might be loaded with an analysis algorithm for license plate recognition, allowing the video processing device to perform license plate analysis tasks on videos using this analysis engine.

[0004] In related art, to enable a video processing device to perform multiple different analysis tasks, different analysis algorithms can be pre-configured for at least two different analysis engines. For example, assuming that the video processing device includes a total of four analysis engines, two of the analysis engines can be configured with analysis algorithms for license plate recognition, and the other two analysis engines can be configured with analysis algorithms for detecting vehicle outlines. Therefore, the video processing device can perform both license plate recognition tasks and vehicle outline detection tasks. Assuming that each engine can simultaneously handle a maximum of eight analysis tasks, the video processing device can simultaneously perform a maximum of 16 license plate recognition tasks and 16 vehicle outline detection tasks.

[0005] However, the number of various analysis tasks required by the video processing device may vary depending on the actual situation. For example, in a certain scenario, the video processing device needs to perform 20 license plate recognition tasks, but only 4 vehicle outline detection tasks. Since the video processing device can only process a maximum of 16 license plate recognition tasks simultaneously, at least 4 license plate recognition tasks cannot be completed in a timely manner. In other words, the real-time performance of the video processing device when performing analysis tasks is poor. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a video analysis method to improve the real-time performance of a video processing device when performing analysis tasks. The specific technical solution is as follows:

[0007] In a first aspect of an embodiment of the present application, a video analysis method is provided, which is applied to a video processing device, wherein the video processing device includes multiple analysis engines, each of which is pre-loaded with an analysis algorithm. The method includes:

[0008] After receiving an analysis task to be executed for a video, determining whether all first-type analysis engines are in a fully loaded state, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed;

[0009] If all first-category analysis engines are in a fully loaded state, selecting a second-category analysis engine in an idle state as a target analysis engine, wherein the second-category analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm;

[0010] Switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm;

[0011] The analysis task to be performed is executed by the target analysis engine.

[0012] In a possible embodiment, before selecting the idle second-type analysis engine as the target analysis engine, the method further includes:

[0013] Determining whether there is an idle second-type analysis engine;

[0014] If there is no second-type analysis engine in an idle state, all analysis tasks being executed by at least one second-type analysis engine are scheduled to second-type analysis engines other than the at least one second-type analysis engine.

[0015] In a possible embodiment, scheduling all tasks being executed by at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine includes:

[0016] For each analysis task being executed by at least one second-type analysis engine, the analysis task is scheduled to the analysis engine with the lowest load among the second-type analysis engines other than the at least one second-type analysis engine that are loaded with the analysis algorithm for executing the analysis task.

[0017] In a possible embodiment, after determining whether all first-type analysis engines that load target analysis algorithms are in a fully loaded state, the method further includes:

[0018] If at least one of the first-category analysis engines is not in a fully loaded state, the to-be-executed analysis task is executed by the first-category analysis engine with the lowest load.

[0019] In a possible embodiment, selecting the second type of analysis engine in an idle state as the target analysis engine includes:

[0020] A second-category analysis engine that is in an idle state and meets preset screening conditions is selected as the target analysis engine, wherein the preset screening conditions include: the remaining performance of the same type is higher than a preset performance threshold, or, the second-category analysis engine with the highest remaining performance of the same type among all second-category analysis engines in an idle state, and the remaining performance of the second-category analysis engine of the same type is used to represent the remaining performance of all analysis engines that load the same analysis algorithm as the second-category analysis engine.

[0021] In a possible embodiment, after switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm, the method further includes:

[0022] When there is an idle first-type analysis engine, the analysis algorithm loaded by the idle first-type analysis engine is switched to the analysis algorithm initially loaded by the target analysis engine.

[0023] In a second aspect of an embodiment of the present application, a video analysis apparatus is provided, which is applied to a video processing device. The video processing device includes multiple analysis engines, each of which is pre-loaded with an analysis algorithm. The apparatus includes:

[0024] A first determination module is configured to, after receiving an analysis task to be executed for a video, determine whether all first-type analysis engines are in a fully loaded state, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed;

[0025] an engine selection module, configured to, if all analysis engines of the first type of analysis algorithm are in a fully loaded state, determine an idle second type of analysis engine as a target analysis engine, wherein the second type of analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm;

[0026] An algorithm switching module, configured to switch the target analysis engine's loaded analysis algorithm to the target analysis algorithm;

[0027] A task allocation module is used to execute the analysis task to be executed through the target analysis engine.

[0028] In a possible embodiment, the apparatus further includes a second determination module, configured to determine whether there is a second type of analysis engine in an idle state;

[0029] The task scheduling module is configured to schedule all analysis tasks being executed by at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine if no second-type analysis engine in an idle state exists.

[0030] In a possible embodiment, the task scheduling module is specifically used to schedule each analysis task being executed by at least one second-class analysis engine to the analysis engine with the lowest load among the second-class analysis engines other than the at least one second-class analysis engine, which is loaded with an analysis algorithm for executing the analysis task.

[0031] In a possible embodiment, the task allocation module is further configured to execute the to-be-executed analysis task by the first-category analysis engine with the lowest load if at least one first-category analysis engine is not in a fully loaded state.

[0032] In a possible embodiment, the engine selection module is specifically used to select a second-category analysis engine that is in an idle state and meets preset screening conditions as a target analysis engine, wherein the preset screening conditions include: the remaining performance of the same type is higher than a preset performance threshold, or, the second-category analysis engine with the highest remaining performance of the same type among all second-category analysis engines in an idle state, and the remaining performance of the second-category analysis engine of the same type is used to represent the remaining performance of all analysis engines that load the same analysis algorithm as the second-category analysis engine.

[0033] In a possible embodiment, the algorithm switching module is further configured to switch the analysis algorithm loaded by the idle first-type analysis engine to the analysis algorithm initially loaded by the target analysis engine when there is an idle first-type analysis engine.

[0034] In a third aspect of an embodiment of the present application, a video processing device is provided, comprising a plurality of analysis engines, a memory, and a processor, wherein each analysis engine is pre-loaded with an analysis algorithm;

[0035] The memory is used to store computer programs;

[0036] The processor is configured to implement any of the method steps described in the first aspect when executing the program stored in the memory.

[0037] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, any of the method steps described in the first aspect is implemented.

[0038] Beneficial effects of the embodiments of the present application:

[0039] The video analysis method, apparatus, and video processing device provided in the embodiments of the present application can automatically convert the second type of analysis engine into the first type of analysis engine when the first type of analysis engine is fully loaded and the second type of analysis engine still has excess computing power. That is, the analysis algorithm loaded by the analysis engine can be automatically configured according to the load status of each analysis engine, thereby making full use of the computing power of each analysis engine to execute the analysis task, thereby effectively improving the real-time performance when executing the analysis task.

[0040] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of a flow chart of a video analysis method provided in an embodiment of the present application;

[0043] Figure 2 Another schematic diagram of a flow chart of a video analysis method provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a video analysis device provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0047] In order to more clearly illustrate the video analysis method provided in the embodiment of the present application, a possible application scenario of the video analysis method provided in the embodiment of the present application will be exemplified below. It can be understood that the following example is only one possible application scenario of the video analysis method provided in the embodiment of the present application. In other possible embodiments, the video analysis method provided in the embodiment of the present application can also be applied to other possible application scenarios, and the following example does not impose any limitations on this.

[0048] In order to facilitate the management of the monitored area, front-end image acquisition devices, such as Internet Protocol Cameras (IPCs), can be used to capture videos of the monitored area. The front-end image acquisition devices transmit the captured videos in the form of video streams in real time to the back-end video processing devices. The video processing devices perform specific analysis tasks on the received video streams in real time, such as license plate recognition, face recognition, behavior recognition, target detection, etc., to obtain execution results. The video processing devices can save the video streams and execution results for relevant personnel to review.

[0049] For the sake of convenience of description, this example assumes that the video processing device includes N analysis engines, where N is any positive integer greater than 1, and each analysis engine can perform up to K analysis tasks simultaneously. It is also assumed that in this example, there are M analysis engines in the video processing device configured to perform face recognition tasks, M is a positive integer less than N, and NM analysis engines are configured to perform behavior recognition tasks.

[0050] Theoretically, the video processing device can simultaneously perform a maximum of K*M face recognition tasks and K*(NM) behavior recognition tasks. Users can pre-set the value of M based on experience to determine the maximum number of face recognition tasks and behavior recognition tasks that the video processing device can perform simultaneously. However, it is often difficult for users to accurately predict the number of face recognition tasks and behavior recognition tasks that the video processing device will need to perform in all scenarios. Therefore, in some scenarios, the following two situations may occur:

[0051] Case 1: The number of face recognition tasks to be performed is greater than K*M, and the number of behavior recognition tasks to be performed is less than K*(NM);

[0052] Case 2: The number of face recognition tasks to be performed is less than K*M, and the number of behavior recognition tasks to be performed is greater than K*(NM).

[0053] For situation one, the video processing device cannot handle all the required face recognition tasks at the same time. In one possible embodiment, the relevant personnel can manually reconfigure an analysis engine that was originally configured to perform behavior recognition tasks to perform face recognition tasks. It is understandable that, on the one hand, the relevant personnel may not be able to reconfigure the analysis engine in a timely manner. For example, situation one occurs during the non-working hours of the relevant personnel. On the other hand, reconfiguration also requires a certain amount of time and labor costs. Therefore, some face recognition tasks will not be executed in real time, and the labor costs consumed by the operation and maintenance of the video processing equipment will be high. Similarly, the video processing equipment in situation two has the problem that the behavior recognition tasks cannot be executed in real time and the labor costs consumed by the operation and maintenance of the video processing equipment are high.

[0054] Based on this, embodiments of the present application provide a video analysis method that is applied to a video processing device. The video processing device herein may refer to any electronic device capable of analyzing and processing videos, including but not limited to an NVR (network video recorder), a DVR (digital video recorder), and an XVR (hybrid video recorder). The video herein may refer to a video stream in the form of streaming media or a video file in a preset video format.

[0055] The video processing device includes multiple analysis engines. An analysis engine, as used herein, refers to any circuit unit capable of performing video analysis tasks, including but not limited to a CPU (general-purpose processor) and a GPU (graphics processing unit). Each analysis engine is preloaded with an analysis algorithm, and at least two analysis engines are preloaded with different analysis algorithms.

[0056] The video analysis method can be found in Figure 1 , Figure 1 The figure shows a flow chart of a video analysis algorithm provided by an embodiment of the present application, which may include:

[0057] S101: After receiving an analysis task to be executed for a video, determine whether all first-type analysis engines are in a fully loaded state.

[0058] S102: If all the first-type analysis engines are in a fully loaded state, select a second-type analysis engine in an idle state as a target analysis engine.

[0059] S103: Switch the analysis algorithm loaded by the target analysis engine to the target analysis algorithm.

[0060] S104: Execute the analysis task to be executed through the target analysis engine.

[0061] By selecting this embodiment, when the first type of analysis engines are all in a fully loaded state and the second type of analysis engines still have excess computing power, the second type of analysis engines can be automatically converted into the first type of analysis engines. That is, the analysis algorithms loaded by the analysis engines can be automatically configured according to the load conditions of each analysis engine, thereby making full use of the computing power of each analysis engine to execute analysis tasks, thereby effectively improving the real-time performance when executing analysis tasks.

[0062] On the other hand, since the video analysis algorithm provided in the embodiment of the present application can automatically configure the analysis engine, there is no need for manual configuration by relevant personnel, thus effectively reducing the labor cost during the operation and maintenance of the video processing equipment.

[0063] Among them, in S101, the first type of analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed. The full load state means that the remaining computing resources of the analysis engine are insufficient to execute new analysis tasks. For example, assuming that an analysis engine has 8 channels, each channel can be used to execute an analysis task, then if the number of channels executing analysis tasks in the analysis engine is 0, the analysis engine is said to be in an idle state, and if the number of channels executing analysis tasks in the analysis engine is 8, the analysis engine is said to be in a fully loaded state.

[0064] If not all first-category analysis engines are fully loaded, that is, if at least one first-category analysis engine is not fully loaded, the pending analysis task can be executed by the at least one first-category analysis engine. In one possible embodiment, the pending task can be executed by the first-category analysis engine with the lowest load, thereby achieving maximum load balancing.

[0065] In S102 , the second type of analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm, that is, the second type of analysis engine may refer to other analysis engines in the video processing device except the first type of analysis engine.

[0066] It is understandable that if all first-class analysis engines are fully loaded, the remaining computing resources of the existing first-class analysis engines are insufficient to execute the received analysis tasks to be executed, so the remaining computing resources of the second-class analysis engines are needed to execute the analysis tasks to be executed.

[0067] When selecting an idle second-category analysis engine, one or more idle second-category analysis engines may be selected. In one possible embodiment, to minimize changes to the analysis engine configuration, one idle second-category analysis engine is selected at a time. For ease of description, the following description uses the example of selecting a single idle second-category analysis engine. The principles for selecting multiple idle second-category analysis engines remain the same and are therefore not further described.

[0068] If there is only one second-category analysis engine in an idle state, then the one second-category analysis engine in an idle state can be selected as the target analysis engine. If there are multiple second-category analysis engines in an idle state, then one second-category analysis engine can be selected as the target analysis engine from the multiple second-category analysis engines in an idle state according to preset rules.

[0069] The preset rules can be set based on actual needs or user experience. For example, the preset rules may include selecting the second-category analysis engine ranked first in order of preset engine numbers from small to large or from large to small. Alternatively, the preset rules may include selecting the second-category analysis engine ranked first in order of performance parameters from high to low or from low to high. In other possible embodiments, the preset rules may also be other rules, and this embodiment does not impose any limitation on this.

[0070] In a possible embodiment, a second-category analysis engine that is in an idle state and meets preset filtering conditions may be selected as a target analysis engine, wherein the preset filtering conditions may include: the remaining performance of the same type is higher than a preset performance threshold, or, the second-category analysis engine with the highest remaining performance of the same type among all second-category analysis engines in an idle state, and the remaining performance of the second-category analysis engine of the same type is used to represent the remaining performance of all analysis engines that load the same analysis algorithm as the second-category analysis engine.

[0071] For example, assume that the video processing device includes 12 analysis engines, which are respectively recorded as analysis engines 1-12, where analysis engines 1-4 load analysis algorithm 1, analysis engines 5-8 load analysis algorithm 2, and analysis engines 8-12 load analysis algorithm 3. If analysis algorithm 1 is the target analysis algorithm, analysis engines 1-4 are first-type analysis engines, and analysis engines 5-12 are second-type analysis engines.

[0072] For any one of analysis engines 5-8, the same type of remaining performance is used to represent the remaining performance of the four analysis engines 5-8, and for any one of analysis engines 9-12, the same type of remaining performance is used to represent the remaining performance of the four analysis engines 9-12.

[0073] The way of expressing the remaining performance may vary depending on the application scenario. For example, it can be expressed as the number of analysis tasks that the analysis engine can still execute concurrently. For example, assuming that each analysis engine in analysis engine 5-8 can execute up to 8 analysis tasks concurrently, that is, analysis engine 5-8 can execute a total of 32 analysis tasks concurrently. Assuming that the current analysis engine 5-8 is executing a total of 10 analysis tasks, the remaining performance of analysis engine 5-8 is 22.

[0074] Assume that analysis engines 5, 6, and 10 of the second-category analysis engine are idle, and the remaining performance of analysis engines 5-8 is 22, and the remaining performance of analysis engines 9-12 is 25. Then, the remaining performance of analysis engines 5 and 6 is 22, and the remaining performance of analysis engine 10 is 25. If the preset filtering condition is the second-category analysis engine with the highest remaining performance among all idle second-category analysis engines, analysis engine 10 can be selected as the target analysis engine. If the preset filtering condition is that the remaining performance is greater than a preset performance threshold, and the preset performance threshold is 20, then one or more analysis engines among analysis engines 5, 6, and 10 can be selected as the target analysis engine.

[0075] For the sake of convenience of description, it is assumed that the analysis algorithm loaded by the target analysis engine is analysis algorithm 2. It can be understood that after the analysis algorithm loaded by the target analysis engine is switched to the target analysis algorithm, the number of analysis engines loaded with analysis algorithm 2 in the video processing device will be reduced. At this time, if a large number of analysis tasks that need to be executed by analysis algorithm 2 are received, these analysis tasks may not be executed in real time due to insufficient remaining performance of the analysis engine loaded with analysis algorithm 2 in the video processing device.

[0076] In this embodiment, when the target analysis engine satisfies the preset screening conditions, it can be considered that the target analysis engine has a high residual performance among similar engines. In this case, even if the analysis algorithm loaded by the target analysis engine is switched to the target analysis algorithm, the residual performance of the analysis engine loaded with analysis algorithm 2 in the video processing device is still high. In this case, even if a large number of analysis tasks requiring execution by analysis algorithm 2 are received, these analysis tasks will not be unable to be executed in real time due to insufficient residual performance of the analysis engine loaded with analysis algorithm 2 in the video processing device. This solves the above-mentioned technical problem.

[0077] In S103, since the target analysis engine is a second-class analysis engine, it cannot be used directly to execute the pending analysis task and needs to be converted to a first-class analysis engine. When switching the analysis algorithm, the target analysis engine can be controlled to cancel the original analysis algorithm and then send the target analysis algorithm to the target analysis engine so that the target analysis engine loads the target analysis algorithm.

[0078] In S104 , the analysis algorithm loaded by the target analysis engine has been switched to the target analysis algorithm, so the target analysis engine can be used to execute the analysis task to be executed.

[0079] Assuming that K is 8, N is 16, and M is 8 in the aforementioned example application scenario, the video processing device can process up to 64 face recognition tasks and 64 behavior recognition tasks simultaneously. Assuming that in a certain scenario, the video processing device is performing 64 face recognition tasks and 32 behavior recognition tasks. If a new task to be executed is received at this time and the task to be executed is a face recognition task, then according to the video analysis method in the relevant technology, since all the first-class analysis engines (i.e., the analysis engines loaded with the analysis algorithms for performing face recognition tasks) are in a fully loaded state, the video processing device cannot process the analysis task to be executed in real time.

[0080] If the video analysis method provided in the embodiment of the present application is selected, an idle analysis engine can be selected from the second type of analysis engine (i.e., an analysis engine loaded with an analysis algorithm for performing behavior recognition tasks) as the target analysis engine, and the analysis algorithm loaded by the target analysis engine is switched to the target analysis algorithm (i.e., the analysis algorithm for performing face recognition tasks), and the analysis task to be performed is executed by the target analysis engine. After switching the analysis algorithm loaded by the target analysis engine, the video processing device can process up to 72 face recognition tasks and 56 behavior recognition tasks at the same time, while the number of face recognition tasks and behavior recognition tasks that the video processing device needs to perform is 65 and 32, respectively. It can be seen that the video processing device can perform all the analysis tasks that need to be performed at the same time, which is obviously more real-time than the relevant technology.

[0081] In a possible embodiment, after S104 , when there is an idle first-type analysis engine, the analysis algorithm loaded by the idle first-type analysis engine is switched to the analysis algorithm initially loaded by the target analysis engine.

[0082] Exemplarily, assuming that the analysis algorithm initially loaded by the target analysis engine is analysis algorithm 2, after S104, when there is an idle first-type analysis engine, the analysis algorithm loaded by the idle first-type analysis engine is switched to analysis algorithm 2.

[0083] It is understood that the analysis algorithms initially configured for each analysis engine are often configured based on actual user needs or experience, and the initial number of analysis engines loaded with each analysis algorithm can be considered reasonable. By selecting this embodiment, the number of analysis engines loaded with each analysis algorithm can be kept consistent with the initial number as much as possible.

[0084] In some scenarios, there may not be an idle second-category analysis engine. For example, taking the above-mentioned example application scenario as an example, it is assumed that before selecting the target analysis engine, 32 behavior recognition tasks are evenly distributed to 8 second-category analysis engines, that is, each second-category analysis engine performs 4 behavior recognition tasks. At this time, none of the 8 second-category analysis engines are in an idle state, so the target analysis engine cannot be directly selected from the second-category analysis engines.

[0085] Based on this, in a possible embodiment, see Figure 2 , Figure 2 FIG. 1 is another flow chart of a video analysis method provided in an embodiment of the present application, which may include:

[0086] S201: After receiving an analysis task to be executed for a video, determine whether all first-type analysis engines are in a fully loaded state.

[0087] This step is the same as S101. Please refer to the relevant description of S101 above and will not be repeated here.

[0088] S202: If all first-category analysis engines are in a fully loaded state, determine whether there is a second-category analysis engine in an idle state. If there is a second-category analysis engine in an idle state, execute S204; if there is no second-category analysis engine in an idle state, execute S203.

[0089] S203: Schedule all analysis tasks being executed by the at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine.

[0090] Among them, the number of at least one second-category analysis engine can be one or more. For the convenience of description, the following text takes the at least one second-category analysis engine as one second-category analysis engine as an example. For the embodiment where at least one second-category analysis engine is multiple second-category analysis engines, since the principle is the same, it will not be repeated.

[0091] It is understandable that when scheduling an analysis task, the analysis engine to which the analysis task is scheduled should be loaded with an analysis algorithm for executing the analysis task. For example, assuming that a face recognition task is scheduled from the second-type analysis engine 1 to the second-type analysis engine 2, the second-type analysis engine 2 should be loaded with an analysis algorithm for executing the face recognition task.

[0092] Moreover, in a possible embodiment, the analysis task may be preferentially scheduled to the analysis engine with the lowest load among the analysis engines other than the at least one second-type analysis engine that are loaded with the analysis algorithm for executing the analysis task, so as to achieve load balancing as much as possible.

[0093] It is possible to arbitrarily select a second-category analysis engine and dispatch all analysis tasks of the second-category analysis engine to a second-category analysis engine other than the second-category analysis engine. In other possible embodiments, it is also possible to select a second-category analysis engine according to a specified selection rule and dispatch all analysis tasks of the second-category analysis engine to a second-category analysis engine other than the second-category analysis engine. The selection rule may vary depending on the application scenario. For example, it may be possible to preferentially select the second-category analysis engine with the least number of analysis tasks being executed, or it may be possible to preferentially select the second-category analysis engine loaded with the analysis algorithm with the largest number of remaining channels.

[0094] Among them, the remaining number of channels of the analysis algorithm refers to the number of channels in all analysis engines loaded with the analysis algorithm that are not used to perform analysis tasks. For example, assume that there are a total of 8 second-class analysis engines, each second-class analysis engine includes 8 channels, of which 5 second-class analysis engines are loaded with analysis algorithms for performing behavior recognition tasks, and the other 3 second-class analysis engines are loaded with analysis algorithms for performing human detection tasks. At this time, the video processing device is performing 32 behavior recognition tasks and 14 human detection tasks. Then the remaining number of channels of the analysis algorithm used to perform behavior recognition tasks is 5*8-32=8, and the remaining number of channels of the analysis algorithm used to perform human detection tasks is 3*8-14=10. Therefore, in this scenario, the second-class analysis engine loaded with the analysis algorithm for performing human detection tasks can be preferentially selected.

[0095] After all analysis tasks being executed by the at least one second-type analysis engine are dispatched to second-type analysis engines other than the at least one second-type analysis engine, the at least one second-type analysis engine is in an idle state.

[0096] S204: Switch the analysis algorithm loaded by the target analysis engine to the target analysis algorithm.

[0097] This step is the same as S103. Please refer to the above description of S103 and will not be repeated here.

[0098] S205: Execute the analysis task to be executed through the target analysis engine.

[0099] This step is the same as S104. Please refer to the above description of S104 and will not be repeated here.

[0100] See also Figure 3 , Figure 3 The figure shows a schematic diagram of the structure of a video analysis device provided in an embodiment of the present application, which may include:

[0101] A first determination module 301 is configured to determine whether all first-type analysis engines are fully loaded after receiving a pending analysis task for a video, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the pending analysis task;

[0102] The engine selection module 302 is configured to, if all analysis engines of the first type of analysis algorithm are in a fully loaded state, determine an idle analysis engine of the second type as a target analysis engine, wherein the second type of analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm;

[0103] An algorithm switching module 303 is configured to switch the target analysis engine's loaded analysis algorithm to the target analysis algorithm;

[0104] The task allocation module 304 is configured to execute the to-be-executed analysis task through the target analysis engine.

[0105] In a possible embodiment, the apparatus further includes a second determination module, configured to determine whether there is a second type of analysis engine in an idle state;

[0106] The task scheduling module is configured to schedule all analysis tasks being executed by at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine if no second-type analysis engine in an idle state exists.

[0107] In a possible embodiment, the task scheduling module is specifically used to schedule each analysis task being executed by at least one second-class analysis engine to the analysis engine with the lowest load among the second-class analysis engines other than the at least one second-class analysis engine, which is loaded with an analysis algorithm for executing the analysis task.

[0108] In a possible embodiment, the task allocation module 304 is further configured to execute the to-be-executed analysis task by the first-category analysis engine with the lowest load if at least one first-category analysis engine is not in a fully loaded state.

[0109] In a possible embodiment, the engine selection module 302 is specifically used to select a second-category analysis engine that is in an idle state and meets preset screening conditions as a target analysis engine, wherein the preset screening conditions include: the remaining performance of the same type is higher than a preset performance threshold, or, the second-category analysis engine with the highest remaining performance of the same type among all second-category analysis engines in an idle state, and the remaining performance of the second-category analysis engine of the same type is used to represent the remaining performance of all analysis engines that load the same analysis algorithm as the second-category analysis engine.

[0110] In a possible embodiment, the algorithm switching module 303 is further used to switch the analysis algorithm loaded by the idle first-type analysis engine to the analysis algorithm initially loaded by the target analysis engine when there is an idle first-type analysis engine.

[0111] The embodiment of the present application also provides a video processing device, such as Figure 4 As shown, it includes multiple analysis engines, a memory 401 and a processor 402, wherein each analysis engine is pre-loaded with an analysis algorithm;

[0112] Memory 401, used for storing computer programs;

[0113] The processor 402 is configured to execute the program stored in the memory 401 by performing the following steps:

[0114] After receiving an analysis task to be executed for a video, determining whether all first-type analysis engines are in a fully loaded state, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed;

[0115] If all first-category analysis engines are in a fully loaded state, selecting a second-category analysis engine in an idle state as a target analysis engine, wherein the second-category analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm;

[0116] Switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm;

[0117] The analysis task to be performed is executed by the target analysis engine.

[0118] The memory mentioned in the video processing device may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the processor.

[0119] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0120] In another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video analysis methods are implemented.

[0121] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the video analysis methods in the above embodiments.

[0122] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0123] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0124] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the embodiments of the apparatus, video processing device, computer-readable storage medium, and computer program product are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, reference can be made to the descriptions of the method embodiments.

[0125] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A video analysis method, characterized in that: Applied to a video processing device, the video processing device includes multiple analysis engines, each analysis engine is pre-loaded with an analysis algorithm, and the method includes: After receiving an analysis task to be executed for a video, determining whether all first-type analysis engines are in a fully loaded state, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed; If all first-category analysis engines are in a fully loaded state, selecting a second-category analysis engine in an idle state as a target analysis engine, wherein the second-category analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm, wherein the idle state means that the number of channels executing analysis tasks is 0; Switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm; Executing the analysis task to be executed by the target analysis engine; Before selecting the second-type analysis engine in an idle state as the target analysis engine, the method further includes: Determining whether there is an idle second-type analysis engine; If there is no second-type analysis engine in an idle state, all analysis tasks being executed by at least one second-type analysis engine are scheduled to second-type analysis engines other than the at least one second-type analysis engine.

2. The method according to claim 1, characterized in that The step of dispatching all tasks being executed by at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine comprises: For each analysis task being executed by at least one second-type analysis engine, the analysis task is scheduled to the analysis engine with the lowest load among the second-type analysis engines other than the at least one second-type analysis engine that are loaded with the analysis algorithm for executing the analysis task.

3. The method according to claim 1, characterized in that After determining whether all first-type analysis engines that load target analysis algorithms are in a fully loaded state, the method further includes: If at least one of the first-category analysis engines is not in a fully loaded state, the to-be-executed analysis task is executed by the first-category analysis engine with the lowest load.

4. The method according to any one of claims 1 to 3, characterized in that: The selecting the second type of analysis engine in an idle state as the target analysis engine includes: A second-category analysis engine that is in an idle state and meets preset screening conditions is selected as the target analysis engine, wherein the preset screening conditions include: the remaining performance of the same type is higher than a preset performance threshold, or, the second-category analysis engine with the highest remaining performance of the same type among all second-category analysis engines in an idle state, and the remaining performance of the second-category analysis engine of the same type is used to represent the remaining performance of all analysis engines that load the same analysis algorithm as the second-category analysis engine.

5. The method according to any one of claims 1 to 3, characterized in that: After switching the analysis algorithm loaded by the target analysis engine to the target analysis algorithm, the method further includes: When there is an idle first-type analysis engine, the analysis algorithm loaded by the idle first-type analysis engine is switched to the analysis algorithm initially loaded by the target analysis engine.

6. A video analysis device, characterized in that: Applied to a video processing device, the video processing device includes multiple analysis engines, each analysis engine is pre-loaded with an analysis algorithm, and the device includes: A first determination module is configured to, after receiving an analysis task to be executed for a video, determine whether all first-type analysis engines are in a fully loaded state, wherein the first-type analysis engine is an analysis engine whose loaded analysis algorithm is a target analysis algorithm, and the target analysis algorithm is an analysis algorithm used to execute the analysis task to be executed; an engine selection module, configured to, if all analysis engines of the first type of analysis algorithm are in a fully loaded state, determine an idle second type of analysis engine as a target analysis engine, wherein the second type of analysis engine is an analysis engine whose loaded analysis algorithm is not the target analysis algorithm, and wherein the idle state means that the number of channels executing analysis tasks is 0; An algorithm switching module, configured to switch the target analysis engine's loaded analysis algorithm to the target analysis algorithm; A task allocation module, configured to execute the analysis task to be executed through the target analysis engine; The device further includes a second determination module for determining whether there is a second type analysis engine in an idle state; The task scheduling module is configured to schedule all analysis tasks being executed by at least one second-type analysis engine to second-type analysis engines other than the at least one second-type analysis engine if no second-type analysis engine in an idle state exists.

7. The device according to claim 6, characterized in that The task scheduling module is specifically configured to schedule, for each analysis task being executed by at least one second-type analysis engine, the analysis task to an analysis engine with the lowest load among the second-type analysis engines other than the at least one second-type analysis engine that are loaded with an analysis algorithm for executing the analysis task; The task allocation module is further configured to execute the to-be-executed analysis task by the first-category analysis engine with the lowest load if at least one first-category analysis engine is not in a fully loaded state; The engine selection module is specifically configured to select a second-category analysis engine that is in an idle state and meets preset screening conditions as a target analysis engine, wherein the preset screening conditions include: a second-category analysis engine having a similar residual performance higher than a preset performance threshold, or a second-category analysis engine having the highest similar residual performance among all second-category analysis engines in an idle state, wherein the similar residual performance of the second-category analysis engine is used to represent the residual performance of all analysis engines that have the same analysis algorithm loaded as the second-category analysis engine; The algorithm switching module is further configured to switch the analysis algorithm loaded by the idle first-type analysis engine to the analysis algorithm initially loaded by the target analysis engine when there is an idle first-type analysis engine.

8. A video processing device, characterized in that: The video processing device includes a plurality of analysis engines, a memory, and a processor, wherein each analysis engine is pre-loaded with an analysis algorithm; The memory is used to store computer programs; The processor is configured to implement the method steps according to any one of claims 1 to 5 when executing the program stored in the memory.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images