A frame rate adjustment detection method and device, a detection device, and a storage medium

Abstract

Embodiments of the present application provide a kind of detection frame rate adjustment method, device, detection equipment and storage medium, the method comprises: judge whether the number of continuous targetless frame of target video acquisition equipment is greater than preset frame number, and whether detection equipment is in high load state;In the number of continuous targetless frame greater than preset frame number, or the case where detection equipment is in high load state, according to the amount of current remaining computing resource of detection equipment, determine the maximum detection frame rate that detection equipment can execute currently;According to maximum detection frame rate, adjust the detection frame rate of the video that detection equipment is collected to target video acquisition equipment.For the video that target video acquisition equipment collects, detection equipment can adjust the detection frame rate of itself adaptively according to the image acquisition condition of video acquisition equipment and the load state of detection equipment, so the flexibility of detection equipment when detecting the video that video acquisition equipment collects is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video detection, and in particular to a detection frame rate adjusting method and device, a detection device and a storage medium. BACKGROUND

[0002] In many application scenarios, it is necessary to detect the video collected by a video collection device to realize target detection, target classification or early warning.

[0003] In the traditional video detection method, a fixed detection frame rate is used, which makes the flexibility of the detection device in detecting the video collected by the video collection device poor. Specifically, on the one hand, in actual applications, there is no target in a large number of video frames collected by the video collection device, but the detection device will detect the video frames according to the fixed detection frame rate regardless of whether there is a target in the video frames, resulting in unnecessary consumption of computing power. On the other hand, when the detection device is in a high load state, the detection device still detects at the fixed detection frame rate, which further increases the load pressure of the detection device, and even if a target is detected, the detection device is difficult to further process the detected target due to the lack of sufficient computing resources, so that the completed detection becomes invalid detection because it cannot be further processed. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a detection frame rate adjusting method, device, detection device and storage medium to improve the flexibility of the detection device in detecting the video collected by the video collection device. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present application provide a detection frame rate adjusting method applied to a detection device connected with at least one video collection device, and the method comprises:

[0006] determining whether the number of continuous non-target frames of a real-time recorded target video collection device is greater than a preset frame number, and determining whether the detection device is in a high load state, wherein the target video collection device is any video collection device in the at least one video collection device;

[0007] in the case that the number of continuous non-target frames is greater than the preset frame number or the detection device is in a high load state, determining the maximum detection frame rate currently executable by the detection device according to the amount of computing resources currently remaining in the detection device;

[0008] adjusting the detection frame rate corresponding to the target video collection device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video collection device is the detection frame rate of the detection device for the video collected by the target video collection device.

[0009] Optionally, the determining whether the detection device is in a high load state comprises:

[0010] calculating a first time consumption according to a current detection frame rate corresponding to the at least one video capture device, a preset single-frame detection time consumption and a preset time length, wherein the current detection frame rate corresponding to the at least one video capture device is a current detection frame rate of the detection device for a video captured by the at least one video capture device, and the first time consumption is a time length required by the detection device for detecting the video captured by the at least one video capture device within the preset time length;

[0011] calculating a frame rate difference between a preset detection frame rate and a current detection frame rate corresponding to the target video capture device, and calculating a product of the frame rate difference, the single-frame detection time consumption and the preset time length as a second time consumption, wherein the current detection frame rate corresponding to the target video capture device is a current detection frame rate of the detection device for a video captured by the target video capture device;

[0012] determining a total time consumption according to the first time consumption and the second time consumption, and determining whether the total time consumption is not greater than the preset time length;

[0013] if the total time consumption is greater than the preset time length, determining that the detection device is in a high load state;

[0014] if the total time consumption is not greater than the preset time length, determining that the detection device is not in a high load state.

[0015] Optionally, before the determining the total time consumption according to the first time consumption and the second time consumption, the method further comprises:

[0016] calculating a third time consumption according to a number of current to-be-processed tasks and a preset single-task processing time consumption, wherein the to-be-processed tasks are processing tasks to be executed by the detection device after completing video frame detection, and the third time consumption is a time length required by the detection device for completing all current to-be-processed tasks;

[0017] the determining the total time consumption according to the first time consumption and the second time consumption comprises:

[0018] determining a sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption.

[0019] Optionally, the determining the maximum detection frame rate currently executable by the detection device according to the amount of currently remaining computing resources of the detection device comprises:

[0020] Subtracting the second time consumption and the fourth time consumption from the total time consumption, a fifth time consumption is obtained, wherein the fourth time consumption is a time length required by the detection device to detect a video collected by the target video collection device within the preset time length at a current detection frame rate corresponding to the target video collection device;

[0021] A difference between the preset time length and the fifth time consumption is calculated as a residual calculation time length within the preset time length;

[0022] A ratio between the residual calculation time length and a target product is calculated, and a maximum detection frame rate currently executable by the detection device is determined according to the ratio, wherein the target product is a product of the preset time length and the single-frame detection time consumption.

[0023] Optionally, the number of continuous non-target frames of the target video collection device recorded in real time is recorded in the following manner:

[0024] In the process of detecting the video collected by the target video collection device, for each video frame, it is detected whether a target exists in the video frame, if the target exists in the video frame, the number of continuous non-target frames of the target video collection device is set to 0, and if the target does not exist in the video frame, the number of continuous non-target frames of the target video collection device is increased by 1.

[0025] Optionally, the detection of whether the target exists in the video frame comprises:

[0026] detecting whether the target exists in a target region of the video frame;

[0027] In the case that the target exists in the target region of the video frame, it is judged whether a confidence of the detected target is greater than a preset confidence threshold;

[0028] If the confidence of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

[0029] Optionally, before the detection frame rate corresponding to the target video collection device is adjusted according to the maximum detection frame rate, the method further comprises:

[0030] According to a preset corresponding relationship between a detection frame rate and a detection rate, it is judged whether a detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold;

[0031] If the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, the step of adjusting the detection frame rate corresponding to the target video collection device according to the maximum detection frame rate is performed;

[0032] If the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, the detection frame rate corresponding to the target video acquisition device is adjusted to a target detection frame rate, where the target detection frame rate is the minimum value of the detection frame rates corresponding to the corresponding relationship and having a detection rate not less than the preset detection rate threshold.

[0033] Optionally, the method further includes:

[0034] In a case where the number of continuous non-target frames is not greater than the preset frame number and the detection device is not in a high load state, the detection frame rate corresponding to the target video acquisition device is adjusted to a preset detection frame rate.

[0035] In a second aspect, an embodiment of the present application provides a detection frame rate adjustment apparatus applied to a detection device, the detection device being connected with at least one video acquisition device, and the apparatus including:

[0036] A first judgment module is configured to judge whether the number of continuous non-target frames of a target video acquisition device recorded in real time is greater than a preset frame number and whether the detection device is in a high load state, where the target video acquisition device is any video acquisition device in the at least one video acquisition device.

[0037] A maximum detection frame rate determination module is configured to determine a maximum detection frame rate currently executable by the detection device according to the amount of computing resources currently remaining in the detection device in a case where the number of continuous non-target frames is greater than the preset frame number or the detection device is in a high load state.

[0038] A detection frame rate adjustment module is configured to adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, where the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the detection device for the video collected by the target video acquisition device.

[0039] Optionally, the first judgment module includes:

[0040] A first time consumption determination unit is configured to calculate a first time consumption according to the current detection frame rate corresponding to the at least one video acquisition device, a preset single-frame detection time consumption, and a preset time length, where the current detection frame rate corresponding to the at least one video acquisition device is the current detection frame rate of the detection device for the video collected by the at least one video acquisition device, and the first time consumption is the time length required by the detection device to detect the video collected by the at least one video acquisition device within the preset time length.

[0041] a second time-consuming determining unit, configured to calculate a frame rate difference value between a preset detection frame rate and a current detection frame rate corresponding to the target video acquisition device, and calculate a product of the frame rate difference value, the single-frame detection time-consuming and the preset time length as a second time-consuming, wherein the current detection frame rate corresponding to the target video acquisition device is a current detection frame rate of a video collected by the detection device for the target video acquisition device;

[0042] a judging unit, configured to determine a total time-consuming according to the first time-consuming and the second time-consuming, and judge whether the total time-consuming is not greater than the preset time length; in a case where the total time-consuming is greater than the preset time length, determine that the detection device is in a high-load state; in a case where the total time-consuming is not greater than the preset time length, determine that the detection device is not in the high-load state.

[0043] Optionally, the first judging module further includes:

[0044] a third time-consuming determining unit, configured to calculate a third time-consuming according to a number of current to-be-processed tasks and a preset processing time-consuming of a single task, wherein the to-be-processed task is a processing task to be executed by the detection device after completing video frame detection, and the third time-consuming is a time length required by the detection device to complete all current to-be-processed tasks;

[0045] The judging unit is specifically configured to determine a sum of the first time-consuming, the second time-consuming and the third time-consuming as the total time-consuming.

[0046] Optionally, the maximum detection frame rate determining module includes:

[0047] a fifth time-consuming determining unit, configured to subtract the second time-consuming and a fourth time-consuming from the total time-consuming to obtain a fifth time-consuming, wherein the fourth time-consuming is a time length required by the detection device to detect a video collected by the target video acquisition device within the preset time length at the current detection frame rate corresponding to the target video acquisition device;

[0048] a remaining calculation time length determining unit, configured to calculate a difference value between the preset time length and the fifth time-consuming as a remaining calculation time length within the preset time length;

[0049] a maximum detection frame rate determining unit, configured to calculate a ratio between the remaining calculation time length and a target product, and determine a maximum detection frame rate currently executable by the detection device according to the ratio, wherein the target product is a product of the preset time length and the single-frame detection time-consuming.

[0050] Optionally, the number of continuous non-target frames of the target video acquisition device recorded in real time is recorded in the following manner:

[0051] In the process of detecting the video collected by the target video collection device, for each video frame, it is detected whether a target exists in the video frame. If the target exists in the video frame, the number of continuous target-free frames of the target video collection device is set to 0. If the target does not exist in the video frame, the number of continuous target-free frames of the target video collection device is increased by 1.

[0052] Optionally, the detection of whether the target exists in the video frame comprises: detection of whether the target exists in a target region of the video frame; in the case where the target exists in the target region of the video frame, judgment of whether the confidence of the detected target is greater than a preset confidence threshold; and if the confidence of the detected target is greater than the preset confidence threshold, determination that the target exists in the video frame.

[0053] Optionally, the apparatus further comprises:

[0054] The second judgment module is configured to judge, according to a preset corresponding relationship between detection frame rates and detection rates, whether a detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold; trigger execution of the detection frame rate adjustment module in the case where the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold; and adjust the detection frame rate corresponding to the target video collection device to a target detection frame rate in the case where the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, wherein the target detection frame rate is the minimum value of detection frame rates corresponding to detection rates not less than the preset detection rate threshold in the corresponding relationship.

[0055] Optionally, the detection frame rate adjustment module is further configured to adjust the detection frame rate corresponding to the target video collection device to a preset detection frame rate in the case where the number of continuous target-free frames is not greater than the preset frame number and the detection device is not in a high-load state.

[0056] In a third aspect, an embodiment of the present application provides a detection device, comprising:

[0057] A memory is configured to store a computer program.

[0058] A processor is configured to execute the program stored in the memory, and implement the method in any of the first aspect.

[0059] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method in any of the first aspect.

[0060] The embodiment of the present application has the following beneficial effects:

[0061] In the solution provided in this application embodiment, the detection device can determine whether the number of consecutive targetless frames recorded in real time by the target video acquisition device exceeds a preset number of frames, and whether the detection device is under high load. The target video acquisition device is any one of at least one video acquisition device connected to the detection device. When the number of consecutive targetless frames exceeds the preset number of frames, or when the detection device is under high load, the maximum executable detection frame rate is determined based on the remaining computing resources of the detection device. The detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, where the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the video acquired by the detection device for the target video acquisition device. Since the detection device can determine the maximum executable detection frame rate in real time based on its remaining computing resources when there are multiple consecutive targetless frames in the video acquired by the video acquisition device or when it is under high load, it can adaptively adjust its detection frame rate for the video acquired by the video acquisition device based on this maximum detection frame rate. This allows the detection frame rate to adapt to changes in the image acquisition situation of the video acquisition device and the load state of the detection device, greatly improving the flexibility of the detection device when detecting the video acquired by the video acquisition device.

[0062] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0064] Figure 1 This is a schematic flowchart of a frame rate adjustment detection method provided in an embodiment of this application;

[0065] Figure 2 For based on Figure 1 A flowchart illustrating a method for determining the load status of a detection device according to an embodiment shown;

[0066] Figure 3 For based on Figure 2 A schematic diagram illustrating a specific process for determining the load status of a detection device in the embodiment shown;

[0067] Figure 4 Based on Figure 1 A flowchart illustrating one method for determining the maximum detection frame rate in the embodiment shown.

[0068] Figure 5 A way of detecting whether a target exists in a video frame based on the embodiment shown in FIG. 1; Figure 1 A flowchart of a way of detecting whether a target exists in a video frame based on the embodiment shown in FIG. 1;

[0069] Figure 6 A specific flowchart of a frame rate adjustment method based on the embodiment shown in FIG. 1; Figure 1 A specific flowchart of a frame rate adjustment method based on the embodiment shown in FIG. 1;

[0070] Figure 7 A structural diagram of a frame rate adjustment device provided by the embodiment of the present application;

[0071] Figure 8 A structural diagram of a detection device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application belong to the scope of protection of the present application.

[0073] In order to improve the flexibility of the detection device when detecting the video collected by the video collection device, the present application provides a frame rate adjustment method, device, detection device, computer readable storage medium and computer program product. First, the frame rate adjustment method provided by the present application will be introduced.

[0074] The frame rate adjustment method provided by the present application can be applied to any detection device connected with at least one video collection device for detecting the video collected by the video collection device, for example, the detection device can be a computer, a server, a computing card, etc., which is not limited here. In order to describe clearly, the following is called a detection device.

[0075] As shown in FIG. 1, a frame rate adjustment method comprises: Figure 1

[0076] S101, judging whether the number of continuous non-target frames of the target video collection device recorded in real time is greater than a preset frame number, and judging whether the detection device is in a high load state. If one of the judgment results is yes, step S102 is executed.

[0077] Among them, the target video collection device is any video collection device in at least one video collection device.

[0078] S102, determining the maximum detection frame rate currently executable by the detection device according to the amount of computing resources currently remaining in the detection device. ​

[0079] S103, adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate.

[0080] The detection frame rate corresponding to the target video acquisition device is a detection frame rate of the detection device for a video collected by the target video acquisition device.

[0081] In the scheme provided by the embodiments of the present application, the detection device can determine whether the number of continuous non-target frames of the target video acquisition device recorded in real time is greater than the preset frame number, and determine whether the detection device is in a high load state, wherein the target video acquisition device is any video acquisition device in at least one video acquisition device connected to the detection device; in the case that the number of continuous non-target frames is greater than the preset frame number, or the detection device is in a high load state, the maximum detection frame rate currently executable by the detection device is determined according to the amount of computing resources currently remaining in the detection device; the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is a detection frame rate of the detection device for a video collected by the target video acquisition device. Since the detection device has continuous multiple non-target frames in the video collected by the video acquisition device or is in a high load state, the maximum detection frame rate currently executable by the detection device can be determined in real time according to the amount of computing resources currently remaining in the detection device, and the detection frame rate of the detection device for the video collected by the video acquisition device is adaptively adjusted according to the maximum detection frame rate. Therefore, the detection frame rate can be adaptively changed with the image acquisition condition of the video acquisition device and the load state of the detection device, greatly improving the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0082] First, the application scenario of the detection frame rate adjustment method provided by the embodiments of the present application is introduced.

[0083] The detection device is connected to at least one video image acquisition device, and for each video image acquisition device, the detection device can detect the video collected by the video image acquisition device at a certain frame rate after receiving the video collected by the video image acquisition device, to determine whether a target exists in the video picture of the video image acquisition device.

[0084] For example, the detection device is connected to a camera A, and the camera A returns 60 video frames per second to the detection device. The detection device extracts 10 video frames from the 60 video frames per second returned by the camera to the detection device for detection at the current detection frame rate of 10 frames per second, to determine whether a target exists in the video picture of the camera A.

[0085] The detection device can record the number of continuous non-target frames of the video image collection device in the process of detecting the video collected by the video image collection device. Specifically, for each extracted video frame, the detection device can detect whether a target exists in the video frame. If a target exists in the video frame, the number of continuous non-target frames of the video collection device for the video frame is set to 0. If a target does not exist in the video frame, the number of continuous non-target frames of the target video collection device for the video frame is increased by 1.

[0086] For example, the detection device extracts 5 video frames from the 60 video frames returned by camera A at the first second for detection, which are video frame a, video frame b, video frame c, video frame d, and video frame e in sequence. When the detection device detects that no target exists in the video frame a, the number of continuous non-target frames of camera A is recorded as 1. When no target exists in the video frame b, the number is increased by 1, i.e., the number of continuous non-target frames of camera A is recorded as 2. When no target exists in the video frame c, the number is increased by 1, i.e., the number of continuous non-target frames of camera A is recorded as 3. When a target exists in the video frame d, the number of continuous non-target frames of camera A is set to 0. When a target exists in the video frame e, the number of continuous non-target frames of camera A is set to 0.

[0087] The detection frame rate adjustment method provided by the embodiments of the present application will be introduced below based on the above application scenario.

[0088] For any video collection device (hereinafter referred to as a target video collection device) of at least one video collection device connected to the detection device, the detection device can perform step S101 before detecting the video collected by the target video collection device, i.e., determining whether the number of continuous non-target frames of the target video collection device recorded in real time through the above method is greater than a preset frame number, i.e., determining whether there is no target in the video collected by the target video collection device for a long time. If the number of continuous non-target frames is greater than the preset frame number, it means that the detection frame rate can be adaptively adjusted at this time to avoid waste of detection resources. In addition, the detection device can also determine whether the detection device is in a high load state by detecting the task currently to be performed by the detection device. If the detection device is in a high load state, it means that the detection frame rate can be adaptively adjusted at this time to reduce the computing resources occupied by detection.

[0089] It should be noted that the above two determination steps can be executed simultaneously, or one of the determination steps can be executed first, and the other determination step can be executed when the determination result of the first determination step is negative. No specific limitation is made herein.

[0090] When the determination results of the above two determination steps are both no, that is, when the number of continuous non-target frames is not greater than the preset frame number and the detection device is not in a high load state, it indicates that the target has appeared in the video collected by the target video collection device in the recent period, and the current computing resource of the detection device is sufficient. On this basis, the detection device can adjust the detection frame rate corresponding to the target video collection device to a preset detection frame rate. The preset detection frame rate can be a detection frame rate that can meet the requirements of the detection rate and the load state of the detection device, which can be preset in the detection device based on the characteristics that the detection frame rate and the detection rate are positively correlated and the detection frame rate and the load are positively correlated in actual application.

[0091] When the determination result of any one of the above determination steps is yes, that is, when the number of continuous non-target frames is greater than the preset frame number, or the detection device is in a high load state, the detection device can further perform step S102 to determine the maximum detection frame rate that can be executed by the detection device according to the current remaining computing resource of the detection device, and then adjust the detection frame rate corresponding to the target video collection device according to the maximum detection frame rate. The specific manner of determining the maximum detection frame rate will be described in detail in subsequent embodiments, which will not be described here.

[0092] For the manner of adjusting the detection frame rate corresponding to the target video collection device according to the maximum detection frame rate, the detection device can be implemented by but not limited to the following manners:

[0093] In the first implementation, the detection device can directly adjust the detection frame rate corresponding to the target video collection device to the maximum detection frame rate.

[0094] In the second implementation, the user can preset a detection frame rate in the detection device based on the characteristics that the detection frame rate and the detection rate are positively correlated and the detection frame rate and the load are positively correlated in actual application, which can meet the requirements of the detection rate and the load state of the detection device. In this case, the detection device can first determine whether the maximum detection frame rate is greater than the preset detection frame rate.

[0095] If the maximum detection frame rate is not greater than the preset detection frame rate, it indicates that the maximum detection frame rate will reduce the detection rate compared with the preset detection frame rate, but the maximum detection frame rate is the maximum detection frame rate that can be executed by the detection device at present. Therefore, in order to avoid overloading operation of the detection device, the detection frame rate corresponding to the target video collection device can be adjusted to the maximum detection frame rate.

[0096] If the maximum detection frame rate is greater than the preset detection frame rate, it indicates that both the maximum detection frame rate and the preset detection frame rate can meet the requirement of the detection rate, but the maximum detection frame rate will increase the load of the detection device compared with the preset detection frame rate. Therefore, in the case that both of them can meet the requirement of the detection rate, in order to reduce the load of the detection device as much as possible, the detection device can adjust the detection frame rate corresponding to the target video acquisition device to the preset detection frame rate.

[0097] In the scheme provided by the embodiments of the present application, when the detection device has continuous multiple target-free frames in the video collected by the video acquisition device or is in a high load state, the maximum detection frame rate that can be currently executed can be determined in real time according to the current remaining computing resource amount of the detection device, and the detection frame rate of the detection device for the video collected by the video acquisition device is adaptively adjusted according to the maximum detection frame rate. Therefore, the detection frame rate can be adaptively changed with the image collection condition of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video collected by the video acquisition device. In addition, by using the above method of flexibly adjusting the detection frame rate, the computing resources of the detection device can be fully utilized to improve the number of image acquisition devices that can be connected by the detection device, and / or improve the complexity of the detection algorithm, so that the detection device can process the video based on the algorithm with higher complexity to more accurately process the video, and / or reduce the situation that the detection device is difficult to further process the detected target due to lack of sufficient computing resources, so that the completed detection becomes invalid detection due to the inability to be further processed.

[0098] As an embodiment of the present application, as shown in Figure 2 When the detection device judges whether it is in a high load state, steps S201-S205 can be used:

[0099] S201, calculating a first time consumption according to the current detection frame rate corresponding to at least one video acquisition device, the preset single-frame detection time consumption and the preset time length.

[0100] When the detection device judges whether it is in a high load state, it can first calculate the time consumption of the detection device in detecting the video collected by the video acquisition device through step S201. Specifically:

[0101] For each of the at least one video capture device connected to the detection device, a first time length required by the detection device to complete detection on a video frame returned by the video capture device in a unit time length can be determined by calculating the product of a current detection frame rate of the video capture device (i.e., a current detection frame rate of the video captured by the detection device for the video capture device) and a preset single-frame detection time consumption. A second time length required by the detection device to complete detection on a video frame returned by the video capture device in a preset time length can be determined by calculating the product of the first time length and the preset time length. Finally, the second time lengths corresponding to the at least one video capture device connected to the detection device are accumulated, and a time length required by the detection device to detect the video captured by the at least one video capture device in the preset time length (first time consumption) can be obtained.

[0102] The preset time length can be a maximum time delay of the detection device when processing the video, can be a unit time length, or can be set according to actual use requirements, which is not limited here.

[0103] In an implementation manner, the detection device can calculate the first time consumption based on the following formula:

[0104]

[0105] wherein, represents the preset time length, represents the number of video capture devices connected to the detection device, represents the single-frame detection time consumption, represents a current detection frame rate of the video captured by the detection device for the i-th video capture device.

[0106] In S202, a frame rate difference between the preset detection frame rate and a current detection frame rate corresponding to the target video capture device is calculated, and the product of the frame rate difference, the single-frame detection time consumption, and the preset time length is calculated as a second time consumption.

[0107] Because in the case that the number of continuous non-target frames is not greater than the preset number of frames and the detection device is not in a high load state, the detection device will adjust the detection frame rate corresponding to the target video capture device to the preset detection frame rate, when determining the load state of the detection device, the time consumption of the detection device to detect the video captured by the target video capture device at the preset detection frame rate needs to be considered.

[0108] ​​For example, the number of continuous non-target frames is not greater than the preset frame number, the current detection frame rate corresponding to the camera A is 8 frames per second, and the preset detection frame rate is 10 frames per second. Assuming that when determining the load state of the detection device, the time consumption of the detection device in detecting the video collected by the target video collection device at a detection frame rate of 8 frames per second is considered, it is determined that the detection device is not in a high load state.

[0109] Then, because the number of continuous non-target frames is not greater than the preset frame number, and the detection device is not in a high load state, the detection device adjusts the detection frame rate corresponding to the target video collection device to 10 frames per second. However, after the adjustment, the detection device needs to extract 2 more frames per second for detection on the video collected by the target video collection device, and the detection work of the 2 more frames per second may cause the detection device to be in a high load state. Therefore, when determining the load state of the detection device, the time consumption of the detection device in detecting the video collected by the target video collection device at a detection frame rate of 10 frames per second needs to be considered.

[0110] Further, because the first time consumption includes the time consumption of the detection device in detecting the video collected by the target video collection device at the current detection frame rate corresponding to the target video collection device, when considering the time consumption of the detection device in detecting the video collected by the target video collection device at the preset detection frame rate, only the time consumption of the detection device in detecting the video collected by the target video collection device at the frame rate difference between the preset detection frame rate and the current detection frame rate (second time consumption) needs to be additionally determined.

[0111] Therefore, the detection device can calculate the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video collection device, and calculate the product of the frame rate difference, the single-frame detection time consumption and the preset time length as the second time consumption.

[0112] In an implementation, the detection device can calculate the second time consumption based on the following formula:

[0113] ;

[0114] wherein, represents the preset time length, represents the preset detection frame rate, represents the current detection frame rate of the detection device for the video collected by the target video collection device, represents the preset detection frame rate, represents the single-frame detection time consumption.

[0115] In S203, the total time consumption is determined according to the first time consumption and the second time consumption, and it is judged whether the total time consumption is not greater than the preset time length. If the judgment result is no, step S204 is executed, and if the judgment result is yes, step S205 is executed.

[0116] After determining the first time consumption and the second time consumption according to step S201 and step S202, the detection device can determine the total time consumption according to the first time consumption and the second time consumption, and determine whether the total time consumption is greater than the preset time length, that is, determine whether the time length required by the detection device for detecting the video collected by the at least one video collection device within the preset time length is greater than the preset time length.

[0117] As for the determination manner of the total time consumption, when the processing of the video by the detection device only includes detecting whether the target exists in the video frame, the total time consumption of the processing of the video by the detection device is the sum of the first time consumption and the second time consumption, in which case, the detection device can sum the first time consumption and the second time consumption, and determine the sum as the total time consumption. When the processing of the video by the detection device includes other processing tasks in addition to detecting whether the target exists in the video frame, the calculation manner of the total time consumption will be described in detail in subsequent embodiments, which will not be described here.

[0118] In an implementation manner, the detection device can calculate the total time consumption based on the following formula:

[0119] ;

[0120] In the formula, T represents the preset time length, F represents the preset detection frame rate, F represents the current detection frame rate of the video collected by the detection device for the i-th video collection device, and T represents the single-frame detection time consumption.

[0121] S204, determining that the detection device is in a high-load state.

[0122] If the total time consumption is greater than the preset time length, it indicates that for the video collected by the at least one video collection device within the preset time length, the detection device cannot complete the detection within the preset time length, and the detection task within the current preset time length will be delayed to continue to be executed in the next preset time length, thereby causing the execution of subsequent detection tasks to be delayed, and the time length of the delay is getting longer and longer. In this case, the detection device can determine that it is in a high-load state.

[0123] S205, determining that the detection device is not in a high-load state.

[0124] ​​​​​​If the total time consumption is not greater than the preset time length, it is indicated that the detection device can complete detection on the video collected by the at least one video collection device within the preset time length, and the detection task in the current preset time length will not be delayed to the next preset time length for execution. In this case, the detection device can determine that it is not in a high load state.

[0125] In the scheme provided by the embodiments of the present application, the first time consumption is determined by the current detection frame rate of each video collection device, the single-frame detection time consumption, and the preset time length, which can accurately evaluate the time consumption of the detection device on the existing detection task. On this basis, the second time consumption is determined by the frame rate difference between the preset detection frame rate and the current detection frame rate, the single-frame detection time consumption, and the preset time length, which further considers the additional load influence caused by the frame rate adjustment, so that the determination of the total time consumption is more scientific and reasonable. Finally, the comparison between the total time consumption and the preset time length can accurately determine the load state of the detection device.

[0126] As an implementation manner of the embodiments of the present application, as shown in Figure 3 Before the detection device executes the step S203, the detection device can further execute a step S301:

[0127] In the step S301, a third time consumption is calculated according to the number of current to-be-processed tasks and the preset processing time consumption of a single task.

[0128] When the processing of the detection device on the video includes not only detecting whether there is a target in the video frame but also other processing tasks after detection, the time consumption of the processing task (to-be-processed task) to be executed by the detection device after completing the detection of the video frame also needs to be further considered when determining the load state of the detection device, and specifically:

[0129] Because the detection task is continuously generated according to the frame rate, new detection tasks will be added every second, so when calculating the time consumption corresponding to the detection task, the preset time length needs to be multiplied. However, compared with this, the to-be-processed task does not necessarily occur every second, and it is related to the detected target. When the target is detected from the video frame, the corresponding to-be-processed task is generated.

[0130] Therefore, when determining the current load state of the detection device, only the current to-be-processed task needs to be considered. On this basis, for each video collection device connected to the detection device among the at least one video collection device, the number of to-be-processed tasks corresponding to the video collection device and the preset processing time consumption of a single task are calculated, so as to determine the third time length required by the detection device to complete the to-be-processed tasks corresponding to the video collection device. Then, the third time lengths corresponding to the at least one video collection device connected to the detection device are accumulated, so as to obtain the time length (third time consumption) required by the detection device to complete all the to-be-processed tasks.

[0131] In an embodiment, the detection device can calculate the third time consumption based on the following formula:

[0132] ;

[0133] wherein, represents the number of video capture devices connected to the detection device, represents the number of tasks to be processed corresponding to the i-th video capture device, represents the number of tasks to be processed corresponding to the i-th video capture device, represents the processing time consumption of a single task.

[0134] It should be noted that the type of task to be processed can be set according to actual use requirements, for example, the task to be processed can be a classification task, a segmentation task, an event judgment task, etc., and the type of task to be processed is not specifically limited in the present application. In addition, when the processing of the detection device on the video includes not only detecting whether the target exists in the video frame, but also other processing tasks after detection, the above-mentioned preset time length can be the maximum delay of the detection device when processing the video.

[0135] Correspondingly, when the detection device executes the above-mentioned step S203, it can be specifically implemented by step S302:

[0136] S302, determining the sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption, and determining whether the total time consumption is not greater than the preset time length.

[0137] After the detection device determines the first time consumption and the second time consumption according to steps S201 and S202, and determines the third time consumption through step S301, the detection device can sum the first time consumption, the second time consumption and the third time consumption, and determine the sum of the three as the total time consumption. And determine whether the total time consumption is greater than the preset time length, that is, determine whether the time length required by the detection device to detect the video captured by at least one video capture device within the preset time length is greater than the preset time length.

[0138] In an embodiment, the detection device can calculate the total time consumption based on the following formula:

[0139] ;

[0140] wherein, represents the preset time length, represents the preset detection frame rate, represents the current detection frame rate of the video captured by the detection device for the i-th video capture device, represents the current detection frame rate of the video captured by the detection device for the i-th video capture device, represents the single-frame detection time consumption, represents the number of video capture devices connected to the detection device, represents the i-th video capture device. a number of to-be-processed tasks corresponding to the video capture device, representing a processing time consumption of a single task.

[0141] In the scheme provided by the embodiments of the present application, the calculation of the third time consumption is further introduced, and the time consumption of other processing tasks after the detection device completes the detection of the video frame is fully considered, and the time evaluation of only the detection of the video frame is no longer limited. The comprehensive consideration makes the judgment of the load state of the detection device more accurate, and can more truly reflect the actual load state of the device in the entire video processing process.

[0142] As an implementation manner of the embodiments of the present application, as shown in Figure 4 When the detection device executes step S103, steps S401-S403 can be specifically implemented:

[0143] S401, subtracting the second time consumption and the fourth time consumption from the total time consumption to obtain a fifth time consumption.

[0144] S402, calculating a difference value between the preset time length and the fifth time consumption as a remaining calculation time length within the preset time length.

[0145] Because the total time consumption includes the time length (the fourth time consumption) required by the detection device to detect the video captured by the target video capture device within the preset time length at the current detection frame rate corresponding to the target video capture device, and the time consumption (the second time consumption) of detecting the video captured by the target video capture device within the preset time length at the frame rate difference between the preset detection frame rate and the current detection frame rate. Therefore, the detection device can first subtract the second time consumption and the fourth time consumption from the total time consumption to obtain the fifth time consumption.

[0146] The total time consumption is the total time length required by the detection device to process the video captured by at least one video capture device within the preset time length, the fourth time consumption is the time consumption of detecting the video captured by the target video capture device within the preset time length at the current detection frame rate corresponding to the target video capture device, and the second time consumption is the additional detection time consumption caused by the frame rate adjustment for the target video capture device.

[0147] When the fifth time consumption is obtained by subtracting the second time consumption and the fourth time consumption from the total time consumption, it is actually the time used for other tasks (such as the detection task of other video capture devices, or the detection task of other video capture devices and other tasks of non-detection links) within the preset time length, in addition to the detection time consumption related to the target video capture device.

[0148] The preset duration is a fixed time period. Subtracting the fifth time consumption from the preset duration gives the remaining time after deducting the time already used (i.e., the fifth time consumption) within the preset duration. This is the remaining calculation time within the preset duration. It can intuitively reflect how much time is available within the preset duration to detect the video captured by the target video acquisition device under the current load state of the detection equipment.

[0149] S403, calculate the ratio between the remaining computation time and the target product, and determine the maximum detection frame rate that the detection device can currently execute based on the ratio.

[0150] As mentioned earlier, the remaining calculation time reflects how much time is left within the preset time to detect the video captured by the target video acquisition device. The first ratio between the remaining calculation time and the preset time reflects how much time is left within the unit time to detect the video captured by the target video acquisition device. Furthermore, the second ratio between the first ratio and the single-frame detection time reflects the maximum number of video frames that can be detected within the unit time (i.e., the maximum detection frame rate).

[0151] Therefore, after obtaining the remaining computation time, the detection device can calculate the ratio between its remaining computation time and the target product (i.e., the product of the preset time and the time consumed per frame detection). The maximum detection frame rate that the detection device can currently execute must not exceed this ratio. For example, after obtaining this ratio, the detection device can round it down to obtain the maximum detection frame rate that the detection device can currently execute.

[0152] In one implementation, when the total time is through Figure 2 In the illustrated embodiment, when the detection device performs steps S401-S403 as described above, the conditions that the maximum detection frame rate must meet can be determined using the following formula:

[0153] ;

[0154] in, The testing equipment is for the first The current executable detection frame rate of the video captured by the video acquisition equipment. Indicates the preset duration. Indicates the time taken for single-frame detection. Indicates that the testing equipment is for the first The current detection frame rate of the video captured by the video acquisition equipment. This indicates the number of video capture devices connected to the detection device.

[0155] In another implementation, when the total time is through Figure 3In the case shown in the embodiment, when the detection device performs the steps S401-S403, the condition required to be met by the maximum detection frame rate can be determined by the following formula:

[0156]

[0157] wherein, the first video acquisition device is the target video acquisition device, the maximum detection frame rate of the detection device currently executable for the video collected by the first video acquisition device, denotes the preset time length, denotes the single-frame detection time consumption, denotes the current detection frame rate of the detection device for the video collected by the first video acquisition device, denotes the number of video acquisition devices connected to the detection device, denotes the number of tasks to be processed by the detection device for the video collected by the first video acquisition device, denotes the processing time consumption of a single task. In the scheme provided by the embodiments of the present application, the second time consumption and the fourth time consumption related to the detection of the target video acquisition device are peeled off from the total time consumption to obtain the fifth time consumption, and then the remaining calculation time length within the preset time length is calculated, and then the maximum detection frame rate is determined based on the ratio of the remaining calculation time length to the target product, so that the detection frame rate can be dynamically adjusted according to the current load of the detection device, so that the detection device can fully utilize the remaining calculation resources when facing complex and changeable video acquisition and processing tasks, avoid resource idling and waste, and effectively prevent the load from being overloaded or other tasks from being completed on time due to excessive allocation of resources to the target video acquisition device.

[0158] As an implementation manner in the embodiments of the present application, as shown in when the detection device detects whether the target exists in the video frame, the steps S501-S504 can be implemented:

[0159] Figure 5 S501, whether the target exists in the target region of the video frame. If the detection result is yes, step S502 is performed, and if the detection result is no, step S504 is performed.

[0160] S501, whether the target exists in the target region of the video frame. If the detection result is yes, step S502 is performed, and if the detection result is no, step S504 is performed.

[0161] ​​It is necessary to set the target region in advance in the video processing process, so that the detection device only detects the target region in the video frame. Specifically, on the one hand, the information covered by the video frame is diverse, and the target concerned by the user only appears in a specific local range. For example, for the picture obtained by the camera for shooting vehicles on the road, the vehicle target will most likely appear in the picture area corresponding to the road, and there is almost no vehicle in the sky, roof and other picture areas. If the entire video frame is comprehensively detected, a large amount of computing resources and time cost will be consumed, and the detection efficiency will be reduced. On the other hand, the target at the edge of the picture is easy to cause feature loss or be disturbed by distortion, which will greatly affect the accuracy and reliability of the detection result. By setting the target region, the adverse effects of these edge regions can be effectively avoided, and the detection can be focused on the core region where the target is more likely to be complete and the features are stable, thereby significantly improving the accuracy and effectiveness of the detection.

[0162] Therefore, when the detection device detects whether the target exists in the video frame, it can only detect whether the target exists in the target region of the video frame. The target can be detected by a target detection algorithm or a pre-trained target detection model, which is not limited here.

[0163] S502, determine whether the confidence of the detected target is greater than the preset confidence threshold. If the result is yes, execute step S503, and if the result is no, execute step S504.

[0164] When the detection device detects the target in the target region of the video frame, it can further determine the confidence of the target. Further, the detection device can determine whether the confidence of the detected target is greater than the preset confidence threshold to determine whether the target indeed exists in the video frame.

[0165] S503, determine that the target exists in the video frame.

[0166] If the target region of the video frame has a target with a confidence greater than the preset confidence threshold, it means that the target is likely to exist in the video frame. In this case, the detection device can determine that the target exists in the video frame.

[0167] S504, determine that the target does not exist in the video frame.

[0168] If the target region of the video frame does not have a target, or the confidence of the target in the target region of the video frame is not greater than the preset confidence threshold, it means that the target is likely to not exist in the video frame. In this case, the detection device can determine that the target does not exist in the video frame.

[0169] In the scheme provided by the embodiments of the present application, the unnecessary calculation amount is greatly reduced through the limited target area detection, the detection efficiency is improved, the detection device can process more video frames in a shorter time. Secondly, the confidence of the detected target is judged, the false detection rate is effectively reduced, and the detection result is more reliable.

[0170] As an embodiment of the present application, as shown in Figure 6 Before step S103 is executed, the detection device can also execute step S601:

[0171] S601, according to the correspondence between the preset detection frame rate and the detection rate, it is judged whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold. If the judgment result is no, step S103 is executed, if the judgment result is yes, step S602 is executed.

[0172] The detection device can pre-store the correspondence between the detection frame rate and the detection rate, wherein the correspondence includes each detection frame rate, and the probability of detecting the target in the video frame (i.e. detection accuracy) when the detection device detects the video collected by the video collection device at each detection frame rate.

[0173] For example, the correspondence between the detection frame rate and the detection rate can be as shown in the following table, wherein the detection frame rate S represents detecting S video frames per second, the detection frame rate S-2 represents detecting (S-2) video frames per second, and the detection frame rate S-8 represents detecting (S-8) video frames per second.

[0174]

[0175] After determining the maximum detection frame rate that the detection device can currently execute, the detection device can determine the detection rate corresponding to the maximum detection frame rate according to the preset correspondence between the detection frame rate and the detection rate, and judge whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, to determine whether the adjusted detection frame rate can meet the requirement of the detection rate when the detection frame rate corresponding to the target video collection device is adjusted according to the maximum detection frame rate.

[0176] If the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, it means that the adjusted detection frame rate can meet the requirement of the detection rate when the detection frame rate corresponding to the target video collection device is adjusted according to the maximum detection frame rate, so in this case, the detection device can execute step S103.

[0177] S602, the detection frame rate corresponding to the target video collection device is adjusted to the target detection frame rate.

[0178] If the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, it indicates that when the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, the adjusted detection frame rate cannot meet the requirement of the detection rate. In this case, the detection device can determine, from the preset corresponding relationship between the detection frame rate and the detection rate, a detection frame rate corresponding to a detection rate not less than the preset detection rate threshold as a candidate detection frame rate.

[0179] Because the detection frame rate and the detection rate are in a positive correlation relationship, the detection frame rate and the load are in a positive correlation relationship, the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, and the detection rate corresponding to the candidate detection frame rate is not less than the preset detection rate threshold, the candidate detection frame rate is greater than the maximum detection frame rate currently executable by the detection device. Therefore, on the basis that the candidate detection frame rate is already greater than the maximum detection frame rate currently executable by the detection device, in order to reduce the load of the detection device as much as possible, the detection device can determine the minimum value in the candidate detection frame rate as the target detection frame rate, and then adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate.

[0180] For example, the preset detection rate threshold is 0.7f, and then the detection device determines the candidate detection frame rate corresponding to a detection rate not less than the preset detection rate threshold from the preset corresponding relationship between the detection frame rate and the detection rate, which can be shown in the following table. The minimum value in each detection frame rate in the table is S-4, and therefore the detection device can adjust the detection frame rate corresponding to the target video acquisition device to S-4.

[0181]

[0182] In the scheme provided in the embodiments of the application, by pre-storing the corresponding relationship between the detection frame rate and the detection rate, after the maximum detection frame rate is determined, whether the detection rate corresponding to the maximum detection frame rate meets the preset detection rate requirement can be judged according to the corresponding relationship. When the requirement is met, the detection frame rate is adjusted according to the maximum detection frame rate. When the requirement is not met, the minimum detection frame rate meeting the preset detection rate threshold requirement is determined as the detection frame rate corresponding to the video acquisition device. In this way, the detection rate requirement can be met, and the positive correlation relationship between the detection frame rate and the load is fully considered. On the premise of guaranteeing the detection rate requirement, the load pressure of the detection device is reduced as much as possible.

[0183] Corresponding to the detection frame rate adjustment method described above, the embodiments of the application further provide a detection frame rate adjustment device. The detection frame rate adjustment device provided in the embodiments of the application is introduced as follows.

[0184] As Figure 7 described, a detection frame rate adjustment device is applied to a detection device connected with at least one video acquisition device. The device can include:

[0185] The first determination module 701 is configured to determine whether the number of continuous non-target frames of the target video acquisition device recorded in real time is greater than a preset frame number, and determine whether the detection device is in a high load state, wherein the target video acquisition device is any one of at least one video acquisition device.

[0186] The maximum detection frame rate determination module 702 is configured to determine a maximum detection frame rate currently executable by the detection device according to the amount of currently remaining computing resources of the detection device in a case where the number of continuous non-target frames is greater than the preset frame number, or the detection device is in the high load state.

[0187] The detection frame rate adjustment module 703 is configured to adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the detection device for the video collected by the target video acquisition device.

[0188] In the scheme provided by the embodiments of the present application, the detection device can determine whether the number of continuous non-target frames of the target video acquisition device recorded in real time is greater than a preset frame number, and determine whether the detection device is in a high load state, wherein the target video acquisition device is any one of at least one video acquisition device connected to the detection device; in a case where the number of continuous non-target frames is greater than the preset frame number, or the detection device is in the high load state, a maximum detection frame rate currently executable by the detection device is determined according to the amount of currently remaining computing resources of the detection device; and the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the detection device for the video collected by the target video acquisition device. Since the detection device can determine the maximum detection frame rate currently executable according to the amount of currently remaining computing resources of the detection device when there are continuous multiple non-target frames in the video collected by the video acquisition device or the detection device is in a high load state, and adaptively adjust the detection frame rate of the detection device for the video collected by the video acquisition device according to the maximum detection frame rate, the detection frame rate can be adaptively changed with the image collection condition of the video acquisition device and the load state of the detection device, thereby greatly improving the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0189] As an implementation manner of the embodiments of the present application, the first determination module 701 can include:

[0190] The first time consumption determining unit is configured to calculate a first time consumption according to a preset detection frame rate, a preset single-frame detection time consumption and a preset time length, wherein the preset detection frame rate is a detection frame rate of the detection device, the preset single-frame detection time consumption is a single-frame detection time consumption of the detection device, and the preset time length is a preset time length.

[0191] The second time consumption determining unit is configured to calculate a frame rate difference between the preset detection frame rate and a current detection frame rate corresponding to the target video acquisition device, and calculate a product of the frame rate difference, the single-frame detection time consumption and the preset time length as a second time consumption, wherein the current detection frame rate corresponding to the target video acquisition device is a current detection frame rate of the detection device for a video collected by the target video acquisition device.

[0192] The judging unit is configured to determine a total time consumption according to the first time consumption and the second time consumption, and determine whether the total time consumption is greater than the preset time length; in a case where the total time consumption is greater than the preset time length, it is determined that the detection device is in a high-load state; and in a case where the total time consumption is not greater than the preset time length, it is determined that the detection device is not in the high-load state.

[0193] As an implementation manner of the embodiment of the present application, the first judging module 701 can further include:

[0194] The third time consumption determining unit is configured to calculate a third time consumption according to a preset single-task processing time consumption and a number of current to-be-processed tasks, wherein the to-be-processed task is a processing task to be executed by the detection device after the video frame detection is completed, and the third time consumption is a time length required by the detection device to complete all current to-be-processed tasks.

[0195] The judging unit is specifically configured to determine a sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption.

[0196] As an implementation manner of the embodiment of the present application, the maximum detection frame rate determining module 702 can include:

[0197] The fifth time consumption determining unit is configured to subtract the second time consumption and a fourth time consumption from the total time consumption to obtain a fifth time consumption, wherein the fourth time consumption is a time length required by the detection device to detect a video collected by the target video acquisition device within the preset time length at the current detection frame rate corresponding to the target video acquisition device.

[0198] The remaining calculation time length determining unit is configured to calculate a difference between the preset time length and the fifth time consumption as a remaining calculation time length within the preset time length.

[0199] The maximum detection frame rate determination unit is configured to calculate a ratio between the remaining calculation duration and a target product, and determine a maximum detection frame rate currently executable by the detection device according to the ratio, wherein the target product is a product of a preset duration and a single-frame detection time consumption.

[0200] As an implementation of the embodiment of the present application, the number of continuous non-target frames of the target video acquisition device recorded in real time can be calculated in the following manner:

[0201] In the process of detecting the video acquired by the target video acquisition device, for each video frame, it is detected whether the target exists in the video frame. If the target exists in the video frame, the number of continuous non-target frames of the target video acquisition device is set to 0, and if the target does not exist in the video frame, the number of continuous non-target frames of the target video acquisition device is increased by 1.

[0202] As an implementation of the embodiment of the present application, the detection of whether the target exists in the video frame can include:

[0203] detecting whether the target exists in the target region of the video frame;

[0204] in the case that the target exists in the target region of the video frame, judging whether the confidence of the detected target is greater than a preset confidence threshold;

[0205] if the confidence of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

[0206] As an implementation of the embodiment of the present application, the apparatus can further include:

[0207] The second judgment module is configured to judge whether the detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold according to a corresponding relationship between the preset detection frame rate and the detection rate, trigger the execution of the detection frame rate adjustment module 703 in the case that the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, and adjust the detection frame rate corresponding to the target video acquisition device to a target detection frame rate in the case that the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, wherein the target detection frame rate is the minimum value of the detection frame rates corresponding to the detection rates not less than the preset detection rate threshold in the corresponding relationship.

[0208] As an implementation of the embodiment of the present application, the detection frame rate adjustment module 703 can also be configured to adjust the detection frame rate corresponding to the target video acquisition device to a preset detection frame rate in the case that the number of continuous non-target frames is not greater than a preset frame number and the detection device is not in a high load state.

[0209] The embodiment of the present application also provides a detection device, as shown in the following figure: Figure 8 The detection device includes:

[0210] a memory 801, configured to store a computer program;

[0211] a processor 802, configured to execute the program stored in the memory 801, and implement the frame rate detection method in any of the above embodiments.

[0212] In the scheme provided by the embodiments of the present application, the detection device can determine whether the number of continuous non-target frames of the target video acquisition device recorded in real time is greater than a preset frame number, and determine whether the detection device is in a high load state, wherein the target video acquisition device is any one of at least one video acquisition device connected to the detection device; in the case that the number of continuous non-target frames is greater than the preset frame number, or the detection device is in a high load state, the maximum detection frame rate currently executable by the detection device is determined according to the amount of computing resources currently remaining in the detection device; and the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the detection device for the video acquired by the target video acquisition device. Since the detection device has continuous non-target frames in the video acquired by the video acquisition device or is in a high load state, the maximum detection frame rate currently executable by the detection device can be determined in real time according to the amount of computing resources currently remaining in the detection device, and the detection frame rate of the detection device for the video acquired by the video acquisition device is adaptively adjusted according to the maximum detection frame rate. Therefore, the detection frame rate can be adaptively changed according to the image acquisition condition of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video acquired by the video acquisition device.

[0213] The detection device can further include a communication bus and / or a communication interface, and the processor 802, the communication interface and the memory 801 can complete mutual communication through the communication bus.

[0214] The communication bus of the detection device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0215] The communication interface is used for communication between the detection device and other devices.

[0216] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0217] The processor described above can be a general processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0218] In yet another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the frame rate detection method in any of the above embodiments.

[0219] In yet another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to perform the frame rate detection method in any of the above embodiments.

[0220] In the embodiments described above, all or some of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or some of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into and executed by a computer, all or some of the processes or functions according to the embodiments described in the specification are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a solid state disk (SSD) and the like.

[0221] It should be noted that, in this document, the terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0222] Each of the embodiments in the specification is described in a related manner, and the same or similar parts between each of the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device, detection device, computer readable storage medium and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0223] The above merely provides the preferred embodiment of the present application, and not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of detecting frame rate adjustment, the method comprising: The method is applied to a detection device connected with at least one video acquisition device, and the method comprises: determining whether the number of continuous non-target frames of a real-time recorded target video acquisition device is greater than a preset frame number, and determining whether the detection device is in a high-load state, wherein the target video acquisition device is any video acquisition device in the at least one video acquisition device; in the case that the number of continuous non-target frames is greater than the preset frame number, or the detection device is in a high-load state, determining the maximum detection frame rate currently executable by the detection device according to the amount of currently remaining computing resources of the detection device; adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is the detection frame rate of the detection device for the video collected by the target video acquisition device; the adjusting of the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate comprises: determining whether the detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold according to the correspondence between the preset detection frame rate and detection rate; if the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate; if the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, adjusting the detection frame rate corresponding to the target video acquisition device to a target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates corresponding to the detection rates not less than the preset detection rate threshold in the correspondence.

2. The method of claim 1, wherein, the determining of whether the detection device is in a high-load state comprises: calculating a first time consumption according to the current detection frame rate corresponding to the at least one video acquisition device, a preset single-frame detection time consumption and a preset time length, wherein the current detection frame rate corresponding to the at least one video acquisition device is the current detection frame rate of the detection device for the video collected by the at least one video acquisition device, and the first time consumption is the time length required by the detection device for detecting the video collected by the at least one video acquisition device within the preset time length; calculating the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculating the product of the frame rate difference, the single-frame detection time consumption and the preset time length as a second time consumption, wherein the current detection frame rate corresponding to the target video acquisition device is the current detection frame rate of the detection device for the video collected by the target video acquisition device; determining a total time consumption according to the first time consumption and the second time consumption, and determining whether the total time consumption is not greater than the preset time length; if the total time consumption is greater than the preset time length, determining that the detection device is in a high-load state; if the total time consumption is not greater than the preset time length, determining that the detection device is not in a high-load state.

3. The method of claim 2, wherein, before the determining of the total time consumption according to the first time consumption and the second time consumption, the method further comprises: According to the number of current to-be-processed tasks and a preset processing time consumption of a single task, a third time consumption is calculated, wherein the to-be-processed tasks are processing tasks to be performed by the detection device after completing video frame detection, and the third time consumption is a time length required by the detection device to complete all current to-be-processed tasks; The total time consumption is determined according to the first time consumption and the second time consumption, including: The sum of the first time consumption, the second time consumption and the third time consumption is determined as the total time consumption.

4. The method according to claim 2 or 3, characterized in that, The maximum detection frame rate currently executable by the detection device is determined according to the amount of current remaining computing resources of the detection device, including: The second time consumption and a fourth time consumption are subtracted from the total time consumption to obtain a fifth time consumption, wherein the fourth time consumption is a time length required by the detection device to detect a video collected by the target video collection device within the preset time length at a current detection frame rate corresponding to the target video collection device; A difference between the preset time length and the fifth time consumption is calculated as a remaining computing time length within the preset time length; A ratio between the remaining computing time length and a target product is calculated, and the maximum detection frame rate currently executable by the detection device is determined according to the ratio, wherein the target product is a product of the preset time length and the single-frame detection time consumption.

5. The method of claim 1, wherein, In the following manner, the number of consecutive non-target frames of the target video collection device is recorded in real time: In the process of detecting the video collected by the target video collection device, for each video frame, it is detected whether the target exists in the video frame, if the target exists in the video frame, the number of consecutive non-target frames of the target video collection device is set to 0, and if the target does not exist in the video frame, the number of consecutive non-target frames of the target video collection device is increased by 1.

6. The method of claim 5, wherein, The detection of whether the target exists in the video frame includes: It is detected whether the target exists in the target region of the video frame; In the case that the target exists in the target region of the video frame, it is judged whether the confidence of the detected target is greater than a preset confidence threshold; If the confidence of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

7. The method according to any one of claims 1-3, 5-6, characterized in that, The method further includes: In the case that the number of consecutive non-target frames is not greater than the preset frame number and the detection device is not in a high-load state, the detection frame rate corresponding to the target video collection device is adjusted to a preset detection frame rate.

8. A frame rate adjustment detecting apparatus characterized by comprising: The device is applied to a detection device connected with at least one video collection device, and includes: A first judgment module is configured to judge whether the number of consecutive non-target frames of a target video collection device recorded in real time is greater than a preset frame number, and to judge whether the detection device is in a high-load state, wherein the target video collection device is any video collection device in the at least one video collection device; A maximum detection frame rate determination module is configured to determine the maximum detection frame rate currently executable by the detection device according to the amount of current remaining computing resources of the detection device in the case that the number of consecutive non-target frames is greater than the preset frame number or the detection device is in a high-load state. The detection frame rate adjustment module is configured to adjust a detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is a detection frame rate of the detection device for a video collected by the target video acquisition device. The second determination module is configured to determine whether a detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold according to a preset corresponding relationship between detection frame rates and detection rates, trigger execution of the detection frame rate adjustment module in a case where the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, and adjust the detection frame rate corresponding to the target video acquisition device to a target detection frame rate in a case where the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, wherein the target detection frame rate is a minimum value of detection frame rates corresponding to detection rates not less than the preset detection rate threshold in the corresponding relationship.

9. The apparatus of claim 8, wherein, The first determination module comprises: The first time consumption determination unit is configured to calculate a first time consumption according to a current detection frame rate corresponding to the at least one video acquisition device, a preset single-frame detection time consumption, and a preset time length, wherein the current detection frame rate corresponding to the at least one video acquisition device is a current detection frame rate of the detection device for a video collected by the at least one video acquisition device, and the first time consumption is a time length required by the detection device for detecting the video collected by the at least one video acquisition device within the preset time length. The second time consumption determination unit is configured to calculate a frame rate difference between a preset detection frame rate and a current detection frame rate corresponding to the target video acquisition device, and calculate a product of the frame rate difference, the single-frame detection time consumption, and the preset time length as a second time consumption, wherein the current detection frame rate corresponding to the target video acquisition device is a current detection frame rate of the detection device for a video collected by the target video acquisition device. The determination unit is configured to determine a total time consumption according to the first time consumption and the second time consumption, and determine whether the total time consumption is not greater than the preset time length, determine that the detection device is in a high-load state in a case where the total time consumption is greater than the preset time length, and determine that the detection device is not in the high-load state in a case where the total time consumption is not greater than the preset time length.

10. The apparatus of claim 9, wherein, The first determination module further comprises: The third time consumption determination unit is configured to calculate a third time consumption according to a number of current to-be-processed tasks and a preset processing time consumption of a single task, wherein the to-be-processed task is a processing task to be executed by the detection device after video frame detection is completed, and the third time consumption is a time length required by the detection device for completing all current to-be-processed tasks. The determination unit is specifically configured to determine a sum of the first time consumption, the second time consumption, and the third time consumption as the total time consumption.

11. The apparatus of claim 9 or 10, wherein, The maximum detection frame rate determination module comprises: a fifth time consumption determination unit, configured to subtract the second time consumption and the fourth time consumption from the total time consumption to obtain a fifth time consumption, wherein the fourth time consumption is a time length required by the detection device to detect a video collected by the target video collection device within the preset time length at a current detection frame rate corresponding to the target video collection device; a residual calculation time length determination unit, configured to calculate a difference between the preset time length and the fifth time consumption as a residual calculation time length within the preset time length; a maximum detection frame rate determination unit, configured to calculate a ratio between the residual calculation time length and a target product, and determine a maximum detection frame rate currently executable by the detection device according to the ratio, wherein the target product is a product of the preset time length and the single-frame detection time consumption.

12. The apparatus of claim 8, wherein, In the following manner, the number of continuous non-target frames of the target video collection device is recorded in real time: In the process of detecting the video collected by the target video collection device, for each video frame, it is detected whether the target exists in the video frame. If the target exists in the video frame, the number of continuous non-target frames of the target video collection device is set to 0. If the target does not exist in the video frame, the number of continuous non-target frames of the target video collection device is increased by 1.

13. The apparatus of claim 12, wherein, The detection of whether the target exists in the video frame includes: detecting whether the target exists in a target region of the video frame; in the case where the target exists in the target region of the video frame, judging whether a confidence degree of the detected target is greater than a preset confidence threshold; if the confidence degree of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

14. The apparatus of any of claims 8-10, 12-13, wherein: the detection frame rate adjustment module is further configured to, in a case where the number of continuous non-target frames is not greater than the preset number of frames and the detection device is not in a high load state, adjust the detection frame rate corresponding to the target video collection device to a preset detection frame rate.

15. A detection device, characterized by comprising: a memory for storing a computer program; a processor for executing the program stored on the memory, so as to implement the method of any of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer program stored in the computer readable storage medium is executed by the processor to implement the method of any of claims 1-7. The computer program stored in the computer readable storage medium is executed by the processor to implement the method of any of claims 1-7.

Images