Detection frame rate adjustment method and device, detection equipment and storage medium

Abstract

The embodiment of the invention provides a detection frame rate adjustment method and device, detection equipment and a storage medium. The method comprises the steps that whether the number of continuous target-free frames of target video collection equipment is larger than a preset frame number or not is judged, and whether the detection equipment is in a high-load state or not is judged; under the condition that the number of continuous target-free frames is greater than a preset frame number, or the detection equipment is in a high-load state, determining a currently executable maximum detection frame rate of the detection equipment according to the current residual calculation resource quantity of the detection equipment; and adjusting the detection frame rate of the detection device for the video collected by the target video collection device according to the maximum detection frame rate. The detection equipment can adaptively adjust the detection frame rate of the video collected by the target video collection equipment according to the image collection condition of the video collection equipment and the load state of the detection equipment, so that the flexibility of the detection equipment in detecting the video collected by the video collection equipment is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of video detection technology, and in particular to a detection frame rate adjustment method, device, detection equipment and storage medium. Background Art

[0002] In many application scenarios, it is necessary to detect the videos captured by video acquisition devices to achieve functions such as target detection, target classification or early warning.

[0003] The traditional video detection method uses a fixed detection frame rate, which makes the detection device less flexible when detecting the video captured by the video acquisition device. Specifically: on the one hand, in practical applications, there is no target in a large number of video frames captured by the video acquisition device, but regardless of whether there is a target in the video frame, the detection device will detect the video frame at a fixed detection frame rate, resulting in unnecessary consumption of computing power. On the other hand, when the detection device is in a high-load state, the detection device still detects at a fixed detection frame rate, which will further increase the load pressure of the detection device, and even if the target is detected, the detection device will find it difficult to further process the detected target due to the lack of sufficient computing resources, making the completed detection become an invalid detection due to the inability to perform further processing. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a detection frame rate adjustment method, device, detection equipment and storage medium to improve the flexibility of the detection equipment when detecting the video captured by the video capture device. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a detection frame rate adjustment method, which is applied to a detection device, wherein the detection device is connected to at least one video acquisition device, and the method includes:

[0006] Determine whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any one of the at least one video acquisition device;

[0007] When the number of the consecutive target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, determining the maximum detection frame rate that the detection device can currently perform according to the amount of computing resources currently remaining in the detection device;

[0008] The detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the detection device for the video acquired by the target video acquisition device.

[0009] Optionally, determining whether the detection device is in a high load state includes:

[0010] Calculate a first time consumption according to the current detection frame rate, preset single-frame detection time consumption and preset duration corresponding to the at least one video acquisition device, wherein the current detection frame rate corresponding to the at least one video acquisition device is: the current detection frame rate of the detection device for the video acquired by the at least one video acquisition device, and the first time consumption is: the duration required for the detection device to detect the video acquired by the at least one video acquisition device within the preset duration;

[0011] Calculate the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculate the product of the frame rate difference, the single frame detection time and the preset time length as the second time consumption, wherein the current detection frame rate corresponding to the target video acquisition device is: the current detection frame rate of the detection device for the video acquired by the target video acquisition device;

[0012] Determine a total time consumption according to the first time consumption and the second time consumption, and determine whether the total time consumption is not greater than the preset time length;

[0013] If the total time consumption is greater than the preset time length, it is determined that the detection device is in a high load state;

[0014] If the total time consumption is not greater than the preset time length, it is determined that the detection device is not in a high load state.

[0015] Optionally, before determining the total time consumption according to the first time consumption and the second time consumption, the method further includes:

[0016] Calculate the third time consumption according to the number of current tasks to be processed and the processing time consumption of a preset single task, wherein the tasks to be processed are processing tasks to be performed by the detection device after completing the video frame detection, and the third time consumption is: the time required for the detection device to complete all the current tasks to be processed;

[0017] The determining the total time consumption according to the first time consumption and the second time consumption includes:

[0018] The sum of the first time consumption, the second time consumption and the third time consumption is determined as the total time consumption.

[0019] Optionally, determining the maximum detection frame rate currently executable by the detection device according to the amount of computing resources currently remaining in the detection device includes:

[0020] The fifth time is obtained by subtracting the second time and the fourth time from the total time, wherein the fourth time is: the time required for the detection device to detect the video captured by the target video capture device within the preset time at the current detection frame rate corresponding to the target video capture device;

[0021] Calculate the difference between the preset duration and the fifth time duration as the remaining calculation duration within the preset duration;

[0022] The ratio between the remaining calculation time and the target product is calculated, and the maximum detection frame rate that the detection device can currently execute is determined according to the ratio, wherein the target product is the product of the preset time and the single-frame detection time.

[0023] Optionally, the number of consecutive target-free frames of the target video acquisition device is recorded in real time in the following manner:

[0024] In the process of detecting the video captured by the target video acquisition device, for each video frame, it is detected whether there is a target in the video frame. If there is a target in the video frame, the number of consecutive non-target frames of the target video acquisition device is set to 0; if there is no target in the video frame, the number of consecutive non-target frames of the target video acquisition device is increased by 1.

[0025] Optionally, the detecting whether there is a target in the video frame includes:

[0026] Detect whether there is a target in the target area of ​​the video frame;

[0027] When there is a target in the target area of ​​the video frame, determining whether the confidence of the detected target is greater than a preset confidence threshold;

[0028] If the confidence of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

[0029] Optionally, before adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, the method further includes:

[0030] According to the corresponding relationship between the preset detection frame rate and the detection rate, it is determined whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold;

[0031] If the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, performing the step of adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate;

[0032] If the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, the detection frame rate corresponding to the target video acquisition device is adjusted to the target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates whose corresponding detection rates included in the corresponding relationship are not less than the preset detection rate threshold.

[0033] Optionally, the method further includes:

[0034] When the number of the consecutive target-free frames is not greater than the preset frame number and the detection device is not in a high-load state, the detection frame rate corresponding to the target video acquisition device is adjusted to the preset detection frame rate.

[0035] In a second aspect, an embodiment of the present application provides a detection frame rate adjustment device, which is applied to a detection device, wherein the detection device is connected to at least one video acquisition device, and the device includes:

[0036] A first judgment module is used to judge whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and to judge whether the detection device is in a high-load state, wherein the target video acquisition device is any one of the at least one video acquisition device;

[0037] A maximum detection frame rate determination module, used to determine the maximum detection frame rate that can be currently executed by the detection device according to the amount of computing resources currently remaining in the detection device when the number of the consecutive target-free frames is greater than the preset number of frames or the detection device is in a high-load state;

[0038] A detection frame rate adjustment module is used to adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the detection device for the video acquired by the target video acquisition device.

[0039] Optionally, the first judgment module includes:

[0040] A first time-consuming determination unit is used to calculate a first time-consuming according to a current detection frame rate corresponding to the at least one video acquisition device, a preset single-frame detection time-consuming and a preset duration, wherein the current detection frame rate corresponding to the at least one video acquisition device is: a current detection frame rate of the detection device for the video acquired by the at least one video acquisition device, and the first time-consuming is: a duration required for the detection device to detect the video acquired by the at least one video acquisition device within the preset duration;

[0041] A second time-consuming determination unit is used to calculate a frame rate difference between a preset detection frame rate and a current detection frame rate corresponding to the target video acquisition device, and calculate the product of the frame rate difference, the single-frame detection time and the preset duration as a second time-consuming, wherein the current detection frame rate corresponding to the target video acquisition device is: the current detection frame rate of the detection device for the video acquired by the target video acquisition device;

[0042] A judgment unit is used to determine the total time consumption based on the first time consumption and the second time consumption, and to judge whether the total time consumption is not greater than the preset time length; if the total time consumption is greater than the preset time length, it is determined that the detection device is in a high-load state; if the total time consumption is not greater than the preset time length, it is determined that the detection device is not in a high-load state.

[0043] Optionally, the first judgment module further includes:

[0044] A third time-consuming determination unit is used to calculate a third time-consuming according to the number of current tasks to be processed and the processing time of a preset single task, wherein the tasks to be processed are processing tasks to be performed by the detection device after completing the video frame detection, and the third time-consuming is: the time required for the detection device to complete all the current tasks to be processed;

[0045] The determining unit is specifically configured to determine the sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption.

[0046] Optionally, the maximum detection frame rate determination module includes:

[0047] a fifth time-consuming determination unit, configured to subtract the second time-consuming and the fourth time-consuming from the total time-consuming to obtain a fifth time-consuming, wherein the fourth time-consuming is: the time required for the detection device to detect the video captured by the target video capture device within the preset time length at the current detection frame rate corresponding to the target video capture device;

[0048] a remaining calculation duration determining unit, configured to calculate a difference between the preset duration and the fifth time consumption as the remaining calculation duration within the preset duration;

[0049] A maximum detection frame rate determination unit is used to calculate the ratio between the remaining calculation time and the target product, and determine the maximum detection frame rate that the detection device can currently execute based on the ratio, wherein the target product is the product of the preset time and the single-frame detection time.

[0050] Optionally, the number of consecutive target-free frames of the target video acquisition device is recorded in real time in the following manner:

[0051] In the process of detecting the video captured by the target video acquisition device, for each video frame, it is detected whether there is a target in the video frame. If there is a target in the video frame, the number of consecutive non-target frames of the target video acquisition device is set to 0; if there is no target in the video frame, the number of consecutive non-target frames of the target video acquisition device is increased by 1.

[0052] Optionally, the detecting whether there is a target in the video frame includes: detecting whether there is a target in a target area of ​​the video frame; when there is a target in the target area of ​​the video frame, determining whether the confidence of the detected target is greater than a preset confidence threshold; if the confidence of the detected target is greater than the preset confidence threshold, determining that there is a target in the video frame.

[0053] Optionally, the device further comprises:

[0054] The second judgment module is used to judge whether the detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold according to the corresponding relationship between the preset detection frame rate and the detection rate; when the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, trigger the execution of the detection frame rate adjustment module; when the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates whose corresponding detection rates included in the corresponding relationship are not less than the preset detection rate threshold.

[0055] Optionally, the detection frame rate adjustment module is also used to adjust the detection frame rate corresponding to the target video acquisition device to a preset detection frame rate when the number of consecutive target-free frames is not greater than the preset number of frames and the detection device is not in a high-load state.

[0056] In a third aspect, an embodiment of the present application provides a detection device, including:

[0057] Memory, used to store computer programs;

[0058] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0059] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0060] Beneficial effects of the embodiments of the present application:

[0061] In the scheme provided by the embodiment of the present application, the detection device can determine whether the number of continuous target-free frames of the target video acquisition device recorded in real time is greater than the preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any video acquisition device in at least one video acquisition device connected to the detection device; when the number of continuous target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, the maximum detection frame rate that the detection device can currently execute is determined according to the amount of computing resources currently remaining in the detection device; the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the video collected by the target video acquisition device. Since the detection device has multiple continuous target-free frames in the video collected by the video acquisition device or is in a high-load state, the detection device can determine the maximum detection frame rate that can be executed in real time according to the current remaining computing resources of itself, and adaptively adjust the detection frame rate of the video collected by the video acquisition device according to the maximum detection frame rate. In this way, the detection frame rate can be adaptively changed with the image acquisition situation of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0062] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0064] Figure 1 A schematic diagram of a flow chart of a detection frame rate adjustment method provided in an embodiment of the present application;

[0065] Figure 2 Based on Figure 1 A schematic diagram of a flow chart of a method for determining a load state of a detection device in the illustrated embodiment;

[0066] Figure 3 Based on Figure 2 A schematic diagram of a specific flow chart of a method for determining a load state of a detection device in the illustrated embodiment;

[0067] Figure 4 Based on Figure 1 A schematic diagram of a flow chart of a method for determining a maximum detection frame rate in the illustrated embodiment;

[0068] Figure 5 Based on Figure 1 A schematic flow chart of a method of detecting whether a target exists in a video frame according to the illustrated embodiment;

[0069] Figure 6 Based on Figure 1 A schematic diagram of a specific flow chart of the detection frame rate adjustment method of the illustrated embodiment;

[0070] Figure 7 A schematic diagram of the structure of a detection frame rate adjustment device provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of the structure of the detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0073] In order to improve the flexibility of the detection device when detecting videos captured by the video acquisition device, the embodiments of the present application provide a detection frame rate adjustment method, device, detection device, computer-readable storage medium and computer program product. The detection frame rate adjustment method provided in the embodiments of the present application is first introduced below.

[0074] The detection frame rate adjustment method provided in the embodiment of the present application can be applied to any detection device connected to at least one video acquisition device for detecting the video acquired by the video acquisition device. For example, the detection device can be a computer, a server, a computing card, etc., which is not specifically limited here. For the sake of clear description, it is referred to as the detection device below.

[0075] like Figure 1 As shown, a detection frame rate adjustment method includes:

[0076] S101, determine whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and determine whether the detection device is in a high-load state. If one of the judgment results is yes, execute step S102.

[0077] The target video acquisition device is any one of the at least one video acquisition device.

[0078] S102: Determine the maximum detection frame rate that can be currently executed by the detection device according to the amount of computing resources currently remaining in the detection device.

[0079] S103: Adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate.

[0080] The detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the video acquired by the detection device for the target video acquisition device.

[0081] In the scheme provided by the embodiment of the present application, the detection device can determine whether the number of continuous target-free frames of the target video acquisition device recorded in real time is greater than the preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any video acquisition device in at least one video acquisition device connected to the detection device; when the number of continuous target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, the maximum detection frame rate that the detection device can currently execute is determined according to the amount of computing resources currently remaining in the detection device; the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the video collected by the target video acquisition device. Since the detection device has multiple continuous target-free frames in the video collected by the video acquisition device or is in a high-load state, the detection device can determine the maximum detection frame rate that can be executed in real time according to the current remaining computing resources of itself, and adaptively adjust the detection frame rate of the video collected by the video acquisition device according to the maximum detection frame rate. In this way, the detection frame rate can be adaptively changed with the image acquisition situation of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0082] Here we first introduce the application scenarios of the detection frame rate adjustment method provided in the embodiments of the present application.

[0083] The detection device is connected to at least one video image acquisition device. For each video image acquisition device, after receiving the video acquired by the video image acquisition device, the detection device can detect the video acquired by the video image acquisition device at a certain frame rate to determine whether there is a target in the video screen of the video image acquisition device.

[0084] For example, the detection device is connected to camera A, and camera A transmits 60 video frames to the detection device per second. The detection device extracts 10 video frames from the 60 video frames transmitted by the camera per second to the detection device according to the current detection frame rate of 10 frames per second for detection to determine whether there is a target in the video image of camera A.

[0085] In the process of detecting the video captured by the video image acquisition device, the detection device can record the number of consecutive target-free frames of the video image acquisition device. Specifically: for each extracted video frame, the detection device can detect whether there is a target in the video frame. If there is a target in the video frame, the number of consecutive target-free frames of the video acquisition device transmitted back to the video frame will be set to 0; if there is no target in the video frame, the number of consecutive target-free frames of the target video acquisition device transmitted back to the video frame will be increased by 1.

[0086] For example, the detection device extracts 5 video frames from the 60 video frames transmitted back by camera A in the first second for detection, which are video frames a, b, c, d, and e. When the detection device detects that there is no target in video frame a, the number of consecutive frames without targets of camera A is recorded as 1. If there is no target in video frame b, the number is increased by 1, that is, the number of consecutive frames without targets of camera A is recorded as 2. If there is no target in video frame c, the number is increased by 1, that is, the number of consecutive frames without targets of camera A is recorded as 3. If there is a target in video frame d, the number of consecutive frames without targets of camera A is set to 0. If there is a target in video frame e, the number of consecutive frames without targets of camera A is set to 0.

[0087] The following will introduce the detection frame rate adjustment method provided in the embodiment of the present application based on the above application scenario.

[0088] For any video acquisition device (hereinafter referred to as the target video acquisition device) of at least one video acquisition device connected to the detection device, before detecting the video acquired by the target video acquisition device, the detection device can execute step S101 to determine whether the number of consecutive target-free frames of the target video acquisition device recorded in real time in the above manner is greater than the preset number of frames, that is, to determine whether there is no target for a long time in the video acquired by the target video acquisition device. If the number of consecutive target-free frames is greater than the preset number of frames, it means that the detection frame rate can be adaptively adjusted at this time to avoid wasting detection resources. In addition, the detection device can also determine whether the detection device is in a high-load state by the task currently to be performed by the detection device. If the detection device is in a high-load state, it means that the detection frame rate can be adaptively adjusted at this time to reduce the computing resources occupied by the detection.

[0089] It should be noted that the above two judgment steps can be executed simultaneously, or one of the judgment steps can be executed first, and when the judgment result of the judgment step is no, the other judgment step can be executed. No specific limitation is made here.

[0090] When the judgment results of the above two judgment steps are both negative, that is, when the number of consecutive target-free frames is not greater than the preset number of frames, and the detection device is not in a high-load state, it means that the target has appeared in the video captured by the target video acquisition device recently, and the current computing resources of the detection device are sufficient. On this basis, the detection device can adjust the detection frame rate corresponding to the target video acquisition device to the preset detection frame rate. The preset detection frame rate can be a detection frame rate that the user can pre-set in the detection device based on the characteristics that the detection frame rate is positively correlated with the detection rate and the detection frame rate is positively correlated with the load in actual applications, and can simultaneously meet the requirements of the detection rate and the load state of the detection device.

[0091] When the judgment result of any of the above judgment steps is yes, that is, when the number of consecutive target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, the detection device can further execute step S102, determine the maximum detection frame rate that the detection device can currently execute according to the remaining computing resources of the detection device, and then adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate. The specific method of determining the maximum detection frame rate will be described in detail in subsequent embodiments and will not be repeated here.

[0092] Regarding the method of adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, the detection device can be implemented by but not limited to the following methods:

[0093] In a first implementation, the detection device may directly adjust the detection frame rate corresponding to the target video acquisition device to the maximum detection frame rate.

[0094] In the second embodiment, the user can pre-set a detection frame rate in the detection device based on the characteristics of the positive correlation between the detection frame rate and the detection rate and the positive correlation between the detection frame rate and the load in actual applications. The detection frame rate can simultaneously meet the requirements of the detection rate and the load state of the detection device. In this case, the detection device can first determine whether the maximum detection frame rate is greater than the preset detection frame rate.

[0095] If the maximum detection frame rate is not greater than the preset detection frame rate, it means that although the maximum detection frame rate will reduce the detection rate compared to the preset detection frame rate, the maximum detection frame rate is already the maximum detection frame rate that the detection device can currently execute. Therefore, in order to avoid overloading the detection device, the detection frame rate corresponding to the target video acquisition device can be adjusted to the maximum detection frame rate.

[0096] If the maximum detection frame rate is greater than the preset detection frame rate, it means that both the maximum detection frame rate and the preset detection frame rate can meet the detection rate requirements, but the maximum detection frame rate will increase the load of the detection device compared to the preset detection frame rate. Therefore, when both can meet the detection rate requirements, in order to minimize the load of the detection device, the detection device can adjust the detection frame rate corresponding to the target video acquisition device to the preset detection frame rate.

[0097] In the scheme provided by the embodiment of the present application, since the detection device has a plurality of continuous target-free frames in the video collected by the video acquisition device or is in a high-load state, the maximum detection frame rate that can be executed can be determined in real time according to the current remaining computing resource amount of itself, and the detection frame rate of the video collected by the video acquisition device itself can be adaptively adjusted according to the maximum detection frame rate. Thus, the detection frame rate can be adaptively changed with the image acquisition situation of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device when detecting the video collected by the video acquisition device. In addition, by the above-mentioned method for flexibly adjusting the detection frame rate, the computing resources of the detection device can be fully utilized to improve the number of image acquisition devices that the detection device can be connected to, and / or, improve the complexity of the detection algorithm, so that the detection device can process the video more accurately based on the algorithm with higher complexity, and / or, reduce the detection device and also due to lack of sufficient computing resources, it is difficult to further process the detected target, so that the completed detection becomes an invalid detection due to the inability to further process.

[0098] As an implementation method of the present application, Figure 2 As shown, when the detection device determines whether it is in a high load state, it can be implemented specifically through steps S201-S205:

[0099] S201, calculating a first time consumption according to a current detection frame rate, a preset single-frame detection time consumption and a preset duration corresponding to at least one video acquisition device.

[0100] When the detection device determines whether it is in a high-load state, it can first calculate the time it takes to detect the video captured by the video capture device through step S201. Specifically:

[0101] For each video acquisition device in at least one video acquisition device connected to the detection device, by calculating the product of the current detection frame rate corresponding to the video acquisition device (i.e., the current detection frame rate of the video acquired by the detection device for the video acquisition device) and the preset single-frame detection time consumption, the first time consumption required for the detection device to complete the detection of the video frame returned by the video acquisition device within the unit time consumption can be determined. By calculating the product between the first time consumption and the preset time consumption, the second time consumption required for the detection device to complete the detection of the video frame returned by the video acquisition device within the preset time consumption can be determined. Finally, the second time consumption corresponding to at least one video acquisition device connected to the detection device is accumulated to obtain the time consumption (first time consumption) required for the detection device to detect the video acquired by at least one video acquisition device within the preset time consumption.

[0102] Among them, the above-mentioned preset duration can be the maximum delay of the detection device when processing the video, or it can be a unit duration, and can also be set accordingly according to actual usage requirements, which is not specifically limited here.

[0103] In one implementation, the detection device may calculate the first time consumption based on the following formula:

[0104] ;

[0105] in, Indicates the preset duration. Indicates the number of video acquisition devices connected to the detection device. Indicates the time taken for single frame detection. Indicates that the testing equipment is for The current detection frame rate of the video captured by the video capture device.

[0106] S202, calculating the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculating the product of the frame rate difference, the single frame detection time and the preset time length as the second time consumption.

[0107] Because when the number of consecutive target-free frames is not greater than the preset number of frames and the detection device is not in a high-load state, the detection device will adjust the detection frame rate corresponding to the target video acquisition device to the preset detection frame rate. Therefore, when determining the load state of the detection device, what needs to be considered is the time it takes for the detection device to detect the video captured by the target video acquisition device at the preset detection frame rate.

[0108] For example, the number of consecutive target-free frames is not greater than the preset number of frames, the current detection frame rate corresponding to camera A is 8 frames / second, and the preset detection frame rate is 10 frames / second. It is assumed that when determining the load state of the detection device, the time taken into account for the detection device to detect the video captured by the target video acquisition device at 8 frames / second is taken into account, and it is determined that the detection device is not in a high-load state.

[0109] Then, because the number of consecutive target-free frames is not greater than the preset number of frames, and the detection device is not in a high-load state, the detection device will adjust the detection frame rate corresponding to the target video acquisition device to 10 frames per second. However, after the adjustment, the detection device will extract 2 more frames per second for detection of the video captured by the target video acquisition device. The detection work of the extra 2 frames per second may cause the detection device to be in a high-load state. Therefore, when determining the load state of the detection device, it is necessary to consider the time it takes for the detection device to detect the video captured by the target video acquisition device at 10 frames per second.

[0110] Furthermore, because the first time consumption includes the time consumption of the detection device to detect the video captured by the target video capture device at the current detection frame rate corresponding to the target video capture device, when considering the time consumption of the detection device to detect the video captured by the target video capture device at the preset detection frame rate, it is only necessary to additionally determine the time consumption (second time consumption) of the detection device to detect the video captured by the target video capture device at the frame rate difference between the preset detection frame rate and the current detection frame rate.

[0111] Therefore, the detection device can calculate the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculate the product of the frame rate difference, the single frame detection time and the preset time length as the second time consumption.

[0112] In one implementation, the detection device may calculate the second time consumption based on the following formula:

[0113] ;

[0114] in, Indicates the preset duration. Indicates the preset detection frame rate. Indicates that the testing equipment is for The current detection frame rate of the video collected by the video acquisition device. Indicates the time taken to detect a single frame.

[0115] S203, determining the total time according to the first time and the second time, and judging whether the total time is not greater than the preset time. If the judgment result is no, executing step S204, if the judgment result is yes, executing step S205.

[0116] After determining the first time consumption and the second time consumption according to step S201 and step S202, the detection device can determine the total time consumption according to the first time consumption and the second time consumption, and judge whether the total time consumption is greater than the preset time length, that is, judge whether the time length required for the detection device to detect the video captured by at least one video capture device within the preset time length is greater than the preset time length.

[0117] As for the method for determining the total time consumption, when the detection device processes the video only including detecting whether there is a target in the video frame, the total time consumption of the detection device processing the video is the sum of the first time consumption and the second time consumption. In this case, the detection device can sum the first time consumption and the second time consumption, and determine the sum of the two as the total time consumption. Among them, when the detection device processes the video in addition to detecting whether there is a target in the video frame, and also includes other processing tasks after detection, the calculation method of the total time consumption will be described in detail in subsequent embodiments and will not be repeated here.

[0118] In one implementation, the detection device may calculate the total time consumption based on the following formula:

[0119] ;

[0120] in, Indicates the preset duration. Indicates the preset detection frame rate. Indicates that the testing equipment is for The current detection frame rate of the video collected by the video acquisition device. Indicates the time taken for single frame detection. Indicates the number of video capture devices connected to the detection device.

[0121] S204, determining that the detection device is in a high load state.

[0122] If the total time is greater than the preset duration, it means that the detection device cannot complete the detection of at least one video captured by the video capture device within the preset duration, and the detection task within the current preset duration will be postponed to the next preset duration, which will cause the execution of subsequent detection tasks to be postponed, and the postponement time will be longer and longer. In this case, the detection device can determine that it is in a high-load state.

[0123] S205, determining that the detection device is not in a high load state.

[0124] If the total time consumption is not greater than the preset duration, it means that for at least one video acquisition device captured within the preset duration, the detection device can complete the detection within the preset duration, and the detection task within the current preset duration will not be postponed to the next preset duration. In this case, the detection device can determine that it is not in a high load state.

[0125] In the solution provided by the embodiment of the present application, the first time consumption is determined by the current detection frame rate, single frame detection time consumption and preset duration of each video acquisition device, which can accurately evaluate the time consumption of the detection device for the existing detection task. On this basis, the second time consumption is determined by the frame rate difference between the preset detection frame rate and the current detection frame rate, the single frame detection time consumption and the preset duration, further considering the additional load impact caused by the frame rate adjustment, so that the determination of the total time consumption is more scientific and reasonable. Finally, based on the comparison of the total time consumption and the preset duration, the load state of the detection device can be accurately judged.

[0126] As an implementation method of the present application, Figure 3 As shown, before executing the above step S203, the detection device may further execute step S301:

[0127] S301, calculating a third time consumption according to the current number of tasks to be processed and a preset processing time consumption of a single task.

[0128] When the detection device processes the video, in addition to detecting whether there is a target in the video frame, and also includes other processing tasks after the detection, when determining the load state of the detection device, it is necessary to further consider the time consumption of the processing tasks (to-be-processed tasks) to be performed by the detection device after completing the video frame detection. Specifically:

[0129] Because detection tasks are continuously generated according to the frame rate, new detection tasks are added every second, so when calculating the time corresponding to the detection task, you need to multiply the preset time. However, in contrast, pending tasks are not necessarily generated every second. Their generation is related to the detected target. When the target is detected from the video frame, the corresponding pending task will be generated.

[0130] Therefore, when determining the current load state of the detection device, only the current pending tasks need to be considered. On this basis, for each video acquisition device in at least one video acquisition device connected to the detection device, by calculating the number of pending tasks corresponding to the video acquisition device and the processing time of a preset single task, the third time required for the detection device to complete the pending tasks corresponding to the video acquisition device can be determined. Then, the third time corresponding to at least one video acquisition device connected to the detection device is accumulated to obtain the time required for the detection device to complete all the current pending tasks (third time).

[0131] In one implementation, the detection device may calculate the third time consumption based on the following formula:

[0132] ;

[0133] in, Indicates the number of video acquisition devices connected to the detection device. Indicates The number of pending tasks corresponding to the video acquisition devices. Indicates the processing time of a single task.

[0134] It should be noted that the type of task to be processed can be set according to actual usage requirements. For example, the task to be processed can be a classification task, a segmentation task, an event judgment task, etc. The type of task to be processed is not specifically limited in this application. In addition, when the detection device processes the video in addition to detecting whether there is a target in the video frame, and also includes other processing tasks after detection, the above-mentioned preset duration can be the maximum delay of the detection device when processing the video.

[0135] Accordingly, when the detection device executes the above step S203, it can be specifically implemented through step S302:

[0136] S302: Determine the sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption, and determine whether the total time consumption is not greater than a preset time length.

[0137] After the detection device determines the first time consumption and the second time consumption according to step S201 and step S202, and determines the third time consumption through step S301, the first time consumption, the second time consumption and the third time consumption can be summed, and the sum of the three can be determined as the total time consumption. And it is judged whether the total time consumption is greater than the preset time length, that is, it is judged whether the time length required for the detection device to detect the video collected by at least one video collection device within the preset time length is greater than the preset time length.

[0138] In one implementation, the detection device may calculate the total time consumption based on the following formula:

[0139] ;

[0140] in, Indicates the preset duration. Indicates the preset detection frame rate. Indicates that the testing equipment is for The current detection frame rate of the video collected by the video acquisition device. Indicates the time taken for single frame detection. Indicates the number of video acquisition devices connected to the detection device. Indicates The number of pending tasks corresponding to the video acquisition devices. Indicates the processing time of a single task.

[0141] In the solution provided in the embodiment of the present application, the third time-consuming calculation is further introduced to fully consider the time-consuming situation of other processing tasks after the detection device completes the video frame detection, and is no longer limited to the time evaluation of the video frame detection link. This comprehensive consideration makes the judgment of the load state of the detection device more accurate and can more truly reflect the actual load state of the device in the entire video processing process.

[0142] As an implementation method of the present application, Figure 4 As shown, when the detection device executes step S103, it can be specifically implemented through steps S401-S403:

[0143] S401, subtract the second time consumption and the fourth time consumption from the total time consumption to obtain the fifth time consumption.

[0144] S402: Calculate the difference between the preset duration and the fifth time consumption as the remaining calculation duration within the preset duration.

[0145] Because the total time consumption includes the time required for the detection device to detect the video captured by the target video capture device within the preset time length at the current detection frame rate corresponding to the target video capture device (the fourth time consumption), and the time consumed by the detection device to detect the video captured by the target video capture device within the preset time length at the frame rate difference between the preset detection frame rate and the current detection frame rate (the second time consumption). Therefore, the detection device can first subtract the second time consumption and the fourth time consumption from the total time consumption to obtain the fifth time consumption.

[0146] The total time consumption is the total time consumption required for the detection device to process the video captured by at least one video acquisition device within the preset time consumption. The fourth time consumption is the time consumption for detecting the video captured by the target video acquisition device within the preset time consumption according to the current detection frame rate corresponding to the target video acquisition device. The second time consumption is the extra detection time consumption for the target video acquisition device due to the frame rate adjustment.

[0147] The fifth time consumed after subtracting the second time consumption and the fourth time consumption from the total time consumption is actually the time used for other tasks (such as detection tasks of other video acquisition devices, or detection tasks of other video acquisition devices and other tasks outside the detection link) within the preset time length, in addition to the detection time consumption related to the target video acquisition device.

[0148] The preset duration is a fixed time period. The difference between the preset duration and the fifth duration is naturally the remaining duration after deducting the occupied duration (i.e., the fifth duration) within the preset duration, that is, the remaining calculation duration within the preset duration. It can intuitively reflect how much time is available within the preset duration to detect the video captured by the target video acquisition device under the current load state of the detection device.

[0149] S403, calculating the ratio between the remaining calculation time and the target product, and determining the maximum detection frame rate that the detection device can currently execute according to the ratio.

[0150] As mentioned above, the remaining calculation time reflects how much time is left within the preset time and can be used to detect the video captured by the target video acquisition device. The first ratio between the remaining calculation time and the preset time reflects how much time is left within the unit time and can be used to detect the video captured by the target video acquisition device. Furthermore, the second ratio between the first ratio and the single-frame detection time reflects the maximum number of video frames that can be detected within the unit time (i.e., the maximum detection frame rate).

[0151] Therefore, after obtaining the remaining calculation time, the detection device can calculate the ratio between the remaining calculation time and the target product (i.e., the product of the preset time and the single-frame detection time), and the maximum detection frame rate that the detection device can currently execute needs to be no greater than the ratio. For example, after obtaining the ratio, the detection device can round the ratio down to obtain the maximum detection frame rate that the detection device can currently execute.

[0152] In one embodiment, when the total time is calculated by Figure 2 In the case of the embodiment shown, when the detection device executes the above steps S401-S403, the conditions that need to be met for the maximum detection frame rate can be determined by the following formula:

[0153] ;

[0154] in, Testing equipment for The current executable detection frame rate of the video collected by the video acquisition device. Indicates the preset duration. Indicates the time taken for single frame detection. Indicates that the testing equipment is for The current detection frame rate of the video collected by the video acquisition device. Indicates the number of video capture devices connected to the detection device.

[0155] In another embodiment, when the total time is calculated by Figure 3In the case of the embodiment shown, when the detection device executes the above steps S401-S403, the conditions that need to be met for the maximum detection frame rate can be determined by the following formula:

[0156] ;

[0157] in, For the The maximum detection frame rate that can be performed by the video detection device currently collected by the video acquisition device. Indicates the preset duration. Indicates the time taken for single frame detection. Indicates that the testing equipment is for The current detection frame rate of the video collected by the video acquisition device. Indicates the number of video acquisition devices connected to the detection device. Testing equipment for The number of pending tasks for videos collected by the video acquisition devices. Indicates the processing time of a single task.

[0158] In the solution provided in the embodiment of the present application, the second time consumption and the fourth time consumption related to the detection of the target video acquisition device are first separated from the total time consumption to obtain the fifth time consumption, and then the remaining calculation time within the preset time length is calculated, and then the maximum detection frame rate is determined based on the ratio of the remaining calculation time to the target product. In this way, the detection frame rate can be dynamically adjusted according to the current load situation of the detection device, so that when the detection device faces complex and changeable video acquisition and processing tasks, it can not only make full use of the remaining computing resources and avoid idle waste of resources, but also effectively prevent load overload or other tasks from being completed on time due to excessive allocation of resources to the target video acquisition device.

[0159] As an implementation method in the present application, Figure 5 As shown, when the detection device detects whether there is a target in the video frame, it can be implemented specifically through steps S501-504:

[0160] S501, detect whether there is a target in the target area of ​​the video frame. If the detection result is yes, execute step S502, if the detection result is no, execute step S504.

[0161] It is very necessary to pre-set the target area during the video processing so that the detection equipment can only detect the target area in the video frame. Specifically: on the one hand, the information covered by the video frame is complex and diverse, and the target that the user is concerned about will only appear in a specific local range. For example, for the picture obtained by the camera used to shoot road vehicles, the vehicle target will most likely only appear in the picture area corresponding to the road. As for the sky, rooftop and other picture areas, it is almost impossible for vehicles to exist. If the entire video frame is fully detected, it will undoubtedly consume a lot of computing resources and time costs, reducing the detection efficiency. On the other hand, targets at the edge of the picture are prone to feature loss or distortion interference, which will greatly affect the accuracy and reliability of the detection results. By setting the target area, the adverse effects of these edge areas can be effectively avoided, so that the detection is focused on the core area where the target is easier to fully present and the features are stable, thereby significantly improving the accuracy and effectiveness of the detection.

[0162] Therefore, when detecting whether there is a target in a video frame, the detection device may only detect whether there is a target in the target area of ​​the video frame. The target may be detected by a target detection algorithm or by a pre-trained target detection model, which is not specifically limited here.

[0163] S502, determine whether the confidence of the detected target is greater than a preset confidence threshold. If the determination result is yes, execute step S503, if the determination result is no, execute step S504.

[0164] When the detection device detects a target in the target area of ​​the video frame, it can further determine the confidence of the target. Furthermore, the detection device can determine whether the confidence of the detected target is greater than a preset confidence threshold to determine whether the target actually exists in the video frame.

[0165] S503: Determine whether there is an object in the video frame.

[0166] If there is a target with a confidence level greater than a preset confidence threshold in the target area of ​​the video frame, it means that there is likely to be a target in the video frame. In this case, the detection device can determine that there is a target in the video frame.

[0167] S504: Determine whether there is no target in the video frame.

[0168] If there is no target in the target area of ​​the video frame, or the confidence of the target in the target area of ​​the video frame is not greater than a preset confidence threshold, it means that there is most likely no target in the video frame. In this case, the detection device can determine that there is no target in the video frame.

[0169] In the solution provided by the embodiment of the present application, by limiting the target area detection, unnecessary calculations are greatly reduced, the detection efficiency is improved, and the detection device can process more video frames in a shorter time. Secondly, the confidence level of the detected target is judged, which effectively reduces the false detection rate and makes the detection result more reliable.

[0170] As an implementation method of the present application, Figure 6 As shown, before executing step S103, the detection device may further execute step S601:

[0171] S601, according to the preset corresponding relationship between the detection frame rate and the detection rate, determine whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold. If the judgment result is no, execute step S103, if the judgment result is yes, execute step S602.

[0172] The detection device can pre-store the correspondence between the detection frame rate and the detection rate, wherein the correspondence includes each detection frame rate and the probability (i.e., detection accuracy) of detecting the target existing in the video frame when the detection device detects the video captured by the video acquisition device at each detection frame rate.

[0173] For example, the correspondence between the detection frame rate and the detection rate can be shown in the following table, where the detection frame rate S means that S video frames are detected per second, the detection frame rate S-2 means that (S-2) video frames are detected per second... The detection frame rate S-8 means that (S-8) video frames are detected per second.

[0174]

[0175] After determining the maximum detection frame rate that the detection device can currently execute, the detection device can determine the detection rate corresponding to the maximum detection frame rate based on the corresponding relationship between the preset detection frame rate and the detection rate, and judge whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, so as to determine whether the adjusted detection frame rate can meet the detection rate requirements when adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate.

[0176] If the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, it means that when the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, the adjusted detection frame rate can meet the detection rate requirement. Therefore, in this case, the detection device can execute step S103.

[0177] S602: Adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate.

[0178] If the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, it means that when the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, the adjusted detection frame rate cannot meet the detection rate requirement. In this case, the detection device can determine the detection frame rate whose corresponding detection rate is not less than the preset detection rate threshold from the correspondence between the preset detection frame rate and the detection rate, as the detection frame rate to be selected.

[0179] Because the detection frame rate is positively correlated with the detection rate, and the detection frame rate is positively correlated with the load, the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, and the detection rate corresponding to the to-be-selected detection frame rate is not less than the preset detection rate threshold, so the to-be-selected detection frame rate is greater than the maximum detection frame rate that the detection device can currently execute. Therefore, on the basis that the to-be-selected detection frame rate is already greater than the maximum detection frame rate that the detection device can currently execute, in order to minimize the load of the detection device, the detection device can determine the minimum value of the to-be-selected detection frame rates as the target detection frame rate, and then adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate.

[0180] For example, if the preset detection rate threshold is 0.7f, then the detection device can determine the corresponding detection frame rate of the candidate detection frame rate that is not less than the preset detection rate threshold from the correspondence between the preset detection frame rate and the detection rate as shown in the following table. The minimum value of each detection frame rate in the table below is S-4. Therefore, the detection device can adjust the detection frame rate corresponding to the target video acquisition device to S-4.

[0181]

[0182] In the solution provided in the embodiment of the present application, by pre-storing the correspondence between the detection frame rate and the detection rate, after determining the maximum detection frame rate, it is possible to judge whether the detection rate corresponding to the maximum detection frame rate meets the preset detection rate requirement based on the correspondence. When the requirement is met, the detection frame rate is adjusted according to the maximum detection frame rate. When the requirement is not met, the minimum detection frame rate that meets the preset detection rate threshold requirement is determined as the corresponding one of the video acquisition device. In this way, the detection rate requirement can be met, and the positive correlation between the detection frame rate and the load is fully taken into account. Under the premise of ensuring the detection rate requirement, the load pressure of the detection equipment can be reduced as much as possible.

[0183] Corresponding to the above-mentioned detection frame rate adjustment method, the embodiment of the present application also provides a detection frame rate adjustment device. The detection frame rate adjustment device provided by the embodiment of the present application is introduced below.

[0184] like Figure 7 The detection frame rate adjustment device is applied to a detection device, and the detection device is connected to at least one video acquisition device. The above device may include:

[0185] The first judgment module 701 is used to judge whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and to judge whether the detection device is in a high-load state, wherein the target video acquisition device is any one of the at least one video acquisition device;

[0186] The maximum detection frame rate determination module 702 is used to determine the maximum detection frame rate that can be currently executed by the detection device according to the amount of computing resources currently remaining in the detection device when the number of consecutive target-free frames is greater than a preset number of frames or the detection device is in a high-load state;

[0187] The detection frame rate adjustment module 703 is used to adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the detection device for the video acquired by the target video acquisition device.

[0188] In the scheme provided by the embodiment of the present application, the detection device can determine whether the number of continuous target-free frames of the target video acquisition device recorded in real time is greater than the preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any video acquisition device in at least one video acquisition device connected to the detection device; when the number of continuous target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, the maximum detection frame rate that the detection device can currently execute is determined according to the amount of computing resources currently remaining in the detection device; the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the video collected by the target video acquisition device. Since the detection device has multiple continuous target-free frames in the video collected by the video acquisition device or is in a high-load state, the detection device can determine the maximum detection frame rate that can be executed in real time according to the current remaining computing resources of itself, and adaptively adjust the detection frame rate of the video collected by the video acquisition device according to the maximum detection frame rate. In this way, the detection frame rate can be adaptively changed with the image acquisition situation of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0189] As an implementation of the embodiment of the present application, the first determination module 701 may include:

[0190] A first time-consuming determination unit is used to calculate a first time-consuming according to a current detection frame rate corresponding to at least one video acquisition device, a preset single-frame detection time-consuming and a preset duration, wherein the current detection frame rate corresponding to at least one video acquisition device is: a current detection frame rate of a video acquired by the detection device for at least one video acquisition device, and the first time-consuming is: a duration required for the detection device to detect a video acquired by at least one video acquisition device within a preset duration;

[0191] The second time-consuming determination unit is used to calculate the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculate the product of the frame rate difference, the single-frame detection time and the preset duration as the second time-consuming, wherein the current detection frame rate corresponding to the target video acquisition device is: the current detection frame rate of the detection device for the video acquired by the target video acquisition device;

[0192] A judgment unit is used to determine the total time consumption based on the first time consumption and the second time consumption, and to judge whether the total time consumption is not greater than a preset time length; when the total time consumption is greater than the preset time length, it is determined that the detection device is in a high-load state; when the total time consumption is not greater than the preset time length, it is determined that the detection device is not in a high-load state.

[0193] As an implementation of the embodiment of the present application, the first determination module 701 may further include:

[0194] A third time-consuming determination unit is used to calculate a third time-consuming according to the number of current tasks to be processed and the preset processing time of a single task, wherein the tasks to be processed are processing tasks to be performed by the detection device after completing the video frame detection, and the third time-consuming is: the time required for the detection device to complete all the current tasks to be processed;

[0195] The judging unit is specifically configured to determine the sum of the first time consumption, the second time consumption and the third time consumption as the total time consumption.

[0196] As an implementation of the embodiment of the present application, the maximum detection frame rate determination module 702 may include:

[0197] A fifth time consumption determining unit is used to subtract the second time consumption and the fourth time consumption from the total time consumption to obtain a fifth time consumption, wherein the fourth time consumption is: the time required for the detection device to detect the video captured by the target video capture device within a preset time length at the current detection frame rate corresponding to the target video capture device;

[0198] A remaining calculation time determination unit, used to calculate the difference between the preset time and the fifth time consumption as the remaining calculation time within the preset time;

[0199] The maximum detection frame rate determination unit is used to calculate the ratio between the remaining calculation time and the target product, and determine the maximum detection frame rate that the detection device can currently execute according to the ratio, wherein the target product is the product of the preset time and the single-frame detection time.

[0200] As an implementation of the embodiment of the present application, the number of consecutive target-free frames of the target video acquisition device can be recorded in real time in the following manner:

[0201] In the process of detecting the video captured by the target video acquisition device, for each video frame, it is detected whether there is a target in the video frame. If there is a target in the video frame, the number of consecutive non-target frames of the target video acquisition device is set to 0. If there is no target in the video frame, the number of consecutive non-target frames of the target video acquisition device is increased by 1.

[0202] As an implementation manner of the embodiment of the present application, the above-mentioned detection of whether there is a target in the video frame may include:

[0203] Detect whether there is a target in the target area of ​​the video frame;

[0204] When there is a target in the target area of ​​the video frame, determining whether the confidence of the detected target is greater than a preset confidence threshold;

[0205] If the confidence of the detected target is greater than a preset confidence threshold, it is determined that the target exists in the video frame.

[0206] As an implementation of the embodiment of the present application, the above device may further include:

[0207] The second judgment module is used to judge whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold according to the corresponding relationship between the preset detection frame rate and the detection rate; when the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, trigger the execution of the detection frame rate adjustment module 703; when the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates whose corresponding detection rates included in the corresponding relationship are not less than the preset detection rate threshold.

[0208] As an implementation of an embodiment of the present application, the above-mentioned detection frame rate adjustment module 703 can also be used to adjust the detection frame rate corresponding to the target video acquisition device to a preset detection frame rate when the number of consecutive target-free frames is not greater than a preset number of frames and the detection device is not in a high-load state.

[0209] The present application also provides a detection device, such as Figure 8 As shown, including:

[0210] Memory 801, used for storing computer programs;

[0211] The processor 802 is used to implement the detection frame rate adjustment method described in any of the above embodiments when executing the program stored in the memory 801.

[0212] In the scheme provided by the embodiment of the present application, the detection device can determine whether the number of continuous target-free frames of the target video acquisition device recorded in real time is greater than the preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any video acquisition device in at least one video acquisition device connected to the detection device; when the number of continuous target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, the maximum detection frame rate that the detection device can currently execute is determined according to the amount of computing resources currently remaining in the detection device; the detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the video collected by the target video acquisition device. Since the detection device has multiple continuous target-free frames in the video collected by the video acquisition device or is in a high-load state, the detection device can determine the maximum detection frame rate that can be executed in real time according to the current remaining computing resources of itself, and adaptively adjust the detection frame rate of the video collected by the video acquisition device according to the maximum detection frame rate. In this way, the detection frame rate can be adaptively changed with the image acquisition situation of the video acquisition device and the load state of the detection device, which greatly improves the flexibility of the detection device in detecting the video collected by the video acquisition device.

[0213] Furthermore, the detection device may further include a communication bus and / or a communication interface, and the processor 802, the communication interface, and the memory 801 communicate with each other via the communication bus.

[0214] The communication bus mentioned in the above detection equipment can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0215] The communication interface is used for communication between the above detection device and other devices.

[0216] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0217] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0218] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the detection frame rate adjustment method described in any of the above embodiments are implemented.

[0219] In another embodiment provided in the present application, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer executes the detection frame rate adjustment method described in any of the above embodiments.

[0220] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0221] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0222] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, detection equipment, computer-readable storage medium and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0223] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A detection frame rate adjustment method, characterized in that: Applied to a detection device, the detection device is connected to at least one video acquisition device, and the method comprises: Determine whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and determine whether the detection device is in a high-load state, wherein the target video acquisition device is any one of the at least one video acquisition device; When the number of the consecutive target-free frames is greater than the preset number of frames, or the detection device is in a high-load state, determining the maximum detection frame rate that the detection device can currently perform according to the amount of computing resources currently remaining in the detection device; The detection frame rate corresponding to the target video acquisition device is adjusted according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the detection device for the video acquired by the target video acquisition device.

2. The method according to claim 1, characterized in that The determining whether the detection device is in a high load state comprises: Calculate a first time consumption according to the current detection frame rate, preset single-frame detection time consumption and preset duration corresponding to the at least one video acquisition device, wherein the current detection frame rate corresponding to the at least one video acquisition device is: the current detection frame rate of the detection device for the video acquired by the at least one video acquisition device, and the first time consumption is: the duration required for the detection device to detect the video acquired by the at least one video acquisition device within the preset duration; Calculate the frame rate difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device, and calculate the product of the frame rate difference, the single frame detection time and the preset time length as the second time consumption, wherein the current detection frame rate corresponding to the target video acquisition device is: the current detection frame rate of the detection device for the video acquired by the target video acquisition device; Determine a total time consumption according to the first time consumption and the second time consumption, and determine whether the total time consumption is not greater than the preset time length; If the total time consumption is greater than the preset time length, it is determined that the detection device is in a high load state; If the total time consumption is not greater than the preset time length, it is determined that the detection device is not in a high load state.

3. The method according to claim 2, characterized in that Before determining the total time consumption according to the first time consumption and the second time consumption, the method further includes: Calculate the third time consumption according to the number of current tasks to be processed and the processing time consumption of a preset single task, wherein the tasks to be processed are processing tasks to be performed by the detection device after completing the video frame detection, and the third time consumption is: the time required for the detection device to complete all the current tasks to be processed; The determining the total time consumption according to the first time consumption and the second time consumption includes: The sum of the first time consumption, the second time consumption and the third time consumption is determined as the total time consumption.

4. The method according to claim 2 or 3, characterized in that: The determining, according to the amount of computing resources currently remaining in the detection device, the maximum detection frame rate currently executable by the detection device, comprises: The fifth time is obtained by subtracting the second time and the fourth time from the total time, wherein the fourth time is: the time required for the detection device to detect the video captured by the target video capture device within the preset time at the current detection frame rate corresponding to the target video capture device; Calculate the difference between the preset duration and the fifth time duration as the remaining calculation duration within the preset duration; The ratio between the remaining calculation time and the target product is calculated, and the maximum detection frame rate that the detection device can currently execute is determined according to the ratio, wherein the target product is the product of the preset time and the single-frame detection time.

5. The method according to claim 1, characterized in that The number of consecutive target-free frames of the target video acquisition device is recorded in real time in the following manner: In the process of detecting the video captured by the target video acquisition device, for each video frame, it is detected whether there is a target in the video frame. If there is a target in the video frame, the number of consecutive non-target frames of the target video acquisition device is set to 0; if there is no target in the video frame, the number of consecutive non-target frames of the target video acquisition device is increased by 1.

6. The method according to claim 5, characterized in that The detecting whether there is a target in the video frame includes: Detect whether there is a target in the target area of ​​the video frame; When there is a target in the target area of ​​the video frame, determining whether the confidence of the detected target is greater than a preset confidence threshold; If the confidence of the detected target is greater than the preset confidence threshold, it is determined that the target exists in the video frame.

7. The method according to any one of claims 1-3, 5-6, characterized in that: Before adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, the method further includes: According to the corresponding relationship between the preset detection frame rate and the detection rate, it is determined whether the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold; If the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, performing the step of adjusting the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate; If the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, the detection frame rate corresponding to the target video acquisition device is adjusted to the target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates whose corresponding detection rates included in the corresponding relationship are not less than the preset detection rate threshold.

8. The method according to any one of claims 1-3, 5-6, characterized in that: The method further comprises: When the number of the consecutive target-free frames is not greater than the preset frame number and the detection device is not in a high-load state, the detection frame rate corresponding to the target video acquisition device is adjusted to the preset detection frame rate.

9. A detection frame rate adjustment device, characterized in that: Applied to a detection device, the detection device is connected to at least one video acquisition device, and the device comprises: A first judgment module is used to judge whether the number of consecutive target-free frames of the target video acquisition device recorded in real time is greater than a preset number of frames, and to judge whether the detection device is in a high-load state, wherein the target video acquisition device is any one of the at least one video acquisition device; A maximum detection frame rate determination module, used to determine the maximum detection frame rate that can be currently executed by the detection device according to the amount of computing resources currently remaining in the detection device when the number of the consecutive target-free frames is greater than the preset number of frames or the detection device is in a high-load state; A detection frame rate adjustment module is used to adjust the detection frame rate corresponding to the target video acquisition device according to the maximum detection frame rate, wherein the detection frame rate corresponding to the target video acquisition device is: the detection frame rate of the detection device for the video acquired by the target video acquisition device.

10. The device according to claim 9, characterized in that The first judgment module includes: a first time-consuming determination unit, which is used to calculate the first time-consuming according to the current detection frame rate corresponding to the at least one video acquisition device, the preset single-frame detection time-consuming and the preset duration, wherein the current detection frame rate corresponding to the at least one video acquisition device is: the current detection frame rate of the video acquired by the detection device for the at least one video acquisition device, and the first time-consuming is: the duration required for the detection device to detect the video acquired by the at least one video acquisition device within the preset duration; a second time-consuming determination unit, which is used to calculate the difference between the preset detection frame rate and the current detection frame rate corresponding to the target video acquisition device , and calculate the product of the frame rate difference, the single frame detection time and the preset time length as the second time length, wherein the current detection frame rate corresponding to the target video acquisition device is: the current detection frame rate of the video acquired by the detection device for the target video acquisition device; a judgment unit, used to determine the total time length according to the first time length and the second time length, and judge whether the total time length is not greater than the preset time length; if the total time length is greater than the preset time length, determine that the detection device is in a high load state; if the total time length is not greater than the preset time length, determine that the detection device is not in a high load state; and / or, The first judgment module further includes: a third time-consuming determination unit, which is used to calculate a third time-consuming according to the number of current tasks to be processed and the processing time of a preset single task, wherein the tasks to be processed are processing tasks to be performed by the detection device after completing the video frame detection, and the third time-consuming is: the time required for the detection device to complete all the current tasks to be processed; the judgment unit is specifically used to determine the sum of the first time-consuming, the second time-consuming and the third time-consuming as the total time-consuming; and / or, The maximum detection frame rate determination module includes: a fifth time consumption determination unit, which is used to subtract the second time consumption and the fourth time consumption from the total time consumption to obtain the fifth time consumption, wherein the fourth time consumption is: the time required for the detection device to detect the video collected by the target video acquisition device within the preset time length at the current detection frame rate corresponding to the target video acquisition device; a remaining calculation time determination unit, which is used to calculate the difference between the preset time length and the fifth time consumption as the remaining calculation time within the preset time length; a maximum detection frame rate determination unit, which is used to calculate the ratio between the remaining calculation time length and the target product, and determine the maximum detection frame rate that the detection device can currently execute according to the ratio, wherein the target product is the product of the preset time length and the single-frame detection time consumption; and / or, The number of consecutive target-free frames of the target video acquisition device is recorded in real time in the following manner: in the process of detecting the video acquired by the target video acquisition device, for each video frame, it is detected whether there is a target in the video frame, if there is a target in the video frame, the number of consecutive target-free frames of the target video acquisition device is set to 0, if there is no target in the video frame, the number of consecutive target-free frames of the target video acquisition device is increased by 1; and / or, The detecting whether there is a target in the video frame includes: detecting whether there is a target in a target area of ​​the video frame; if there is a target in the target area of ​​the video frame, determining whether the confidence of the detected target is greater than a preset confidence threshold; if the confidence of the detected target is greater than the preset confidence threshold, determining that there is a target in the video frame; and / or, The device also includes: a second judgment module, which is used to judge whether the detection rate corresponding to the maximum detection frame rate is less than a preset detection rate threshold according to the corresponding relationship between the preset detection frame rate and the detection rate; if the detection rate corresponding to the maximum detection frame rate is not less than the preset detection rate threshold, trigger the execution of the detection frame rate adjustment module; if the detection rate corresponding to the maximum detection frame rate is less than the preset detection rate threshold, adjust the detection frame rate corresponding to the target video acquisition device to the target detection frame rate, wherein the target detection frame rate is the minimum value of the detection frame rates whose corresponding detection rates included in the corresponding relationship are not less than the preset detection rate threshold; and / or, The detection frame rate adjustment module is also used to adjust the detection frame rate corresponding to the target video acquisition device to the preset detection frame rate when the number of consecutive target-free frames is not greater than the preset frame number and the detection device is not in a high-load state.

11. A detection device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-8 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images