Method and apparatus for personnel safety detection based on dynamic resource allocation

Abstract

The present disclosure relates to a personnel safety detection method and device based on dynamic resource allocation, which does not require additional hardware acceleration equipment and can realize multi-region high-precision monitoring relying on a single host, thereby reducing deployment costs. The method comprises: extracting video frames from camera video streams corresponding to a plurality of safety-restricted areas respectively; performing frame inference tasks corresponding to the plurality of safety-restricted areas respectively according to a first inference frame rate to obtain personnel safety detection results; in the case that the personnel safety detection results represent that there are persons in the safety-restricted areas, determining an abnormal safety-restricted area in which persons exist; suspending the execution of frame inference tasks corresponding to other safety-restricted areas, and performing frame inference tasks corresponding to the abnormal safety-restricted area according to a second inference frame rate to obtain new personnel safety detection results corresponding to the abnormal safety-restricted area, wherein the second inference frame rate is higher than the first inference frame rate.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of safety detection, and in particular to a personnel safety detection method and device based on dynamic resource allocation. BACKGROUND

[0002] In industrial production, warehousing logistics, public facilities and other occasions, personnel safety monitoring is an important link to ensure production order and personal safety. Traditional monitoring systems usually rely on manual duty or simple motion detection algorithms, but these methods have problems such as low efficiency, high false alarm rate, and inability to process complex scenes in real time. In recent years, neural network-based machine vision technology has been gradually applied to the field of personnel safety detection. By capturing video streams through cameras and using deep learning models for target detection and behavior analysis, automatic and intelligent safety monitoring can be achieved.

[0003] However, existing technologies still have significant limitations in multi-region monitoring scenarios. For example, when multiple safety-restricted areas need to be monitored simultaneously, a single host machine cannot perform real-time inference on all areas at a high frame rate due to limitations on GPU (Graphics Processing Unit) computing resources. Taking a common 3070 model GPU as an example, a single frame inference takes about 10 milliseconds, and if 20 areas need to be monitored, the allocated frame rate for each area is only 5 frames per second. Such low frame rate inference may result in slow response speed to abnormal situations (such as illegal entry of personnel), missing critical intervention opportunities. SUMMARY

[0004] To overcome the problems in the related art, the embodiments of the present disclosure provide a personnel safety detection method and device based on dynamic resource allocation to solve the defects in the related art.

[0005] According to a first aspect of the embodiments of the present disclosure, a personnel safety detection method based on dynamic resource allocation is provided, comprising:

[0006] extracting video frames from video streams corresponding to a plurality of safety-restricted areas respectively;

[0007] performing frame inference tasks corresponding to the plurality of safety-restricted areas respectively at a first inference frame rate to obtain personnel safety detection results corresponding to each safety-restricted area, wherein the frame inference task corresponding to a safety-restricted area is used to infer the video frame corresponding to the safety-restricted area through a pre-trained neural network, and the personnel safety detection result is used to represent whether there is a person in the safety-restricted area;

[0008] determining an abnormal safety-restricted area with a person present when the personnel safety detection result represents that there is a person in the safety-restricted area;

[0009] suspend execution of a frame inference task corresponding to a security-restricted area other than the abnormal security-restricted area among the plurality of security-restricted areas, and perform the frame inference task corresponding to the abnormal security-restricted area at a second inference frame rate to obtain a new personnel safety detection result corresponding to the abnormal security-restricted area, wherein the second inference frame rate is higher than the first inference frame rate.

[0010] In some embodiments, the performing the frame inference task corresponding to the abnormal security-restricted area at the second inference frame rate comprises:

[0011] In response to detecting the abnormal security-restricted area, adding the abnormal security-restricted area to an abnormal queue, and determining whether a length of the abnormal queue exceeds a maximum abnormal area number threshold;

[0012] If the length of the abnormal queue does not exceed the maximum abnormal area number threshold, performing the frame inference task corresponding to the abnormal security-restricted area at the second inference frame rate;

[0013] If the length of the abnormal queue exceeds the maximum abnormal area number threshold, determining an inference priority of the plurality of abnormal security-restricted areas, removing abnormal security-restricted areas in the abnormal queue in order from low to high according to the inference priority until the length of the abnormal queue is equal to the maximum abnormal area number threshold, and performing the frame inference task corresponding to the abnormal security-restricted area in the abnormal queue at the second inference frame rate.

[0014] In some embodiments, the maximum abnormal area number threshold is a result of dividing a maximum inference frame rate of a graphics processor performing the inference task by a minimum frame rate required by a target tracking algorithm in the inference task, and then rounding down.

[0015] In some embodiments, the determining the inference priority of the plurality of abnormal security-restricted areas comprises:

[0016] For each of the abnormal security-restricted areas, determining an abnormal confidence, an abnormal category, and an abnormal duration of the abnormal security-restricted area output by the neural network, and performing weighted calculation on the abnormal confidence, the abnormal category, and the abnormal duration based on a first preset weight to obtain an abnormal degree of the abnormal security-restricted area, wherein the abnormal category is used to represent a personnel behavior category in the abnormal security-restricted area.

[0017] For each of the abnormal security restricted areas, a regional asset value, a personnel density, a historical anomaly probability and a compliance level of the abnormal security restricted area are determined, and the regional asset value, the personnel density, the historical anomaly probability and the compliance level are weighted and calculated based on a second preset weight to obtain a business importance degree of the abnormal security restricted area.

[0018] The inference priorities of the plurality of abnormal security restricted areas are determined at least according to the abnormal degrees and the business importance degrees of the plurality of abnormal security restricted areas respectively.

[0019] In some embodiments, further comprising:

[0020] The time decay constants and the scene sensitivity coefficients corresponding to the plurality of abnormal security restricted areas respectively are obtained, wherein the time decay constant is inversely proportional to the resource load of an image processor performing the inference task, and the scene sensitivity coefficient is obtained through reinforcement learning, and a reward value of the reinforcement learning is obtained based on a device loss risk, a personnel loss risk and a compliance risk.

[0021] The inference priorities of the plurality of abnormal security restricted areas are determined at least according to the abnormal degrees and the business importance degrees of the plurality of abnormal security restricted areas respectively.

[0022] For each of the abnormal security restricted areas, an abnormal decay time value corresponding to the abnormal security restricted area is determined according to a time decay constant and an abnormal duration corresponding to the abnormal security restricted area, and the abnormal degree, the business importance degree, the scene sensitivity coefficient and the abnormal decay time value of the abnormal security restricted area are multiplied to obtain a priority score of the abnormal security restricted area.

[0023] The inference priorities of the plurality of abnormal security restricted areas are determined according to the priority scores, wherein the priority score is proportional to the inference priority.

[0024] In some embodiments, the performing of the frame inference task corresponding to the abnormal security restricted area at the second inference frame rate comprises:

[0025] The appearance features and the motion features of a target in the abnormal security restricted area are extracted at the second inference frame rate, and the target is feature-encoded through a deep feature descriptor;

[0026] The motion trajectory of the target is predicted based on a Kalman filter and the feature encoding, and the position change and the speed change of the target in consecutive frames are calculated based on the predicted motion trajectory to establish a motion model;

[0027] identify an abnormal target that does not appear continuously or has an abnormal motion trajectory based on the motion model;

[0028] filter the abnormal target.

[0029] In some embodiments, performing the frame inference task corresponding to the abnormal safety-restricted area at the second inference frame rate comprises:

[0030] predicting a remaining resource usage duration of the GPU based on the abnormal occurrence frequency, the utilization rate of the GPU and the memory bandwidth occupation for performing the inference task within a preset historical duration;

[0031] in a case where the remaining resource usage duration is less than a preset duration threshold, taking the second inference frame rate as an initial target inference frame rate, performing a loop process that: subtracts a preset frame rate threshold from the target inference frame rate to obtain a third inference frame rate, predicts a new remaining resource usage duration of the GPU based on the third inference frame rate, the abnormal occurrence frequency, the utilization rate of the GPU and the memory bandwidth occupation, and in a case where the new remaining resource usage duration is less than the preset duration threshold, takes the third inference frame rate as a new target inference frame rate, until the new remaining resource usage duration is greater than or equal to the preset duration threshold;

[0032] performing the frame inference task corresponding to the abnormal safety-restricted area at the third inference frame rate at the end of the loop process.

[0033] In some embodiments, the method further comprises:

[0034] allocating the inference tasks corresponding to the plurality of safety-restricted areas to different threads through a multi-thread framework, wherein one thread is used to perform the frame inference tasks corresponding to one or more safety-restricted areas; or allocating the inference tasks corresponding to the plurality of safety-restricted areas to different streams through a CUDA stream mechanism, wherein one stream is used to perform the frame inference tasks corresponding to one or more safety-restricted areas.

[0035] According to a second aspect of the embodiments of the present disclosure, a personnel safety detection device based on dynamic resource allocation is provided, comprising:

[0036] a frame extraction module configured to extract video frames from camera video streams corresponding to a plurality of safety-restricted areas respectively;

[0037] The first detection module is configured to perform a plurality of frame inference tasks corresponding to the plurality of safety-restricted areas respectively at a first inference frame rate, to obtain personnel safety detection results corresponding to each safety-restricted area, wherein the frame inference task corresponding to a safety-restricted area is configured to perform inference on a video frame corresponding to the safety-restricted area by using a pre-trained neural network, and the personnel safety detection result is configured to represent whether there is a person in the safety-restricted area;

[0038] The anomaly determination module is configured to determine an abnormal safety-restricted area in which a person exists, in a case where the personnel safety detection result represents that there is a person in the safety-restricted area.

[0039] The second detection module is configured to suspend the execution of the frame inference tasks corresponding to the safety-restricted areas other than the abnormal safety-restricted area among the plurality of safety-restricted areas, and perform the frame inference task corresponding to the abnormal safety-restricted area at a second inference frame rate, to obtain a new personnel safety detection result corresponding to the abnormal safety-restricted area, wherein the second inference frame rate is higher than the first inference frame rate.

[0040] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, which includes a memory and a processor, the memory is configured to store computer instructions executable on the processor, and the processor is configured to implement the method of any one of the first aspect when executing the computer instructions.

[0041] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0042] The personnel safety detection method based on dynamic resource allocation provided by the embodiments of the present disclosure dynamically adjusts inference resource allocation according to the real-time state of the area by combining the first inference frame rate and the second inference frame rate. First, in the normal monitoring state, all areas are uniformly inferred at a lower first inference frame rate; once an abnormality (such as a person breaking in) is detected in a certain area, the second inference frame rate is switched to, and the computing resources are concentrated in the area. In this way, by dynamically adjusting the inference frame rate, the waste of resources caused by fixed frame rate allocation is avoided, so that a single host can monitor more areas, improve the GPU resource utilization rate, the detection accuracy, and the response speed, and is suitable for large-scale deployment in industrial production, warehouse logistics, public facilities, and the like. In addition, the method of the present disclosure does not require additional hardware acceleration devices, and can realize multi-area high-precision monitoring relying on a single host, thereby reducing the deployment cost. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate one embodiment consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0044] Figure 1 is a flow chart of a personnel safety detection method based on dynamic resource allocation according to an example embodiment of the present disclosure;

[0045] Figure 2 is a system workflow diagram corresponding to the personnel safety detection method based on dynamic resource allocation according to an example embodiment of the present disclosure;

[0046] Figure 3 is a schematic diagram of a dynamic inference frame rate adjustment flow in the personnel safety detection method based on dynamic resource allocation according to an example embodiment of the present disclosure;

[0047] Figure 4 is a structural schematic diagram of a personnel safety detection device based on dynamic resource allocation according to an example embodiment of the present disclosure;

[0048] Figure 5 is a structural block diagram of an electronic device according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to designate the same elements, unless otherwise indicated. The embodiments described in the following example embodiments are not representative of all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0050] The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in the present disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0051] It should be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used merely to distinguish one from another. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining."

[0052] As the background art, the prior art still has significant limitations in the multi-region monitoring scene. The low frame rate inference may result in slow response speed for abnormal situations (such as illegal entry of personnel), missing the key intervention opportunity. Some solutions attempt to increase the frame rate by reducing the model complexity or using hardware acceleration devices, but these methods often sacrifice detection accuracy at the cost of high hardware cost, making it difficult to deploy on a large scale. For example, model lightweight techniques (such as MOBILENET, EFFICIENTNET) can improve inference speed, but the detection accuracy in complex scenes will decrease significantly. Hardware acceleration devices (such as FPGA (Field Programmable Gate Array), special AI (Artificial Intelligence) chips) can improve performance, but the deployment cost is high, and the compatibility requirements for existing systems are high.

[0053] Therefore, how to realize multi-region high-precision personnel safety detection under limited computing resources is still a technical problem to be solved.

[0054] Therefore, how to realize multi-region high-precision personnel safety detection under limited computing resources is still a technical problem to be solved. Figure 1 It shows the flow of the method, including steps S101 to S104.

[0055] In step S101, video frames are extracted from the camera video streams corresponding to the plurality of safety-restricted areas respectively.

[0056] In step S102, the frame inference tasks corresponding to the plurality of safety-restricted areas are executed at a first inference frame rate to obtain personnel safety detection results corresponding to the safety-restricted areas. The frame inference task corresponding to the safety-restricted area is used to infer the video frame corresponding to the safety-restricted area through the pre-trained neural network, and the personnel safety detection result is used to represent whether there is a person in the safety-restricted area.

[0057] In step S103, in the case that the personnel safety detection result represents that there is a person in the safety-restricted area, an abnormal safety-restricted area with a person is determined.

[0058] In step S104, the frame inference tasks corresponding to the safety-restricted areas other than the abnormal safety-restricted area in the plurality of safety-restricted areas are suspended, and the frame inference task corresponding to the abnormal safety-restricted area is executed at a second inference frame rate to obtain new personnel safety detection results corresponding to the abnormal safety-restricted area, wherein the second inference frame rate is higher than the first inference frame rate.

[0059] For example, the plurality of cameras are distributed in different security-restricted areas to collect video streams in real time. Each security-restricted area corresponds to a frame inference task, so that the personnel safety detection result of each security-restricted area can be obtained. If the personnel safety detection result of any security-restricted area indicates that there is a person, it is determined that the security-restricted area is abnormal. Thus, in a normal monitoring state, all areas are uniformly inferred at a low frame rate (i.e., a first inference frame rate); once it is detected that a certain area is abnormal (e.g., a person intrudes), the full-time inference mode (i.e., a second inference frame rate) is switched to, and the computing resources are concentrated in the area, so that the inference frame rate can be increased to 100 frames per second, thereby supporting high-precision target tracking and anomaly detection.

[0060] In some embodiments, in response to detecting the abnormal security-restricted area, the abnormal security-restricted area is added to an abnormal queue, and it is determined whether the length of the abnormal queue exceeds a maximum abnormal area number threshold; if the length of the abnormal queue does not exceed the maximum abnormal area number threshold, the frame inference task corresponding to the abnormal security-restricted area is executed at the second inference frame rate; if the length of the abnormal queue exceeds the maximum abnormal area number threshold, the inference priorities of the plurality of abnormal security-restricted areas are determined, the abnormal security-restricted areas in the abnormal queue are removed in order from low to high according to the inference priorities, until the length of the abnormal queue is equal to the maximum abnormal area number threshold, and the frame inference tasks corresponding to the abnormal security-restricted areas in the abnormal queue are executed at the second inference frame rate.

[0061] Thus, an abnormal area priority management mechanism is introduced, and a maximum area number threshold for full-time inference is set. When multiple areas are abnormal at the same time, resources are allocated according to preset priority rules (such as abnormal severity and area importance), to ensure high-frame-rate inference of key areas, while avoiding resource exhaustion.

[0062] In some embodiments, the maximum abnormal area number threshold is the result of the maximum inference frame rate of a graphics processor performing the inference task divided by the minimum frame rate required by a target tracking algorithm in the inference task, rounded down. That is, the maximum abnormal area number threshold can be obtained by the following calculation formula:

[0063] (1) ;

[0064] wherein, is the maximum abnormal area number threshold, is the maximum inference frame rate of a graphics processor performing the inference task, is the minimum frame rate required by a target tracking algorithm in the inference task.

[0065] In some embodiments, determining the reasoning priorities of the plurality of abnormal safety-restricted areas comprises: determining, for each abnormal safety-restricted area, an abnormal confidence, an abnormal category, and an abnormal duration of the abnormal safety-restricted area output by the neural network, and performing weighted calculation on the abnormal confidence, the abnormal category, and the abnormal duration based on a first preset weight to obtain an abnormal degree of the abnormal safety-restricted area; determining, for each abnormal safety-restricted area, a regional asset value, a personnel density, a historical abnormal probability, and a compliance level of the abnormal safety-restricted area, and performing weighted calculation on the regional asset value, the personnel density, the historical abnormal probability, and the compliance level based on a second preset weight to obtain a business importance degree of the abnormal safety-restricted area; and determining the reasoning priorities of the plurality of abnormal safety-restricted areas according to at least the abnormal degrees and the business importance degrees of the plurality of abnormal safety-restricted areas respectively.

[0066] The abnormal category is used to represent a personnel behavior category in the abnormal safety-restricted area, such as personnel intrusion, personnel retention, personnel fighting, etc.

[0067] For example, the first preset weight includes preset weights corresponding to the abnormal confidence, the abnormal category, and the abnormal duration respectively. The second preset weight includes preset weights corresponding to the regional asset value, the personnel density, the historical abnormal probability, and the compliance level respectively.

[0068] For example, the abnormal degree of the abnormal safety-restricted area can be obtained according to the following calculation formula:

[0069] (2);

[0070] wherein, is the abnormal degree of the abnormal safety-restricted area, is the abnormal confidence, is the abnormal category, is the abnormal duration, , and are preset weights corresponding to the abnormal confidence, the abnormal category, and the abnormal duration respectively.

[0071] For example, the business importance degree of the abnormal safety-restricted area can be obtained according to the following calculation formula:

[0072] (3);

[0073] wherein, is the business importance degree of the abnormal safety-restricted area, is the regional asset value, is the personnel density, is the historical abnormal probability, is the compliance level, , , and are preset weights corresponding to the regional asset value, the personnel density, the historical anomaly probability, and the compliance level, respectively.

[0074] In some embodiments, the method further comprises: obtaining time decay constants and scene sensitivity coefficients corresponding to the plurality of anomalous security-restricted areas, respectively, wherein the time decay constant is inversely proportional to the resource load of the image processor performing the inference task, and the scene sensitivity coefficient is obtained through online adjustment of reinforcement learning, and the reward value of the reinforcement learning is based on the device loss risk, the personnel loss risk, and the compliance risk. Accordingly, determining the inference priority of the plurality of anomalous security-restricted areas based on at least the anomaly degree and the business importance degree of each anomalous security-restricted area comprises: for each anomalous security-restricted area, determining an anomaly decay time value of the anomalous security-restricted area based on the time decay constant and the anomaly duration corresponding to the anomalous security-restricted area, and multiplying the anomaly degree, the business importance degree, the scene sensitivity coefficient, and the anomaly decay time value of the anomalous security-restricted area to obtain a priority score of the anomalous security-restricted area; and determining the inference priority of the plurality of anomalous security-restricted areas based on the priority score, wherein the priority score is directly proportional to the inference priority.

[0075] It should be understood that if the priority score does not decay over time, it will regard "just breaking in" and "having been lingering for 10 minutes" as equally important, increasing the risk of delayed detection. In addition, the risk of anomaly is completely different for the same production line in "daytime full load" and "nighttime maintenance" scenarios. Through the scene sensitivity coefficient, "illegal intrusion into a dangerous chemical warehouse" and "on-duty personnel inspection" can be distinguished, reducing false positives or false negatives.

[0076] For example, the reward value of the reinforcement learning can be obtained by weighting the device loss risk, the personnel loss risk, and the compliance risk. The device loss risk can be represented by the number of device downtime minutes, the personnel loss risk can be represented by the actual number of casualties caused by the anomaly, and the compliance risk can be represented by the penalty for violation. The weights for the weighted calculation can be set according to actual needs, such as setting the weight value of the device loss risk to 0.6, the weight value of the personnel loss risk to 0.2, and the weight value of the compliance risk to 0.2, which is not limited in the present disclosure.

[0077] For example, the anomaly decay time value can be a linear decay value or an exponential decay value. Taking the exponential decay value as an example, the anomaly decay time value can be obtained by the following calculation formula:

[0078] (4);

[0079] wherein, anomaly duration, anomaly duration, time decay constant.

[0080] For example, the priority score of the anomaly safety restricted area can be obtained by the following calculation formula:

[0081] (5);

[0082] wherein, priority score, scene sensitivity coefficient.

[0083] In some embodiments, performing a frame inference task corresponding to the anomaly safety restricted area at a second inference frame rate comprises: extracting appearance features and motion features of a target in the anomaly safety restricted area at the second inference frame rate, and encoding the target through a deep feature descriptor; predicting a motion trajectory of the target based on a Kalman filter and the feature encoding, and calculating a position change and a speed change of the target in consecutive frames based on the predicted motion trajectory, and establishing a motion model; identifying an anomaly target that does not appear continuously or has an abnormal motion trajectory based on the motion model; and filtering the anomaly target.

[0084] It should be understood that the low frame rate inference of the prior art cannot effectively handle the "ghost" phenomenon (such as false detection caused by light changes, object shaking, etc.) in dynamic scenes, and it is difficult to implement a target tracking algorithm, resulting in a decrease in the continuity and accuracy of personnel detection. In the embodiments of the present disclosure, at the second inference frame rate, a target tracking algorithm (such as SORT or DEEPSORT) can be combined to continuously detect personnel, and "ghost" interference can be eliminated through trajectory analysis and feature matching. The continuous frame data generated by high frame rate inference is used to establish a motion trajectory model of the target, and targets that do not appear continuously are eliminated, thereby effectively suppressing the false detection phenomenon in dynamic scenes.

[0085] In some embodiments, performing the frame inference task corresponding to the abnormal safety restricted area at the second inference frame rate comprises: predicting a remaining resource usage duration of the graphics processor based on the frequency of abnormality occurrence within a preset historical duration, the utilization rate of the graphics processor for performing the inference task, and the memory bandwidth occupation; in a case where the remaining resource usage duration is less than a preset duration threshold, taking the second inference frame rate as an initial target inference frame rate, and performing a loop process: subtracting a preset frame rate threshold from the target inference frame rate to obtain a third inference frame rate, predicting a new remaining resource usage duration of the graphics processor based on the third inference frame rate, the frequency of abnormality occurrence, the utilization rate of the graphics processor, and the memory bandwidth occupation, and in a case where the new remaining resource usage duration is less than the preset duration threshold, taking the third inference frame rate as a new target inference frame rate, until the new remaining resource usage duration is greater than or equal to the preset duration threshold; and performing the frame inference task corresponding to the abnormal safety restricted area at the third inference frame rate at the end of the loop process.

[0086] For example, a drop amplitude can be preset, such as 5 frames per second.

[0087] In this way, the risk of resource exhaustion can be discovered in advance, and high frame rate inference can be ensured as much as possible in the case of sufficient resources.

[0088] In some embodiments, the inference tasks corresponding to the plurality of safety restricted areas can also be respectively distributed to different threads by a multi-thread framework, wherein one thread is used to perform the frame inference task corresponding to one or more safety restricted areas; or the inference tasks corresponding to the plurality of safety restricted areas are respectively distributed to different streams by using a CUDA stream mechanism, wherein one stream is used to perform the frame inference task corresponding to one or more safety restricted areas. In this way, by multi-thread task scheduling, the inference tasks are distributed to different GPU streams (STREAM), ensuring parallel execution of the inference tasks, while reducing the overhead of task switching.

[0089] The personnel safety detection method based on dynamic resource allocation provided by the present disclosure will be described below through another embodiment.

[0090] For example, the system architecture of the method comprises the following core modules:

[0091] Camera distribution: a plurality of cameras are distributed in different safety restricted areas to collect video streams in real time.

[0092] Edge host: receiving the video stream of the camera, responsible for inference calculation and alarm triggering.

[0093] GPU resource manager: dynamically allocating inference tasks and controlling the inference frame rate of each area.

[0094] Alarm management module: responsible for alarm triggering and recording of abnormal conditions.

[0095] As Figure 2 shown below is the workflow of the system:

[0096] 1. Initialization phase: the camera establishes a connection with the edge host and starts video streaming. The host loads a pre-trained neural network model (such as YOLOv8 or EfficientDet). Configure the security restricted range of each area (defined by coordinate points). Initialize the GPU resource manager and set dynamic inference parameters (such as uniform inference frame rate, full-time inference frame rate, maximum number of abnormal regions, etc.).

[0097] 2. Uniform inference phase: the edge host extracts frames from each camera video stream at fixed intervals. Use the pre-trained neural network to infer each frame and detect the bounding box (box coordinates) of the person. If the box coordinates of the detected person are located within the restricted area defined in advance, an alarm is triggered and the full-time inference mode is entered. The inference frame rate is dynamically allocated according to the GPU performance and the number of monitoring areas (for example, a 3070 GPU monitors 20 areas, each area is allocated 5 frames / second).

[0098] 3. Full-time inference phase: when an abnormality is detected in a region, the GPU resource manager immediately increases the inference frame rate of that region to 100 frames / second, while suspending the inference tasks of other regions. In full-time inference mode, combine target tracking algorithms (such as DeepSORT) to continuously detect people and eliminate "ghost" phenomena through trajectory analysis. If multiple regions have abnormalities at the same time, resources are allocated according to pre-set priority rules to ensure high frame rate inference in key areas.

[0099] 4. Resource recovery mechanism: when the abnormal region returns to normal (such as when the person leaves the restricted area), the GPU resource manager switches the region back to the uniform inference mode and resumes monitoring other regions.

[0100] As Figure 3 shown in the disclosure, the dynamic inference frame rate adjustment process includes uniform inference frame rate (i.e. the first inference frame rate mentioned above) and full-time inference frame rate (i.e. the second inference frame rate mentioned above). The inference frame rate allocation follows the following calculation formula:

[0101] (6);

[0102] where, is the actual inference frame rate used, is the uniform inference frame rate (such as 5 frames / second), is the full-time inference frame rate (such as 100 frames / second).

[0103] In the full-time reasoning mode, the target tracking algorithm is used for continuous detection of personnel, and the "ghost" is eliminated through the following steps:

[0104] 1. Feature extraction: Extract the appearance features (such as color, shape) and motion features (such as speed, direction) of the target. Use deep feature descriptors (such as ResNet-34 in DeepSORT) to encode the features of the target.

[0105] 2. Trajectory modeling: Based on Kalman filter, the motion trajectory of the target is predicted and updated. Calculate the position change and speed change of the target in consecutive frames, and establish a motion model.

[0106] 3. Ghost elimination: For targets that do not appear continuously or have abnormal motion trajectories, mark them as "ghosts" and eliminate them. By setting a minimum frame threshold (such as 5 frames) for continuous appearance, filter out short-term false detection targets.

[0107] In order to improve the utilization of GPU resources, the present disclosure adopts asynchronous reasoning and resource scheduling optimization technology:

[0108] 1. Multi-thread task scheduling: Use multi-thread framework (such as OpenMP or C++11 thread library) to distribute reasoning tasks to different threads. Each thread is responsible for the reasoning task of one or more cameras, ensuring parallel execution of tasks.

[0109] 2. GPU stream management: Use CUDA stream mechanism to distribute reasoning tasks to different streams to achieve parallel execution of tasks. Through asynchronous memory copy and kernel launch, reduce the overhead of task switching.

[0110] 3. Task priority queue: The reasoning task of the abnormal area has the highest priority, ensuring the timely processing of critical tasks. The reasoning task of the normal area is allocated resources in a round-robin manner, ensuring the monitoring coverage of all areas.

[0111] In order to achieve high-precision detection under limited resources, the present invention combines model lightweight and knowledge distillation technology:

[0112] 1. Model lightweight: Use lightweight detection models (such as YOLOv8-N or EfficientDet-Lite) as the base model to reduce reasoning delay. Further compress the model size through model pruning and quantization technology to improve reasoning efficiency.

[0113] 2. Knowledge distillation: Migrate the feature representation of large models (such as YOLOv8-L) to lightweight models to maintain high accuracy while reducing computational complexity. Use soft labels and feature map distillation methods to ensure that the detection performance of lightweight models approaches that of large models.

[0114] To reduce inference latency, the present invention adopts the following optimization strategies:

[0115] 1. Batch Inference: In the uniform inference phase, frame data from multiple regions are organized into batches and sent to GPU inference in batches, reducing inference overhead. The batch size is dynamically adjusted to balance latency and throughput.

[0116] 2. Memory Management Optimization: Use shared memory and zero-copy technology to reduce data transfer overhead. Pre-allocate memory space required for inference to avoid latency caused by dynamic memory allocation.

[0117] 3. Operator Fusion Optimization: Fuse consecutive operators in the neural network (such as convolution and activation function) into a single calculation unit to reduce kernel call overhead. Use CUDA's Tensor Core technology to accelerate matrix operations and improve inference efficiency.

[0118] To improve GPU resource utilization, the present invention adopts the following strategies:

[0119] 1. Load Balancing: Dynamically adjust the allocation of inference tasks for each region to ensure balanced GPU load. During abnormal region processing, dynamically recycle idle resources for other tasks.

[0120] 2. Task Prefetching and Caching: Prefetch the next frame data and cache it to reduce task switching latency. Use GPU cache to store frequently accessed data (such as model weights) to improve access speed.

[0121] 3. Asynchronous Data Processing: While performing inference tasks, asynchronously process the next frame data (such as decoding and preprocessing) to improve overall throughput.

[0122] In a production line monitoring scenario in a certain factory, a personnel safety detection system based on the present disclosure is deployed: Number of cameras: 20 cameras covering all safety-restricted areas. GPU configuration: Single host with 3070 GPU. Uniform inference frame rate: 5 frames / second. Full-time inference frame rate: 100 frames / second. Maximum number of abnormal regions threshold: 5.

[0123] System performance is as follows: In normal monitoring state, all regions perform uniform inference at a frame rate of 5 frames / second, with no resource bottlenecks. When a region detects a personnel intrusion, it immediately switches to full-time inference mode at a frame rate of 100 frames / second, and the target tracking algorithm successfully eliminates the "ghost" phenomenon, achieving a detection accuracy of 99.5%. When multiple regions simultaneously appear abnormal, the system allocates resources according to priority rules to ensure high frame rate inference in critical regions, without resource depletion.

[0124] In addition, the present disclosure supports the following extensions:

[0125] 1. Multi-host cluster deployment: Support multiple hosts to work together through distributed task scheduling, further expand the number of monitoring areas.

[0126] 2. Model upgrade: Support online updating of detection model without restarting the system.

[0127] 3. Function extension: Can integrate behavior analysis (such as running, climbing), identity recognition and other functions to meet more scene requirements.

[0128] In addition, the present disclosure can be compatible with the following hardware and software platforms: Hardware platform: Support NVIDIA GPU (such as 3070, 3090, A100) and mainstream CPU architecture (such as x86, ARM). Software platform: Support Windows, Linux operating system, compatible with ONNX Runtime, TensorRT and other inference engines.

[0129] Therefore, through dynamic inference frame rate adjustment, abnormal area priority management, target tracking and ghost elimination, etc. Innovative technology, multi-area high-precision personnel safety detection is realized under limited GPU resources. Compared with the prior art, the present application has significant improvement in resource utilization, detection accuracy, response speed, etc. It is suitable for large-scale deployment in industrial production, warehouse logistics, public facilities and other scenes. And through system performance optimization and scalability design, it can flexibly adapt to monitoring requirements of different scales and complexities, providing an efficient and low-cost solution for personnel safety monitoring field.

[0130] According to a second aspect of the embodiments of the present disclosure, a personnel safety detection device based on dynamic resource allocation is provided. Please refer to the accompanying Figure 4 The personnel safety detection device 400 based on dynamic resource allocation comprises:

[0131] The frame extraction module 401 is configured to extract video frames from the video streams of the cameras corresponding to the plurality of safety restricted areas respectively;

[0132] The first detection module 402 is configured to perform frame inference tasks corresponding to the plurality of safety restricted areas respectively at a first inference frame rate, to obtain personnel safety detection results corresponding to each safety restricted area, wherein the frame inference task corresponding to the safety restricted area is used to infer the video frames corresponding to the safety restricted area through a pre-trained neural network, and the personnel safety detection result is used to represent whether there is a person in the safety restricted area;

[0133] The anomaly determination module 403 is configured to determine an abnormal safety restricted area with a person present when the personnel safety detection result represents that there is a person in the safety restricted area;

[0134] The second detection module 404 is configured to suspend execution of the frame inference task corresponding to the other security-restricted areas except the abnormal security-restricted area among the plurality of security-restricted areas, and perform the frame inference task corresponding to the abnormal security-restricted area at a second inference frame rate to obtain a new personnel safety detection result corresponding to the abnormal security-restricted area, where the second inference frame rate is higher than the first inference frame rate.

[0135] In some embodiments of the present disclosure, the abnormal security-restricted area is multiple, and the second detection module 404 is further configured to:

[0136] In response to detecting the abnormal security-restricted area, adding the abnormal security-restricted area to an exception queue, and determining whether a length of the exception queue exceeds a maximum exception area number threshold;

[0137] If the length of the exception queue does not exceed the maximum exception area number threshold, performing the frame inference task corresponding to the abnormal security-restricted area at the second inference frame rate;

[0138] If the length of the exception queue exceeds the maximum exception area number threshold, determining an inference priority of the plurality of abnormal security-restricted areas, removing the abnormal security-restricted areas in the exception queue in order from low to high according to the inference priority until the length of the exception queue is equal to the maximum exception area number threshold, and performing the frame inference task corresponding to the abnormal security-restricted areas in the exception queue at the second inference frame rate.

[0139] In some embodiments of the present disclosure, the maximum exception area number threshold is a result of dividing a maximum inference frame rate of a graphics processor performing the inference task by a minimum frame rate required by a target tracking algorithm in the inference task, and then rounding down.

[0140] In some embodiments of the present disclosure, the second detection module 404 is further configured to:

[0141] For each abnormal security-restricted area, determining an abnormal confidence, an abnormal category, and an abnormal duration of the abnormal security-restricted area output by the neural network, and performing weighted calculation on the abnormal confidence, the abnormal category, and the abnormal duration based on a first preset weight to obtain an abnormal degree of the abnormal security-restricted area, where the abnormal category is used to represent a personnel behavior category in the abnormal security-restricted area.

[0142] For each of the abnormal security restricted areas, a regional asset value, a personnel density, a historical anomaly probability and a compliance level of the abnormal security restricted area are determined, and the regional asset value, the personnel density, the historical anomaly probability and the compliance level are weighted and calculated based on a second preset weight to obtain a business importance degree of the abnormal security restricted area.

[0143] The inference priorities of the plurality of abnormal security restricted areas are determined according to at least the abnormal degrees and the business importance degrees of the plurality of abnormal security restricted areas respectively.

[0144] In some embodiments of the present disclosure, the method further comprises a obtaining module configured to:

[0145] obtain a time decay constant and a scene sensitivity coefficient corresponding to each of the plurality of abnormal security restricted areas, wherein the time decay constant is inversely proportional to a resource load of an image processor performing the inference task, and the scene sensitivity coefficient is obtained through reinforcement learning, and a reward value of the reinforcement learning is obtained based on a device loss risk, a personnel loss risk and a compliance risk;

[0146] The second detection module 404 is further configured to:

[0147] For each of the abnormal security restricted areas, an abnormal decay time value corresponding to the abnormal security restricted area is determined according to a time decay constant and an abnormal duration corresponding to the abnormal security restricted area, and a priority score of the abnormal security restricted area is obtained by multiplying the abnormal degree, the business importance degree, the scene sensitivity coefficient and the abnormal decay time value of the abnormal security restricted area.

[0148] The inference priorities of the plurality of abnormal security restricted areas are determined according to the priority scores, wherein the priority scores are directly proportional to the inference priorities.

[0149] In some embodiments of the present disclosure, the second detection module 404 is further configured to:

[0150] The appearance features and the motion features of the target in the abnormal security restricted area are extracted at a second inference frame rate, and the target is feature-encoded through a deep feature descriptor;

[0151] Based on a Kalman filter and the feature encoding, a motion trajectory of the target is predicted, and a position change and a speed change of the target in consecutive frames are calculated based on the predicted motion trajectory to establish a motion model;

[0152] Based on the motion model, an abnormal target that does not appear continuously or has an abnormal motion trajectory is identified;

[0153] filter the abnormal target.

[0154] In some embodiments of the present disclosure, the second detection module 404 is further configured to:

[0155] predict a remaining resource usage duration of the graphics processor based on the abnormal occurrence frequency, a utilization rate of the graphics processor for performing the inference task, and a memory bandwidth occupation;

[0156] in a case where the remaining resource usage duration is less than a preset duration threshold, taking the second inference frame rate as an initial target inference frame rate, performing a loop process of: subtracting a preset frame rate threshold from the target inference frame rate to obtain a third inference frame rate, predicting a new remaining resource usage duration of the graphics processor based on the third inference frame rate, the abnormal occurrence frequency, the utilization rate of the graphics processor, and the memory bandwidth occupation, and in a case where the new remaining resource usage duration is less than the preset duration threshold, taking the third inference frame rate as a new target inference frame rate, until the new remaining resource usage duration is greater than or equal to the preset duration threshold;

[0157] performing the frame inference task corresponding to the abnormal safety restricted region according to the third inference frame rate at the end of the loop process.

[0158] In some embodiments of the present disclosure, further comprising a parallel module configured to:

[0159] allocating, by a multi-thread framework, the inference tasks corresponding to the plurality of safety restricted regions respectively to different threads, wherein one thread is used to perform the frame inference tasks corresponding to one or more safety restricted regions; or allocating, by a CUDA stream mechanism, the inference tasks corresponding to the plurality of safety restricted regions respectively to different streams, wherein one stream is used to perform the frame inference tasks corresponding to one or more safety restricted regions.

[0160] As to the apparatus in the above embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method in the first aspect, and will not be described herein in detail.

[0161] According to a third aspect of the embodiments of the present disclosure, refer to the accompanying drawings Figure 5 which shows a block diagram of an electronic device 700. The electronic device 700 can include a processor 701 and a memory 702. The electronic device 700 can also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0162] The processor 701 is configured to control overall operations of the electronic device 700 to complete all or part of the steps of any of the above-described methods. The memory 702 is configured to store various types of data to support operations of the electronic device 700, which can include, for example, instructions for operating any application or method on the electronic device 700, and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk. The multimedia component 703 can include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 702 or transmitted through the communication component 705. The audio component also includes at least one speaker configured to output audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 705 is configured to perform wired or wireless communication between the electronic device 700 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so the corresponding communication component 705 can include a Wi-Fi module, a Bluetooth module, an NFC module.

[0163] In an exemplary embodiment, the electronic device 700 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-mentioned personnel safety detection method based on dynamic resource allocation.

[0164] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of any of the above-mentioned methods. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, which can be executed by the processor 701 of the electronic device 700 to complete any of the above-mentioned methods.

[0165] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a processor, and when the computer program is executed by the processor, the steps of any of the above-mentioned methods are implemented.

[0166] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details in the above-described embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0167] In addition, it should be noted that each specific technical feature described in the above-described specific embodiments can be combined in any appropriate manner without contradiction, and in order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0168] Furthermore, any combination of the various different embodiments of the present disclosure can also be made, as long as it does not deviate from the idea of the present disclosure, and it should also be considered as disclosed by the present disclosure.

Claims

1. A method for personnel safety detection based on dynamic resource allocation, characterized in that, The method comprises: extracting video frames from camera video streams corresponding to a plurality of security-restricted areas respectively; performing frame inference tasks corresponding to the plurality of security-restricted areas respectively according to a first inference frame rate to obtain personnel safety detection results corresponding to each security-restricted area, wherein a frame inference task corresponding to a security-restricted area is used to infer video frames corresponding to the security-restricted area by a pre-trained neural network, and the personnel safety detection result is used to represent whether there is a person in the security-restricted area; in a case where the personnel safety detection result represents that there is a person in the security-restricted area, determining an abnormal security-restricted area in which a person exists; suspending the performance of frame inference tasks corresponding to security-restricted areas other than the abnormal security-restricted area among the plurality of security-restricted areas, and performing a frame inference task corresponding to the abnormal security-restricted area according to a second inference frame rate to obtain a new personnel safety detection result corresponding to the abnormal security-restricted area, wherein the second inference frame rate is higher than the first inference frame rate; the performing of the frame inference task corresponding to the abnormal security-restricted area according to the second inference frame rate comprises: in response to detecting the abnormal security-restricted area, adding the abnormal security-restricted area to an abnormal queue, and determining whether the length of the abnormal queue exceeds a maximum abnormal area number threshold; if the length of the abnormal queue does not exceed the maximum abnormal area number threshold, performing the frame inference task corresponding to the abnormal security-restricted area according to the second inference frame rate; if the length of the abnormal queue exceeds the maximum abnormal area number threshold, determining inference priorities of a plurality of abnormal security-restricted areas, removing abnormal security-restricted areas in the abnormal queue in order from low to high according to the inference priorities until the length of the abnormal queue is equal to the maximum abnormal area number threshold, and performing frame inference tasks corresponding to abnormal security-restricted areas in the abnormal queue according to the second inference frame rate.

2. The method of claim 1, wherein, The maximum abnormal area number threshold is obtained by dividing the maximum inference frame rate of a graphics processing unit performing the inference task by the minimum frame rate required by a target tracking algorithm in the inference task and then taking the floor of the result.

3. The method of claim 1, wherein, The determination of the inference priorities of the plurality of abnormal security-restricted areas comprises: for each abnormal security-restricted area, determining an abnormal confidence, an abnormal category, and an abnormal duration of the abnormal security-restricted area output by the neural network, and performing weighted calculation on the abnormal confidence, the abnormal category, and the abnormal duration based on a first preset weight to obtain an abnormal degree of the abnormal security-restricted area, wherein the abnormal category is used to represent a personnel behavior category in the abnormal security-restricted area; for each abnormal security-restricted area, determining a regional asset value, a personnel density, a historical abnormal probability, and a compliance level of the abnormal security-restricted area, and performing weighted calculation on the regional asset value, the personnel density, the historical abnormal probability, and the compliance level based on a second preset weight to obtain a business importance degree of the abnormal security-restricted area. Determine reasoning priorities of the multiple abnormal security restricted areas according to abnormal degrees and business importance degrees corresponding to the multiple abnormal security restricted areas respectively.

4. The method of claim 3, wherein, Further comprising: Obtain time decay constants and scene sensitivity coefficients corresponding to the multiple abnormal security restricted areas respectively, wherein the time decay constants are inversely proportional to resource loads of an image processor performing the reasoning task, and the scene sensitivity coefficients are obtained through reinforcement learning, and a reward value of the reinforcement learning is obtained based on device loss risk, personnel loss risk and compliance risk weighting; The determination of the reasoning priorities of the multiple abnormal security restricted areas according to the abnormal degrees and the business importance degrees corresponding to the multiple abnormal security restricted areas respectively comprises: For each abnormal security restricted area, determine an abnormal decay time value corresponding to the abnormal security restricted area according to a time decay constant and an abnormal duration corresponding to the abnormal security restricted area, and multiply the abnormal degree, the business importance degree, the scene sensitivity coefficient and the abnormal decay time value of the abnormal security restricted area to obtain a priority score of the abnormal security restricted area; Determine the reasoning priorities of the multiple abnormal security restricted areas according to the priority scores, wherein the priority scores are directly proportional to the reasoning priorities.

5. The method of personnel safety detection based on dynamic resource allocation according to any one of claims 1-4, characterized in that, The execution of the frame reasoning task corresponding to the abnormal security restricted area at the second reasoning frame rate comprises: Extract appearance features and motion features of a target in the abnormal security restricted area at the second reasoning frame rate, and encode the target through a deep feature descriptor; Predict a motion trajectory of the target based on a Kalman filter and the feature encoding, and calculate a position change and a speed change of the target in consecutive frames based on the predicted motion trajectory to establish a motion model; Identify an abnormal target that does not appear continuously or has an abnormal motion trajectory based on the motion model; Filter the abnormal target.

6. The method of personnel safety detection based on dynamic resource allocation according to any one of claims 1-4, characterized in that, The execution of the frame reasoning task corresponding to the abnormal security restricted area at the second reasoning frame rate comprises: Predict a remaining resource usage duration of a graphics processing unit used to perform the reasoning task based on an abnormal occurrence frequency within a preset historical duration, a utilization rate of the graphics processing unit and a memory bandwidth occupation; In a case where the remaining resource usage duration is less than a preset duration threshold, set the second reasoning frame rate as an initial target reasoning frame rate, and perform a loop process that subtracts a preset frame rate threshold from the target reasoning frame rate to obtain a third reasoning frame rate, predicts a new remaining resource usage duration of the graphics processing unit based on the third reasoning frame rate, the abnormal occurrence frequency, the utilization rate of the graphics processing unit and the memory bandwidth occupation, and sets the third reasoning frame rate as a new target reasoning frame rate in a case where the new remaining resource usage duration is less than the preset duration threshold, until the new remaining resource usage duration is greater than or equal to the preset duration threshold; Execute the frame reasoning task corresponding to the abnormal security restricted area at the third reasoning frame rate at the end of the loop process.

7. The method of personnel safety detection based on dynamic resource allocation according to any one of claims 1-4, characterized in that, Further comprising: The inference tasks corresponding to the plurality of security-restricted areas are respectively allocated to different threads through a multi-thread framework, wherein one thread is used to execute the frame inference tasks corresponding to one or more security-restricted areas; or the inference tasks corresponding to the plurality of security-restricted areas are respectively allocated to different streams by using a CUDA stream mechanism, wherein one stream is used to execute the frame inference tasks corresponding to one or more security-restricted areas.

8. A personnel safety detection apparatus based on dynamic resource allocation, characterized by, The method comprises the following steps: a frame extraction module is configured to extract video frames from camera video streams corresponding to a plurality of security-restricted areas; a first detection module is configured to execute frame inference tasks corresponding to the plurality of security-restricted areas at a first inference frame rate to obtain personnel safety detection results corresponding to each security-restricted area, wherein the frame inference tasks corresponding to a security-restricted area are used to infer video frames corresponding to the security-restricted area by using a pre-trained neural network, and the personnel safety detection results are used to represent whether there is a person in the security-restricted area; an abnormality determination module is configured to determine an abnormal security-restricted area in which a person exists in a case where the personnel safety detection results represent that there is a person in the security-restricted area; a second detection module is configured to pause the execution of frame inference tasks corresponding to security-restricted areas other than the abnormal security-restricted area among the plurality of security-restricted areas, and execute the frame inference tasks corresponding to the abnormal security-restricted area at a second inference frame rate to obtain new personnel safety detection results corresponding to the abnormal security-restricted area, wherein the second inference frame rate is higher than the first inference frame rate; The second detection module is further configured to: in response to detecting the abnormal security-restricted area, add the abnormal security-restricted area to an abnormality queue, and determine whether the length of the abnormality queue exceeds a maximum abnormal area number threshold; if the length of the abnormality queue does not exceed the maximum abnormal area number threshold, execute the frame inference tasks corresponding to the abnormal security-restricted area at the second inference frame rate; if the length of the abnormality queue exceeds the maximum abnormal area number threshold, determine the inference priorities of the plurality of abnormal security-restricted areas, remove abnormal security-restricted areas in the abnormality queue in order from low to high according to the inference priorities, until the length of the abnormality queue is equal to the maximum abnormal area number threshold, and execute the frame inference tasks corresponding to the abnormal security-restricted areas in the abnormality queue at the second inference frame rate.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is used to store computer instructions executable on the processor, and the processor is used to implement the steps of the method of any one of claims 1-7 when executing the computer instructions.

Images