Personnel falling detection method, device, equipment and storage medium

Abstract

The present application provides a kind of personnel fall detection method, device, equipment and storage medium, the method comprises: obtaining the real-time monitoring data that camera combination is collected for target area;Wherein, the camera combination includes at least two cameras;Personnel fall detection is carried out based on the real-time monitoring data of each camera respectively;When detecting the personnel fall detection result corresponding to any one of the camera as personnel fall, personnel fall alarm is triggered.The present application simultaneously carries out personnel fall detection to target area by using multiple cameras, and when the detection result corresponding to any one of the camera is personnel fall, alarm processing is carried out, so as to avoid the missed detection caused by angle problem, so as to improve the accuracy of personnel fall detection without increasing the performance of camera itself.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to a method, apparatus, equipment, and storage medium for detecting people falling to the ground. Background Technology

[0002] With increasing demands for safety and improved economic conditions, video surveillance technology is becoming increasingly widespread. Among these advancements, the demand for human behavior recognition, especially fall detection, is also growing.

[0003] In existing technology 1, a camera capable of extracting depth information is used to create a 3D model of the scene, and then a human body is detected. The height of the human body from the ground is then used to determine whether the person has fallen. This technology is highly accurate because it uses a special camera capable of 3D modeling. However, due to the high cost of the camera and its limited scope of application, it is not easy to promote and cannot utilize existing ordinary cameras.

[0004] In existing technology 2, deep learning human detection technology is used to detect human bodies in the monitoring image of a monocular camera, and then the change in the aspect ratio of the human body detection box is used to determine whether the person has fallen. This method has low accuracy. When the direction in which the person falls is consistent with the direction of the camera's monitoring, the change in the aspect ratio before and after the person falls is not obvious, which will lead to missed detections. When the person's arms are outstretched, the change in the aspect ratio will cause false detections.

[0005] In existing technology 3, deep learning object detection technology is used in the monitoring footage of a monocular camera to directly identify the fallen human body as a target, or after detecting the human body, a classifier for fallen and other categories is trained using deep learning object classification technology to finally confirm whether it is a case of falling. This method has low accuracy. These two methods are currently the most commonly used and are suitable for simple scenarios where accuracy requirements are not high. However, since only a regular camera is used, even with the most advanced deep learning technology, it is difficult to improve accuracy, making it unsuitable for complex scenarios.

[0006] In summary, existing fall detection technologies suffer from low accuracy or are difficult to widely apply due to the high performance requirements of the cameras. Summary of the Invention

[0007] The present invention aims to provide a method, apparatus, device and storage medium for detecting people falling to the ground, so as to solve the above-mentioned technical problems and improve the accuracy of detecting people falling to the ground without increasing the performance of the camera itself.

[0008] To address the aforementioned technical problems, this invention provides a method for detecting a person falling to the ground, comprising:

[0009] Acquire real-time monitoring data collected by a camera array targeting a target area; wherein the camera array includes at least two cameras;

[0010] Personnel fall detection is performed based on real-time monitoring data from each of the aforementioned cameras;

[0011] When a person is detected to have fallen to the ground by any of the cameras, a person fall alarm is triggered.

[0012] Furthermore, the step of detecting a person falling to the ground based on real-time monitoring data from each of the cameras includes:

[0013] The system uses a pre-defined human detection model to detect human bodies in the target area and calculates the aspect ratio of the human body bounding box for each detected person.

[0014] When the ratio of the width to height of the target person's body frame is greater than a preset threshold, it is determined that a person has fallen to the ground.

[0015] Furthermore, the determination that a person has fallen to the ground occurs when the aspect ratio of the target person's body frame is greater than a preset threshold is as follows:

[0016] When the aspect ratio of the target person's body frame is determined to be greater than a preset threshold, the target person is marked as a suspected person who has fallen to the ground.

[0017] The suspected fallen person's limb joint points are detected, and the body outline of the suspected fallen person is corrected after the detected target joint points are removed.

[0018] The aspect ratio of the suspected fallen person's body frame is recalculated based on the corrected body frame. When the aspect ratio of the suspected fallen person's body frame is greater than the preset threshold, it is determined that a person has fallen.

[0019] Furthermore, the step of triggering a person fall alarm when any of the cameras detects a person falling includes:

[0020] When a person is detected falling to the ground by any of the cameras, the target person who has fallen is tracked and detected.

[0021] When it is determined that the fallen person's fallen state continues for more than a preset time threshold, a person falling alarm is triggered.

[0022] Furthermore, the step of triggering a person fall alarm when any of the cameras detects a person falling includes:

[0023] When at least two cameras are detected to have caused a person to fall within a preset time period, the multiple fall alarm events triggered within the preset time period are deduplicated.

[0024] Furthermore, the deduplication process for multiple fall alarm events triggered within the preset time period specifically includes:

[0025] Multiple fall alarm events triggered within the preset time period are merged into a single fall alarm event.

[0026] Furthermore, the camera assembly consists of three cameras, and the angle between the line connecting each adjacent camera and the center of the target area is within the range of 80° to 130°.

[0027] The present invention also provides a person falling detection device, comprising:

[0028] A data acquisition module is used to acquire real-time monitoring data collected by the camera assembly for a target area; wherein the camera assembly includes at least two cameras;

[0029] The fall detection module is used to detect people falling based on the real-time monitoring data of each of the cameras.

[0030] The alarm triggering module is used to trigger a person falling alarm when the detection result of any of the cameras corresponding to a person falling to the ground is detected.

[0031] The present invention also provides a terminal device, including a processor and a memory storing a computer program, wherein the processor executes the computer program to implement any of the personnel fall detection methods described above.

[0032] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the personnel fall detection methods described above.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] This invention provides a method, apparatus, device, and storage medium for detecting fallen persons. The method includes: acquiring real-time monitoring data collected by a camera array targeting a target area; wherein the camera array includes at least two cameras; performing fallen person detection based on the real-time monitoring data of each camera; and triggering a fallen person alarm when the detection result corresponding to any of the cameras indicates that a person has fallen. This invention, by employing multiple cameras to simultaneously detect fallen persons in a target area and triggering an alarm when the detection result corresponding to any of the cameras indicates a fallen person, avoids missed detections due to angle issues, thereby improving the accuracy of fallen person detection without increasing the performance of the cameras themselves. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a method for detecting a person falling to the ground according to an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of the structure of a personnel fall detection device provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Please see Figure 1 This invention provides a method for detecting a person falling to the ground, which may include the following steps:

[0039] S1. Acquire real-time monitoring data collected by the camera combination for the target area; wherein the camera combination includes at least two cameras; further, the number of cameras in the camera combination is three, and the angle between the line connecting each of two adjacent cameras and the center of the target area is within the range of 80° to 130°.

[0040] S2. Perform personnel fall detection based on the real-time monitoring data of each of the cameras;

[0041] S3. When the detection result of a person falling to the ground is detected by any of the cameras, a person falling to the ground alarm is triggered.

[0042] It should be noted that two or more cameras are positioned at a certain angle to detect people falling in the target area from different angles, thus avoiding missed detections. Person fall detection is performed based on real-time monitoring data from each camera, which can be understood as multiple detection threads running simultaneously. When any thread detects a person falling, an alarm event is triggered.

[0043] The personnel fall detection method provided in this embodiment of the invention uses multiple cameras to simultaneously detect personnel falls in a target area and issues an alarm when the detection result of any camera indicates that a person has fallen. This avoids missed detections due to angle issues and improves the accuracy of personnel fall detection without increasing the performance of the cameras themselves.

[0044] In this embodiment of the invention, the step of detecting a person falling to the ground based on the real-time monitoring data of each of the cameras further includes:

[0045] The system uses a pre-defined human detection model to detect human bodies in the target area and calculates the aspect ratio of the human body bounding box for each detected person.

[0046] When the ratio of the width to height of the target person's body frame is greater than a preset threshold, it is determined that a person has fallen to the ground.

[0047] It should be noted that existing deep learning-based human detection technologies can be used for detecting fallen persons. In this embodiment of the invention, the method of comparing the aspect ratio of the human body bounding box with a preset threshold is used for detecting fallen persons.

[0048] In this embodiment of the invention, the step of determining that a person has fallen to the ground when the aspect ratio of the target person's body frame is greater than a preset threshold is specifically as follows:

[0049] When the aspect ratio of the target person's body frame is determined to be greater than a preset threshold, the target person is marked as a suspected person who has fallen to the ground.

[0050] The suspected fallen person's limb joint points are detected, and the body outline of the suspected fallen person is corrected after the detected target joint points are removed.

[0051] The aspect ratio of the suspected fallen person's body frame is recalculated based on the corrected body frame. When the aspect ratio of the suspected fallen person's body frame is greater than the preset threshold, it is determined that a person has fallen.

[0052] It should be noted that the aspect ratio of the human body frame changes when the arms are outstretched, which can easily lead to false detections. To overcome this problem, during the fall detection process, individuals whose aspect ratio is initially greater than a threshold are marked as suspected fall victims. After detecting and removing target joints (such as wrists and elbows) of these suspected fall victims, the aspect ratio is recalculated (i.e., upper limbs and arms are ignored when calculating the aspect ratio). If the aspect ratio is still greater than the preset threshold, it indicates that a fall is very likely to have occurred, thus reducing the possibility of false detections to some extent.

[0053] In this embodiment of the invention, the step of triggering a person falling alarm when the detection result of any one of the cameras corresponding to a person falling to the ground is detected includes:

[0054] When a person is detected falling to the ground by any of the cameras, the target person who has fallen is tracked and detected.

[0055] When it is determined that the fallen person's fallen state continues for more than a preset time threshold, a person falling alarm is triggered.

[0056] It should be noted that in some cases, detecting a person falling does not necessarily require an immediate alarm. This could be due to the person getting up immediately after falling, or a false alarm being detected at a moment due to the angle of the fall. Therefore, no alarm is triggered initially when a fall is detected. Alarms are only triggered when the fallen person's condition persists for a preset time threshold. This effectively avoids false alarms or triggering unnecessary alarms.

[0057] In this embodiment of the invention, the step of triggering a person falling alarm when the detection result of any one of the cameras corresponding to a person falling to the ground is detected includes:

[0058] When at least two cameras are detected to have caused a person to fall within a preset time period, the multiple fall alarm events triggered within the preset time period are deduplicated.

[0059] It should be noted that since multiple cameras are used to detect falls simultaneously, the same target may generate multiple alarm events at the same time (or within a period of time). In this case, it is necessary to deduplicate these multiple alarm events within a short period of time to avoid multiple alarms for a single fall or frequent alarms within a short period of time.

[0060] In this embodiment of the invention, the step of deduplicating multiple fall alarm events triggered within the preset time period specifically includes:

[0061] Multiple fall alarm events triggered within the preset time period are merged into a single fall alarm event.

[0062] Furthermore, different methods can be used to deduplicate fall alarm events. For example, within a certain time period, the latest alarm event is always used as the final alarm event. Another example is that, in this embodiment of the invention, multiple fall alarm events triggered within a preset time period can be merged into a single fall alarm event to deduplicate multiple alarm events within a short period. Additionally, the merged alarm event can be configured with an alarm format distinct from regular alarm events (i.e., corresponding to situations where no multiple alarm events occur within a short period).

[0063] Based on the above scheme, and to facilitate a better understanding of the personnel fall detection method provided in the embodiments of the present invention, the following detailed description is provided:

[0064] To address the technical problems raised in the background, the purpose of this invention is to utilize existing surveillance cameras in the environment, without adding special equipment (high-performance cameras) or increasing costs, to maintain high accuracy of the fall detection algorithm in complex scenarios.

[0065] This invention utilizes multiple monocular cameras to simultaneously analyze data from different angles to reduce missed detections. Furthermore, it combines human limb detection technology to correct the aspect ratio of the human body, thereby reducing false alarms. Specifically, this can be achieved through the following steps:

[0066] The first step is to deploy cameras. Make sure there are at least two cameras in the monitoring environment, and the angle between two adjacent cameras is within the range of 80-130 degrees, or deploy three cameras with an angle of 120 degrees between each pair.

[0067] The second step is to analyze the real-time video stream of each camera separately.

[0068] The third step is to use a human detection model to detect human bodies and calculate the aspect ratio of the human bounding box. If the aspect ratio wh_ratio is greater than the preset threshold ratio_thd (for example, the default value is set to 1.3), the target is marked as suspected of falling.

[0069] The fourth step is to detect the joints of the suspected fallen human target, then remove the joints such as the left and right wrists and elbows and correct the human body frame. Then, calculate the new aspect ratio. If the new aspect ratio is still greater than the threshold, it is confirmed that a person has fallen.

[0070] The fifth step is to start timing the target that has been confirmed to be a person who has fallen to the ground. An alarm event will be triggered after the person has been in a fallen state for a certain period of time (which can be set to the default value of 4 seconds).

[0071] Step 6: Alarm event filtering. Since multiple cameras are used for simultaneous analysis, the same target may generate multiple alarm events. In this case, we need to deduplicate the alarm events generated within a certain period of time, keeping only one alarm. This time period can be set according to actual needs.

[0072] It should be noted that the embodiments of the present invention have the following beneficial effects:

[0073] (1) Multi-angle monocular camera collaborative analysis reduces missed reports.

[0074] (2) The aspect ratio of the human body frame after the human body key point detection correction is used to judge the fall of the person, which reduces false alarms and has a high accuracy rate.

[0075] It should be noted that, for the sake of simplicity, the above methods or process embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0076] Please see Figure 2 This invention also provides a person falling detection device, comprising:

[0077] Data acquisition module 1 is used to acquire real-time monitoring data collected by the camera assembly for the target area; wherein, the camera assembly includes at least two cameras;

[0078] The fall detection module 2 is used to detect people falling based on the real-time monitoring data of each of the cameras.

[0079] The alarm triggering module 3 is used to trigger a person falling alarm when the detection result of any of the cameras corresponding to a person falling to the ground is detected.

[0080] Furthermore, the fall detection module 2 is specifically used for:

[0081] The system uses a pre-defined human detection model to detect human bodies in the target area and calculates the aspect ratio of the human body bounding box for each detected person.

[0082] When the ratio of the width to height of the target person's body frame is greater than a preset threshold, it is determined that a person has fallen to the ground.

[0083] Furthermore, the fall detection module 2 is specifically used for:

[0084] When the aspect ratio of the target person's body frame is determined to be greater than a preset threshold, the target person is marked as a suspected person who has fallen to the ground.

[0085] The suspected fallen person's limb joint points are detected, and the body outline of the suspected fallen person is corrected after the detected target joint points are removed.

[0086] The aspect ratio of the suspected fallen person's body frame is recalculated based on the corrected body frame. When the aspect ratio of the suspected fallen person's body frame is greater than the preset threshold, it is determined that a person has fallen.

[0087] Furthermore, the alarm triggering module 3 is specifically used for:

[0088] When a person is detected falling to the ground by any of the cameras, the target person who has fallen is tracked and detected.

[0089] When it is determined that the fallen person's fallen state continues for more than a preset time threshold, a person falling alarm is triggered.

[0090] Furthermore, the alarm triggering module 3 is specifically used for:

[0091] When at least two cameras are detected to have caused a person to fall within a preset time period, the multiple fall alarm events triggered within the preset time period are deduplicated.

[0092] Furthermore, the alarm triggering module 3 is specifically used for:

[0093] Multiple fall alarm events triggered within the preset time period are merged into a single fall alarm event.

[0094] Furthermore, the camera assembly consists of three cameras, and the angle between the line connecting each adjacent camera and the center of the target area is within the range of 80° to 130°.

[0095] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention. The personnel fall detection device provided by the embodiments of the present invention can implement the personnel fall detection method provided by any one of the method embodiments of the present invention.

[0096] The present invention also provides a terminal device, including a processor and a memory storing a computer program, wherein the processor executes the computer program to implement any of the personnel fall detection methods described above.

[0097] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the personnel fall detection methods described above.

[0098] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0099] Those skilled in the art will clearly understand that, for convenience and simplicity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0100] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0101] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0102] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0103] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0104] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting a person falling to the ground, characterized in that, include: Acquire real-time monitoring data collected by a camera array targeting a target area; wherein the camera array includes at least two cameras; The system uses a pre-defined human detection model to detect human bodies in the target area and calculates the aspect ratio of the human body bounding box for each detected person. When the aspect ratio of the target person's body frame is determined to be greater than a preset threshold, the target person is marked as a suspected person who has fallen to the ground. The suspected fallen person's limb joint points are detected, and the body outline of the suspected fallen person is corrected after the detected target joint points are removed. The aspect ratio of the suspected fallen person's body frame is recalculated based on the corrected body frame. When the aspect ratio of the suspected fallen person's body frame is greater than the preset threshold, it is determined that a person has fallen. When a person is detected to have fallen to the ground by any of the cameras, a person fall alarm is triggered.

2. The method for detecting a person falling to the ground according to claim 1, characterized in that, When a person is detected to have fallen at any of the cameras, a person fall alarm is triggered, including: When a person is detected falling to the ground by any of the cameras, the target person who has fallen is tracked and detected. When it is determined that the fallen person's fallen state continues for more than a preset time threshold, a person falling alarm is triggered.

3. The method for detecting a person falling to the ground according to claim 1, characterized in that, When a person is detected to have fallen at any of the cameras, a person fall alarm is triggered, including: When at least two cameras are detected to have caused a person to fall within a preset time period, the multiple fall alarm events triggered within the preset time period are deduplicated.

4. The method for detecting a person falling to the ground according to claim 3, characterized in that, The process of deduplicating multiple fall alarm events triggered within the preset time period specifically includes: Multiple fall alarm events triggered within the preset time period are merged into a single fall alarm event.

5. The method for detecting a person falling to the ground according to any one of claims 1-4, characterized in that, The camera combination consists of 3 cameras, and the angle between the line connecting each adjacent camera and the center of the target area is between 80° and 130°.

6. A device for detecting when a person falls to the ground, characterized in that, include: A data acquisition module is used to acquire real-time monitoring data collected by the camera assembly for a target area; wherein the camera assembly includes at least two cameras; The fall detection module is used to perform human detection in a target area using a preset human detection model and calculate the aspect ratio of the human body frame of each detected person. When the aspect ratio of the human body frame of a target person is greater than a preset threshold, the target person is marked as a suspected fall victim. The module then performs limb joint detection on the suspected fall victim and removes the detected joints to correct the human body frame of the suspected fall victim. Based on the corrected human body frame, the aspect ratio of the human body frame of the suspected fall victim is recalculated. When the aspect ratio of the human body frame of the suspected fall victim is greater than the preset threshold, it is determined that a fall has occurred. The alarm triggering module is used to trigger a person falling alarm when the detection result of any of the cameras corresponding to a person falling to the ground is detected.

7. A terminal device, comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the personnel fall detection method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the personnel fall detection method as described in any one of claims 1 to 5.

Images