Falling behavior detection method, device, electronic device and storage medium

Abstract

The present application provides a fall behavior detection method, device, electronic device and storage medium. The method includes: obtaining surveillance video data captured by a camera based on a queue window; inputting the surveillance video data into a trained fall detection model to obtain a detection result corresponding to the timing of the queue window output by the fall detection model, wherein the fall detection model includes a YOLOv5 model, and the Backbone of the YOLOv5 model is provided with a recursive gated convolution gnConv module; when the detection result indicates that there is a fall behavior in the surveillance video data, an alarm prompt corresponding to the danger level is issued according to the danger level in the detection result. In this way, the characteristics of the fall in the time dimension can be reflected in the detection result, which is conducive to others to promptly discover the occurrence of the fall, so as to facilitate timely rescue of the fallen person.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a fall behavior detection method, device, electronic device and storage medium. Background Art

[0002] Accidental falls can cause stroke, paralysis, or even death in the elderly. Failure to detect a fall or improper self-rescue can lead to secondary injuries. Current research on fall detection is categorized into three types: environmental sensor-based, wearable sensor-based, and machine vision-based. These approaches, however, suffer from high false positive rates, expensive deployment, and inconvenient wear, hindering widespread adoption.

[0003] Traditional machine vision-based fall detection methods extract features from video frames and feed them into a classifier for detection. These features can include single or mixed features such as human figure, optical flow, RGB, posture, and skeleton. The model is computationally complex and susceptible to external interference, resulting in suboptimal real-time performance and accuracy. Furthermore, it fails to capture the temporal characteristics of falls, hindering the timely detection of falls. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present application is to provide a fall behavior detection method, device, electronic device and storage medium, which can improve the problem that the characteristics of the fall in the time dimension cannot be reflected and is not conducive to timely detection of fall behavior.

[0005] To achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for detecting a fall, the method comprising:

[0007] Obtain surveillance video data captured by the camera based on the queue window;

[0008] Inputting the monitored video data into a trained fall detection model to obtain a detection result output by the fall detection model corresponding to the time sequence of the queue window, wherein the fall detection model includes a YOLOv5 model, and the YOLOv5 model includes an input module Input, a backbone network Backbone, a feature fusion layer Neck, and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module;

[0009] When the detection result indicates that there is a fall in the monitoring video data, an alarm prompt corresponding to the danger level is issued according to the danger level in the detection result.

[0010] In conjunction with the first aspect, in some optional implementations, inputting the surveillance video data into a trained fall detection model to obtain a detection result output by the fall detection model corresponding to the time sequence of the queue window includes:

[0011] Outputting the image frames in the monitoring video data to the Backbone via the Input;

[0012] Performing feature extraction on image frames in the surveillance video data using the Backbone to obtain extracted first-class image features, wherein the gnConv module is used to extend the two-order interaction in self-attention to any order;

[0013] fusing the first type of image features through the Neck to obtain the second type of image features;

[0014] Behavior detection is performed on the second type of image features through the Output to obtain the detection result corresponding to the time sequence of the queue window.

[0015] In combination with the first aspect, in some optional implementations, performing behavior detection on the second type of image features through the Output to obtain the detection result corresponding to the time sequence of the queue window includes:

[0016] Performing behavior detection on the second type of image features using the Output to obtain a recognition result corresponding to each image frame in the queue window;

[0017] According to the recognition result of each image frame, the window state of the queue window and the detection result are determined, and the window state is expressed as:

[0018]

[0019] Among them, s refers to the window status; S1 indicates that the image frames in the queue window are all unrecognized frames; S2 indicates that the image frames in the queue window contain both recognized frames and unrecognized frames; S3 indicates that the image frames in the queue window are all recognized frames; x refers to the number of recognized frames in the queue window; y refers to the number of unrecognized frames in the queue window; m refers to the width of the queue window.

[0020] In conjunction with the first aspect, in some optional implementations, before acquiring the surveillance video data captured by the camera based on the queue window, the method further includes:

[0021] Acquire a training data set, the training data set including a first type of image set with a label representing a fall behavior, and a second type of image set with a label representing a non-fall behavior;

[0022] The YOLOv5 model is trained using the first image set and the second image set to obtain a trained YOLOv5 model as the fall detection model, wherein the backbone network of the YOLOv5 model is provided with the gnConv module.

[0023] In combination with the first aspect, in some optional embodiments, the danger level includes a first danger level indicating that the falling behavior is in an emergency state, a second danger level indicating that the falling behavior is continuous and in a non-emergency state, and a third danger level indicating that the falling behavior is resolved.

[0024] In conjunction with the first aspect, in some optional implementations, the method further includes:

[0025] When the danger level in the detection result is the first danger level, an alarm is sent to a designated user terminal, and preset position data of the camera is sent to the designated user terminal as the location where the fall occurs.

[0026] In conjunction with the first aspect, in some optional implementations, the method further includes:

[0027] A video segment corresponding to the falling behavior is cropped from the monitoring video data captured by the camera, and the cropped video segment is sent to the designated user terminal.

[0028] In a second aspect, an embodiment of the present application further provides a fall behavior detection device, the device comprising:

[0029] An acquisition unit, configured to acquire surveillance video data captured by a camera based on a queue window;

[0030] A detection unit, configured to input the surveillance video data into a trained fall detection model, and obtain a detection result output by the fall detection model corresponding to the time sequence of the queue window, wherein the fall detection model comprises a YOLOv5 model, and the YOLOv5 model comprises an input module Input, a backbone network Backbone, a feature fusion layer Neck, and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module;

[0031] A prompt unit is used to issue an alarm prompt corresponding to the danger level according to the danger level in the detection result when the detection result indicates that there is a fall behavior in the monitoring video data.

[0032] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device executes the above method.

[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is run on a computer, the computer executes the above method.

[0034] The invention adopting the above technical solution has the following advantages:

[0035] In the technical solution provided by this application, surveillance video data captured by a camera is obtained based on a queue window; the surveillance video data is input into a trained fall detection model to obtain a detection result corresponding to the time sequence of the queue window output by the fall detection model. In this way, the characteristics of the fall in the time dimension can be reflected in the detection result. In addition, when the detection result indicates that there is a fall in the surveillance video data, an alarm prompt corresponding to the danger level in the detection result is issued. This helps others to promptly discover the occurrence of a fall, so that the fallen person can be rescued in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present application may be further illustrated by the non-limiting embodiments provided in the accompanying drawings. It should be understood that the following drawings illustrate only certain embodiments of the present application and are therefore not to be construed as limiting the scope of the present application. It is understood that a person skilled in the art can derive other relevant drawings from these drawings without inventive effort.

[0037] Figure 1 Schematic diagram of the network structure of the traditional YOLOv5 model.

[0038] Figure 2 A schematic diagram of the network structure of the improved YOLOv5 model provided in an embodiment of the present application.

[0039] Figure 3 A flowchart of the fall behavior detection method provided in an embodiment of the present application.

[0040] Figure 4 A schematic diagram describing the window status of a queue window provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts in the drawings or descriptions are numbered the same. Implementations not shown or described in the drawings are known to those of ordinary skill in the art. In the description of this application, the terms "first," "second," etc. are used solely to distinguish descriptions and are not to be construed as indicating or implying relative importance.

[0042] An embodiment of the present application provides an electronic device including a processing module and a storage module. The storage module stores a computer program, which, when executed by the processing module, enables the electronic device to perform corresponding steps in the following fall behavior detection method.

[0043] The electronic device may be, but is not limited to, a smartphone, a personal computer, a server, or other devices. The electronic device may be communicatively connected to a camera and may obtain real-time surveillance video captured by the camera from the camera. Furthermore, the electronic device may also establish a communication connection with a designated user terminal to facilitate sending an alarm message indicating a fall to the designated user terminal. The designated user terminal may be a smartphone or other device, and is not specifically limited here.

[0044] For example, when the application scenario is a residential area or shopping mall, the designated user terminal can be a property staff smartphone. When the application scenario is a park or other outdoor place, the designated user terminal can be a terminal at the local emergency center.

[0045] In this embodiment, the electronic device stores an improved YOLOv5 model. This improved YOLOv5 model can automatically detect falls in surveillance video and identify their danger level. For ease of description, the traditional YOLOv5 model is referred to as the first YOLOv5 model, and the improved YOLOv5 model is referred to as the second YOLOv5 model.

[0046] Please refer to Figure 1 and Figure 2 , Figure 1 The network structure of the YOLOv5 model shown is a conventional structure. Figure 2 The YOLOv5 model shown is an improved network structure. Understandably, in the improved second YOLOv5 model, only the backbone network Backbone is different from the Backbone of the first YOLOv5 model.

[0047] Please refer to Figure 2In the second YOLOv5 model, Backbone adds a recursive gated convolution (gnConv) module. The input of the gnConv module is connected to the SPP via CSP1_3; the output of the gnConv module is connected to CSP1_2 in the feature fusion layer Neck. CSP1_3, SPP, and CSP1_2 represent the conventional network structure of the traditional YOLOv5 model, and the functions of each functional block are not detailed here.

[0048] In this embodiment, the gnConv module is constructed using standard convolution, linear projection, and element-wise multiplication. This module is then integrated into the backbone of the traditional YOLOv5 model to form a new YOLOv5 model. The gnConv module extends the two-order interactions in self-attention to arbitrary orders, improving modeling capabilities. Furthermore, the gnConv module utilizes gated convolution and recursive design to achieve adaptive input and high-order spatial interactions, providing compatibility and flexibility without introducing additional computational overhead. Consequently, this improved second YOLOv5 model can more accurately locate the exact position of objects of interest, facilitating better behavior detection performance.

[0049] Please refer to Figure 3 The present application also provides a fall behavior detection method that can be applied to the above-mentioned electronic device, and each step of the method is executed or implemented by the electronic device. The fall behavior detection method can include the following steps:

[0050] Step 110: Acquire surveillance video data captured by the camera based on the queue window;

[0051] Step 120: Input the surveillance video data into a trained fall detection model to obtain a detection result output by the fall detection model corresponding to the time sequence of the queue window, wherein the fall detection model includes a YOLOv5 model, and the YOLOv5 model includes an input module Input, a backbone network Backbone, a feature fusion layer Neck, and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module;

[0052] Step 130: When the detection result indicates that there is a fall in the monitoring video data, an alarm corresponding to the danger level in the detection result is issued.

[0053] The following is a detailed description of the steps of the fall behavior detection method:

[0054] When executing step 110, the fall detection model in the electronic device has been trained and tested. If the fall detection model has not been trained, a model training step may be included before step 110.

[0055] For example, before acquiring the surveillance video data captured by the camera based on the queue window, the method may further include:

[0056] Acquire a training data set, the training data set including a first type of image set with a label representing a fall behavior, and a second type of image set with a label representing a non-fall behavior;

[0057] The YOLOv5 model is trained using the first image set and the second image set to obtain a trained YOLOv5 model as the fall detection model, wherein the backbone network of the YOLOv5 model is provided with the gnConv module.

[0058] Understandably, the first type of image set is used as positive sample data, and the second type of image set is used as a negative sample data set. Adding rich negative sample data to the training data set is conducive to improving the richness of the data set. In order to improve the problem of model overfitting and poor generalization ability caused by the single composition of the data set and the lack of negative samples. In this embodiment, pictures of other non-falling actions can be added to the data set as negative samples, which include 40 common non-falling actions (such as walking, running, standing from sitting position, etc.), and the number of images of each action is about 200-300. In the end, the entire training data set can be a total of about 12,600 picture samples, of which about 4,800 are positive samples of falling and about 7,800 are negative samples of other actions. The rich sample data composition is conducive to improving the stability of the model and reducing the occurrence of false detection.

[0059] The first set of images is labeled with a fall action label, while the second set is labeled with a non-fall action label, to facilitate fall detection. In other words, after training the YOLOv5 model using both the first and second sets of images, the improved YOLOv5 model can now detect falls in surveillance videos.

[0060] In step 110, the electronic device can periodically acquire surveillance video data captured by the camera using a queue window. The size of the queue window can be understood as the duration of each acquired surveillance video data or the number of acquired image frames, i.e., the width of the queue window. The size of the queue window can be flexibly determined based on actual circumstances. The period for acquiring surveillance video data can be flexibly determined based on actual circumstances and is not specifically limited here.

[0061] In this embodiment, step 120 may include:

[0062] Outputting the image frames in the monitoring video data to the Backbone via the Input;

[0063] Performing feature extraction on image frames in the surveillance video data using the Backbone to obtain extracted first-class image features, wherein the gnConv module is used to extend the two-order interaction in self-attention to any order;

[0064] fusing the first type of image features through the Neck to obtain the second type of image features;

[0065] Behavior detection is performed on the second type of image features through the Output to obtain the detection result corresponding to the time sequence of the queue window.

[0066] In the improved YOLOv5 model, the introduction of the high-order self-attention gnConv module can improve the performance of YOLOv5; the gnConv module uses gated convolution and recursive design to achieve adaptive input and high-order spatial interaction, and has compatibility and flexibility without introducing additional computational burden; the gnConv module extends the two-order interaction in self-attention to any order, improving modeling capabilities.

[0067] In this embodiment, the detection result may include whether a fall has occurred, as well as the danger level if a fall has occurred. It is understandable that a fall is a temporal process. A complete fall takes some time, and there are many different situations during a fall, such as falling, moving after falling, remaining still after falling, and getting up after falling. Different alarm messages should be issued for different situations.

[0068] The danger level can be flexibly determined based on actual conditions. For example, the danger level may include a first danger level indicating that the fall is in an emergency state, a second danger level indicating that the fall is ongoing and in a non-emergency state, and a third danger level indicating that the fall has resolved.

[0069] Please refer to Figure 4 , the fall detection model can read and perform recognition detection on a frame-by-frame basis for the image frames in the queue window. For each image frame identified, the one with a confidence level greater than a threshold is marked as a recognized frame (for example, marked in red), otherwise it is marked as an unrecognized frame (for example, marked in gray). The confidence level threshold can be flexibly determined based on actual conditions to measure whether the image frame is a posture in a fall behavior. If the confidence level exceeds the threshold, it indicates that there is a posture of a fall behavior in the image frame. The electronic device can determine whether the window state has switched based on the change in the number of recognized frames in the window and issue an alarm message corresponding to the state switch.

[0070] The step of performing behavior detection on the second type of image features through the output to obtain the detection result corresponding to the time sequence of the queue window includes:

[0071] Performing behavior detection on the second type of image features using the Output to obtain a recognition result corresponding to each image frame in the queue window;

[0072] According to the recognition result of each image frame, the window state of the queue window and the detection result are determined, and the window state is expressed as:

[0073]

[0074] Among them, s refers to the window status; S1 refers to the image frames in the queue window are all unrecognized frames, which can be expressed as "empty"; S2 refers to the image frames in the queue window contain both recognized frames and unrecognized frames, which can be expressed as "mixed"; S3 refers to the image frames in the queue window are all recognized frames, which can be expressed as "full"; x refers to the number of recognized frames in the queue window; y refers to the number of unrecognized frames in the queue window; m refers to the width of the queue window.

[0075] Understandably, the queue window dynamically retains the detection results of the first w frames (the window size, which can be flexibly determined based on actual conditions) of the improved YOLOv5 fall detection model. Frames where a fall is detected are marked as recognized, and frames otherwise as unrecognized. The parameters T_full (how long the window remains 'full') and T_mixed (how long the window remains 'mixed') are set to measure the duration of the fall state.

[0076] Please refer again Figure 4 , define the warning level and alarm information corresponding to the state transition. Define the state switching description corresponding to the window state switching (previous frame - current frame) and update the parameters T_full, T_mixed, define the corresponding danger level (Ⅰ, Ⅱ, Ⅲ) of the state switching description, you can refer to Figure 4 Ⅰ, Ⅱ, Ⅲ, respectively represent the first danger level, the second danger level and the third danger level. After obtaining the window status of the queue window, you can Figure 4 The logic shown is used to determine whether there is a fall in the video clips in the queue window, and the danger level of the fall.

[0077] In step 130, the camera is typically connected to an on-site monitoring platform, which may be equipped with a prompt module (e.g., a speaker, etc.) that can issue an alarm under the control of an electronic device. By issuing corresponding alarms for different danger levels, it is helpful for personnel who hear the alarm to distinguish different danger levels based on the different alarm prompts.

[0078] In this embodiment, the method may further include: when the danger level in the detection result is the first danger level, alerting a designated user terminal and transmitting the preset location data of the camera to the designated user terminal as the location where the fall occurred. This facilitates relevant personnel to rush to the scene in a timely manner based on the address of the fall to provide assistance or rescue to the fallen person.

[0079] In this embodiment, the method may further include: cropping a video segment corresponding to the falling behavior from the monitoring video data captured by the camera, and sending the cropped video segment to the designated user terminal.

[0080] Understandably, the electronic device can send video clips of a period of time before and after the fall to a designated user terminal so that the video files of the fall can be manually reviewed in a timely manner.

[0081] After completing the improvement and training of the YOLOv5 model, the inventors also conducted experimental verification. Specifically, the indicators used for verification were precision (P), recall (R), average average precision (mAP), and detection speed (FPS). The verification method can be expressed as follows:

[0082]

[0083]

[0084]

[0085]

[0086] Where TP stands for true positive, meaning a positive class is considered positive; FP stands for false positive, meaning a negative class is considered positive; FN stands for false negative, meaning a positive class is considered negative; and TN stands for true negative, meaning a negative class is considered negative. AP stands for the area under the PR curve, which represents the average precision. FPS stands for the average time required to detect M frames. K represents the number of classes to be detected; and t represents the frame number.

[0087] To verify the performance of the traditional YOLOv5 model and the improved YOLOv5 model (YOLOv5-gnConv), training parameters were uniformly set to 400 epochs and 32 batches. Finally, a queue-window-based fall detection and alarm framework, using the improved YOLOv5-gnConv model as the core algorithm, performed real-time fall detection, achieving an average detection speed of 40 frames per second, meeting the requirements for real-time detection. The accuracy of the two models is shown in Table 1, which shows the accuracy and average accuracy of YOLOv5 and YOLOv5-gnConv.

[0088] Table 1

[0089]

[0090] Based on Table 1, we can see that the performance of the improved model YOLOv5-gnConv is better than the traditional YOLOv5.

[0091] Based on the above design, the method provided in the embodiment of the present application proposes to add rich negative samples to the data set to improve the sample richness of the data set to address the problem of high false detection rate of the model; to address the problem of redundant calculation of model features, it proposes to introduce a high-order self-attention gnConv module, and use the self-attention mechanism to enhance the model's ability to focus on key features, thereby improving the performance of the fall detection model; to address the problem that the model cannot characterize the temporal and dimensional characteristics of falls, a queue window-based method is proposed for fall status judgment and alarm.

[0092] The present application also provides a fall detection device, which includes at least one software function module that can be stored in a storage module in the form of software or firmware, or embedded in an operating system (OS) of an electronic device. A processing module is configured to execute the executable module stored in the storage module, such as the software function module and computer program included in the fall detection device.

[0093] The fall behavior detection device includes an acquisition unit, a detection unit, and a prompt unit. The functions of each unit can be as follows:

[0094] An acquisition unit, configured to acquire surveillance video data captured by a camera based on a queue window;

[0095] A detection unit, configured to input the surveillance video data into a trained fall detection model, and obtain a detection result output by the fall detection model corresponding to the time sequence of the queue window, wherein the fall detection model comprises a YOLOv5 model, and the YOLOv5 model comprises an input module Input, a backbone network Backbone, a feature fusion layer Neck, and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module;

[0096] A prompt unit is used to issue an alarm prompt corresponding to the danger level according to the danger level in the detection result when the detection result indicates that there is a fall behavior in the monitoring video data.

[0097] Optionally, the detection unit can be used to:

[0098] Outputting the image frames in the monitoring video data to the Backbone via the Input;

[0099] Performing feature extraction on image frames in the surveillance video data using the Backbone to obtain extracted first-class image features, wherein the gnConv module is used to extend the two-order interaction in self-attention to any order;

[0100] fusing the first type of image features through the Neck to obtain the second type of image features;

[0101] Behavior detection is performed on the second type of image features through the Output to obtain the detection result corresponding to the time sequence of the queue window.

[0102] Optionally, the detection unit can be used to:

[0103] Performing behavior detection on the second type of image features using the Output to obtain a recognition result corresponding to each image frame in the queue window;

[0104] According to the recognition result of each image frame, the window state of the queue window and the detection result are determined, and the window state is expressed as:

[0105]

[0106] Among them, s refers to the window status; S1 indicates that the image frames in the queue window are all unrecognized frames; S2 indicates that the image frames in the queue window contain both recognized frames and unrecognized frames; S3 indicates that the image frames in the queue window are all recognized frames; x refers to the number of recognized frames in the queue window; y refers to the number of unrecognized frames in the queue window; m refers to the width of the queue window.

[0107] Optionally, the fall behavior detection device may further include a training unit. Before the acquisition unit acquires the surveillance video data captured by the camera based on the queue window, the acquisition unit may further be configured to: acquire a training dataset, wherein the training dataset includes a first type of image set with a label representing a fall behavior and a second type of image set with a label representing a non-fall behavior; and the training unit is configured to: train a YOLOv5 model using the first type of image set and the second type of image set to obtain a trained YOLOv5 model as the fall detection model, wherein the backbone network of the YOLOv5 model is configured with the gnConv module.

[0108] Optionally, the prompt unit can also be used to: when the danger level in the detection result is the first danger level, alert a designated user terminal, and send the preset position data of the camera to the designated user terminal as the location where the fall occurred.

[0109] Optionally, the prompt unit may also be configured to: crop a video segment corresponding to the falling behavior from the monitoring video data captured by the camera, and send the cropped video segment to the designated user terminal.

[0110] In this embodiment, the processing module can be an integrated circuit chip with signal processing capabilities. The above-mentioned processing module can be a general-purpose processor. For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0111] The storage module may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc. In this embodiment, the storage module may be used to store surveillance video data, a fall detection model, etc. Of course, the storage module may also be used to store programs, which the processing module executes upon receiving an execution instruction.

[0112] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding processes of each step in the aforementioned method, and will not be elaborated here.

[0113] The present application also provides a computer-readable storage medium that stores a computer program, which, when executed on a computer, causes the computer to execute the fall behavior detection method described in the above embodiment.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0115] In summary, the embodiments of the present application provide a method, device, electronic device and storage medium for detecting a fall behavior. In this solution, surveillance video data captured by a camera is obtained based on a queue window; the surveillance video data is input into a trained fall detection model to obtain a detection result corresponding to the timing of the queue window output by the fall detection model. In this way, the characteristics of the fall in the time dimension can be reflected in the detection result. In addition, when the detection result indicates that there is a fall behavior in the surveillance video data, an alarm prompt corresponding to the danger level is issued according to the danger level in the detection result. In this way, it is helpful for others to discover the occurrence of a fall in time, so that the fallen person can be rescued in time.

[0116] In the embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can also be implemented in other ways. The device, system and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and a part of the module, program segment or code includes one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0117] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A fall behavior detection method, characterized in that: The method comprises: Obtain surveillance video data captured by the camera based on the queue window; Input the monitoring video data into a trained fall detection model to obtain a detection result output by the fall detection model corresponding to the timing of the queue window, wherein the fall detection model includes a YOLOv5 model, and the YOLOv5 model includes an input module Input, a backbone network Backbone, a feature fusion layer Neck and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module, the input end of the gnConv module is connected to SPP through CSP1_3, and the output end of the gnConv module is connected to CSP1_2 in the feature fusion layer Neck; When the detection result indicates that there is a fall in the monitoring video data, according to the danger level in the detection result, an alarm prompt corresponding to the danger level is issued; The monitoring video data is input into a trained fall detection model to obtain a detection result output by the fall detection model corresponding to the timing of the queue window, including: Outputting the image frames in the monitoring video data to the Backbone via the Input; Performing feature extraction on the image frames in the surveillance video data through the Backbone to obtain extracted first-class image features, wherein the gnConv module is used to extend the two-order interaction in the self-attention to any order; By fusing the first type of image features through the Neck, a second type of image features is obtained; Performing behavior detection on the second type of image features through the Output to obtain the detection result corresponding to the timing of the queue window; Performing behavior detection on the second type of image features through the Output to obtain the detection result corresponding to the timing of the queue window includes: Performing behavior detection on the second type of image features through the Output to obtain a recognition result corresponding to each image frame in the queue window; According to the recognition result of each image frame, the window state of the queue window and the detection result are determined, and the window state is expressed as: Among them, after obtaining the window state of the queue window, according to the window state switching from the previous frame to the current frame, it is obtained whether there is a falling behavior in the video clip in the queue window and the danger level of the falling behavior, s refers to the window state; S1 indicates that the image frames in the queue window are all unrecognized frames; S2 indicates that the image frames in the queue window contain both recognized frames and unrecognized frames; S3 indicates that the image frames in the queue window are all recognized frames; x refers to the number of recognized frames in the queue window; y refers to the number of unrecognized frames in the queue window; m refers to the width of the queue window.

2. The method according to claim 1, characterized in that Before acquiring the surveillance video data captured by the camera based on the queue window, the method further includes: Acquire a training data set, wherein the training data set includes a first type of image set with a label representing a fall behavior, and a second type of image set with a label representing a non-fall behavior; The YOLOv5 model is trained using the first category image set and the second category image set to obtain a trained YOLOv5 model as the fall detection model, wherein the backbone network of the YOLOv5 model is provided with the gnConv module.

3. The method according to claim 1, characterized in that The danger levels include a first danger level indicating that the falling behavior is in an emergency state, a second danger level indicating that the falling behavior is continuous and in a non-emergency state, and a third danger level indicating that the falling behavior is resolved.

4. The method according to claim 3, characterized in that The method further comprises: When the danger level in the detection result is the first danger level, an alarm is sent to the designated user terminal, and the preset position data of the camera is sent to the designated user terminal as the location where the fall occurs.

5. The method according to claim 4, characterized in that The method further comprises: A video segment corresponding to the falling behavior is cropped from the monitoring video data captured by the camera, and the cropped video segment is sent to the designated user terminal.

6. A fall behavior detection device, characterized in that: The device comprises: An acquisition unit, used for acquiring surveillance video data shot by a camera based on a queue window; A detection unit, used for inputting the monitoring video data into a trained fall detection model, and obtaining a detection result output by the fall detection model corresponding to the timing of the queue window, wherein the fall detection model comprises a YOLOv5 model, and the YOLOv5 model comprises an input module Input, a backbone network Backbone, a feature fusion layer Neck and an output module Output, wherein the Backbone is provided with a recursive gated convolution gnConv module, an input end of the gnConv module is connected to SPP through CSP1_3, and an output end of the gnConv module is connected to CSP1_2 in the feature fusion layer Neck; a prompting unit, configured to, when the detection result indicates that there is a fall in the monitoring video data, issue an alarm prompt corresponding to the danger level according to the danger level in the detection result; Wherein, the detection unit is also used for: Outputting the image frames in the monitoring video data to the Backbone via the Input; Performing feature extraction on the image frames in the surveillance video data through the Backbone to obtain extracted first-class image features, wherein the gnConv module is used to extend the two-order interaction in the self-attention to any order; By fusing the first type of image features through the Neck, a second type of image features is obtained; Performing behavior detection on the second type of image features through the Output to obtain the detection result corresponding to the timing of the queue window; Performing behavior detection on the second type of image features through the Output to obtain the detection result corresponding to the timing of the queue window includes: Performing behavior detection on the second type of image features through the Output to obtain a recognition result corresponding to each image frame in the queue window; According to the recognition result of each image frame, the window state of the queue window and the detection result are determined, and the window state is expressed as: Among them, after obtaining the window state of the queue window, according to the window state switching from the previous frame to the current frame, it is obtained whether there is a falling behavior in the video clip in the queue window and the danger level of the falling behavior, s refers to the window state; S1 indicates that the image frames in the queue window are all unrecognized frames; S2 indicates that the image frames in the queue window contain both recognized frames and unrecognized frames; S3 indicates that the image frames in the queue window are all recognized frames; x refers to the number of recognized frames in the queue window; y refers to the number of unrecognized frames in the queue window; m refers to the width of the queue window.

7. An electronic device, characterized in that: The electronic device comprises a processor and a memory coupled to each other, wherein the memory stores a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5.

Images