A fall detection method, apparatus, device, and storage medium
By combining radar sensors and event vision sensors to acquire radar data and event data of the target object, performing dual detection and fusion, the problems of detection accuracy and privacy protection in existing technologies are solved, and fall detection with high accuracy and low privacy leakage is achieved.
Patent Information
- Application Number
- CN202510593117.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-10
AI Technical Summary
Existing fall detection methods cannot simultaneously ensure accuracy, reliability, and protection of the privacy of monitored individuals. Methods based on radar sensors suffer from low accuracy and cannot confirm falls, while methods combining radar sensors and APS pose a risk of privacy leaks.
Radar and event vision sensors are used to acquire radar data and event data of the target object, and fall behavior detection is performed separately. The detection results are fused to obtain the fall detection result of the target object. A pre-trained classifier and feature extraction technology are used to improve the detection accuracy, and the detection result is optimized by weight adjustment.
It improves the accuracy and reliability of fall detection, while reducing the risk of privacy leaks and enhancing the user experience.
Smart Images

Figure CN123058710A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a fall detection method, apparatus, device and storage medium. Background Technology
[0002] Fall detection refers to the use of various technologies to monitor and identify whether an individual has fallen. It is commonly used in fields such as smart homes, healthcare, and elderly care, and has significant social and medical implications.
[0003] In existing technologies, radar sensors are typically installed in the environment where the target is located. Fall detection is performed using data output from these radar sensors, or by combining radar sensor data with images output from an active pixel sensor (APS). While radar sensor-based fall detection effectively protects personal privacy, it cannot provide a live view of the fall, lacks accuracy, and cannot confirm whether a fall has occurred or the specific circumstances of the fall, resulting in low accuracy and reliability. Furthermore, while the combination of radar sensors and APS can provide real-time images, it also carries the risk of exposing the privacy information of the monitored individual.
[0004] Therefore, existing fall detection methods cannot simultaneously ensure the accuracy and reliability of fall detection as well as the protection of the privacy information of the monitored individuals. Summary of the Invention
[0005] This invention provides a fall detection method, apparatus, device, and storage medium to improve the accuracy and reliability of fall detection, while also reducing the risk of leakage of the target's privacy information.
[0006] In a first aspect, the present invention provides a fall detection method, comprising:
[0007] The system acquires radar data and event data of the target object, whereby the radar data is collected by a radar sensor and the event data is collected by an event vision sensor; it detects the fall behavior of the target object based on the radar data to obtain a first detection result, and detects the fall behavior of the target object based on the event data to obtain a second detection result; it then fuses the first and second detection results to obtain a fall detection result.
[0008] Furthermore, the first detection result includes a first fall probability. Accordingly, based on radar data, the fall behavior of the target object is detected to obtain the first detection result, which includes:
[0009] Feature extraction is performed on the radar data to obtain the first feature data; the first feature data is input into the pre-trained first classifier to obtain the first fall probability, wherein the first fall probability is the probability that the first feature data matches the preset fall behavior.
[0010] Furthermore, before extracting features from the radar data to obtain the first feature data, the fall detection method also includes:
[0011] The radar data undergoes a first preprocessing step, which includes dynamic noise filtering, static clutter suppression, and conversion from polar coordinates to Cartesian coordinates.
[0012] Furthermore, the second detection result includes a second fall probability. Accordingly, based on event data, the fall behavior of the target object is detected to obtain the second detection result, which includes:
[0013] Feature extraction is performed on the event data to obtain second feature data; the second feature data is input into a pre-trained second classifier to obtain a second fall probability, wherein the second fall probability is the probability that the second feature data matches the preset fall behavior.
[0014] Furthermore, the first detection result includes a first fall probability, the second detection result includes a second fall probability, and the fall detection result includes a target fall probability. Accordingly, the first and second detection results are fused to obtain the fall detection result, which includes:
[0015] Obtain the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability; based on the first weight and the second weight, perform a weighted calculation on the first fall probability and the second fall probability to obtain the target fall probability.
[0016] Further, the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability are obtained, including:
[0017] Obtain a preset weight dataset, which includes the data mapping relationship between the ambient brightness of the target object, the number of target objects, and the first weight and the second weight; obtain the ambient brightness of the target object and the number of target objects; based on the ambient brightness of the target object and the number of target objects, obtain the corresponding first weight and second weight from the preset weight dataset.
[0018] Furthermore, fall detection methods also include:
[0019] Based on the fall detection results, determine whether the target object has fallen; if the target object has fallen, send an alarm message to the corresponding monitoring device; in response to the confirmation message sent by the monitoring device, send event data to the monitoring device, or send event data and scene images to the monitoring device, wherein the scene images are acquired by an active pixel sensor.
[0020] Furthermore, the confirmation information includes first confirmation information and second confirmation information. Accordingly, in response to the confirmation information sent by the monitoring device, event data is sent to the monitoring device, or event data and scene images are sent to the monitoring device, including:
[0021] If the confirmation information sent by the monitoring device is the first confirmation information, the event data related to the fall behavior will be sent to the monitoring device and displayed on the monitoring device in the form of an event image; if the confirmation information sent by the monitoring device is the second confirmation information, the scene image related to the fall behavior will be acquired using an active pixel sensor, and the event data and scene image related to the fall behavior will be sent to the monitoring device, and the event data will be displayed on the monitoring device in the form of an event image, and / or, the scene image will be displayed on the monitoring device.
[0022] Secondly, embodiments of the present invention also provide a fall detection device, comprising:
[0023] The acquisition module is used to acquire radar data and event data of the target object. The radar data is acquired by radar sensors, and the event data is acquired by event vision sensors.
[0024] The data processing and detection module is used to detect the fall behavior of the target object based on radar data to obtain a first detection result, and is also used to detect the fall behavior of the target object based on event data to obtain a second detection result;
[0025] The fusion module is used to fuse the first detection result and the second detection result to obtain the fall detection result.
[0026] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:
[0027] At least one processor; and a memory communicatively connected to the at least one processor;
[0028] The memory stores a computer program that can be executed by at least one processor, which enables the electronic device to perform the fall detection method as described in any of the first aspects.
[0029] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the fall detection method as described in any of the first aspects.
[0030] Fifthly, the present invention provides a computer program product comprising computer instructions that, when executed on a computer, cause the computer to perform the fall detection method as provided in the first aspect.
[0031] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the fall detection device, or it may be packaged separately from the processor of the fall detection device; this invention does not impose any limitations on this.
[0032] The technical solution of this invention achieves radar monitoring of target objects within a preset area by acquiring radar data of target objects within the field of view of a radar sensor, and event monitoring of target objects by acquiring event data of target objects. Neither the radar data nor the event data involves the privacy information of the target objects, achieving dual real-time monitoring of the target objects without infringing on their privacy. A first detection result is determined by using radar data to perform fall detection on the target objects, and a second detection result is determined by using event data to perform fall detection on the target objects, thus achieving two fall detections of the target objects. The first and second detection results are fused to obtain the final fall detection result of the target objects, improving the accuracy and reliability of fall detection. Therefore, using radar data acquired by a radar sensor and event data acquired by an event vision sensor to perform fall detection on target objects can improve the accuracy and reliability of fall detection while reducing the risk of privacy leakage of target objects, thus improving the user experience.
[0033] The descriptions of the second, third, fourth, and fifth aspects of this invention can be referred to the detailed description of the first aspect; and the beneficial effects described in the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.
[0034] In this invention, the name of the aforementioned fall detection device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this invention, it falls within the scope of the claims of this invention and its equivalents.
[0035] These or other aspects of the invention will become more apparent from the following description. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 A flowchart of a fall detection method provided in an embodiment of the present invention;
[0038] Figure 2 A flowchart of another fall detection method provided in an embodiment of the present invention;
[0039] Figure 3 This is a schematic diagram of the structure of a fall detection device provided in an embodiment of the present invention;
[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0041] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0042] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0043] The terms "first" and "second," etc., used in the specification and drawings of this invention are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.
[0044] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.
[0045] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0046] It should be noted that in the embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0047] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.
[0048] In existing technologies, fall detection can be implemented in two ways: First, based on radar sensors, fall detection can be performed using radar data acquired by the sensors. This method effectively protects personal privacy; however, it is susceptible to environmental factors and the number of people present, presenting certain limitations. Furthermore, the lack of crucial real-time full-duplex audio and video footage hinders in-depth understanding of the specific situation, resulting in lower detection accuracy. Second, based on radar sensors and an APS (Advanced Perspective Screen), initial detection is performed using radar data, followed by further analysis of posture and behavior using scene images obtained from the APS. This method offers higher accuracy, but carries risks of algorithmic misjudgment and privacy breaches. For example, false alarms may occur when someone actively sits down to tidy up, crawls to clean, or picks up items, and privacy leaks may result from unauthorized access to the APS. Additionally, APS is significantly affected by lighting and environmental conditions.
[0049] Therefore, the present invention provides a fall detection method that improves the accuracy and reliability of fall detection while reducing the risk of leakage of the target object's privacy information.
[0050] Figure 1 This is a flowchart illustrating a fall detection method provided in an embodiment of the present invention. This embodiment is applicable to situations where the accuracy and reliability of fall detection are improved while reducing the risk of leakage of the target object's privacy information. The method can be executed by a fall detection device, such as... Figure 1As shown, the specific steps include the following:
[0051] Step 110: Obtain radar data and event data of the target object.
[0052] Among them, radar data is collected by radar sensors, and event data is collected by event vision sensors.
[0053] Understandably, the radar sensor can be a millimeter-wave radar, and the event-based vision sensor (EVS) can be either a pure event camera or a fusion camera that integrates multiple operating modes, including at least the event camera mode (EVS mode). For example, a fusion camera that integrates EVS mode and APS mode can function as an event camera operating in EVS mode, or as an active pixel sensor operating in APS mode. The field of view of both the millimeter-wave radar and the EVS is a preset area. The target object can be understood as an object appearing within the preset area.
[0054] Radar data is the output data of a millimeter-wave radar after it transmits millimeter waves to scan a preset area. Generally, radar data includes point cloud data, velocity information, and reflection intensity. Point cloud data includes the three-dimensional coordinates (range, azimuth, and elevation angles) of each target within the radar's field of view; velocity information includes the radial velocity of each target within the radar's field of view; and reflection intensity is the strength of the reflected signal from each target within the radar's field of view.
[0055] Event data is the data output by EVS in response to changes in light intensity within its field of view. It's important to note that, unlike APS, EVS does not perceive the environment by capturing images; instead, it perceives the environment by recording events of light intensity changes. Therefore, event data does not involve the privacy information of the target object. EVS works by detecting changes in light intensity at the pixel level. When the light intensity increases or decreases and the change exceeds a set light intensity threshold, the corresponding pixel triggers EVS to output event data. Event data includes a timestamp, pixel location, and brightness change polarity. The timestamp indicates the time the event data was generated; the pixel location indicates the position of the pixel that generated the event data; and the brightness change polarity indicates whether the light intensity change that triggered the event data was an increase or decrease.
[0056] Specifically, radar data of objects within its field of view can be acquired based on radar sensors, i.e., radar data of target objects within a preset area can be acquired based on millimeter-wave radar. Event data of objects within its field of view can also be acquired based on event vision sensors, i.e., event data of target objects within a preset area can be acquired based on EVS, enabling continuous monitoring of each target object within the preset area. Furthermore, the radar data and event data of the target objects can be used for fall detection.
[0057] In this embodiment of the invention, by acquiring radar data and event data of the target object, dual monitoring of a preset area is achieved, thus providing dual monitoring protection for the target object without infringing on its privacy within the preset area.
[0058] Step 120: Detect the falling behavior of the target object based on radar data to obtain the first detection result; detect the falling behavior of the target object based on event data to obtain the second detection result.
[0059] Specifically, after acquiring the radar data and event data of the target object, fall behavior detection is performed on the target object based on the radar data and event data.
[0060] Specifically, when detecting fall behavior in a target object based on radar data, the radar data can first be preprocessed and its features extracted. Then, based on the extracted feature data, the motion pattern of the target object is analyzed to determine whether the target object has fallen and the probability of falling, thus obtaining a first detection result. Similarly, when detecting fall behavior in a target object based on event data, the event data can also be preprocessed first. Then, features are extracted from the preprocessed event data, and based on the extracted feature data, the motion pattern of the target object is identified to determine whether the target object has fallen and the probability of falling, thus obtaining a second detection result.
[0061] In this embodiment of the invention, a first detection result is determined by performing fall detection on the target object using radar data of the target object, and a second detection result is determined by performing fall detection on the target object using event data of the target object, thereby realizing the detection of two fall behaviors of the target object.
[0062] Step 130: Combine the first and second detection results to obtain the fall detection results.
[0063] To adjust the emphasis of fall detection on radar and event vision sensors, a first weight can be assigned to the first detection result, and a second weight to the second detection result. When fall detection balances the fall behavior detection of both radar and event vision sensors, the first weight can be set equal to the second weight; when fall detection emphasizes radar sensor detection, the first weight can be set greater than the second weight; and when fall detection emphasizes event vision sensor detection, the first weight can be set less than the second weight. Of course, the sum of the first and second weights is always 1. The specific values of the first and second weights can be set according to actual needs and are not specifically limited here.
[0064] Specifically, after determining the first detection result and the second detection result of the target object, the first detection result and the second detection result can be fused based on the first weight of the first detection result and the second weight of the second detection result to obtain the fall detection result of the target object. Specifically, when the first detection result includes the first fall probability of the target object falling and the second detection result includes the second fall probability of the target object falling, the target fall probability of the target object can be determined based on the first weight and the first fall probability, as well as the second weight and the second fall probability. Therefore, the target fall probability of the target object can be determined as the fall detection result of the target object.
[0065] In this embodiment of the invention, the fall detection result of the target object is obtained by fusing the first detection result and the second detection result with the first weight corresponding to the first detection result and the second weight corresponding to the second detection result. This allows the fall detection result of the target object to integrate the first detection result and the second detection result, thereby improving the accuracy and reliability of fall detection.
[0066] In this embodiment of the invention, radar data of target objects within the field of view of radar sensors and event vision sensors are acquired to achieve radar monitoring of target objects within a preset area. Event data of target objects is acquired to achieve event monitoring of target objects. Neither radar data nor event data involves the privacy information of the target objects. This achieves dual real-time monitoring of target objects without infringing on their privacy. A first detection result is determined by using radar data to perform fall detection on the target objects, and a second detection result is determined by using event data to perform fall detection on the target objects. The first and second detection results are then fused to obtain the final fall detection result, improving the accuracy and reliability of fall detection. Therefore, using radar data acquired by radar sensors and event data acquired by event vision sensors to perform fall detection on target objects can improve the accuracy and reliability of fall detection while reducing the risk of privacy leakage and enhancing user experience.
[0067] Figure 2 This is a flowchart of another fall detection method provided by an embodiment of the present invention. This embodiment is a specific modification based on the above embodiments. Figure 2 As shown, in this embodiment, the method may further include:
[0068] Step 210: Obtain radar data and event data of the target object.
[0069] If the light intensity within the preset area remains unchanged, or if the light intensity changes but the change is not greater than the light intensity threshold, EVS will not output event data. In this case, the target object is not moving within the preset area, nor is it possible for it to fall within the preset area. If the light intensity within the preset area changes and the change is greater than the light intensity threshold, the corresponding pixels will generate event data, and EVS can output the event data. In this case, it is determined that the target object within the preset area may have moved, or the brightness of the light source within the preset area may have changed. Since the brightness of the light source generally does not change, it is determined that there may be a moving object within the preset area. That is, the target object within the preset area may be moving within the preset area, or it may have fallen within the preset area.
[0070] By performing on-chip processing on the event data output from all the acquired pixels, a contour image of the target object's movement process can be formed based on multiple event data with the same timestamp. This image is the event image.
[0071] Therefore, given the EVS output event image, if it is determined that the target object in the preset area has moved or the brightness of the light source in the preset area has changed (the change in brightness of the light source is generally negligible), it can be determined that the target object in the preset area has moved and may have fallen. In this case, it is necessary to perform fall detection on the target object to obtain the radar data and event data of the target object.
[0072] In this embodiment of the invention, radar data and event data of the target object are acquired only when the target object moves within a preset area. The radar data and event data are used to detect fall behavior of the target object, which can reduce the power consumption and computational load of fall detection.
[0073] Step 220: Detect the falling behavior of the target object based on radar data to obtain the first detection result.
[0074] In one implementation, step 220 may specifically include:
[0075] The radar data undergoes a first preprocessing step, which includes dynamic noise filtering, static clutter suppression, and transformation from polar coordinates to Cartesian coordinates. Feature extraction is performed on the radar data to obtain first feature data. The first feature data is input into a pre-trained first classifier to obtain a first fall probability, where the first fall probability is the probability that the first feature data matches a preset fall behavior. The first fall probability is then determined as the first detection result.
[0076] The first feature data may include kinematic features, micro-Doppler features, and spatiotemporal joint features. In embodiments of the present invention, kinematic features include features that change significantly when the target object falls, such as vertical velocity, height change, and center of mass trajectory tilt angle; micro-Doppler features include the temporal spectrum and spectral energy distribution associated with the target object's motion pattern; and spatiotemporal joint features include the target object's trajectory sequence and the duration of the trajectory sequence.
[0077] Specifically, directly acquired radar data contains a certain amount of noise interference, and radar data is usually in polar coordinate system. In order to improve the efficiency and accuracy of feature extraction from radar data, a first preprocessing can be performed on the radar data before extracting the first feature data of the target object based on the radar data. Specifically, dynamic noise filtering and static clutter suppression can be performed on the radar data to reduce the disturbance terms in the radar data. The radar data can also be transformed from polar coordinate system to Cartesian coordinate system to realize the coordinate transformation of the radar data.
[0078] Secondly, the primary characteristic data of the target object can be extracted based on radar data. Specifically, the Doppler effect of the radar data can be analyzed to determine the target object's motion pattern. This can be achieved by analyzing the changes in the Doppler frequency, spectral width, and spectral shape of the radar data over time to determine the target object's motion description information. When the motion description information is determined to be a fall, feature extraction of the target object using radar data can determine its primary characteristic data, thereby identifying the target object's trajectory, speed changes, and body posture.
[0079] Furthermore, the first feature data can be input into the pre-trained first classifier. The first classifier first preprocesses the first feature data, specifically performing feature normalization and feature selection. Feature normalization performs Z-score standardization on the first feature data, eliminating differences in the dimensions of different features and improving the convergence speed and accuracy of the first classifier. Feature selection removes redundant features, reducing the computational load of the first classifier. Moreover, feature selection can use mutual information analysis to retain the top n features most highly correlated with fall behavior. Then, the first classifier will perform inference based on the preprocessed first feature data. Specifically, it can compare the first feature data with the feature data corresponding to preset fall behaviors. Based on the similarity between the first feature data and the feature data corresponding to preset fall behaviors, it determines the first similarity between the target object's current behavior and each preset fall behavior. If the first similarity between the target object's current behavior and any preset fall behavior is greater than the first threshold, it can be determined that the first detection result is that the target object has fallen, and the fall probability is the first similarity. If the first similarity between the target object's current behavior and each preset fall behavior is not greater than the first threshold, it can be determined that the first detection result is that the target object has not fallen.
[0080] Of course, since falls are usually accompanied by rapid changes in speed and abnormal movement patterns, the fall detection described above is achieved by analyzing time-series data from radar data.
[0081] It should be noted that, depending on the application scenario or requirements, the embodiments of the present invention may select an appropriate first classifier to infer the first fall probability. For example, in scenarios requiring lightweight models, XGBoost or lightweight neural networks (such as MobileNet variants) can be used as the first classifier; in high-precision scenarios, a two-stream network can be used as the first classifier. The two streams of the two-stream network process different feature data respectively, and the two feature processing results are weighted and concatenated through a fusion layer to obtain the first fall probability.
[0082] More specifically, after the first feature data is input into the first classifier, the first classifier can perform fall behavior detection based on machine learning algorithms or deep learning algorithms and output the first fall probability. XGBoost or lightweight neural networks generally correspond to machine learning algorithms, first using ReliefF or Random Forest to evaluate the feature importance of the first feature data, and then using Support Vector Machines, Decision Trees, etc., to classify the first feature data. Two-stream networks generally correspond to deep learning algorithms, using CNN to process the spectrogram and LSTM to process the trajectory sequence to process the first feature data.
[0083] It should be noted that, in this embodiment of the invention, a training dataset is acquired and a first classifier is trained based on the training dataset. The training dataset includes positive samples and negative samples. The positive samples contain multiple sets of radar data corresponding to various types of fall behaviors, such as forward falls, side falls, and slips. The negative samples contain multiple sets of radar data corresponding to various types of non-fall behaviors (disruptive actions), such as sitting down quickly, bending over, and jumping. In some embodiments, the training dataset can be expanded and the data enhanced by adding noise events, simulating occlusion, and adjusting the time scaling.
[0084] In this embodiment of the invention, a first detection result is obtained by detecting the fall behavior of the target object using radar data, thereby realizing the detection of the fall behavior of the target object based on radar data.
[0085] Step 230: Detect the fall behavior of the target object based on the event data to obtain the second detection result.
[0086] In one implementation, step 230 may specifically include:
[0087] Feature extraction is performed on the event data to obtain second feature data; the second feature data is input into a pre-trained second classifier to obtain a second fall probability, wherein the second fall probability is the probability that the second feature data matches a preset fall behavior; the second fall probability is determined as the second detection result.
[0088] Among them, the event data is the asynchronous event stream data of the target object within the preset area obtained by EVS, which includes timestamps, pixel positions and brightness change polarity.
[0089] Specifically, a spiking neural network (SNN) can first be used to detect the centroid of the event data, determining the centroid of the event stream corresponding to the event data, i.e., the centroid of the target object, thus achieving target object localization. Alternatively, the centroid trajectory of the target object can be determined based on continuously acquired event data. The centroid trajectory of the target object determined through event data is the second feature data of the target object. A fixed-length centroid trajectory is input into a second classifier, which can output binary labels of "fallen" or "not fallen," where the "fallen" binary label includes a second fall probability. Therefore, the second detection result can be determined as either "no fall occurred" or "fallen" with a second fall probability.
[0090] It should be noted that there is an overlap between the centroid trajectories of fixed lengths.
[0091] In some embodiments, before extracting features from the event data to obtain the second feature data, background activity noise in the event data can be filtered out. Specifically, when filtering background activity noise from the event data, a suitable filter can be selected according to the application scenario. Since the amplitude of movement varies at different parts of the target object, and some parts generate less event data, a filter with strong signal retention capability is preferred in this embodiment. It is understood that filtering out noise from the event data before feature extraction can improve the accuracy of feature extraction.
[0092] SNNs do not require the conversion of event streams to event frames. They directly process event data based on a series of interconnected convolutional layers, max-pooling layers, convolutional layers, max-pooling layers, global average pooling layers, and an output fully connected layer. The output of the output fully connected layer indicates whether there is a moving object or not. By processing event data through the SNN, the weights of the output fully connected layer and the output feature map of the second max-pooling layer are used to calculate the SNN's Class Activation Map (CAM). The center of the calculated CAM is used as the centroid of the event stream. The center of the CAM can be understood as the average position of a specified proportion of pixels with larger CAM pixel values.
[0093] The training process of an SNN may include: establishing a Convolutional Neural Network (CNN) with the same network topology as the SNN. The input to the CNN is event images (event frames) accumulated from event data. Neurons in the CNN use continuous real values to transmit information. Except for the last fully connected output layer, neurons in all other layers use the ReLU activation function, while neurons in the last fully connected output layer use the Softmax activation function. Event images containing moving human bodies are used as positive samples, and event images containing only noise are used as negative samples. These positive and negative samples constitute the training samples to complete the training of the CNN. The CNN is then converted to an SNN to obtain the trained SNN.
[0094] Furthermore, this embodiment of the invention employs stochastic gradient descent to train the CNN, setting a learning rate and training for several epochs. For example, with a learning rate of 0.01, the accuracy reaches 90% after 20 epochs.
[0095] It should be noted that an event image can be understood as an image composed of multiple event data with the same timestamp after processing.
[0096] It should also be noted that the classification algorithm used by the second classifier can be one of the following: k Nearest Neighbor (KNN) algorithm, Gaussian Naive Bayes (GNB) algorithm, Decision Tree (DT) algorithm, and Random Forest (RF) algorithm.
[0097] In this embodiment of the invention, a second detection result is obtained by detecting the fall behavior of the target object through event data, thereby realizing the detection of the fall behavior of the target object based on event data.
[0098] It should be noted that after step 210, steps 220 and 230 can be executed. The execution order of steps 220 and 230 is not specifically limited. Step 220 can be executed first and then step 230, or step 230 can be executed first and then step 220, or steps 220 and 230 can be executed simultaneously.
[0099] Step 240: Combine the first and second detection results to obtain the fall detection results.
[0100] In one implementation, step 240 may specifically include:
[0101] Obtain the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability; based on the first weight and the second weight, perform a weighted calculation on the first fall probability and the second fall probability to obtain the target fall probability.
[0102] Specifically, after determining the first detection result and the second detection result, if both the first and second detection results indicate that the target object has fallen, then the target fall probability is determined based on the first fall probability and the first weight, as well as the second fall probability and the second weight. More specifically, the first and second detection results can be fused based on the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability to obtain the target object's fall detection result. That is, the target fall probability can be determined by weighted summation of the first and second fall probabilities based on the first and second weights.
[0103] If either the first or second detection result includes the target object falling, then when it is determined that the first detection result includes the target object falling, the first fall probability included in the first detection result can be determined as the target fall probability; and when it is determined that the second detection result includes the target object falling, the second fall probability included in the second detection result can be determined as the target fall probability.
[0104] Furthermore, the target fall probability and the target threshold can be compared. If the target fall probability is greater than the target threshold, the fall detection result of the target object is determined to be that the target object has fallen and the fall probability is the target fall probability. Otherwise, the fall detection result of the target object is determined to be that the target object has not fallen.
[0105] Of course, if neither the first nor the second detection result includes the target object falling, it can be determined that the target object's fall detection result is that the target object did not fall.
[0106] Further, the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability are obtained, including:
[0107] Obtain a preset weight dataset, which includes the data mapping relationship between the ambient brightness of the target object, the number of target objects, and the first weight and the second weight; obtain the ambient brightness of the target object and the number of target objects; based on the ambient brightness of the target object and the number of target objects, obtain the corresponding first weight and second weight from the preset weight dataset.
[0108] The brightness of the environment in which the target object is located within the preset area and the number of target objects within the preset area will affect the accuracy of the first detection result determined based on radar data and the second detection result determined based on event data. Therefore, the first weight of the first detection result determined based on radar data and the second weight of the second detection result determined based on event data can be adaptively adjusted according to the brightness of the environment in which the target object is located within the preset area and the number of target objects within the preset area.
[0109] Specifically, the brightness of the environment in which the target object is located within the preset area will affect the accuracy of the second detection result determined based on event data, while the brightness of the environment in which the target object is located within the preset area will hardly affect the accuracy of the first detection result determined based on radar data.
[0110] More specifically, when the detection threshold of the event vision sensor remains unchanged, the event data it collects under different ambient brightness conditions will differ. For the same action, due to the limitation of ambient brightness, the change in light intensity collected under dim light is relatively small, and the corresponding event polarity may be 0, that is, it is considered that there is no change in light intensity. However, the change in light intensity collected under strong light may be relatively large, and the corresponding event polarity may be +1 or -1, that is, it is considered that the light intensity has become brighter or darker.
[0111] In other words, when the detection threshold of the event vision sensor remains unchanged, the ambient brightness affects the probability and quantity of valid event data (event data with an event polarity of +1 or -1) collected. For the same action, fewer valid event data are collected in low light, which may lead to missed event data collection and potentially result in missed or false detections of fall behavior, resulting in low accuracy of the corresponding detection results. On the other hand, more valid event data are collected in strong light, which almost eliminates the possibility of missed event data collection and reduces the probability of missed or false detections of fall behavior, thus improving the accuracy of the corresponding detection results.
[0112] More specifically, radar sensors transmit electromagnetic waves. Aside from a few visible light signals like lasers, most signals are invisible electromagnetic waves, such as millimeter waves and microwaves. Changes in ambient light or light sources generally have minimal interference with these invisible electromagnetic waves. Even for lidar, the interference from changes in ambient light or light sources on the laser signal is negligible. Therefore, fluctuations in ambient light or light sources have a negligible impact on radar sensor signals or radar data.
[0113] In other words, changes in ambient brightness within the preset area have almost no impact on radar data, and consequently, no impact on the accuracy of the detection results determined based on radar data.
[0114] As the above analysis shows, with the default event visual sensor detection threshold unchanged, the accuracy of the fall detection result determined based on event data is affected by the ambient brightness of the target object within the preset area. Fall detection results are less accurate in low light and more accurate in strong light. Therefore, when fusing the first and second detection results, the second weight corresponding to the second detection result needs to be adaptively adjusted according to the ambient brightness of the target object within the preset area. The second weight should be increased appropriately when the ambient brightness is high and decreased appropriately when the ambient brightness is low, so that the final fused fall detection result has high accuracy. Specifically, the number of target objects within the preset area has almost no impact on the accuracy of the second detection result determined based on event data, while the number of target objects within the preset area affects the accuracy of the first detection result determined based on radar data.
[0115] More specifically, event vision sensors primarily extract temporal motion information, especially contour information. In other words, event data includes the contour information of moving objects. Event vision sensors can effectively detect various moving targets within their field of view; they can sense any motion or change in light intensity, and are virtually unaffected by the number of target objects. Therefore, the number of target objects within a preset area does not affect the acquisition of event data or valid event data, and consequently, it does not affect the accuracy of the second detection result determined based on the event data.
[0116] More specifically, radar sensors collect data based on reflected signals that are returned after the transmitted signal encounters an object. In other words, radar data consists of reflected signals. When the operating parameters of the radar sensor remain constant, and the number of objects or moving objects within the field of view is small, there is little or no interference between the reflected signals, resulting in accurate radar data acquisition and matching, and high precision of the detection results. However, when the number of moving objects within the field of view is large, there is significant mutual interference between the reflected signals, making radar data acquisition and matching more prone to errors, and resulting in lower precision of the detection results.
[0117] As the above analysis shows, with the default operating parameters of the visual sensor and the radar sensor remaining unchanged, the number of target objects within the preset area has almost no impact on the accuracy of the second detection result determined based on event data, but it does affect the accuracy of the first detection result determined based on radar data. A higher number of target objects results in lower fall detection accuracy, while a lower number results in higher accuracy. Therefore, when fusing the first and second detection results, the first weight corresponding to the first detection result needs to be adaptively adjusted according to the number of target objects within the preset area. When the number of target objects is high, the first weight should be appropriately lowered; when the number of target objects is low, the first weight should be appropriately increased to ensure high accuracy of the final fused fall detection result. Therefore, in this embodiment of the invention, to improve the accuracy of the fused fall detection result, the first weight of the first detection result and the second weight of the second detection result can be adaptively adjusted according to the brightness of the environment where the target objects are located within the preset area and the number of target objects within the preset area.
[0118] It should be noted that the specific data mapping relationship between the brightness of the environment where the target object is located within the preset area, the number of target objects within the preset area, and the first and second weights can be set through pre-calibration or adjusted in real time according to the relative changes of parameters, and is not limited here.
[0119] In an optional embodiment of the present invention, if the ambient brightness is greater than a first brightness threshold, the second weight is increased according to the difference between the ambient brightness and the first brightness threshold; if the ambient brightness is less than the second brightness threshold, the second weight is decreased according to the difference between the second brightness threshold and the ambient brightness; if the ambient brightness is between the first brightness threshold and the second brightness threshold, the default second weight is used. Similarly, if the number of target objects is greater than a first quantity threshold, the first weight is decreased according to the difference between the number of target objects and the first quantity threshold; if the number of target objects is less than the second quantity threshold, the first weight is increased according to the difference between the second quantity threshold and the number of target objects; if the number of target objects is between the first quantity threshold and the second quantity threshold, the default first weight is used.
[0120] When performing pre-calibration settings to ensure high-precision fall detection results, the data mapping relationship between the brightness of the environment where the target object is located within the pre-calibrated preset area, the number of target objects within the preset area, and the first and second weights is stored in a preset weight dataset. Therefore, after determining the brightness of the environment where the target object is located within the preset area and the number of target objects within the preset area, the preset weight dataset can be directly accessed to determine the corresponding first and second weights based on the ambient brightness and the number of target objects.
[0121] In this embodiment of the invention, the fall detection result of the target object is obtained by fusing the first detection result and the second detection result with a first weight corresponding to the first detection result and a second weight corresponding to the second detection result. This allows the fall detection result of the target object to integrate the first and second detection results, improving the accuracy and reliability of fall detection. Furthermore, the first and second weights are adaptively adjusted according to the brightness of the environment in which the target object is located within a preset area and the number of target objects within the preset area, further improving the accuracy and reliability of fall detection.
[0122] Step 250: Determine whether the target object has fallen based on the fall detection results.
[0123] Specifically, if the fall detection results include the target object falling, then it is determined that the target object has fallen.
[0124] Step 260: In the event that the target object falls, send an alarm message to the corresponding monitoring device.
[0125] The monitoring equipment can be understood as the monitoring equipment corresponding to the target object, that is, the terminal equipment of the guardian corresponding to the target object. The alarm information can include event data, fall warning information and confirmation information.
[0126] Specifically, after confirming that a target has fallen, in order for the target's guardian to respond promptly to the fall, an alarm message can be sent to the monitoring device corresponding to the guardian. The alarm message can include event data, a fall alert, and a confirmation message. The fall alert is used to notify the guardian that the target has fallen; for example, it could be "The target is suspected of having fallen." The confirmation message is used to confirm whether further scene images should be obtained; for example, it could be "Do you allow further detection of the fall based on scene image sensors and the presentation of scene images containing privacy?" A complete alarm message could be: event data and "The target is suspected of having fallen; do you allow further detection of the fall based on scene image sensors and the presentation of scene images containing privacy?"
[0127] Step 270: In response to the confirmation message sent by the monitoring device, send the event data to the monitoring device, or send the event data and scene image to the monitoring device.
[0128] The scene images were acquired by an active pixel sensor.
[0129] The scene image sensor can be understood as an image sensor used to acquire scene images. For example, it can be an APS. Compared with EVS, APS has higher imaging quality and can output RGB color images containing more information.
[0130] In one implementation, the confirmation information includes first confirmation information and second confirmation information. Accordingly, step 270 may specifically include:
[0131] If the confirmation information sent by the monitoring device is the first confirmation information, the event data related to the fall behavior will be sent to the monitoring device and displayed on the monitoring device in the form of an event image; if the confirmation information sent by the monitoring device is the second confirmation information, the scene image related to the fall behavior will be acquired using an active pixel sensor, and the event data and scene image related to the fall behavior will be sent to the monitoring device, and the event data will be displayed on the monitoring device in the form of an event image, and / or, the scene image will be displayed on the monitoring device.
[0132] Since the scene image contains the privacy information of the target object, the scene image sensor is activated to acquire the scene image after receiving the second confirmation message confirming the acquisition of the scene image.
[0133] Specifically, after receiving confirmation information input by the guardian, the monitoring device can upload the confirmation information. If the confirmation information is the first confirmation information, the event data related to the fall behavior is sent to the monitoring device, which can display the event data in the form of event images. If the confirmation information is the second confirmation information, the APS is activated to obtain scene images related to the fall behavior, and the event data and scene images related to the fall behavior are sent to the monitoring device. The monitoring device can display the event data and / or scene images. That is, the monitoring device can display the event data and scene images simultaneously in the form of event images, or it can display the event data only in the form of event images, or it can display only scene images.
[0134] In practical applications, after acquiring the scene image, auxiliary detection is performed based on the acquired scene image. The target object's posture is recognized according to the scene image, and the target object's posture is output. That is, based on the aforementioned fall detection, fall posture recognition is performed to assist in determining the target object's fall behavior. If the target object's posture is determined to be a fall posture, the scene image is sent to the monitoring device; otherwise, it is not necessary to send the scene image to the monitoring device. This corrects the aforementioned fall detection misjudgments and improves the accuracy of fall detection.
[0135] Furthermore, if no confirmation is received within a preset time, the system acquires scene images based on the APS (Advanced Positioning System). Upon determining that the target object's posture is a fall posture through posture recognition based on the acquired scene images, an early warning message is sent to the monitoring system corresponding to the target object's guardian. This warning message may include event data and scene images. In cases where the guardian fails to respond promptly to the target object's fall, the guardian can provide timely assistance, preventing incidents of untimely rescue after a fall due to the guardian's failure to respond, thus reducing safety risks.
[0136] Understandably, the guardianship institution could be a nursing home or emergency care center in the area where the target person lives.
[0137] In this embodiment of the invention, when it is determined that a target object has fallen, an alarm message is sent to the monitoring device to alert the target object's guardian that a fall has occurred. After receiving confirmation information from the guardian based on the monitoring device, event data is sent to the monitoring device, or event data and scene images are sent to the monitoring device. This allows for the acquisition of scene images containing private information with the guardian's consent, thereby improving the security of fall detection.
[0138] The fall detection method provided in this invention determines that a target object may be moving within a preset area when the event visual sensor outputs event data. In this case, radar data and event data of the target object can be acquired. Conversely, if the event visual sensor does not output event data, the target object is not moving within the preset area, and acquiring radar data and event data is unnecessary. Radar data and event data are only acquired when the target object is moving within the preset area, reducing the power consumption and computational load of fall detection. A first detection result is determined by performing fall detection on the target object using radar data, and a second detection result is determined by performing fall detection on the target object using event data. Furthermore, neither radar data nor event data involves the target object's privacy information. This achieves the detection of two fall behaviors of the target object without infringing on the target object's privacy. The first detection result is then used to determine the second fall behavior. The second weight corresponding to the second detection result is used to fuse the first and second detection results to obtain the fall detection result of the target object. This fuses the first and second detection results, improving the accuracy and reliability of fall detection. When it is determined that the target object has fallen based on the fall detection result, an alarm message is sent to the monitoring device to remind the target object's guardian of the fall. After receiving the confirmation message sent by the guardian based on the monitoring device, the event data is sent to the monitoring device, or the event data and scene images are sent to the monitoring device. This allows for the acquisition of scene images containing private information with the guardian's consent, protecting the target object's privacy and improving the security of fall detection. Furthermore, it facilitates the guardian's accurate assessment of the target object's current state, providing accurate scene records for the next critical rescue step.
[0139] Figure 3 This is a schematic diagram of a fall detection device provided in an embodiment of the present invention. The device can be applied to situations where the accuracy and reliability of fall detection are improved while reducing the risk of leakage of the target object's privacy information. The device can be implemented by software and / or hardware and is generally integrated into electronic devices, such as fall detection devices.
[0140] like Figure 3 As shown, the device includes:
[0141] The acquisition module 310 is used to acquire radar data and event data of the target object, wherein the radar data is acquired by a radar sensor and the event data is acquired by an event vision sensor.
[0142] The data processing and detection module 320 is used to detect the fall behavior of the target object based on radar data to obtain a first detection result, and is also used to detect the fall behavior of the target object based on event data to obtain a second detection result;
[0143] The fusion module 330 is used to fuse the first detection result and the second detection result to obtain the fall detection result.
[0144] The fall detection device provided in this embodiment achieves radar monitoring of target objects within a preset area by acquiring radar data of target objects within the field of view of radar sensors and event vision sensors, and event monitoring of target objects by acquiring event data of target objects. Neither radar data nor event data involves the privacy information of the target objects, achieving dual real-time monitoring of the target objects without infringing on their privacy. A first detection result is determined by using radar data to perform fall detection on the target objects, and a second detection result is determined by using event data to perform fall detection on the target objects, thus achieving two fall detections of the target objects. The first and second detection results are fused to obtain the final fall detection result of the target objects, improving the accuracy and reliability of fall detection. Therefore, using radar data acquired by radar sensors and event data acquired by event vision sensors to perform fall detection on target objects can improve the accuracy and reliability of fall detection while reducing the risk of privacy information leakage of target objects, thus improving user experience.
[0145] Based on the above embodiments, the first detection result includes a first fall probability. Accordingly, the data processing and detection module 320 is specifically used for:
[0146] The radar data undergoes a first preprocessing step, which includes dynamic noise filtering, static clutter suppression, and transformation from polar coordinates to Cartesian coordinates. Feature extraction is performed on the radar data to obtain first feature data. The first feature data is then input into a pre-trained first classifier to obtain a first fall probability, where the first fall probability is the probability that the first feature data matches a preset fall behavior.
[0147] Based on the above embodiments, the second detection result includes a second fall probability. Accordingly, the data processing and detection module 320 is specifically used for:
[0148] Feature extraction is performed on the event data to obtain second feature data; the second feature data is input into a pre-trained second classifier to obtain a second fall probability, wherein the second fall probability is the probability that the second feature data matches the preset fall behavior.
[0149] Based on the above embodiments, the first detection result includes a first fall probability, the second detection result includes a second fall probability, and the fall detection result includes a target fall probability. Accordingly, the fusion module 330 is specifically used for:
[0150] Obtain the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability; based on the first weight and the second weight, perform a weighted calculation on the first fall probability and the second fall probability to obtain the target fall probability.
[0151] In one embodiment, obtaining the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability includes:
[0152] Obtain a preset weight dataset, which includes the data mapping relationship between the ambient brightness of the target object, the number of target objects, and the first weight and the second weight; obtain the ambient brightness of the target object and the number of target objects; based on the ambient brightness of the target object and the number of target objects, obtain the corresponding first weight and second weight from the preset weight dataset.
[0153] Based on the above embodiments, the device further includes:
[0154] The judgment and alarm module is used to determine whether the target object has fallen based on the fall detection results; if the target object has fallen, it sends an alarm message to the corresponding monitoring device; in response to the confirmation message sent by the monitoring device, it sends event data to the monitoring device, or sends event data and scene images to the monitoring device, wherein the scene images are acquired by an active pixel sensor.
[0155] In one embodiment, the confirmation information includes first confirmation information and second confirmation information. Accordingly, the step of sending event data to the monitoring device in response to the confirmation information sent by the monitoring device, or sending event data and scene images to the monitoring device, includes:
[0156] If the confirmation information sent by the monitoring device is the first confirmation information, the event data related to the fall behavior will be sent to the monitoring device and displayed on the monitoring device in the form of an event image; if the confirmation information sent by the monitoring device is the second confirmation information, the scene image related to the fall behavior will be acquired using an active pixel sensor, and the event data and scene image related to the fall behavior will be sent to the monitoring device, and the event data will be displayed on the monitoring device in the form of an event image, and / or, the scene image will be displayed on the monitoring device.
[0157] The fall detection device provided in this embodiment of the invention can execute the fall detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the fall detection method.
[0158] It is worth noting that in the above embodiments of the fall detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.
[0159] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0160] like Figure 4 As shown, electronic device 4 is represented in the form of a general-purpose computing electronic device. The components of electronic device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0161] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0162] Electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 4, including volatile and non-volatile media, removable and non-removable media.
[0163] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0164] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.
[0165] Electronic device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 4, and / or with any device that enables electronic device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with electronic device 4 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0166] Processing unit 16 executes various functional applications and page displays by running programs stored in system memory 28, such as implementing the fall detection method provided in this embodiment of the invention, which includes:
[0167] The system acquires radar data and event data of the target object. The radar data is collected by radar sensors, and the event data is collected by event vision sensors.
[0168] The first detection result is obtained by detecting the fall behavior of the target object based on radar data, and the second detection result is obtained by detecting the fall behavior of the target object based on event data.
[0169] The first and second test results are combined to obtain the fall detection results.
[0170] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the fall detection method provided in any embodiment of the present invention.
[0171] This invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements, for example, the fall detection method provided in this invention, which includes:
[0172] The system acquires radar data and event data of the target object. The radar data is collected by radar sensors, and the event data is collected by event vision sensors.
[0173] The first detection result is obtained by detecting the fall behavior of the target object based on radar data, and the second detection result is obtained by detecting the fall behavior of the target object based on event data.
[0174] The first and second test results are combined to obtain the fall detection results.
[0175] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0176] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0177] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0178] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0179] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0180] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with the relevant provisions of national laws and regulations.
[0181] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.
Claims
1. A fall detection method, characterized in that, include: The radar data and event data of the target object are acquired, wherein the radar data is collected by a radar sensor and the event data is collected by an event vision sensor. The fall behavior of the target object is detected based on the radar data to obtain a first detection result; the fall behavior of the target object is detected based on the event data to obtain a second detection result. The first detection result and the second detection result are fused together to obtain the fall detection result.
2. The fall detection method according to claim 1, characterized in that, The first detection result includes a first fall probability. Correspondingly, the step of detecting the fall behavior of the target object based on the radar data to obtain the first detection result includes: Feature extraction is performed on the radar data to obtain the first feature data; The first feature data is input into a pre-trained first classifier to obtain the first fall probability, wherein the first fall probability is the probability that the first feature data matches a preset fall behavior.
3. The fall detection method according to claim 2, characterized in that, Before performing feature extraction on the radar data to obtain the first feature data, the fall detection method further includes: The radar data undergoes a first preprocessing step, which includes dynamic noise filtering, static clutter suppression, and conversion from polar coordinates to Cartesian coordinates.
4. The fall detection method according to claim 1, characterized in that, The second detection result includes a second fall probability. Accordingly, the step of detecting the fall behavior of the target object based on the event data to obtain the second detection result includes: Feature extraction is performed on the event data to obtain second feature data; The second feature data is input into a pre-trained second classifier to obtain the second fall probability, wherein the second fall probability is the probability that the second feature data matches a preset fall behavior.
5. The fall detection method according to claim 1, characterized in that, The first detection result includes a first fall probability, the second detection result includes a second fall probability, and the fall detection result includes a target fall probability. Accordingly, fusing the first detection result and the second detection result to obtain the fall detection result includes: Obtain the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability; Based on the first weight and the second weight, the first fall probability and the second fall probability are weighted and calculated to obtain the target fall probability.
6. The fall detection method according to claim 5, characterized in that, The step of obtaining the first weight corresponding to the first fall probability and the second weight corresponding to the second fall probability includes: Obtain a preset weight dataset, wherein the preset weight dataset includes the data mapping relationship between the ambient brightness of the target object, the number of the target object and the first weight and the second weight; Obtain the ambient brightness of the target object and the number of the target objects; Based on the ambient brightness of the target object and the number of the target objects, the corresponding first weight and second weight are obtained from the preset weight dataset.
7. The fall detection method according to claim 1, characterized in that, The fall detection method also includes: Based on the fall detection results, determine whether the target object has fallen. In the event that the target object falls, an alarm message is sent to the corresponding monitoring device; In response to a confirmation message sent by the monitoring device, the event data is sent to the monitoring device, or the event data and a scene image are sent to the monitoring device, wherein the scene image is acquired by an active pixel sensor.
8. The fall detection method according to claim 7, characterized in that, The confirmation information includes first confirmation information and second confirmation information. Accordingly, the step of sending the event data to the monitoring device in response to the confirmation information sent by the monitoring device, or sending the event data and scene image to the monitoring device, includes: If the confirmation information sent by the monitoring device is the first confirmation information, then the event data related to the fall behavior is sent to the monitoring device and displayed on the monitoring device in the form of an event image; If the confirmation information sent by the monitoring device is the second confirmation information, then the active pixel sensor is used to acquire the scene image related to the fall behavior, and the event data related to the fall behavior and the scene image are sent to the monitoring device. Then the event data is displayed on the monitoring device in the form of an event image, and / or the scene image is displayed on the monitoring device.
9. A fall detection device, characterized in that, include: The acquisition module is used to acquire radar data and event data of the target object, wherein the radar data is acquired by a radar sensor and the event data is acquired by an event vision sensor. The data processing and detection module is used to detect the fall behavior of the target object based on the radar data to obtain a first detection result, and is also used to detect the fall behavior of the target object based on the event data to obtain a second detection result; The fusion module is used to fuse the first detection result and the second detection result to obtain the fall detection result.
10. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the electronic device to implement the fall detection method as described in any one of claims 1-8.
11. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to implement the fall detection method as described in any one of claims 1-8.
