A fall alarm method, device, apparatus and storage medium
By combining an event vision sensor and an active pixel sensor, the system can initially detect and confirm fall behavior, solving the problem of low accuracy in fall detection in existing technologies. This achieves high-accuracy and low-power fall detection, ensuring timely alerts while maintaining privacy and security.
Patent Information
- Application Number
- CN202510596664.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-10
AI Technical Summary
In existing technologies, fall detection has low accuracy and guardians cannot understand the specific situation of the target, resulting in a poor user experience.
An event vision sensor is used to initially detect the behavior of the target object. Feature extraction and classification models are used to determine whether it is a fall. If a fall is confirmed, the video stream from the active pixel sensor is then acquired for posture analysis to ensure the accuracy of the detection. An alarm message is sent after confirmation.
It improves the accuracy of fall detection, reduces power consumption, reduces the risk of privacy leaks, enables timely and effective reminders of target objects falling, and enhances the user experience.
Smart Images

Figure CN123072142A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a fall alarm method, apparatus, device, and storage medium. Background Technology
[0002] Fall detection refers to the use of various technologies to monitor and identify whether an individual has fallen. It is commonly used in smart homes, healthcare, and elderly care, and has significant social and medical implications. Once a fall is detected, an alarm message needs to be sent to the individual's guardian so that the guardian can respond promptly.
[0003] In existing technologies, radar sensors are typically installed in the environment where the target is located. The data output by the radar sensors is used to detect fall behavior, and an alarm message is sent to the guardian after the fall detection result is determined to be that the target has fallen.
[0004] However, this alarm method has low accuracy in determining whether the target has fallen, and the guardian cannot know the specific situation of the target through the alarm information, resulting in a poor user experience. Summary of the Invention
[0005] This invention provides a fall alarm method, device, equipment, and storage medium that reduces power consumption while improving the accuracy of fall detection of target objects, and provides timely and effective reminders of the fall behavior of target objects.
[0006] In a first aspect, embodiments of the present invention provide a fall alarm method, comprising:
[0007] Obtain event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0008] After determining that the target behavior is a fall, the video stream of the target object within a preset time period is acquired, and the behavior of the target object is detected based on the video stream to determine again whether the target behavior is a fall.
[0009] After confirming that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to the remote terminal device.
[0010] Furthermore, behavior detection is performed on the target object based on event data to determine whether the target object's target behavior is a fall, including:
[0011] Feature extraction is performed on the event data to obtain the feature data time series;
[0012] Determine whether the feature data time series matches the preset feature data time series of the fall behavior. If yes, the corresponding target behavior is the fall behavior; otherwise, the corresponding target behavior is not the fall behavior.
[0013] Furthermore, behavior detection is performed on the target object based on the video stream to further determine whether the target behavior is a fall, including:
[0014] The pose sequence of the target object is obtained from the video stream, and it is determined whether the pose in the pose sequence matches the preset pose after the fall.
[0015] After a pose in the pose sequence conforms to a preset pose, obtain the duration corresponding to the pose that conforms to the preset pose;
[0016] Compare the duration with the preset duration. If the duration is greater than or equal to the preset duration, the target behavior is a fall. If the duration is less than the preset duration, the target behavior is not a fall.
[0017] Furthermore, fall alarm methods also include:
[0018] After confirming that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to the remote monitoring and management center.
[0019] Furthermore, fall alarm methods also include:
[0020] After confirming that the target behavior is a fall, the target object is subjected to state detection based on the video stream to determine whether the video stream involves human privacy information.
[0021] After determining that the video stream involves human privacy information, the video stream is deprived of privacy to obtain the target video stream, which is then sent to the remote terminal device. A deprived audio and video communication connection is established between the on-site monitoring device corresponding to the target object and the remote terminal device.
[0022] Alternatively, after determining that the video stream involves human privacy information, the video stream can be deleted, and an audio communication connection can be established between the on-site monitoring equipment and the remote terminal equipment.
[0023] Furthermore, the video stream undergoes privacy-de-enhancing processing to obtain the target video stream, including:
[0024] The human privacy information in the video stream is blurred or replaced to obtain the target video stream.
[0025] Furthermore, fall alarm methods also include:
[0026] After confirming that the video stream does not involve human privacy information, the video stream is sent to the remote terminal device, and an audio and video communication connection is established between the on-site monitoring device and the remote terminal device.
[0027] Secondly, embodiments of the present invention also provide a fall alarm device, comprising:
[0028] The first processing module is used to acquire event data of the target object and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0029] The second processing module is used to acquire the video stream of the target object within a preset time period after determining that the target behavior is a fall behavior, and to perform behavior detection on the target object based on the video stream in order to determine again whether the target behavior is a fall behavior.
[0030] The output module is used to send alarm information and event data related to the fall to the remote terminal device after it is confirmed again that the target behavior is a fall.
[0031] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:
[0032] At least one processor; and a memory communicatively connected to said at least one processor;
[0033] The memory stores a computer program that can be executed by at least one processor to enable the electronic device to perform the fall alarm method as described in any of the first aspects.
[0034] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the fall alarm method as described in any of the first aspects.
[0035] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the fall alarm method as provided in the first aspect.
[0036] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the fall alarm device, or it may be packaged separately from the processor of the fall alarm device; this application does not impose any limitations on this.
[0037] The technical solution of this invention first acquires event data of the target object, performs behavior detection on the target object based on the event data, determines the target behavior of the target object, and achieves initial behavior detection of the target object. After determining that the target behavior of the target object is a fall, in order to further confirm whether the target behavior is indeed a fall, a video stream of the target object within a preset time period can be acquired, and behavior detection of the target object can be performed again based on the video stream to determine the target behavior again, thus achieving secondary behavior detection of the target object. Furthermore, this secondary behavior detection is performed only after the target behavior of the target object has been determined to be a fall based on the event data. Fall detection primarily uses an event vision sensor, supplemented by an active pixel sensor. The active pixel sensor is only activated for re-detection after a fall is detected based on event data from the event vision sensor. Compared to fall detection using a single event vision sensor, this improves detection accuracy and reduces power consumption, while also minimizing the risk of privacy breaches. Once a fall is confirmed, an alarm message and related event data are sent to a remote terminal device, providing timely and effective alerts for the target's fall. This allows caregivers to monitor the target's fall behavior promptly, enhancing the user experience.
[0038] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.
[0039] In this application, the name of the aforementioned fall alarm device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.
[0040] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 A flowchart of a fall alarm method provided in an embodiment of the present invention;
[0043] Figure 2 A flowchart of another fall alarm method provided in an embodiment of the present invention;
[0044] Figure 3 This is a schematic diagram of the structure of a fall alarm device provided in an embodiment of the present invention;
[0045] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0046] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0047] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0048] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.
[0049] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.
[0050] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0051] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0052] In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0053] In existing technologies, fall detection technology can be implemented in two ways: one is based on radar sensors, specifically using acquired radar data for fall detection. The advantage of this method is that it effectively protects personal privacy; however, it is easily affected by the environment and the number of people, has certain limitations, and its accuracy is relatively low. Furthermore, after confirming a fall, the information used to alert guardians lacks crucial real-time full-duplex audio and video, preventing guardians from gaining a deeper understanding of the situation. The second method is based on radar and visual sensors. Initial detection is performed using radar data acquired by the radar sensor, followed by further analysis of posture and behavior using video streams acquired by the visual sensor. The advantage of this method is higher accuracy, but it is more expensive.
[0054] Therefore, this application provides a fall detection method that improves the accuracy of fall detection while reducing costs.
[0055] The fall alarm method proposed in this application will be described in detail below with reference to the illustrations and embodiments.
[0056] Figure 1 This is a flowchart illustrating a fall alarm method provided in an embodiment of the present invention. This embodiment is applicable to situations requiring improved accuracy of fall detection and effectiveness of fall behavior alarms. The method can be executed by a fall alarm device, such as... Figure 1 As shown, the specific steps include the following:
[0057] Step 110: Obtain event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0058] Event data can be understood as the data output by an event-based vision sensor, also known as an event camera (EVS). EVS senses the environment by recording changes in light intensity. When the light intensity in a preset area changes and the change exceeds a light intensity threshold, the corresponding pixel triggers the EVS to output event data. At this point, the target object has moved, indicating a potential fall. Event data includes a timestamp, pixel position, and brightness change polarity. The timestamp indicates the time the event data was generated; the pixel position indicates the location of the pixel that generated the event data; and the brightness change polarity indicates whether the light intensity change that triggered the event data was stronger or weaker. On-chip processing of the event data output from all acquired pixels creates a black-and-white outline image of the target object's movement; this image is the event image.
[0059] Specifically, after EVS outputs event data, it can perform behavior detection on target objects based on the event data. Specifically, when performing behavior detection on target objects based on event data, firstly, feature data can be obtained by extracting features from the event data; secondly, the target behavior of the target object can be determined based on the feature data. For example, the similarity between the target behavior and each preset fall behavior can be determined based on the feature data corresponding to the target behavior and the preset feature data corresponding to each preset fall behavior. If the similarity between the feature data and any preset feature data is greater than a similarity threshold, the target behavior is determined to be a fall behavior; if the similarity between the feature data and each preset feature data is not greater than a similarity threshold, the target behavior is determined not to be a fall behavior.
[0060] Alternatively, feature vectors composed of feature data from multiple frames of event data can be input into a pre-trained classification model. The classification model can then determine the target behavior based on the feature vectors. Specifically, the classification model can determine the similarity between the target behavior and each preset fall behavior by analyzing the feature vector and the preset feature vectors corresponding to those preset fall behaviors. If the similarity between the feature vector and any preset feature vector is greater than or equal to a similarity threshold, the target behavior of the target object can be determined to be a fall behavior. If the similarity between the feature vector and each preset feature vector is not greater than the similarity threshold, the target behavior of the target object is determined not to be a fall behavior.
[0061] It should be noted that the specific value of the similarity threshold can be set according to actual needs, and no specific limit is made here.
[0062] In this embodiment of the invention, the target object's behavior is detected by using the event data of the target object, and the target behavior of the target object is determined. This method can simply and efficiently detect whether the target behavior of the target object is a falling behavior.
[0063] Step 120: After determining that the target behavior is a fall, acquire the video stream of the target object within a preset time period, and perform behavior detection on the target object based on the video stream to determine again whether the target behavior is a fall.
[0064] The video stream can be understood as the data acquired by the Active Pixel Sensor (APS). Compared to the EVS, the APS has higher imaging quality and can output an RGB color image containing more information. The EVS used to acquire event data of the target object and the APS used to acquire the video stream of the target object have the same field of view, which is a preset area.
[0065] Specifically, after determining that the target behavior of the target object is a fall based on the event data of the target object, in order to perform behavior detection on the target object again, the video stream of the target object within a preset time period can be obtained. The preset time period can be understood as a time period of preset length after the moment when the target behavior of the target object is determined to be a fall. Then, behavior detection of the target object can be performed based on the video stream.
[0066] After a target object falls, it typically maintains a relatively fixed posture for a period of time. This posture can be pre-stored as a preset posture. If the target object maintains the preset posture for a duration no less than the preset time, then the target behavior is determined to be a falling behavior.
[0067] Therefore, when performing behavior detection on a target object based on a video stream, the first step is to obtain the target object's pose sequence from the video stream. Secondly, it can be determined whether the pose in the pose sequence is a preset pose. If the pose in the pose sequence is determined to be a preset pose, the duration for which the target object maintains that pose is determined. If the duration is not less than the preset duration, the target object's target behavior can be determined to be a fall. In this embodiment of the invention, after determining that the target object's target behavior is a fall based on event data, behavior detection is performed on the target object again, further improving the accuracy of behavior detection. Furthermore, performing behavior detection only after determining that the target object's target behavior is a fall based on event data reduces power consumption and further reduces detection costs compared to continuously using the active pixel sensor for fall detection.
[0068] Step 130: After confirming again that the target behavior is a fall, send alarm information and event data related to the fall behavior to the remote terminal device.
[0069] In this context, remote terminal equipment can be understood as the terminal equipment of the guardian corresponding to the target object.
[0070] Specifically, after confirming that the target behavior of the individual is a fall, in order to remind the corresponding guardian to take timely action against the fall, an alarm message can be sent to a remote terminal device. The alarm message may include the behavioral detection results used to alert the guardian to the fall.
[0071] To ensure guardians are promptly informed of the specific circumstances surrounding a fall, event data related to the fall can be sent to a remote terminal device. This device can display alarm information and event data as images.
[0072] In this embodiment of the invention, after confirming again that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to a remote terminal device to achieve timely reminder of the target object's fall behavior.
[0073] The fall alarm method provided in this invention first acquires event data of a target object, performs behavior detection on the target object based on the event data, determines the target object's target behavior, and achieves initial behavior detection. After determining that the target object's target behavior is a fall, to further confirm whether the target behavior is indeed a fall, a video stream of the target object within a preset time period can be acquired, and behavior detection on the target object can be performed again based on the video stream to determine the target behavior again, achieving secondary behavior detection. Furthermore, this secondary behavior detection is performed only after the target object's target behavior has been determined to be a fall based on the event data. In other words, fall detection primarily uses an event vision sensor, supplemented by an active pixel sensor. The active pixel sensor is only activated for re-detection after a fall is detected based on event data from the event vision sensor. This reduces power consumption while improving the accuracy of behavior detection. Furthermore, the active pixel sensor is only triggered after the first fall is detected, resulting in minimal collection of privacy information and a low risk of privacy breaches. Once the fall is confirmed again, an alarm message and related event data are sent to a remote terminal device, providing timely and effective alerts for the target's fall behavior. This allows caregivers to monitor the target's fall behavior promptly, enhancing the user experience.
[0074] Figure 2 This is a flowchart illustrating another fall alarm method provided by an embodiment of the present invention. This embodiment is a specific modification based on the above embodiments. Figure 2 As shown, in this embodiment, the method may further include:
[0075] Step 210: Obtain event data of the target object.
[0076] Specifically, event data of the target object can be obtained based on EVS.
[0077] Step 220: Perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0078] In one implementation, step 220 may specifically include:
[0079] Feature extraction is performed on the event data to obtain a feature data time series; it is then determined whether the feature data time series matches the preset feature data time series of the fall behavior. If it does, the corresponding target behavior is the fall behavior; otherwise, the corresponding target behavior is not the fall behavior.
[0080] Falling behavior is generally a continuous behavior, and the event data of falling behavior usually corresponds to a continuous time series. Therefore, the feature data corresponding to the event data of falling behavior is relatively fixed and can be stored in advance as a preset feature data time series.
[0081] Specifically, when performing behavior detection on a target object based on event data, the event data can first be used to extract features to obtain a feature data time series. Then, the feature data time series can be compared with the preset feature data time series corresponding to the fall behavior. If the feature data time series matches the preset feature data time series corresponding to the fall behavior, then the target behavior of the target object is determined to be a fall behavior; otherwise, the target behavior of the target object is determined not to be a fall behavior.
[0082] In this embodiment of the invention, the target object's behavior is detected by using the event data of the target object, thereby determining the target object's target behavior and improving the accuracy of behavior detection.
[0083] Step 230: After determining that the target behavior is a fall, acquire the video stream of the target object within a preset time period.
[0084] Specifically, after determining that the target behavior of the target object is a fall based on the event data of the target object, in order to improve the accuracy of fall detection, the target object can be subjected to behavior detection again. That is, the video stream of the target object within a preset time period can be further obtained. The preset time can be understood as a time period of preset length after the moment when the target behavior of the target object is determined to be a fall. Then, behavior detection of the target object can be performed based on the video stream.
[0085] It should be noted that the preset time length can be set according to actual needs, and no specific limit is made here.
[0086] Step 240: Perform behavior detection on the target object based on the video stream to further determine whether the target behavior is a fall.
[0087] In one implementation, step 240 may specifically include:
[0088] The pose sequence of the target object is obtained from the video stream, and it is determined whether the pose in the pose sequence matches the preset pose after the fall behavior. After the pose in the pose sequence matches the preset pose, the duration corresponding to the pose that matches the preset pose is obtained. The duration is compared with the preset duration. If the duration is greater than or equal to the preset duration, the target behavior is a fall behavior. If the duration is less than the preset duration, the target behavior is not a fall behavior.
[0089] As described in the previous embodiment, the target object will generally maintain a relatively fixed posture for a period of time after falling.
[0090] Therefore, when performing behavior detection on a target object based on a video stream, the first step is to obtain the target object's posture sequence from the video stream. The second step is to determine whether the posture in the posture sequence is the preset posture after falling. If the posture in the posture sequence is determined to be the preset posture after falling, the duration for which the target object maintains this posture is determined. Then, the duration can be compared with the preset duration. If the duration is greater than or equal to the preset duration, the target object's target behavior can be determined to be a falling behavior.
[0091] Of course, if the pose in the pose sequence is not the preset pose, or if the pose in the pose sequence is the preset pose, but the target object maintains the pose for a duration shorter than the preset duration, then the target behavior of the target object is determined not to be a falling behavior.
[0092] Specifically, after an initial detection of a fall based on event data collected by an event vision sensor, the system performs further detection based on video streams collected by an active pixel sensor. The fall behavior is determined by combining the target object's posture and the duration of the posture. Because the postures of different actions vary and the changes in posture over time are significant, the system can effectively distinguish falls from other similar fall behaviors. This can eliminate false detections caused by actions such as lying down to pick up something, lying down to clean, or sitting down to tidy up, thus improving the accuracy of fall detection.
[0093] It should be noted that the fall detection algorithm here differs from conventional fall detection algorithms. Conventional fall detection algorithms focus on changes in the feature parameters of the fall action, mainly judging whether the behavioral feature sequence of the target object matches the preset feature sequence of the fall behavior. If they match, the target behavior is a fall. However, the fall detection algorithm here focuses on the temporal changes in posture after the fall. The target object lies on the floor and maintains the fall posture for a period of time, without immediately getting up or changing posture. Even if it gets up, the corresponding posture is different from other similar behaviors. Therefore, the fall behavior can be judged by combining the target object's posture and the duration of the posture.
[0094] The posture after falling or the posture after getting up can be obtained by training based on a large amount of sample data.
[0095] In this embodiment of the invention, after determining that the target behavior of the target object is a fall based on event data, the target object is re-detected by the video stream of the target object within a preset time period, and the target behavior of the target object is determined again, which further improves the accuracy of behavior detection. Moreover, the behavior detection of the target object is only performed after the target behavior of the target object is determined to be a fall based on event data. Compared with continuously turning on the active pixel sensor for fall detection, this reduces power consumption and detection costs.
[0096] Step 250: After confirming again that the target behavior is a fall, send alarm information and event data related to the fall behavior to the remote terminal device.
[0097] The alarm information may include the behavior detection results used to alert the target object to a fall. For example, the behavior detection result may be "the target object has fallen".
[0098] As described in the previous embodiment, after confirming again that the target behavior of the target object is a fall, in order to remind the corresponding guardian of the target object to take timely action against the fall, alarm information and event data related to the fall can be sent to a remote terminal device. The alarm information may include the behavior detection results used to remind the guardian of the fall, and the event data related to the fall may include the specific circumstances of the fall. The remote terminal device can display the alarm information and can also display the event data in the form of event images.
[0099] In one implementation, after reaffirming that the target behavior is a falling behavior, the method further includes:
[0100] Send alarm information and event data related to fall behavior to the remote monitoring and management center.
[0101] The remote monitoring and management center can be understood as the management system of the target nursing home or rescue organization.
[0102] After confirming that the target behavior is a fall and sending an alarm message to the remote terminal device, the guardian may not be able to respond in a timely manner. To prevent the target from being in a dangerous situation due to the guardian's failure to respond promptly, after confirming that the target behavior is a fall again, an alarm message and event data related to the fall behavior can be sent to the remote monitoring and management center.
[0103] In one implementation, after reaffirming that the target behavior is a falling behavior, the method further includes:
[0104] The system performs state detection on the target object based on the video stream to determine whether the video stream involves human privacy information. If it is determined that the video stream involves human privacy information, it undergoes privacy-de-sampling processing to obtain the target video stream, which is then sent to a remote terminal device. A privacy-de-sampling audio and video communication connection is established between the on-site monitoring device corresponding to the target object and the remote terminal device. Alternatively, if it is determined that the video stream involves human privacy information, it is deleted, and an audio communication connection is established between the on-site monitoring device and the remote terminal device. If it is determined that the video stream does not involve human privacy information, it is sent to the remote terminal device, and an audio and video communication connection is established between the on-site monitoring device and the remote terminal device.
[0105] Furthermore, the video stream undergoes privacy-de-enhancing processing to obtain the target video stream, including:
[0106] The human privacy information in the video stream is blurred or replaced to obtain the target video stream.
[0107] Human privacy information includes facial information and information about private parts. In order to improve user privacy and security, if the obtained video stream contains human privacy information, on the one hand, the video stream can be de-privacy processed before being sent to a remote terminal device, and on the other hand, the privacy-related content in the video stream can be deleted before being sent to a remote terminal device.
[0108] Specifically, after obtaining the video stream of the target object, the status of the video stream can be monitored frame by frame to determine whether the video stream contains facial information or private part information of the target object. If the video stream contains facial information and / or private part information of the target object, the video stream needs to be processed.
[0109] On the one hand, in order to facilitate the guardian to view the specific circumstances of the target's fall, it is necessary to send the video stream to a remote terminal device. At this time, the video stream can be de-privacy processed, specifically by blurring or replacing the human privacy information in the video stream to obtain the target video stream. The target video stream is then sent to the remote terminal device, and a de-privacy audio and video communication connection is established between the on-site monitoring device corresponding to the target and the remote terminal device. Based on this communication connection, when the video stream in subsequent real-time communication involves the human privacy information of the target, the same de-privacy processing method and the same video transmission processing method are used.
[0110] On the other hand, in order to improve the privacy and security of the target, it is necessary to delete relevant frames involving human privacy information in the video stream and establish a video communication connection between the on-site monitoring equipment and the remote terminal equipment. Based on this communication connection, the same frame deletion process will be used when the real-time video stream of subsequent communications involves human privacy information of the target.
[0111] Of course, in order to further enhance the privacy and security of the target, after determining that the video stream involves human privacy information, the video stream can be deleted, and an audio communication connection can be established between the on-site monitoring equipment and the remote terminal equipment. The audio communication based on this audio communication connection will hardly involve the target's privacy information, and there is no risk of privacy leakage.
[0112] Furthermore, if the video stream does not contain facial information or private parts information of the target object, the video stream can be directly sent to a remote terminal device, and then an audio and video communication connection can be established between the on-site monitoring device corresponding to the target object and the remote terminal device. The audio and video communication based on this audio and video communication connection will hardly involve the target object's private information, and there is no risk of privacy leakage.
[0113] In this embodiment of the invention, after determining that the target behavior is a fall, an alarm message is sent to a remote terminal device to promptly alert the guardian to the fall, facilitating timely intervention by the guardian. Sending the alarm message to a remote monitoring and management center provides a backup solution, preventing the target from being in danger due to the guardian's failure to address the fall in a timely manner. Furthermore, by sending fall-related data and audio / video communication to the remote terminal device, a clear display of the specific circumstances of the fall is achieved, allowing the guardian to clearly understand the target's situation. Simultaneously, privacy-protecting processing is performed on video streams or real-time video communication streams containing private information, effectively reducing the risk of privacy leaks.
[0114] The fall alarm method provided in this invention first acquires event data of a target object, performs behavior detection on the target object based on the event data, determines the target behavior of the target object, and achieves initial behavior detection of the target object. After determining that the target behavior of the target object is a fall, in order to further confirm whether the target behavior is indeed a fall, a video stream of the target object within a preset time period can be acquired, and behavior detection of the target object can be performed again based on the video stream to determine the target behavior again, achieving secondary behavior detection of the target object. Moreover, the secondary behavior detection of the target object is only performed after the target behavior of the target object has been determined to be a fall based on the event data. In other words, fall detection is based on visual transmission of events. The system primarily uses a visual sensor, supplemented by an active pixel sensor. The active pixel sensor is only activated for re-detection after a fall is detected by the event data from the event visual sensor. This reduces power consumption while improving the accuracy of behavior detection. Furthermore, the active pixel sensor only triggers after the first fall is detected, resulting in minimal collection of privacy information and a low risk of privacy breaches. Once the fall is confirmed again, an alarm message, fall-related event data, and a video stream without personal privacy information are sent to a remote terminal device. This provides timely and effective alerts for the target's fall, allowing caregivers to monitor the fall behavior while ensuring the target's privacy, thus enhancing the user experience.
[0115] Furthermore, after confirming that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to the remote monitoring and management center to enable timely handling of the target's fall behavior and prevent the target from being in a dangerous situation due to the guardian's failure to respond to the target's fall behavior in a timely manner.
[0116] Figure 3 This is a schematic diagram of a fall alarm device provided in an embodiment of the present invention. The device can be applied to situations where it is necessary to improve the accuracy of fall detection and the effectiveness of fall behavior alarm. The device can be implemented by software and / or hardware and is generally integrated into electronic devices, such as fall detection devices.
[0117] like Figure 3 As shown, the device includes:
[0118] The first processing module 310 is used to acquire event data of the target object and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0119] The second processing module 320 is used to acquire the video stream of the target object within a preset time period after determining that the target behavior is a fall behavior, and to perform behavior detection on the target object based on the video stream in order to determine again whether the target behavior is a fall behavior.
[0120] The output module 330 is used to send alarm information and event data related to the fall behavior to the remote terminal device after it is determined again that the target behavior is a fall behavior.
[0121] The fall alarm device provided in this embodiment first acquires event data of the target object, performs behavior detection on the target object based on the event data, determines the target object's target behavior, and achieves initial behavior detection of the target object. After determining that the target object's target behavior is a fall, in order to further confirm whether the target behavior is indeed a fall, it can acquire a video stream of the target object within a preset time period, and perform behavior detection on the target object again based on the video stream to determine the target behavior again, achieving secondary behavior detection of the target object. Furthermore, this secondary behavior detection is only performed after the target object's target behavior has been determined to be a fall based on the event data. The fall detection system primarily uses an event vision sensor, supplemented by an active pixel sensor. The active pixel sensor only activates for re-detection after a fall has been detected based on event data from the event vision sensor. Compared to fall detection using a single event vision sensor, this improves detection accuracy and reduces power consumption and privacy risks. Once a fall is confirmed, an alarm message and related event data are sent to a remote terminal device, providing timely and effective alerts for potential falls. This allows caregivers to monitor their child's fall behavior promptly, enhancing the user experience.
[0122] Based on the above embodiments, the first processing module 310 is specifically used for:
[0123] Acquire event data of the target object; extract features from the event data to obtain a feature data time series; determine whether the feature data time series matches the preset feature data time series of the fall behavior. If yes, the corresponding target behavior is a fall behavior; otherwise, the corresponding target behavior is not a fall behavior.
[0124] Based on the above embodiments, the second processing module 320 is specifically used for:
[0125] After determining that the target behavior is a fall, acquire the video stream of the target object within a preset time period; acquire the posture sequence of the target object based on the video stream, and determine whether the posture in the posture sequence matches the preset posture after the fall; after the posture in the posture sequence matches the preset posture, acquire the duration corresponding to the posture that matches the preset posture; compare the duration with the preset duration. If the duration is greater than or equal to the preset duration, the target behavior is a fall; if the duration is less than the preset duration, the target behavior is not a fall.
[0126] Based on the above embodiments, the device further includes:
[0127] The sending module is used to send alarm information and event data related to the fall to the remote monitoring and management center after it is confirmed again that the target behavior is a fall.
[0128] Based on the above embodiments, the second processing module 320 is further configured to:
[0129] After confirming that the target behavior is a fall, the system performs state detection on the target object based on the video stream to determine if the video stream involves human privacy information. If it is determined that the video stream involves human privacy information, the video stream is de-privacy processed to obtain the target video stream, which is then sent to the remote terminal device, and a de-privacy audio and video communication connection is established between the on-site monitoring device corresponding to the target object and the remote terminal device. Alternatively, if it is determined that the video stream involves human privacy information, the video stream is deleted, and an audio communication connection is established between the on-site monitoring device and the remote terminal device. If it is determined that the video stream does not involve human privacy information, the video stream is sent to the remote terminal device, and an audio and video communication connection is established between the on-site monitoring device and the remote terminal device.
[0130] In one implementation, privacy-de-enhancing processing is performed on the video stream to obtain the target video stream, including:
[0131] The human privacy information in the video stream is blurred or replaced to obtain the target video stream.
[0132] The fall alarm device provided in the embodiments of the present invention can execute the fall alarm method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the fall alarm method.
[0133] It is worth noting that in the above embodiments of the fall alarm device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.
[0134] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0135] like Figure 4As shown, electronic device 4 is represented in the form of a general-purpose computing electronic device. The components of electronic device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0136] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0137] Electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 4, including volatile and non-volatile media, removable and non-removable media.
[0138] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0139] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.
[0140] Electronic device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 4, and / or with any device that enables electronic device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with electronic device 4 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0141] Processing unit 16 executes various functional applications and page displays by running programs stored in system memory 28, such as implementing the fall alarm method provided in this embodiment of the invention, which includes:
[0142] Obtain event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0143] After determining that the target behavior is a fall, the video stream of the target object within a preset time period is acquired, and the behavior of the target object is detected based on the video stream to determine again whether the target behavior is a fall.
[0144] After confirming that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to the remote terminal device.
[0145] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the fall alarm method provided in any embodiment of the present invention.
[0146] This invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements, for example, the fall alarm method provided in this invention, which includes:
[0147] Obtain event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall.
[0148] After determining that the target behavior is a fall, the video stream of the target object within a preset time period is acquired, and the behavior of the target object is detected based on the video stream to determine again whether the target behavior is a fall.
[0149] After confirming that the target behavior is a fall, an alarm message and event data related to the fall behavior are sent to the remote terminal device.
[0150] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0151] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0152] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0153] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0154] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0155] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with the relevant provisions of national laws and regulations.
[0156] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.
Claims
1. A fall alarm method, characterized in that, include: Obtain event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall. After determining that the target behavior is the fall behavior, a video stream of the target object within a preset time period is obtained, and behavior detection of the target object is performed based on the video stream to determine again whether the target behavior is the fall behavior; After confirming again that the target behavior is the fall, an alarm message and the event data related to the fall are sent to the remote terminal device.
2. The fall alarm method according to claim 1, characterized in that, The step of performing behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall includes: Feature extraction is performed on the event data to obtain a feature data time series; Determine whether the time series of the feature data matches the preset time series of the fall behavior. If yes, the corresponding target behavior is the fall behavior; otherwise, the corresponding target behavior is not the fall behavior.
3. The fall alarm method according to claim 1, characterized in that, The step of performing behavior detection on the target object based on the video stream to further determine whether the target behavior is the fall behavior includes: The pose sequence of the target object is obtained from the video stream, and it is determined whether the pose in the pose sequence matches the preset pose after the fall. After a pose in the pose sequence conforms to the preset pose, the duration corresponding to the pose that conforms to the preset pose is obtained; The duration is compared with a preset duration. If the duration is greater than or equal to the preset duration, the target behavior is the fall behavior. If the duration is less than the preset duration, the target behavior is not the fall behavior.
4. The fall alarm method according to claim 1, characterized in that, The fall alarm method also includes: After confirming again that the target behavior is the fall, the alarm information and the event data related to the fall behavior are sent to the remote monitoring and management center.
5. The fall alarm method according to claim 1, characterized in that, The fall alarm method also includes: After confirming that the target behavior is the fall behavior again, the target object is subjected to state detection based on the video stream to determine whether the video stream involves human privacy information. After determining that the video stream involves the human privacy information, the video stream is deprived of privacy to obtain the target video stream, the target video stream is sent to the remote terminal device, and a deprived audio and video communication connection is established between the on-site monitoring device corresponding to the target object and the remote terminal device. Alternatively, after determining that the video stream involves the human privacy information, the video stream can be deleted, and an audio communication connection can be established between the on-site monitoring equipment and the remote terminal equipment.
6. The fall alarm method according to claim 5, characterized in that, The process of performing privacy-de-sensitizing processing on the video stream to obtain the target video stream includes: The human privacy information in the video stream is blurred or replaced to obtain the target video stream.
7. The fall alarm method according to claim 5, characterized in that, The fall alarm method also includes: After determining that the video stream does not involve the human privacy information, the video stream is sent to the remote terminal device, and an audio and video communication connection is established between the on-site monitoring device and the remote terminal device.
8. A fall alarm device, characterized in that, include: The first processing module is used to acquire event data of the target object, and perform behavior detection on the target object based on the event data to determine whether the target object's target behavior is a fall. The second processing module is used to, after determining that the target behavior is the fall behavior, acquire the video stream of the target object within a preset time period, and perform behavior detection on the target object based on the video stream, so as to determine again whether the target behavior is the fall behavior; The output module is used to send alarm information and event data related to the fall to a remote terminal device after it is determined again that the target behavior is the fall behavior.
9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the electronic device to perform the fall alarm method as described in any one of claims 1-7.
10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the fall alarm method as described in any one of claims 1-7.
