Fall detection method and device based on depth anomaly perception and kinematic rule verification, electronic equipment and storage medium

Abstract

The application provides a fall detection method and device based on deep abnormality perception and kinematic rule verification, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a target video stream to be detected; detecting a human target in the target video stream to obtain human basic feature information; determining a deep abnormal change feature based on the human basic feature information, and constructing a multi-dimensional fusion feature according to the deep abnormal change feature; performing fall detection on the human target based on the multi-dimensional fusion feature to obtain a preliminary fall detection result; verifying the preliminary fall detection result, and generating a target fall detection result according to the verification result, so as to trigger a warning mechanism based on the target fall detection result. The application can effectively improve the reliability of fall detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a fall detection method, device, electronic device, and storage medium based on depth anomaly perception and kinematic rule verification. Background Technology

[0002] As society ages, the automatic identification and timely warning of falls among the elderly has become an important research direction in the field of intelligent care. In scenarios such as home-based elderly care, medical care, elderly care institutions, and public safety management, if falls are not detected in time, it may lead to delayed treatment, secondary injuries, or even endanger lives. Existing fall detection technologies mainly fall into three categories: those based on wearable sensors, those based on environmental sensors, and those based on computer vision. Methods based on wearable sensors (such as accelerometers, gyroscopes, and barometers) detect falls by collecting data on peak acceleration, changes in angular velocity, or changes in posture from human motion data. While these methods are not sensitive to light or occlusion, their effectiveness heavily relies on continuous wear by the user. Elderly individuals often forget to wear the device, resist wearing it, find charging inconvenient, or find it inconvenient to wear during bathing or sleeping, thus limiting their actual continuous monitoring capabilities. Methods based on environmental sensors (such as millimeter-wave radar, infrared sensors, ultrasonic sensors, and piezoelectric flooring) do not require users to wear devices, but they are easily affected by factors such as furniture obstruction, pet activity, multipath reflection, installation location, and modifications to the floor structure. Furthermore, the high cost of modifying piezoelectric flooring and similar solutions makes large-scale deployment in ordinary homes and existing hospital wards difficult. Fall detection methods based on computer vision, using depth anomaly perception and kinematic rule verification, acquire human images via cameras and determine whether a fall has occurred through human target detection, key point extraction, pose estimation, and action recognition models. These methods offer advantages such as being non-contact, flexible in deployment, and suitable for continuous monitoring, making them a promising technological approach. However, existing fall detection technologies still suffer from high false negative rates. Summary of the Invention

[0003] This invention provides a fall detection method, device, electronic device, and storage medium based on depth anomaly perception and kinematic rule verification, in order to solve the technical problem that existing fall detection technologies still have a high rate of missed fall detections.

[0004] This invention provides a fall detection method based on depth anomaly perception and kinematic rule verification, comprising the following steps: Acquire the target video stream to be detected; Human targets in the target video stream are detected to obtain basic human feature information; Based on the aforementioned basic human body features, depth anomaly change features are determined, and multi-dimensional fusion features are constructed based on these depth anomaly change features. Based on the multi-dimensional fusion features, fall detection is performed on the human target to obtain preliminary fall detection results; The preliminary fall detection results are verified, and a target fall detection result is generated based on the verification results, so as to trigger an early warning mechanism based on the target fall detection result.

[0005] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification is provided, wherein the basic human body feature information includes at least bounding box temporal information and key point temporal information; The process of determining depth anomaly variation features based on the aforementioned basic human body feature information, and constructing multi-dimensional fusion features based on these depth anomaly variation features, includes: Based on the bounding box time-series information, the bounding box area change rate characteristics and the bounding box aspect ratio characteristics are determined. Depth anomaly change characteristics are determined based on the bounding box area change rate characteristics; Obtain the key point trajectory features from the key point time-series information; Multidimensional fusion features are constructed by fusing the bounding box area change rate feature, the bounding box aspect ratio feature, the depth anomaly change feature, and the key point trajectory feature.

[0006] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification is provided, wherein determining the bounding box area change rate feature based on the bounding box temporal information includes: Based on the bounding box timing information, calculate the bounding box area of ​​the human target in the current frame, and obtain the historical bounding box area within a preset historical window length before the current frame. Calculate the historical average area based on the historical bounding box area; Calculate the rate of change of the bounding box area based on the bounding box area and the historical average area; Based on the rate of change of the bounding box area, the characteristics of depth anomaly changes are determined.

[0007] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification is provided, wherein determining depth anomaly change features based on the bounding box area change rate includes: Obtain the preset area shrinkage threshold and the preset area expansion threshold; If the rate of change of the bounding box area is less than the preset area shrinkage threshold or greater than the preset area expansion threshold, then the depth anomaly change feature is set to detect an anomaly. If the rate of change of the bounding box area is greater than or equal to the preset area shrinkage threshold and less than or equal to the preset area expansion threshold, then the depth anomaly change feature is set to no anomaly detected.

[0008] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification is provided, wherein the fall detection of the human target is performed based on the multi-dimensional fusion features to obtain preliminary fall detection results, including: Obtain the preset probability threshold; The multidimensional fusion features are input into a pre-trained spatiotemporal action recognition model to obtain the predicted fall probability; Based on the preset probability threshold and the predicted fall probability, the initial fall detection result is determined.

[0009] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification includes verifying the preliminary fall detection result and generating a target fall detection result based on the verification result, comprising: When the preliminary fall detection results indicate that the human target is at risk of falling, determine whether the conditions for exemption from dangerous features are met. When the hazard feature exemption conditions are met, the preliminary fall detection result is taken as the target detection result; When the hazard feature exemption condition is not met, and at least one of the chassis stability rejection condition or the lower limb suspension rejection condition is met, the target fall detection result is set to "not fallen". When the hazard feature exemption condition is not met, and the chassis stability rejection condition and the lower limb suspension rejection condition are not met at the same time, the preliminary fall detection result is taken as the target fall detection result.

[0010] According to the present invention, a fall detection method based on depth anomaly perception and kinematic rule verification is provided. The hazard feature exemption condition is: the aspect ratio of the bounding box of the human target is greater than or equal to a preset aspect ratio threshold, or the depth anomaly change feature is detected as an anomaly; the chassis stability rejection condition is: the longitudinal displacement of the hip key point of the human target is less than a preset longitudinal displacement threshold; the lower limb suspension rejection condition is: the relative height ratio of the lower limb key point to the hip key point of the human target is greater than a preset suspension ratio threshold.

[0011] The present invention also provides a fall detection device based on depth anomaly perception and kinematic rule verification, comprising the following modules: The acquisition module is used to acquire the target video stream to be detected; The detection module is used to detect human targets in the target video stream and obtain basic human feature information; The construction module is used to determine the depth anomaly change features based on the human body basic feature information, and to construct multi-dimensional fusion features based on the depth anomaly change features; The detection module is also used to perform fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; The verification module is used to verify the preliminary fall detection results and generate a target fall detection result based on the verification results, so as to trigger an early warning mechanism based on the target fall detection result.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fall detection method based on depth anomaly perception and kinematic rule verification as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fall detection method based on depth anomaly perception and kinematic rule verification as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the fall detection method based on depth anomaly perception and kinematic rule verification as described above.

[0015] This invention provides a fall detection method, device, electronic device, and storage medium based on depth anomaly perception and kinematic rule verification. The method involves: acquiring a target video stream to be detected; detecting human targets in the target video stream to obtain basic human feature information; determining depth anomaly change features based on the basic human feature information and constructing multi-dimensional fusion features based on the depth anomaly change features; performing fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; verifying the preliminary fall detection results and generating a target fall detection result based on the verification results, thereby triggering an early warning mechanism based on the target fall detection result. This invention addresses the high false negative rate of existing fall detection technologies. Compared to existing technologies, this invention extracts abnormal depth change features, enabling the detection of abnormal human movements along the camera's optical axis without the need for additional depth sensors. This effectively reduces the false negative rate of falls such as forward and backward leans. Furthermore, based on these features, a multi-dimensional fusion feature is constructed for preliminary detection, and a verification mechanism is used to confirm the preliminary results. This ensures that genuine fall events are detected promptly while reducing false alarms caused by fall-like actions, thus achieving more reliable and balanced fall detection performance in low-cost monocular vision deployment scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts of the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention.

[0018] Figure 2 This is a detailed flowchart illustrating the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the pseudo-depth sensing geometry principle of the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention, which describes anomaly changes in the depth direction.

[0020] Figure 4 This is a waveform diagram showing the temporal evolution of fall features in the fall detection method based on depth anomaly perception and kinematic rule verification provided by this invention.

[0021] Figure 5 This is the second flowchart of the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention.

[0022] Figure 6 This is a schematic diagram of a multi-level fall detection judgment method based on depth anomaly perception and kinematic rule verification provided by the present invention.

[0023] Figure 7 This is a schematic diagram of the fall detection device based on depth anomaly perception and kinematic rule verification provided by the present invention.

[0024] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] The following is combined with Figure 1 and Figure 5This invention describes a fall detection method based on depth anomaly perception and kinematic rule verification. This method is applicable to any fall detection. The execution subject of this method can be an electronic device or a fall detection device based on depth anomaly perception and kinematic rule verification installed in the electronic device. The fall detection device based on depth anomaly perception and kinematic rule verification can be implemented by software, hardware, or a combination of both.

[0027] Figure 1 This is one of the flowcharts of the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 101: Obtain the target video stream to be detected; It should be noted that the target video stream refers to a continuous sequence of images (i.e., continuous video frames) captured in real time or pre-recorded by a monocular 2D camera, which serves as the raw input for the fall detection algorithm. This video stream contains the human movement scene to be detected, such as surveillance footage from environments like home-based elderly care, medical institution wards, or nursing homes. Specifically, the target video stream can be obtained by real-time reading of video files from a USB camera, a network camera (IP camera), or a local video file.

[0028] Step 102: Detect human targets in the target video stream to obtain basic human feature information; It should be noted that basic human body feature information includes at least bounding box temporal information and keypoint temporal information. Bounding box temporal information refers to the sequence of coordinates of the two-dimensional bounding box (top-left corner x and y coordinates, width, and height) of the human target in consecutive video frames, used to describe the position and scale changes of the human body in the image plane. Keypoint temporal information refers to the sequence of coordinates and confidence scores of human joints (such as the head, shoulders, elbows, wrists, hips, knees, and ankles) in consecutive frames, used to describe the fine movements of human posture. This feature information is the raw data for subsequent depth variation analysis and kinematic rule verification.

[0029] In the specific implementation, a deep learning-based human detector (such as YOLO, SSD, or Faster R-CNN) is first used to extract the human bounding boxes, obtaining the top-left corner coordinates (x, y), width w, and height h of each human target. That is, the i-th human target in the t-th frame can be represented as: ,in, and These represent the coordinates of the top left corner of the human body bounding box. Indicates the width of the human body bounding box. The height of the human bounding box is represented, and then human targets in consecutive frames are tracked. Target numbers are assigned to the same human target to obtain the temporal sequence of the bounding boxes of the same human target. Simultaneously, for each image region within a bounding box, a human pose estimation network (such as OpenPose, HRNet, or MediaPipe) is used to detect K keypoints. Each keypoint contains an x-coordinate u, a y-coordinate v, and a confidence score c. ,in, Represents the x-coordinate of the key point. Represents the ordinate of the key point. This represents the confidence level of key points, ultimately forming a set of human body key point coordinates. Meanwhile, multi-target tracking algorithms (such as SORT and DeepSORT) are used to assign unique numbers to the same human body in different frames, thereby forming bounding box time series and key point time series.

[0030] Step 103: Based on the basic human body feature information, determine the depth anomaly change features, and construct multi-dimensional fusion features based on the depth anomaly change features; It should be noted that depth anomaly change features refer to using abrupt changes in the bounding box area over time to characterize the forward or backward movement of a human body along the camera's optical axis (Z-axis). Since monocular cameras lack depth information, this solution simulates depth changes through the area change rate: the area expands when the human body approaches the camera and contracts when it moves away, thus indirectly perceiving abnormal movement in the depth direction. Multidimensional fusion features involve concatenating multiple features, such as the bounding box area change rate, bounding box aspect ratio, depth anomaly change markers, and keypoint trajectories, along the temporal dimension to form a high-dimensional feature tensor for subsequent fall detection and recognition.

[0031] Step 104: Based on the multi-dimensional fusion features, perform fall detection on the human target to obtain preliminary fall detection results; It's important to note that fall detection refers to using classification or regression models to determine whether a person's current action constitutes a fall. The initial fall detection result is the model output before rule validation, such as a binary label (fall / not fall) or a probability value. This next step utilizes a deep learning model to automatically learn the fall movement pattern from fused features.

[0032] In practical implementation, a pre-trained spatiotemporal action recognition model can analyze multi-dimensional fusion features to capture the joint changes in bounding box shape, keypoint trajectory, and depth direction of a fall, thus providing a preliminary judgment. This preliminary result has high sensitivity but may be affected by fall-like actions, leading to false alarms. Specifically, the multi-dimensional fusion features are input into a spatiotemporal action recognition model composed of a convolutional neural network and a long short-term memory network. The model outputs a fall probability value between 0 and 1. This probability value is compared with a preset probability threshold (e.g., 0.7). If it is greater than the threshold, a fall is preliminarily judged; otherwise, it is judged as not a fall.

[0033] Step 105: Verify the preliminary fall detection results and generate target fall detection results based on the verification results, so as to trigger an early warning mechanism based on the target fall detection results.

[0034] It should be noted that verification refers to a secondary review of the preliminary results based on human kinematics rules, used to eliminate false alarms caused by fall-related actions such as sitting down or lifting a leg. The target fall detection result is the final judgment after verification, and has higher accuracy. The early warning mechanism refers to the alarm signal issued by the system after confirming a fall event, such as push notifications or sound alerts.

[0035] In practical implementation, kinematic rules can be introduced to simulate the basic laws of human physical movement, logically constraining the output of the deep learning model and thus correcting potential misjudgments. The verification process can significantly reduce false alarms without lowering the actual fall recall rate, making the system more reliable in real-world deployments.

[0036] In practice, once the system confirms a fall event, it generates an alarm data packet containing a timestamp, a screenshot of the fall, and a video clip. The alarm information is then pushed to a preset software client UI or WeChat mini-program via a network interface, triggering a screen highlight pop-up, vibration, or message notification. Test and accident video archives are also generated synchronously on the local computer or in the cloud backend.

[0037] In specific implementations, such as Figure 2The fall detection process is as follows: First, the input video stream is preprocessed, and basic human features (including bounding box temporal information and key point temporal information) are extracted. Then, pseudo-depth features are calculated, i.e., depth anomaly change features are constructed based on the bounding box area change rate. Next, the multi-dimensional fusion features are input into the neural network spatiotemporal action recognition model to obtain a preliminary fall prediction result. If the preliminary prediction is not a fall, the current monitoring cycle ends and the next frame is processed. If the preliminary prediction is a fall, the final motion logic judgment module is entered. This module executes three rules in parallel: chassis stability rejection, lower limb suspension rejection, and hazard feature exemption. A comprehensive logical judgment is performed based on the outputs of these three rules, and the final fall detection result is output, triggering an alarm push.

[0038] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention involves: acquiring a target video stream to be detected; detecting human targets in the target video stream to obtain basic human feature information; determining depth anomaly change features based on the basic human feature information and constructing multi-dimensional fusion features based on the depth anomaly change features; performing fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; verifying the preliminary fall detection results and generating a target fall detection result based on the verification results, thereby triggering an early warning mechanism based on the target fall detection result. This invention addresses the high false negative rate of existing fall detection technologies. Compared to existing technologies, this invention extracts abnormal depth change features, enabling the detection of abnormal human movements along the camera's optical axis without the need for additional depth sensors. This effectively reduces the false negative rate of falls such as forward and backward leans. Furthermore, based on these features, a multi-dimensional fusion feature is constructed for preliminary detection, and a verification mechanism is used to confirm the preliminary results. This ensures that genuine fall events are detected promptly while reducing false alarms caused by fall-like actions, thus achieving more reliable and balanced fall detection performance in low-cost monocular vision deployment scenarios.

[0039] Based on any of the above embodiments, the basic human body feature information includes at least bounding box temporal information and key point temporal information; The process of determining depth anomaly variation features based on the aforementioned basic human body feature information, and constructing multi-dimensional fusion features based on these depth anomaly variation features, includes: Based on the bounding box time-series information, the bounding box area change rate characteristics and the bounding box aspect ratio characteristics are determined. Depth anomaly change characteristics are determined based on the bounding box area change rate characteristics; Obtain the key point trajectory features from the key point time-series information; Multidimensional fusion features are constructed by fusing the bounding box area change rate feature, the bounding box aspect ratio feature, the depth anomaly change feature, and the key point trajectory feature.

[0040] It should be noted that the bounding box area change rate feature is the ratio of the current frame bounding box area to the historical average area, used to characterize the scale change of the human body's projection in the image; the bounding box aspect ratio feature is the ratio of the bounding box width to its height, used to describe the degree of the human body's rollover or upright / lying position.

[0041] Understandably, depth anomaly change features are a binary indicator that shows whether the human body has undergone rapid displacement along the camera's optical axis (such as falling forward or backward). Keypoint trajectory features refer to the coordinate sequence of various joints of the human body in multiple consecutive frames (which may be the original coordinates or normalized coordinates), used to describe the fine-grained process of human posture changes.

[0042] In the specific implementation, the formula for calculating the aspect ratio feature of the bounding box is as follows: In the formula, Indicates the first Personal physical goals in The aspect ratio of the frame's bounding box; Indicates the width of the human body bounding box. This indicates the height of the human body bounding box.

[0043] In its implementation, during the feature extraction stage of the fall detection method based on depth anomaly perception and kinematic rule verification, after detecting human key points from video frames, the system first performs normalization and occlusion tolerance processing to ensure that even when some key points are occluded or have low confidence, stable and scale-consistent key point features can still be obtained for subsequent spatiotemporal action recognition models and kinematic rule verification. Specifically, the system first determines whether the confidence of the hip key point or pelvic key point is greater than or equal to a preset confidence threshold. If so, the coordinates of the hip key point are used as the anchor point of the local coordinate system. If the conditions are not met, i.e., the primary reference point is severely occluded or the confidence level is too low, then all other keypoints in the current frame that meet the confidence level are collected, and the geometric centers of these keypoints are calculated as temporary anchor points. Next, the system calculates the Euclidean distance between the neck keypoints and the anchor points as the torso length, which is used for scale normalization. For each detected human keypoint, the system subtracts the anchor point coordinates from its original coordinates and then divides by the torso length to obtain the normalized keypoint coordinates. After the above processing, the coordinates of all key points are transformed into a local coordinate system centered on the human body, eliminating scale differences caused by distance from the camera. This mechanism ensures that the extracted key point features remain stable and consistent even with partial occlusion or pose distortion, thereby improving the robustness of subsequent fall detection models.

[0044] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention integrates multi-dimensional temporal features such as bounding box area change rate, aspect ratio, depth anomaly markers, and keypoint trajectories. Specifically, the bounding box area change rate can perceive depth movements such as forward and backward leaning, the aspect ratio characterizes the tendency to roll over, the depth anomaly markers provide discrete risk warnings, and the keypoint trajectories finely describe posture changes. This multi-feature fusion allows the model to utilize both global contour and local skeletal information simultaneously, significantly improving the accuracy and robustness of fall action recognition and providing rich and reliable input for subsequent intelligent judgment.

[0045] Based on any of the above embodiments, determining the bounding box area change rate feature based on the bounding box time series information includes: Based on the bounding box timing information, calculate the bounding box area of ​​the human target in the current frame, and obtain the historical bounding box area within a preset historical window length before the current frame. Calculate the historical average area based on the historical bounding box area; Calculate the rate of change of the bounding box area based on the bounding box area and the historical average area; Based on the rate of change of the bounding box area, the characteristics of depth anomaly changes are determined.

[0046] In the specific implementation, the formula for calculating the bounding box area change rate feature is as follows: In the formula, Indicates the first Personal physical goals in Rate of change of the bounding box area of ​​the frame; This represents the bounding box area of ​​the i-th human target in frame t (i.e., the bounding box area of ​​the current frame). This represents the historical average area.

[0047] In the specific implementation, the formula for calculating the area of ​​the bounding box in the current frame is as follows: In the formula, Indicates the width of the human body bounding box. This indicates the height of the human body bounding box.

[0048] In practice, the formula for calculating the historical average area is as follows: In the formula, The preset history window length is set (i.e., the area of ​​the historical bounding box of the human target in the previous M frames is used as the history reference window).

[0049] It is understandable that in the pinhole imaging model of a typical monocular camera, the actual height of the human body is assumed to be... The camera focal length is The distance between the human body and the camera is Then the height h and width of the human body in the image They can be represented as: Among them, the area of ​​the image bounding box It can be approximated as: Understandably, in a short period of time, the actual size of the human body remains essentially unchanged. At this time, the area A of the bounding box is approximately inversely proportional to the square of the distance Z: Therefore, when a person falls forward or backward, Sudden changes can lead to drastic changes in the bounding box area; this embodiment utilizes the rate of change of the bounding box area, i.e., to determine... Axial anomalies can be detected without additional depth sensors to capture fall characteristics in the longitudinal direction.

[0050] In specific implementations, such as Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the geometric principle of depth anomaly detection provided in an embodiment of the present invention. A three-dimensional coordinate system is established with the optical center of the camera as the origin, where the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the direction of the camera's optical axis (i.e., the depth direction). The area A of the bounding box of the human body in the image plane is approximately inversely proportional to the square of the distance Z from the human body to the camera, i.e., A ∝ 1 / Z 2 When a human body falls forward or backward, the Z-value changes abruptly, causing a sharp change in the bounding box area A. By analyzing the rate of change of the bounding box area, abnormal movement in the Z-axis direction can be detected without the aid of a depth sensor, providing a theoretical basis for depth fall detection.

[0051] In specific implementations, such as Figure 4 As shown, Figure 4This table presents a time-series evolution waveform data of fall characteristics provided in this embodiment of the invention. The table uses time t as the horizontal axis and records the numerical changes in the bounding box area change rate R, the center of gravity height (i.e., the vertical coordinate V_hip of the hip key point), and the final judgment signal Y_final. At t=0, R is 1.0, the center of gravity height is 100, and the final judgment signal is 0 (no fall). At t=1, R rises to 1.2, the center of gravity height drops to 80, and the final judgment signal remains 0. At t=2, R further increases to 1.5, exceeding the area expansion threshold (e.g., 1.35), while the center of gravity height suddenly drops to 50, indicating a forward fall. At this point, the final judgment signal jumps to 1 (fall alarm). This time-series data visually demonstrates the dynamic process of the coordinated change of depth anomaly features and center of gravity height, triggering the final judgment.

[0052] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention calculates the historical average bounding box area using a sliding window and compares it with the area of ​​the current frame to obtain the area change rate, thereby determining depth anomaly change characteristics. This mechanism can detect forward or backward movements of the human body along the optical axis of the camera without a depth sensor, effectively solving the problem of high false negative rates for falls in the depth direction in traditional monocular vision. Simultaneously, using historical averaging can suppress noise caused by detection box jitter, improving the robustness and accuracy of depth anomaly detection.

[0053] Based on any of the above embodiments, determining the depth anomaly change characteristics based on the bounding box area change rate includes: Obtain the preset area shrinkage threshold and the preset area expansion threshold; If the rate of change of the bounding box area is less than the preset area shrinkage threshold or greater than the preset area expansion threshold, then the depth anomaly change feature is set to detect an anomaly. If the rate of change of the bounding box area is greater than or equal to the preset area shrinkage threshold and less than or equal to the preset area expansion threshold, then the depth anomaly change feature is set to no anomaly detected.

[0054] It should be noted that the preset area contraction threshold is a positive number less than 1 (typically 0.70), used to determine whether the human body exhibits abnormal contraction movement away from the camera; the preset area expansion threshold is a positive number greater than 1 (typically 1.35), used to determine whether the human body exhibits abnormal expansion movement closer to the camera. These two thresholds together constitute the boundary of the normal depth movement range. When the rate of change of the bounding box area is less than the contraction threshold or greater than the expansion threshold, it indicates that the human body has undergone violent displacement along the camera's optical axis, corresponding to a forward or backward fall; when the rate of change is between the two thresholds, it indicates that the depth movement is within the normal range of activity. This range-based judgment method can effectively identify depth anomalies without the need for a depth sensor.

[0055] It is understandable that when a human body leans forward or backward along the optical axis of the camera, the distance between the human target and the camera changes within a short period of time, causing the bounding box area in the two-dimensional image plane to appear significantly enlarged or reduced. Therefore, this invention uses the rate of change of the bounding box area as a visual representation of abnormal movement in the depth direction of the human body.

[0056] In practical implementation, the characteristics of depth anomaly changes can be represented as follows: In the formula, Indicates the preset area shrinkage threshold; Indicates the preset area expansion threshold; when This indicates that anomalies have been detected due to changes in depth. This indicates that no abnormality was detected in the depth anomaly characteristics.

[0057] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention compares the rate of change of the bounding box area with the normal range by setting asymmetric area contraction and expansion thresholds: when the rate of change is less than the contraction threshold or greater than the expansion threshold, it is determined to be a depth anomaly; otherwise, it is considered normal. This mechanism can sensitively capture two types of fall patterns: forward fall (area expansion) and backward fall (area contraction), breaking through the depth blind spot of monocular vision, while effectively filtering area fluctuations during normal activities, thus improving the accuracy and robustness of depth anomaly detection.

[0058] Based on any of the above embodiments, the step of performing fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results includes: Obtain the preset probability threshold; The multidimensional fusion features are input into a pre-trained spatiotemporal action recognition model to obtain the predicted fall probability; Based on the preset probability threshold and the predicted fall probability, the initial fall detection result is determined.

[0059] It should be noted that the preset probability threshold is a constant between 0 and 1 (typically 0.7), used to convert the continuous probability values ​​output by the spatiotemporal action recognition model into binary classification results. This threshold can be adjusted according to the system's tolerance for false positives and false negatives: lowering the threshold can improve recall (reduce false negatives) but may increase false positives; raising the threshold has the opposite effect.

[0060] It should be noted that the spatiotemporal action recognition model is a deep learning model that has been trained on a large-scale dataset of falls and non-falls. Its input is a multi-dimensional fused feature tensor, and its output is a scalar probability value P_fall (i.e., the predicted fall probability), representing the confidence that the current action belongs to the fall category. Specifically, the spatiotemporal action recognition model can employ convolutional neural networks, long short-term memory networks, graph convolutional networks, temporal convolutional networks, or combinations thereof.

[0061] In practical implementation, multidimensional fusion features can be represented as: In the formula, Represents the trajectory features of key points in the human body. This indicates the rate of change of the bounding box area. This represents the aspect ratio feature of the bounding box. This indicates the characteristics of abnormal changes in depth.

[0062] In practical implementation, multi-dimensional fused features are input into the spatiotemporal action recognition model to obtain the predicted fall probability: In the formula, This represents a pre-trained spatiotemporal action recognition model. Indicates model parameters.

[0063] In practice, when the predicted probability of falling is greater than or equal to a preset probability threshold, the initial fall detection result is set to indicate that there is a risk of falling; when the predicted probability of falling is not greater than the preset probability threshold, the fall detection result is set to indicate that there is no risk of falling.

[0064] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention uses a pre-trained spatiotemporal action recognition model to intelligently analyze multi-dimensional fusion features, outputting fall probabilities, and converting continuous probabilities into binary preliminary results by combining an adjustable probability threshold. This method achieves rapid and automated preliminary screening, flexibly adjusting the threshold according to the scenario to balance false negatives and false positives, and providing a reliable initial judgment for subsequent kinematic rule verification, effectively improving overall detection efficiency and robustness.

[0065] Figure 5 This is the second flowchart of the fall detection method based on depth anomaly perception and kinematic rule verification provided by the present invention, as shown in Figure 5. Step 105 further includes steps 1051 to 1054: Step 1051: When the preliminary fall detection results indicate that the human target is at risk of falling, determine whether the conditions for exemption from danger feature are met; It should be noted that, to avoid overly rejecting genuine fall events during rule validation, when a human target exhibits obvious lateral roll or abnormal changes in depth, the risk of fall can be directly confirmed. Specifically, the hazard characteristic exemption rule can be expressed as: In the formula, This indicates the preset aspect ratio threshold; Indicates the first Personal physical goals in The aspect ratio feature of the frame's bounding box; This indicates the characteristics of abnormal changes in depth.

[0066] Understandably, when This indicates that anomalies have been detected due to changes in depth. This indicates that no anomalies were detected in the depth anomaly pattern. This indicates that it can be directly confirmed as a fall. Step 1052: When the hazard feature exemption conditions are met, the preliminary fall detection result is taken as the target detection result; Understandably, in order to avoid overly rejecting real fall events during rule validation, when a human target shows obvious side roll or abnormal changes in depth, it is directly confirmed that the human target is at risk of falling, and the chassis stability rejection rule and the lower limb suspension rejection rule are no longer applied.

[0067] Step 1053: When the hazard feature exemption condition is not met, and at least one of the chassis stability rejection condition or the lower limb suspension rejection condition is met, the target fall detection result is set to "not fallen". It should be noted that the chassis stability rejection condition refers to a very small longitudinal displacement of the human hips, indicating that the human body is sitting stably on a support (such as a sofa or the edge of a bed); the lower limb suspension rejection condition refers to an excessively large height ratio of the lower limbs relative to the hips, indicating that the human body is in a posture such as raising the legs, crossing the legs, or turning over in bed.

[0068] In practical implementation, the chassis stability veto condition can be expressed as: In the formula, This indicates that the human target is more likely to be in a stable sitting position, at which point the initial fall suggestion is rejected; Indicates the longitudinal displacement of key points in the buttocks; Indicates the threshold for longitudinal displacement of the buttocks; This indicates that preliminary fall detection results suggest a risk of falling on the human target.

[0069] In the specific implementation, the formula for calculating the longitudinal displacement of the hip key points is as follows: In the formula, This indicates the number of historical frames used to calculate longitudinal displacement; keypoints at the center of the buttocks, hips, or pelvis can be selected as reference keypoints near the body's center of gravity (buttock keypoints), denoted as... Its ordinate is .

[0070] In practical implementation, the lower limb suspension veto rule can be expressed as: In the formula, If the target is more likely to be in a non-falling posture such as raising their leg, crossing their leg, or rolling over in bed, then the initial fall suggestion should be rejected. This indicates the relative height ratio between the key points of the lower limbs and the key points of the buttocks; This indicates the preset threshold for the percentage of floating elements. This indicates that preliminary fall detection results suggest a risk of falling on the human target.

[0071] In practical implementation, the formula for calculating the relative height ratio between the lower limb key points and the hip key points is as follows: In the formula, This represents the height of the bounding box of the human target; keypoints at the ankle or knee can be selected as lower limb keypoints, denoted as... Its ordinate is .

[0072] Understandably, when the exemption conditions are not met (i.e. there are no obvious danger signals) and the human body exhibits non-fall characteristics such as a stable sitting posture or raised lower limbs, the system will determine it as a false alarm and correct the final result to indicate that the human target has not fallen.

[0073] Step 1054: When the hazard feature exemption condition is not met, and the chassis stability rejection condition and the lower limb suspension rejection condition are not met at the same time, the preliminary fall detection result is taken as the target fall detection result.

[0074] It is understandable that when the exemption conditions for dangerous characteristics are not met, and the rejection conditions for chassis stability and lower limb suspension are not met at the same time, the preliminary fall detection result will be taken as the target detection result (i.e., the target detection result is a human target fall).

[0075] In the specific implementation, the final decision logic can be expressed as: In specific implementations, such as Figure 6As shown, when an initial fall proposal is determined to pose a fall risk, a hazard feature exemption check is first performed: if the aspect ratio of the human target's bounding box reaches or exceeds a preset aspect ratio threshold, or if a depth anomaly feature is marked as abnormal, a fall is directly determined without further verification. If the exemption condition is not met, the chassis stability rejection rule and the lower limb suspension rejection rule are executed sequentially: if the longitudinal displacement of the human hip key point is less than a preset longitudinal displacement threshold, the fall proposal is rejected; otherwise, it continues to determine whether the relative height ratio between the lower limb key point and the hip key point is greater than a preset suspension ratio threshold. If it is satisfied, the fall is also rejected. If neither of the above two rejection conditions is met, a fall is finally determined. This process effectively balances false negatives and false positives in fall detection through a tiered verification process of hazard priority exemption and step-by-step rejection filtering.

[0076] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention improves detection reliability through a three-layer verification mechanism: First, it prioritizes exemptions for dangerous features, directly confirming a fall when a rollover or depth anomaly occurs, ensuring zero false negatives for high-risk events. If the exemption is not met, it filters false positives such as sitting down or leg raising using conditions of chassis stability and lower limb suspension. If neither condition is met, the preliminary result is maintained. This hierarchical strategy effectively balances false positives and false negatives, significantly improving the accuracy of fall detection under monocular vision.

[0077] Based on any of the above embodiments, the hazard feature exemption condition is: the aspect ratio of the bounding box of the human target is greater than or equal to a preset aspect ratio threshold, or the depth anomaly change feature is detected as abnormal; the chassis stability rejection condition is: the longitudinal displacement of the hip key point of the human target is less than a preset longitudinal displacement threshold; the lower limb suspension rejection condition is: the relative height ratio between the lower limb key point and the hip key point of the human target is greater than a preset suspension ratio threshold.

[0078] It should be noted that the preset aspect ratio threshold is a value greater than 0 (typically 0.8). When the aspect ratio feature of the bounding box is greater than or equal to the preset aspect ratio threshold, it indicates that the human body in the image is close to a "lying down" state, i.e., it has rolled over. The depth anomaly change feature indicates that when an anomaly is detected, it means that the human body has a drastic displacement along the Z-axis.

[0079] It should be noted that the preset vertical displacement threshold is a small pixel value (e.g., 5 pixels), indicating that the hips have hardly moved. When the vertical displacement of the hip keypoint of the human target is less than this threshold, it indicates that there is no significant vertical position change of the hips between consecutive frames. This is a typical manifestation of a person sitting stably on a support such as a sofa, bed edge, or chair. If the initial fall detection result is judged as a fall, this condition will trigger a chassis stability rejection to filter false alarms of fall-like actions such as sitting down quickly.

[0080] It should be noted that the preset suspension ratio threshold is a value between 0 and 1 (typically 0.6), representing the upper limit of the relative height ratio between the lower limb keypoint and the hip keypoint. When the absolute value of the difference between the ordinates of the lower limb keypoints (such as the ankle or knee) and the hip keypoints of the human target, divided by the bounding box height, is greater than this threshold, it indicates that the lower limbs are raised to a relatively high degree relative to the hips. This feature is commonly seen in non-falling postures such as crossing one's legs, raising one's legs, and turning over in bed. In these situations, if the initial fall detection result is a fall, a lower limb suspension rejection is triggered to exclude false alarms caused by the above-mentioned fall-like actions.

[0081] The fall detection method based on depth anomaly perception and kinematic rule verification provided in this invention constructs a hazard exemption mechanism by using the aspect ratio of the bounding box and changes in depth anomalies. This mechanism prioritizes the identification of high-risk states such as side rolls and forward / backward falls, ensuring zero false negatives of actual falls. Simultaneously, it establishes a chassis stability and lower limb suspension ratio rejection rule using longitudinal hip displacement and lower limb suspension ratio, effectively filtering false alarms such as rapid sitting down and leg raising. This method requires no additional depth hardware and can achieve highly sensitive, low-false-positive fall detection under monocular vision.

[0082] The fall detection device based on depth anomaly perception and kinematic rule verification provided by this invention will be described below. The fall detection device based on depth anomaly perception and kinematic rule verification described below can be referred to in correspondence with the fall detection method based on depth anomaly perception and kinematic rule verification described above. Figure 7 As shown, the fall detection device based on depth anomaly perception and kinematic rule verification includes: Acquisition module 10 is used to acquire the target video stream to be detected; Detection module 20 is used to detect human targets in the target video stream and obtain basic human feature information; The construction module 30 is used to determine the depth anomaly change features based on the human body basic feature information, and to construct multi-dimensional fusion features based on the depth anomaly change features; The detection module 20 is also used to perform fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; The verification module 40 is used to verify the preliminary fall detection result and generate a target fall detection result based on the verification result, so as to trigger an early warning mechanism based on the target fall detection result.

[0083] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a fall detection method based on depth anomaly perception and kinematic rule verification. This method includes: acquiring a target video stream to be detected; detecting human targets in the target video stream to obtain basic human feature information; determining depth anomaly change features based on the basic human feature information, and constructing multi-dimensional fusion features based on the depth anomaly change features; performing fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; verifying the preliminary fall detection results, and generating a target fall detection result based on the verification result, thereby triggering an early warning mechanism based on the target fall detection result.

[0084] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the fall detection method based on depth anomaly perception and kinematic rule verification provided by the above methods. The method includes: acquiring a target video stream to be detected; detecting human targets in the target video stream to obtain basic human feature information; determining depth anomaly change features based on the basic human feature information, and constructing multi-dimensional fusion features based on the depth anomaly change features; performing fall detection on the human target based on the multi-dimensional fusion features to obtain a preliminary fall detection result; verifying the preliminary fall detection result, and generating a target fall detection result based on the verification result, so as to trigger an early warning mechanism based on the target fall detection result.

[0086] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a fall detection method based on depth anomaly perception and kinematic rule verification provided by the above methods. The method includes: acquiring a target video stream to be detected; detecting a human target in the target video stream to obtain basic human feature information; determining depth anomaly change features based on the basic human feature information, and constructing multi-dimensional fusion features based on the depth anomaly change features; performing fall detection on the human target based on the multi-dimensional fusion features to obtain a preliminary fall detection result; verifying the preliminary fall detection result, and generating a target fall detection result based on the verification result, so as to trigger an early warning mechanism based on the target fall detection result.

[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fall detection method based on depth anomaly perception and kinematic rule verification, characterized in that, include: Acquire the target video stream to be detected; Human targets in the target video stream are detected to obtain basic human feature information; Based on the aforementioned basic human body features, depth anomaly change features are determined, and multi-dimensional fusion features are constructed based on these depth anomaly change features. Based on the multi-dimensional fusion features, fall detection is performed on the human target to obtain preliminary fall detection results; The preliminary fall detection results are verified, and a target fall detection result is generated based on the verification results, so as to trigger an early warning mechanism based on the target fall detection result.

2. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 1, characterized in that, The basic human body feature information includes at least bounding box temporal information and key point temporal information; The process of determining depth anomaly variation features based on the aforementioned basic human body feature information, and constructing multi-dimensional fusion features based on these depth anomaly variation features, includes: Based on the bounding box time-series information, the bounding box area change rate characteristics and the bounding box aspect ratio characteristics are determined. Depth anomaly change characteristics are determined based on the bounding box area change rate characteristics; Obtain the key point trajectory features from the key point time-series information; Multidimensional fusion features are constructed by fusing the bounding box area change rate feature, the bounding box aspect ratio feature, the depth anomaly change feature, and the key point trajectory feature.

3. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 2, characterized in that, The step of determining the bounding box area change rate feature based on the bounding box time series information includes: Based on the bounding box timing information, calculate the bounding box area of ​​the human target in the current frame, and obtain the historical bounding box area within a preset historical window length before the current frame. Calculate the historical average area based on the historical bounding box area; Calculate the rate of change of the bounding box area based on the bounding box area and the historical average area; Based on the rate of change of the bounding box area, the characteristics of depth anomaly changes are determined.

4. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 3, characterized in that, The determination of depth anomaly change characteristics based on the bounding box area change rate includes: Obtain the preset area shrinkage threshold and the preset area expansion threshold; If the rate of change of the bounding box area is less than the preset area shrinkage threshold or greater than the preset area expansion threshold, then the depth anomaly change feature is set to detect an anomaly. If the rate of change of the bounding box area is greater than or equal to the preset area shrinkage threshold and less than or equal to the preset area expansion threshold, then the depth anomaly change feature is set to no anomaly detected.

5. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 1, characterized in that, The method of performing fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results includes: Obtain the preset probability threshold; The multidimensional fusion features are input into a pre-trained spatiotemporal action recognition model to obtain the predicted fall probability; Based on the preset probability threshold and the predicted fall probability, the initial fall detection result is determined.

6. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 1, characterized in that, The step of verifying the preliminary fall detection results and generating target fall detection results based on the verification results includes: When the preliminary fall detection results indicate that the human target is at risk of falling, determine whether the conditions for exemption from dangerous features are met. When the hazard feature exemption conditions are met, the preliminary fall detection result is taken as the target detection result; When the hazard feature exemption condition is not met, and at least one of the chassis stability rejection condition or the lower limb suspension rejection condition is met, the target fall detection result is set to "not fallen". When the hazard feature exemption condition is not met, and the chassis stability rejection condition and the lower limb suspension rejection condition are not met at the same time, the preliminary fall detection result is taken as the target fall detection result.

7. The fall detection method based on depth anomaly perception and kinematic rule verification according to claim 6, characterized in that, The exemption condition for the dangerous feature is: the aspect ratio of the bounding box of the human target is greater than or equal to a preset aspect ratio threshold, or the abnormal change in depth is detected as abnormal; the rejection condition for chassis stability is: the longitudinal displacement of the hip key point of the human target is less than a preset longitudinal displacement threshold; the rejection condition for lower limb suspension is: the relative height ratio between the lower limb key point and the hip key point of the human target is greater than a preset suspension ratio threshold.

8. A fall detection device based on depth anomaly perception and kinematic rule verification, characterized in that, include: The acquisition module is used to acquire the target video stream to be detected; The detection module is used to detect human targets in the target video stream and obtain basic human feature information; The construction module is used to determine the depth anomaly change features based on the human body basic feature information, and to construct multi-dimensional fusion features based on the depth anomaly change features; The detection module is also used to perform fall detection on the human target based on the multi-dimensional fusion features to obtain preliminary fall detection results; The verification module is used to verify the preliminary fall detection results and generate a target fall detection result based on the verification results, so as to trigger an early warning mechanism based on the target fall detection result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the fall detection method based on depth anomaly perception and kinematic rule verification as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fall detection method based on depth anomaly perception and kinematic rule verification as described in any one of claims 1 to 7.

Images