Fall detection method based on spatial point cloud pose analysis, and computer system
By combining point cloud video detection and target tracking algorithms, the problem of fall detection caused by the disappearance of the target after a person has stopped is solved, and accurate identification and early warning of fall status are achieved.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- ULTIMATE IOT (SHANGHAI) TECH LTD
- Filing Date
- 2025-10-31
- Publication Date
- 2026-07-23
AI Technical Summary
In existing technologies, when a person suddenly falls after being stationary, the target is easily lost, making it difficult to identify the fall state.
Combining point cloud video detection and target tracking algorithms, target detection and clustering are performed on point cloud frame images, and target tracking is performed using Kalman filtering and DBSCAN clustering algorithms. Combined with pre-fall judgment and fall posture analysis algorithms, it is determined whether the target has fallen.
It can accurately identify the state of a fall, avoid losing the target after it has come to rest, and improve the accuracy and reliability of fall detection.
Smart Images
Figure CN2025131648_23072026_PF_FP_ABST
Abstract
Description
A fall detection method and computer system based on spatial point cloud attitude analysis
[0001] Cross-references to related applications
[0002] This disclosure claims priority to Chinese Patent Application No. 2025100734130, filed on January 17, 2025, entitled "A Fall Detection Method and Computer System Based on Spatial Point Cloud Attitude Analysis", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This disclosure relates to a fall detection method and computer system based on spatial point cloud attitude analysis, belonging to the field of action and behavior recognition technology. Background Technology
[0004] Accidental falls, especially for the elderly, can cause serious injury. Being able to detect an accidental fall promptly when alone at home can greatly help in providing assistance.
[0005] In the field of smart homes, after detecting whether a place is occupied or unoccupied, the behavior and height changes of the target person can be analyzed by recognizing point clouds, which can detect a person's fall and report a fall warning.
[0006] Chinese invention patent authorization announcement document with authorization announcement number CN116840835B discloses a fall detection method. The method includes: clustering acquired point cloud data to obtain human morphological information; determining the trajectory of the human body's center of gravity based on the morphological information; determining the vertical velocity of the human body's center of gravity and comparing it with a preset velocity threshold to filter out suspected fall information; inputting the suspected fall information into a trained variational autoencoder network for fall judgment; and having the variational autoencoder network output the final fall judgment result. By utilizing an unsupervised variational autoencoder network to further determine whether a fall event has occurred from the filtered suspected fall information, the method improves the efficiency and accuracy of fall detection, and has good practicality and ease of use.
[0007] A Chinese invention patent authorization announcement document with authorization number CN114359814B also discloses a fall detection method. The method includes: when a target person enters a preset detection area, acquiring a video image of the target person through a preset camera, and acquiring human body point cloud data of the target person through a preset radar; performing fall behavior recognition based on the video image to obtain a video image recognition result, and also performing fall behavior recognition based on the human body point cloud data to obtain a human body point cloud data recognition result; if both the video image recognition result and the human body point cloud data recognition result simultaneously determine that the target person has engaged in fall behavior, then it is determined that the target person has engaged in fall behavior.
[0008] However, with existing methods, if a person remains still for a period of time after entering the area, the target can easily disappear, and it is difficult to detect a fall if the person suddenly falls after the target disappears.
[0009] Application content
[0010] The purpose of this disclosure is to provide a fall detection method and computer system based on spatial point cloud attitude analysis, in order to solve the problem of difficulty in identifying the fall state when a person suddenly falls after being lost due to being stationary.
[0011] To achieve the above objectives, the disclosed solution includes:
[0012] This disclosure provides a fall detection method based on spatial point cloud attitude analysis. The method includes: detecting human targets based on point cloud video and tracking the targets; for each point cloud video frame, firstly, target detection is performed based on the point cloud data of the point cloud frame image, and dynamic point cloud clusters that meet the conditions are formed into clustered targets; the clustered targets are matched and associated with the tracking targets of the target tracking algorithm according to their spatial positions; after matching and association, fall data configured for fall detection based on the clustered targets is added to the class of the corresponding matched and associated tracking targets; thereby determining whether the target has fallen based on the fall data of the corresponding targets in multiple consecutive frames.
[0013] Optionally, the method further includes: if a tracking target is detected in the current frame, the clustered targets are divided into strong detection targets and weak detection targets according to the number of dynamic point clouds on the clustered targets in the current frame, wherein the number of dynamic point clouds on the strong detection targets is greater than the number of dynamic point clouds on the weak detection targets; and the spatial position conditions for matching the weak detection targets with the corresponding tracking targets require a closer distance than the spatial position conditions for matching the strong detection targets with the corresponding tracking targets.
[0014] Optionally, the step of matching and associating the clustered target with the tracking target of the target tracking algorithm based on spatial location includes: performing cascade matching of the strong detection target and the tracking target in the tracker, and performing cascade matching of the weak detection target and the tracking target in the tracker.
[0015] Optionally, the step of matching and associating the clustered target with the tracking target of the target tracking algorithm according to spatial location, and then adding the fall data configured for fall detection obtained according to the clustered target to the class of the corresponding matched and associated tracking target, includes: if the tracking target is not tracked in the current frame, and if there is a clustered target in the current frame with a dynamic point cloud quantity that meets a preset condition near the position of the corresponding tracking target in the previous frame where the tracking target could be tracked, then the clustered target is associated with the tracking target, the clustered target in the current frame is updated according to the feature data of the corresponding tracking target in the previous frame where it could be tracked, and the fall data of the clustered target in the current frame with a dynamic point cloud quantity that meets the preset condition is added to the class of the associated tracking target.
[0016] Optionally, the step of determining whether the target has fallen includes: a pre-fall judgment process based on the change in the target's height and a fall posture analysis algorithm judgment process based on the target's posture. If the fall data of the corresponding target matches the pre-fall judgment, then the fall posture analysis algorithm judgment is performed. If the corresponding target further meets the fall posture analysis algorithm judgment, then the target is confirmed to have fallen.
[0017] Optionally, the fall data includes the height data of the corresponding target and the acceleration direction data of the target height change.
[0018] Optionally, the height data is the average height of the corresponding target in the most recent few frames.
[0019] Optionally, the pre-fall judgment process includes: when the target height is less than a certain threshold or one-third of the original target height, determining whether the acceleration direction of the target height change is continuously downward.
[0020] Optionally, the fall posture analysis algorithm determines the following: the overall point cloud posture is tilted forward, and during the downward process, the centroid of the target point cloud shifts in the horizontal direction, and the starting position of the fall is greater than a set distance.
[0021] Optionally, the fall posture analysis algorithm further includes determining that the target height does not rise beyond a certain range within a set number of frames after the fall is considered to have occurred.
[0022] Optionally, the step of detecting human targets based on point cloud video and tracking the targets includes: performing coordinate transformation from point cloud space to real coordinate system space on the point cloud video; initializing the tracker configured for target tracking; and detecting human targets through the tracker and tracking the targets.
[0023] This disclosure also provides a computer system including a processor configured to perform steps implementing the fall detection method based on spatial point cloud attitude analysis as described above.
[0024] The beneficial effects of this disclosure are as follows:
[0025] When a person falls within the detection area, the system can accurately detect the location of the fall and issue a corresponding warning. By using the target initialization conditions of the tracking algorithm, it avoids the problem of easily losing the target when it is stationary. Simultaneously, by combining the target's height, centroid, and other fall data from consecutive frames of corresponding clustered targets, it analyzes the target's state to determine whether the target has fallen, thus avoiding the problem of difficulty in detecting sudden falls after the target has been lost when stationary.
[0026] This disclosure uses a target-strong repair algorithm to identify targets that have been stationary for a long time and accurately identify the state of a fall.
[0027] Optionally, this disclosure employs a pre-screening mechanism to pre-screen point cloud frames that may fall by changing the target height, thereby reducing the resource consumption of the device. Attached Figure Description
[0028] Figure 1 is a flowchart of a fall detection method based on spatial point cloud attitude analysis disclosed in this invention;
[0029] Figure 2 is a schematic diagram of the structure of an embodiment of the computer system. Detailed Implementation
[0030] To make the purpose, technical solution and advantages of this disclosure clearer, the following detailed description of this disclosure is provided in conjunction with the accompanying drawings and embodiments.
[0031] The present invention combines a fall detection algorithm with a target tracking algorithm. It detects human targets based on point cloud video and tracks them. As shown in Figure 1, for each point cloud video frame, target detection is first performed based on the point cloud data of that frame. Moving point clouds (potentially dynamic human targets) that meet certain conditions are clustered into clustered targets. Then, a multi-target tracking algorithm is used to match and associate these clustered targets with the tracking targets of the target tracking algorithm. After association, fall detection data (e.g., cluster centroid position, cluster target height, etc.) obtained from the clustered targets is added to the corresponding matched and associated target class. This allows the tracking target to be consistently tracked across multiple consecutive frames, while also enabling... When the clustering target obtained by clustering based on the current frame is matched to obtain the fall data of the corresponding clustering target, the fall data of the corresponding target in multiple consecutive frames is used to determine whether the target has fallen. Even if the corresponding tracking target is not tracked in several frames, the tracking target is associated with the clustering target in the current frame near the position of the target in the previous frame where the target can be tracked. When the association is successful, the feature data of the tracking target in the previous frame is used to update the feature data of the tracking target in the current frame. At the same time, the tracking target can still obtain the fall data obtained by clustering target in the current frame, and can still determine whether the target has fallen based on the fall data of the corresponding target in multiple consecutive frames.
[0032] By adjusting the target initialization conditions of the corresponding tracking algorithm, the target is easily lost after it becomes stationary. At the same time, by combining the fall data such as the height and centroid of the corresponding clustered target in multiple consecutive frames, the state of the target is analyzed to determine whether the target has fallen. This avoids the problem of the target being difficult to detect when it suddenly falls after being lost due to being stationary.
[0033] Optionally, as shown in Figure 1, a fall prediction (pre-fall judgment) is performed based on fall data including target height and downward acceleration. If the pre-fall judgment is met, the posture analysis of the corresponding target is performed according to the fall posture analysis algorithm. If the posture of the corresponding target meets the fall posture, the target fall state is updated to fall.
[0034] By pre-judging potential falls, targets that may fall are identified in advance, and then fall posture analysis is performed, which reduces equipment resource consumption while ensuring the accuracy of fall judgment.
[0035] This disclosure provides a fall detection method based on spatial point cloud attitude analysis. This embodiment uses point cloud data of the detected space acquired by millimeter-wave radar as an example to illustrate the method. In this method, the fall detection algorithm is embedded in a target tracking algorithm based on point cloud video. The point cloud video consists of frame-by-frame point cloud images. In this embodiment, the millimeter-wave radar acquires millimeter-wave point cloud images at a rate of five frames per second.
[0036] This embodiment of a fall detection method based on spatial point cloud attitude analysis includes the following parts.
[0037] 1. Since the installation of millimeter-wave radar is often at a certain angle to the spatial coordinate system, after obtaining the point cloud video, the first step is to convert the point cloud space in the video to the real coordinate system space.
[0038] The transfer matrix T is obtained by using the calculation formula between two-dimensional matrices based on the rotation angle of the millimeter-wave radar.
[0039] 2. Initialize the tracker configured for target tracking.
[0040] The tracker identifies moving and micro-moving points in each frame of point cloud data based on preset conditions. It then identifies targets containing a certain number of moving and micro-moving points (considered potential human beings entering the space; these moving and micro-moving points are collectively referred to as dynamic point clouds). Based on parameters such as spatial IoU distance, the tracker uses a Kalman filter algorithm to track the target across consecutive point cloud frames. When a target is tracked consecutively for a set number of frames, it is initialized as a tracking target (confirmed by the algorithm as a human being entering the space) and tracking continues. The trajectory of the tracked target across consecutive point cloud frames is called a track.
[0041] Initialization parameters: minimum number of points in the point cloud, spatial IoU (Intersection over Union) distance, maximum number of times the object cannot be detected, number of consecutive detections during initialization, detection area, shielded area, and number of detection areas.
[0042] Among them, the minimum number of points in the point cloud refers to the number of moving / micro-moving points that the target tracking algorithm needs to have between consecutive point cloud image frames to identify as the tracked target; the spatial IoU distance refers to the conditions under which the IoU distance between detected targets between two consecutive point cloud image frames must meet to track the two targets as the same target; the maximum number of times the target cannot be detected (tracked) is the number of consecutive image frames after which the target is considered to have disappeared; the number of consecutive detections during initialization is the number of consecutive frames after which the target is initialized as a real target to be tracked, otherwise it is considered interference and not tracked; the detection area is the preset area within the millimeter-wave radar detection range that needs to be detected (tracked), and the shielded area is the preset area within the millimeter-wave radar detection range that does not need to be detected (tracked) to exclude some environmental interference.
[0043] 3. Perform data protocol analysis on the acquired point cloud data.
[0044] (1) Perform spatial position transformation on all points in the point cloud to convert them into their positions in the real coordinate system. Specific method: multiply the coordinates of the points by the transformation matrix T.
[0045] (2) Obtain relevant data according to the data transmission protocol. This includes the positions of all moving points, all micro-moving points, and the target track.
[0046] 4. The tracker uses a Kalman filter to track targets in the current frame. When the target tracking value is greater than 0 (i.e., there is a tracked target in the current frame), the tracker updates the current frame and processes personnel detection information within the area. Targets tracked using the Kalman filter in the current frame are referred to as tracked targets.
[0047] (1) Perform clustering, filtering, and aggregation analysis on the point cloud data obtained in the current frame, including:
[0048] a) Analyze the number of dynamic point clouds (i.e., moving points) and static point clouds (i.e., micro-moving points) within each track area; use the DBSCAN clustering algorithm to perform cluster analysis on the point cloud data, and remove clusters with fewer than 5 moving points (considered as interference) and related noise to reduce the amount of computation;
[0049] b) Based on the distance from the centroid of the clustered target to the corresponding target detected by the tracker in the current frame, the target center detected in the current frame is modified to the centroid of the cluster; the target within the corresponding track is referred to as the clustered target.
[0050] c) Aggregate the micro-moving points and moving point information of the clustered targets, along with clustering information including location, to obtain input information configured for target detection and classification. The data type of the input information is a list. Each input information includes [x-coordinate, y-coordinate, height z, number of moving points of the clustered target, total number of micro-moving points and moving points of the clustered target, and the region where it is located]. The x-coordinate, y-coordinate, and height z represent the position of the corresponding clustered target in the real coordinate system; the region where it is located refers to the specific region within the detection area or the masking area where the corresponding clustered target is located.
[0051] (2) Point cloud frame update based on DeepSort target tracking algorithm.
[0052] a) Perform target detection and classification on the input information:
[0053] When the total number of all micro-moving points and moving points in the list of input information is greater than 10 or the number of moving point clouds (moving points) is greater than 4, it is a strongly marked target.
[0054] Otherwise, the target is a weakly labeled target.
[0055] After the point cloud data of each frame is clustered, filtered, and aggregated to form input information, the targets obtained by classifying based on the input information are referred to as labeled targets.
[0056] The specific configuration, such as the number of moving points and micro-moving points for target detection and classification, can be based on the performance of the millimeter-wave radar used.
[0057] b) Update the tracker.
[0058] i. For each tracked target within the tracker, first update the "count of undetectable targets": increment the "count of undetectable targets" by 1 for all tracked targets.
[0059] ii. Cascade matching between strong detection targets and tracked targets within the tracker;
[0060] When performing multi-target matching, the matching is mainly based on spatial distance and the Hungarian algorithm. When the spatial distance IoU is less than 1.5 meters and the distance difference on both the x-axis and y-axis is less than 0.5 meters, the strongly labeled target and the corresponding tracked target are successfully matched. When the spatial distance IoU is less than 1.5 meters and the distance difference on either the x-axis or y-axis is greater than 0.5 meters but less than 0.8 meters, the number of moving points of the target must be greater than 30 for the strong target to be considered successfully matched.
[0061] The specific IoU distance threshold and other conditions mentioned above take into account that at the millimeter-wave radar operating frequency of five frames per second, the relevant millimeter-wave radar data needs to be uploaded to the cloud for calculation. During the data transmission process, there may be one or two frames lost. In this case, the settings are made in combination with the general speed of a moving person. The relevant conditions can be set according to the actual situation, and this disclosure does not limit them.
[0062] iii. Cascade matching between weakly detected targets and tracked targets within the tracker;
[0063] The principle is similar to that of strongly labeled target matching, but the parameters are different. When the spatial distance IoU is less than 1.5 meters and the distance difference on both the x-axis and y-axis is less than 0.5 meters, the weakly detected target and the corresponding tracked target are successfully matched. When the spatial distance IoU is less than 1.5 meters and the distance difference on either the x-axis or y-axis is greater than 0.5 meters but less than 0.7 meters, then the total number of micro-moving points and moving points must be greater than 30 to be considered a successful weak target match.
[0064] For two tracking targets and one marker target, if the distance difference between the marker target and the two tracking targets on the x-axis and y-axis is less than 0.5 meters, the tracking target with more moving points detected in the most recent frame will be matched first.
[0065] iv. Update the tracked targets that have completed cascading matching within the tracker;
[0066] The location information and point cloud information of the matched marked targets are added to the tracking object class of the corresponding successfully matched tracking target.
[0067] Once the tracked target matches the marked target (including strong detection targets and weak detection targets), this tracked target will be referred to as the matching object. Subsequently, the falling data of the matching object will be updated in each frame, and the corresponding matching object will be judged to have fallen after the falling data meets the falling conditions.
[0068] Determine if the matching object is within the detection area, and update the detection area in which it is located.
[0069] The "Number of Undetectable Counts" for a matching target is updated to 0; that is, when a marked target (including strong and weak detection targets) matches a tracking target (a matching target is obtained), the "Number of Undetectable Counts" for the corresponding tracking target is cleared to zero, indicating that the target was detected and matched successfully in the current frame. If a tracking target is not detected for a long time, and the "Number of Undetectable Counts" reaches the "Maximum Number of Undetectable Counts" in the parameters, the tracking target will be deleted.
[0070] v. Update the fall data of matched objects within the tracker;
[0071] Based on the variance, abnormal and large deviation height data in a set number of consecutive frames of the matching object are removed. The average height of the remaining heights is used as the target height of the matching object. At the same time, the acceleration direction of the target height change is calculated based on the target height difference and time difference between adjacent frames.
[0072] Update the height information, downward acceleration direction information, and fall rejection information in the fall data of the corresponding matched object (if the target height is greater than a certain threshold again after the fall condition is met, the target is considered to have stood up again, and the judgment of the corresponding target falling is removed).
[0073] If the current target height is less than a certain threshold, the falling data of the matched object (including the target height of the corresponding target obtained from the previous frame and the acceleration direction of the corresponding target obtained from the two adjacent frames) is analyzed to see if the downward acceleration array is continuously downward. If it is continuously downward and the target height of the current frame is less than the preset threshold or one-third of the target height, then the falling algorithm analysis is performed on the target.
[0074] The fall algorithm analysis includes using the fall data of the matched objects to determine if the target has fallen when it meets all of the following fall conditions.
[0075] Falling conditions:
[0076] 1) The direction of acceleration due to the change in target height is downward multiple times.
[0077] 2) The overall point cloud attitude is tilted forward; specifically, the displacement of the centroid of the circumscribed ellipse of the target point cloud on the x-axis and y-axis planes is greater than the set value.
[0078] 3) The target's height reaches a certain threshold for the last time.
[0079] 4) During the downward process, the centroid of the point cloud shifts and the position where it begins to fall is greater than 0.6 meters.
[0080] 5) The target height does not rise above a certain range within 60 frames after the fall is considered to have occurred.
[0081] vi. Effective data updates within the tracker;
[0082] The matched object satisfies the condition that the target disappears:
[0083] The system determines the number of times a matching object exhibits only minor movement. If the target is not a strong detection target, and the number of times (in frames) exhibiting only minor movement reaches 300 (in this example, with 5 frames per minute, 300 times equals 1 minute), the target status is marked as disappeared, and the corresponding strongly marked region list is cleared. If the target is a strong detection target, and the number of times exhibiting only minor movement reaches 1800 (in 6 minutes), the matching object status is marked as deleted.
[0084] For targets that include both matched objects and general tracking targets that do not match:
[0085] If a target is not detected for five consecutive frames (or for one second) outside the detection area, it is marked for deletion. Targets larger than one extreme edge of the detection area are also marked for deletion to prevent targets that have passed outside the detection area but have not entered it from being tracked.
[0086] If two targets are within 0.5 meters of each other, only the target with the larger number of moving points is retained and initialized.
[0087] If the tracked target does not match the clustered target (the labeled target after the clustered target is classified, including strong detection target and weak detection target), the tracked target is initialized. The initialization requires that the target be tracked for 3 consecutive frames (or 3 consecutive times if frames are lost). Each time, the number of dynamic point clouds detected is greater than 30, and there are no other tracked targets within 0.6 meters.
[0088] The "count of undetectable" for the tracked target after initialization is cleared to zero. The target will not be deleted immediately but will continue to be cascaded and matched in each frame.
[0089] Iterate through the trackers and delete targets whose target status is deleted. Retain the most recent feature information for each tracked object (including matched objects), including: location, tracking sequence number, total number of point clouds, number of dynamic point clouds, number of micro-motion point clouds, and centroid information after clustering.
[0090] c) Output information within the region based on the updated tracker.
[0091] Iterate through all matched objects (and may include other tracked objects depending on the needs of the upper-layer application), and return feature information and fall data based on whether the matched object is within the detection area, the last time it was tracked, or whether it is within the masked area. Specifically:
[0092] When outside the shielded area, if within the detection area, matching objects whose last detected moving point was within 200 frames (40 seconds) are added to the return information for output to the upper-layer application. If outside the detection area, matching objects whose last detected moving point was within 10 frames (2 seconds) are added to the return information for output to the upper-layer application.
[0093] 5. When the Kalman target detection value is equal to 0 (i.e. no target is tracked in the current frame), the tracker is updated and the personnel detection information in the area is processed.
[0094] (1) Perform clustering, filtering, and aggregation analysis on the point cloud data obtained in the current frame, including:
[0095] a) Analyze the number of moving points and micro-moving points within each track area; perform cluster analysis on the point cloud data using the DBSCAN clustering algorithm, and remove clusters with fewer than 5 moving points and related noise to reduce computational load;
[0096] b) Based on the distance from the centroid of the clustered target to the corresponding target detected by the tracker in the current frame, the target center detected in the current frame is modified to the centroid of the cluster; the target within the corresponding track is referred to as the clustered target.
[0097] c) Aggregate the micro-moving points and moving point information of the clustered targets, along with clustering information including location, to obtain input information configured for target detection and classification. The data type of the input information is a list. Each input information includes [x-coordinate, y-coordinate, height z, number of moving points of the clustered target, total number of micro-moving points and moving points of the clustered target, and the region where it is located]. The x-coordinate, y-coordinate, and height z represent the position of the corresponding clustered target in the real coordinate system; the region where it is located refers to the specific region within the detection area or the masking area where the corresponding clustered target is located.
[0098] (2) Point cloud frame update based on DeepSort target tracking algorithm.
[0099] Since the target is not tracked in the current frame, the cluster target obtained by clustering the moving points and micro-moving points in the current frame is no longer matched with the target being tracked. Instead, the tracker is updated directly.
[0100] i. For each tracked target within the tracker, first update the "count of undetectable targets": increment the "count of undetectable targets" by 1 for all tracked targets.
[0101] ii. Determine whether the cluster target corresponding to each input information is a strongly labeled object. The algorithm for confirming strongly labeled objects is as follows: Based on the position data of the most recent 100 frames of the track corresponding to the cluster target, if less than 100 frames, obtain all position data. If the track enters the detection area from outside the detection area and is detected a certain number of times within the area, and the point cloud data density on the target in each frame is greater than a preset value, then the cluster target is set as a strongly labeled object. If the cluster target moves out of the area, and there is only one frame outside the area, the label of the strongly labeled object is deleted.
[0102] In this point cloud frame, if there are clustered targets with 5 or more moving points or 13 or more micro-moving points near the tracking target (the tracking target is not tracked in this frame, but it is tracked in the previous frame or earlier and has not been deleted), and the clustered target is a strong tracking object, the relevant feature data of the tracking target in the tracker will be updated to the corresponding clustered target in this frame. That is, the clustered target in this frame will be associated with the tracking target that meets the above conditions.
[0103] Add the location information and point cloud information of the successfully associated clustered targets to the tracking object class of the corresponding successfully associated tracking targets.
[0104] After the tracked target is associated with the clustered target obtained in the current frame, this tracked target is referred to as the associated object. Subsequently, the fall data of the associated object will be updated in each frame, and the corresponding associated object will be judged to have fallen after the fall data meets the fall condition.
[0105] Determine if the associated object is within the detection area, and update the detection area in which it is located.
[0106] The "Number of Undetectable Counts" for associated objects is updated to 0. That is, when a clustered target is associated with a tracking target (an associated object is obtained), the "Number of Undetectable Counts" for the corresponding tracking target is cleared to zero. This indicates that although the tracking target was not tracked in the current frame, it was successfully associated with a target detected in this frame (clustered target), and fall detection analysis continues for that target. If a tracking target is not detected for an extended period, and the "Number of Undetectable Counts" reaches the "Maximum Number of Undetectable Counts" parameter, the tracking target will be deleted.
[0107] iii. Update fall data of associated objects within the tracker;
[0108] Use the target height of the target corresponding to the associated object in the previous frame as the target height in the current frame, and set the downward acceleration of the associated object in the current frame to 0.
[0109] Update the height information, acceleration direction information of the target height change in the fall data of the corresponding matched object, and whether the target has fallen and should be removed (if the target height is greater than a certain threshold again after the fall condition is met, the target is considered to have stood up again and the judgment of the target falling is removed).
[0110] If the current target height is less than a certain threshold, the fall data of the associated object (including the target height of the corresponding target obtained from the previous frame and the acceleration direction of the corresponding target obtained from the two adjacent frames) is analyzed to see if the downward acceleration array is continuously downward. If it is continuously downward and the target height of the current frame is less than the preset threshold or one-third of the target height, then the fall algorithm analysis is performed on the target.
[0111] The fall algorithm analysis includes using the fall data of the matched objects to determine if the target has fallen when it meets all of the following fall conditions.
[0112] Falling conditions:
[0113] 1) The direction of acceleration due to the change in target height is downward multiple times.
[0114] 2) The overall point cloud attitude is tilted forward; specifically, the displacement of the centroid of the circumscribed ellipse of the target point cloud on the x-axis and y-axis planes is greater than the set value.
[0115] 3) The target's height reaches a certain threshold for the last time.
[0116] 4) During the downward process, the centroid of the point cloud shifts and the position where it begins to fall is greater than 0.6 meters.
[0117] 5) The target height does not rise above a certain range within 60 frames after the fall is considered to have occurred.
[0118] iv. Valid data updates within the tracker;
[0119] The associated object satisfies the condition that the target disappears:
[0120] The system determines the number of times an associated object exhibits only micro-movements. When the number of times a micro-movement occurs reaches 1800 (6 minutes), the associated object is marked as deleted.
[0121] For targets that include both associated objects and general tracking targets that are not associated:
[0122] If a target is not detected for five consecutive frames (or for one second) outside the detection area, it is marked for deletion. Targets larger than one extreme edge of the detection area are also marked for deletion to prevent targets that have passed outside the detection area but have not entered it from being tracked.
[0123] If two targets are within 0.5 meters of each other, only the target with the larger number of moving points is retained and initialized.
[0124] If there are no clustered targets with more than 5 moving points or 13 micro-moving points near the target being tracked (not tracked in this frame, but tracked in the previous frame or earlier and not yet deleted), the target being tracked will only have its data updated based on whether it needs to be deleted according to the "maximum number of times it cannot be detected".
[0125] If the target being tracked is not associated with a cluster target, the target being tracked is initialized. Initialization requires that the target be tracked for 3 consecutive frames (or 3 consecutive frames if frames are lost), and that the number of dynamic point clouds detected each time is greater than 30, and that no other target being tracked appears within 0.6 meters.
[0126] The "count of undetectable" for the tracked target after initialization is cleared to zero. The target will not be deleted immediately, but will continue to attempt to associate with the clustered target in each frame.
[0127] Iterate through the tracker and delete any associated objects whose status is "deleted". Retain the most recent feature information for each associated object, including: location, tracking sequence number, total number of point clouds, number of dynamic point clouds, number of micro-motion point clouds, and centroid information after clustering.
[0128] d) Output information within the region based on the updated tracker.
[0129] Iterate through all associated objects (and may include other tracked objects depending on the needs of the upper-layer application), and return feature information and fall data based on whether the associated object is within the detection area, the last time it was tracked, or whether it is within the masked area. Specifically:
[0130] When outside the shielded area, if within the detection area, associated objects from the last detected point cloud frames within 200 frames (40 seconds) are added to the returned information for output to the upper-layer application. If outside the detection area, associated objects from the last detected detection frames within 10 frames (2 seconds) are added to the returned information for output to the upper-layer application.
[0131] 6. Add processing time and header-related data to the returned information, and output it to the upper-level application in combination with the returned region information data.
[0132] This disclosure also provides a computer system, as shown in FIG2, including a processor and a memory, wherein the processor is configured to execute computer-readable instructions stored in the memory to implement a fall detection method based on spatial point cloud attitude analysis described in the method embodiments of this disclosure. Industrial applicability
[0133] This disclosure provides a fall detection method and computer system based on spatial point cloud attitude analysis, which can identify targets that have been stationary for a long time and accurately identify the fall state.
Claims
A fall detection method based on spatial point cloud attitude analysis, characterized in that, The method includes: detecting human targets based on point cloud video and tracking the targets; for each point cloud video frame, firstly, target detection is performed based on the point cloud data of the point cloud frame image, and dynamic point cloud clusters that meet the conditions are formed into clustered targets, and the clustered targets are matched and associated with the tracking targets of the target tracking algorithm according to their spatial positions, and after matching and association, fall data configured for fall detection based on the clustered targets is added to the class of the corresponding matched and associated tracking targets; thereby, it is determined whether the target has fallen based on the fall data of the corresponding targets in multiple consecutive frames. The fall detection method based on spatial point cloud attitude analysis according to claim 1 is characterized in that, The method further includes: if a tracking target is tracked in the current frame, the clustered targets are divided into strong detection targets and weak detection targets according to the number of dynamic point clouds on the clustered targets in the current frame, wherein the number of dynamic point clouds on the strong detection targets is greater than the number of dynamic point clouds on the weak detection targets; the spatial position conditions for matching the weak detection targets with the corresponding tracking targets are closer in distance than the spatial position conditions for matching the strong detection targets with the corresponding tracking targets. The fall detection method based on spatial point cloud attitude analysis according to claim 2 is characterized in that, The step of matching and associating the clustered target with the tracking target of the target tracking algorithm based on spatial location includes: cascading the strong detection target with the tracking target in the tracker, and cascading the weak detection target with the tracking target in the tracker. The fall detection method based on spatial point cloud attitude analysis according to any one of claims 1-3 is characterized in that, The step of matching and associating the clustered target with the tracking target of the target tracking algorithm based on spatial location, and then adding the fall data configured for fall detection obtained from the clustered target to the class of the corresponding matched and associated tracking target, includes: if the tracking target is not tracked in the current frame, and if there is a clustered target in the current frame with a dynamic point cloud quantity that meets a preset condition near the position of the corresponding tracking target in the previous frame where the tracking target could be tracked, then the clustered target is associated with the tracking target, the clustered target in the current frame is updated according to the feature data of the corresponding tracking target in the previous frame where it could be tracked, and the fall data of the clustered target in the current frame with a dynamic point cloud quantity that meets the preset condition is added to the class of the associated tracking target. The fall detection method based on spatial point cloud attitude analysis according to any one of claims 1-4 is characterized in that, The step of determining whether the target has fallen includes: a pre-fall judgment process based on changes in the target's height and a fall posture analysis algorithm judgment process based on the target's posture. If the fall data of the corresponding target matches the pre-fall judgment, then the fall posture analysis algorithm judgment is performed. If the corresponding target further meets the fall posture analysis algorithm judgment, then the target is confirmed to have fallen. The fall detection method based on spatial point cloud attitude analysis according to claim 5 is characterized in that, The fall data includes the height data of the corresponding target and the acceleration direction data of the target height change. The fall detection method based on spatial point cloud attitude analysis according to claim 6 is characterized in that, The height data is the average height of the corresponding target in the most recent few frames. The fall detection method based on spatial point cloud attitude analysis according to claim 6 is characterized in that, The pre-fall judgment process includes: when the target height is less than a certain threshold or one-third of the original target height, it is then judged whether the acceleration direction of the target height change is continuously downward. The fall detection method based on spatial point cloud attitude analysis according to claim 6 is characterized in that, The fall posture analysis algorithm determines the following: the overall point cloud posture tilts forward, and during the downward process, the centroid of the target point cloud shifts in the horizontal direction, and the starting fall position is greater than a set distance. The fall detection method based on spatial point cloud attitude analysis according to claim 8 is characterized in that, The fall posture analysis algorithm also includes determining whether the target height does not rise beyond a certain range within a set number of frames after the fall is considered to have occurred. The method according to any one of claims 1-10, characterized in that, The steps of detecting human targets based on point cloud video and tracking the targets include: performing coordinate transformation from point cloud space to real coordinate system space on the point cloud video; initializing the tracker configured for target tracking; and detecting and tracking human targets through the tracker. A computer system, comprising a processor, characterized in that, The processor is configured to perform the steps of implementing the fall detection method based on spatial point cloud attitude analysis as described in any one of claims 1-11.