Fall detection method based on spatial point cloud attitude analysis and computer system

Abstract

The invention relates to a fall detection method based on spatial point cloud attitude analysis and a computer system, and belongs to the technical field of action and behavior recognition. According to the scheme, a human body target is detected according to a point cloud video, and the target is tracked; for each point cloud video frame, firstly, target detection is carried out according to point cloud data of a point cloud frame image, dynamic point clouds meeting conditions are clustered to form a clustered target, and the clustered target and a tracking target of a target tracking algorithm are matched and associated according to spatial positions; after matching association, adding tumble data used for tumble detection and obtained according to the clustering target into a class of the tracking target corresponding to the matching association; and judging whether the target falls down or not according to the continuous multi-frame falling data of the corresponding target. According to the algorithm, a target which is static for a long time can be identified through a target strong repair algorithm, and a falling state is accurately identified.

Description

Technical Field

[0001] The present invention relates to a fall detection method and a computer system based on spatial point cloud posture analysis, belonging to the technical field of action and behavior recognition. Background Art

[0002] Accidental falls, especially for the elderly, can cause serious injuries. Being able to detect an accidental fall in time when alone at home can be of great help in rescue.

[0003] In the field of smart home, after detecting whether an area is occupied or unoccupied, the target person's posture and height changes are analyzed through point cloud recognition, which can detect the person's fall status and report a fall warning.

[0004] The Chinese invention patent authorization announcement document with the authorization announcement number CN116840835B discloses a fall detection method. The method includes: clustering the acquired point cloud data to obtain human body shape information, determining the trajectory of the human body's center of gravity based on the human body shape information, and then determining the vertical speed of the human body's center of gravity, and comparing it with the preset speed threshold to screen out suspected human body fall information; inputting the suspected human body fall information into the trained variational autoencoder network for fall judgment, and the variational autoencoder network outputs the final fall judgment result, and further judges whether a fall event has occurred by using the unsupervised variational autoencoder network to further judge whether the screened suspected human body fall information has occurred, thereby improving the efficiency and accuracy of fall detection, and having good practicality and ease of use.

[0005] The Chinese invention patent authorization announcement document with the authorization announcement number CN114359814B also discloses a fall detection method. The method includes: when the target person enters the preset detection area, the video image of the target person is obtained through the preset camera, and the human body point cloud data of the target person is obtained through the preset radar; the fall behavior is identified according to the video image to obtain the video image recognition result, and the fall behavior is also identified according to the human body point cloud data to obtain the human body point cloud data recognition result; if the target person is determined to have a fall behavior according to the video image recognition result and the human body point cloud data recognition result at the same time, then it is determined that the target person has a fall behavior.

[0006] However, with existing methods, after a person enters an area, if the person remains still for a period of time, the target may easily disappear, and if the person suddenly falls after the target disappears, it is difficult to detect the fall state. Summary of the invention

[0007] The purpose of the present invention is to provide a fall detection method and computer system based on spatial point cloud posture analysis, so as to solve the problem that it is difficult to identify the fall state when a person suddenly falls down after losing the target due to being stationary.

[0008] To achieve the above object, the solution of the present invention includes: The present invention discloses a technical solution for a fall detection method based on spatial point cloud posture analysis. The method detects a human target based on a point cloud video and tracks the target. For each point cloud video frame, the target is firstly detected based on the point cloud data of the point cloud frame image, and the dynamic point clouds that meet the conditions are clustered to form cluster targets. The cluster targets are matched and associated with the tracking targets of the target tracking algorithm according to the spatial position. After the matching and association, the fall data for fall detection obtained according to the cluster targets are added to the class of the tracking targets on the corresponding matching association. Thus, it is judged whether the target falls according to the fall data of the corresponding target in multiple consecutive frames.

[0009] Furthermore, if the current frame tracks the tracking target, the clustered targets are divided into strong detection targets and weak detection targets according to the number of dynamic point clouds on the clustered targets in the current frame, and the number of dynamic point clouds on the strong detection targets is greater than the number of dynamic point clouds on the weak detection targets; the spatial position condition for matching the weak detection targets with the corresponding tracking targets is closer than the spatial position condition for matching the strong detection targets with the corresponding tracking targets.

[0010] Furthermore, if the tracking target is not tracked in the current frame, if there is a cluster target in the current frame whose number of dynamic point clouds meets a preset condition near the position of the corresponding tracking target in the frame in the previous frame in which the tracking target can be tracked, then the cluster target is associated with the tracking target, and the cluster target in the current frame is updated according to the feature data of the corresponding tracking target in the previous frame that can be tracked, and the fall data of the cluster target in the current frame whose number of dynamic point clouds meets the preset condition is added to the class of the associated tracking target.

[0011] Furthermore, judging whether the target falls includes a pre-fall judgment process based on the change in target height and a fall posture analysis algorithm judgment process based on the target posture. If the fall data of the corresponding target meets the pre-fall judgment, the fall posture analysis algorithm judgment is performed. If the corresponding target further meets the fall posture analysis algorithm judgment, the target is confirmed to have fallen.

[0012] Furthermore, the fall data includes height data of the corresponding target and acceleration direction data of the target height change.

[0013] Furthermore, the height data is an average value of the height of the corresponding target in the latest several frames.

[0014] Furthermore, the pre-fall judgment includes, when the target height is less than a certain threshold or one third of the original target height, judging whether the acceleration direction of the target height change is continuously downward.

[0015] Furthermore, the falling posture analysis algorithm determines that the target point cloud center of mass shifts in the horizontal direction from the starting falling position greater than a set distance during the overall point cloud posture tilting forward and downward.

[0016] Furthermore, the fall posture analysis algorithm also includes determining that the target height is no longer higher than a certain range within a set number of frames after the fall is considered.

[0017] A technical solution of a computer system of the present invention includes a processor, wherein the processor is used to execute the steps of implementing the fall detection method based on spatial point cloud posture analysis as described above.

[0018] The beneficial effects of the present invention are: When a person falls in the detection area, the present invention can accurately detect the position where the person falls and issue a corresponding warning. The tracking target initialization condition of the corresponding tracking algorithm is used to avoid the target being easily lost after it stops. At the same time, the state of the target is analyzed by combining the height, center of mass and other fall data of the corresponding clustering target in multiple consecutive frames of the corresponding tracking target, so as to judge whether the target has fallen, and avoid the problem that the target is difficult to detect after it stops and suddenly falls.

[0019] The present invention can identify targets that remain stationary for a long time and accurately identify the falling state through the target strong repair algorithm.

[0020] Furthermore, the present invention adopts a pre-screening mechanism to screen out point cloud frames where falls may occur in advance through changes in target height, thereby reducing resource consumption of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a method flow chart of a fall detection method based on spatial point cloud posture analysis of the present invention; Figure 2 It is a structural principle diagram of an embodiment of a computer system. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail in a clear and complete manner in conjunction with the accompanying drawings and embodiments.

[0023] The idea of ​​the present invention is to combine the fall detection algorithm with the target tracking algorithm. The human target is detected based on the point cloud video and the target is tracked; at the same time, Figure 1As shown in the figure, for each point cloud video frame, the target detection of the frame is first performed according to the point cloud data of the point cloud frame image, and the moving point cloud (which may be a dynamic target of the human body) that meets the conditions is clustered to form a cluster target. Then, the multi-target tracking algorithm is used to match and associate the cluster target with the tracking target of the target tracking algorithm. After the association, the fall data (such as the cluster centroid position, the cluster target height, etc.) obtained according to the cluster target for fall detection is added to the class of the tracking target corresponding to the matching association; thereby, the tracking target can always be tracked in multiple consecutive frames, and can be matched with the clustered target obtained according to the current frame. When clustering targets to obtain fall data corresponding to the clustered targets, it is determined whether the target has fallen based on the fall data of the corresponding targets in multiple consecutive frames; even if the corresponding tracking target cannot be tracked in several frames, the tracking target is associated with the clustered targets in the current frame near the position of the target in the previous frame in which the target can be tracked. When the association is successful, the feature data of the tracking target in the previous frame is used to update the feature data of the tracking target in the current frame. At the same time, the tracking target can still obtain the fall data obtained according to the clustered targets in the current frame, and can still determine whether the target has fallen based on the fall data of the corresponding targets in multiple consecutive frames.

[0024] The target initialization conditions of the corresponding tracking algorithm are used to avoid the target being lost after it becomes stationary. At the same time, the target's state is analyzed by combining the height, center of mass and other fall data of the corresponding clustered targets in multiple consecutive frames to determine whether the target has fallen, thus avoiding the problem of the target being lost due to a sudden fall caused by the target being stationary and difficult to detect.

[0025] Furthermore, if Figure 1 As shown, a fall prediction (pre-fall judgment) is performed based on the fall data including the target height and the downward acceleration. If the pre-fall judgment is met, the posture analysis of the corresponding target is performed according to the fall posture analysis algorithm. If the posture of the corresponding target meets the fall posture, the target fall state is updated to fall.

[0026] Through pre-fall judgment, targets that may fall are screened out in advance, and then the falling posture analysis is performed to reduce the equipment resource usage while ensuring the accuracy of fall judgment.

[0027] Method Example: The present invention provides a fall detection method based on spatial point cloud posture analysis. This embodiment uses the point cloud data of the detected space acquired by the millimeter wave radar as an example to illustrate the method of the present invention. In the method of the present invention, the fall detection algorithm is embedded in the target tracking algorithm based on the point cloud video. The point cloud video is composed of point cloud images frame by frame. In this embodiment, the millimeter wave radar collects millimeter wave point cloud images at a speed of five frames per second.

[0028] A fall detection method based on spatial point cloud posture analysis of this embodiment includes the following parts.

[0029] 1. Since the installation of millimeter-wave radar is often at a certain angle to the spatial coordinate system, after obtaining the point cloud video, the point cloud space in the video is first converted to the real coordinate system space.

[0030] The transfer matrix T is obtained according to the rotation angle of the millimeter wave radar through the calculation formula between two-dimensional matrices.

[0031] 2. Initialize the tracker for target tracking.

[0032] The tracker will find the moving points and micro-moving points in each frame of point cloud data and preset conditions, and find targets including a certain number of moving points and micro-moving points according to the preset conditions (it is considered that they may be human bodies entering the space, and moving points and micro-moving points are collectively referred to as dynamic point clouds); according to parameters such as spatial IoU distance, the Kalman filter algorithm is used to track the target in continuous point cloud frames. When the target is tracked in a set number of frames continuously, the target is initialized as the tracking target (the algorithm confirms it as a human body entering the space) and continues to track. The movement trajectory of the tracking target in continuous point cloud frames is called a track.

[0033] Initialization parameters: minimum number of point cloud points, spatial IoU (Intersection over Union) distance, maximum number of undetectable times, number of consecutive detections during initialization, detection area, shielded area, and number of detection areas.

[0034] Among them, the minimum number of point cloud points is the number of moving points / micro-moving points between consecutive point cloud image frames received by the target tracking algorithm, when it can be identified as the tracked target; the spatial IoU distance is the IoU distance between the detected targets between two consecutive frames of point cloud images, when the two targets can be tracked as the same target; the maximum number of undetectable times is the number of consecutive frames of images in which the corresponding target cannot be detected (tracked) before the target is considered to have disappeared; the number of consecutive detections during initialization is the number of consecutive frames in which the corresponding target can be detected before the target is initialized as a real target to be tracked, otherwise it will be treated as interference and not tracked; the detection area is the area preset within the detection range of the millimeter-wave radar that needs to be detected (tracked), and the shielding area is the area preset within the detection range of the millimeter-wave radar that does not need to be detected (tracked) to exclude some environmental interference.

[0035] 3. Perform data protocol analysis on the acquired point cloud data.

[0036] (1) Perform spatial position transformation on all points in the point cloud and convert them into positions in the real coordinate system space. Specific method: multiply the coordinates of the point by the transformation matrix T.

[0037] (2) Obtain relevant data according to the data transmission protocol, including the positions of all moving points, all micro-moving points, and target tracks.

[0038] 4. The tracker uses Kalman filtering to track the target in the current frame. When the target tracking value is greater than 0 (that is, there is a tracked target in the current frame), the tracker of the current frame is updated and the personnel detection information in the area is processed. The target tracked in the current frame using Kalman filtering is referred to as the tracking target below.

[0039] (1) Perform clustering, screening and aggregation analysis on the point cloud data obtained in the current frame, including: a) Analyze the number of dynamic point clouds (i.e. moving points) and static point clouds (i.e. slightly moving points) in each track area; use the DBSCAN clustering algorithm to perform cluster analysis on the point cloud data, and remove clusters with less than 5 moving points (considered to be interference) and related noise to reduce the amount of calculation; b) Perform target correction based on the distance from the cluster centroid of the cluster target to the corresponding target detected by the tracker in the current frame, and modify the center of the target detected in the current frame to the centroid of the cluster; the target in the corresponding track is hereinafter referred to as the cluster target; c) Aggregate the micro-motion points and moving point information of the clustered target and the clustering information including the position, etc., to obtain the input information for target detection and classification. The data type of the input information is a list. Each input information contains [abscissa x, abscissa y, height z, number of moving points of the target after clustering, total number of micro-motion points and moving points of the target after clustering, and the region where it is located]. The abscissa x, abscissa y, and height z are the positions of the corresponding clustered targets in the real coordinate system; the region where it is located is the position of the corresponding clustered target, that is, which specific region in the detection area or shielded area it is located in.

[0040] (2) Point cloud frame update based on DeepSort target tracking algorithm.

[0041] a) Perform target detection and classification on the input information: When the total number of all micro-motion points and moving points in the list of input information is greater than 10 or the number of motion point clouds (moving points) is greater than 4, it is a strongly marked target.

[0042] Otherwise the target is a weakly marked target.

[0043] After the point cloud data of each frame is clustered, filtered, and aggregated to form input information, the target obtained by classification based on the input information is referred to as the marked target below.

[0044] The standards for the number of moving points and micro-moving points used for target detection and classification can be adjusted according to the performance of the adopted millimeter-wave radar, the characteristics of the detected object, etc. according to the actual situation, and the specific standards are not limited in the present invention.

[0045] b) Update the tracker.

[0046] i. For each tracked target in the tracker, first update the “number of undetectable times”: add 1 to the “number of undetectable times” of all tracked targets.

[0047] ii. Cascade matching between strongly marked targets and tracking targets in the tracker; When performing multi-target matching, targets are matched mainly based on spatial distance and the Hungarian algorithm. When the spatial distance IoU is less than 1.5 meters and the distance difference on the x-axis and y-axis is less than 0.5 meters, the strong marked target and the corresponding tracked target are matched successfully; when the spatial distance IoU is less than 1.5 meters and the distance difference on either the x-axis or the y-axis is greater than 0.5 meters but less than 0.8 meters, the number of moving points of the target must be greater than 30 to be considered a strong target match.

[0048] The above specific IoU distance threshold and other conditions are based on the fact that at the working frequency of the millimeter-wave radar of five frames per second, the relevant millimeter-wave radar collected data needs to be uploaded to the cloud for calculation, and 1 to 2 frames may be lost during the data transmission process. In this case, the setting is combined with the general speed of a moving person. The relevant conditions can be specifically set according to the actual situation, and the present invention is not limited thereto.

[0049] iii. Cascade matching between weakly marked targets and tracking targets in the tracker; The principle is similar to that of strong target matching, but the parameters are different. When the spatial distance IoU is less than 1.5 meters and the distance difference on the x-axis and y-axis is less than 0.5 meters, the weak target and the corresponding tracking target are matched successfully; when the spatial distance IoU is less than 1.5 meters and the distance difference on either the x-axis or the y-axis is greater than 0.5 meters but less than 0.7 meters, the total number of micro-motion points and moving points must be greater than 30 to consider the weak target matched successfully.

[0050] For two tracking targets and one marked target, when the distance difference between the marked target and the two tracking targets on the x-axis and y-axis is less than 0.5 meters, the tracking target with more moving points detected in the most recent frame is matched first.

[0051] iv. The tracking target that completes the cascade matching in the tracker is updated; The location information and point cloud information of the matched marked target are added to the tracking object class of the corresponding successfully matched tracking target.

[0052] After the tracking target matches the marked target (including strong marked targets and weak marked targets), this tracking target is referred to as the matching object below. The fall data of the matching object will be updated in each frame, and when the fall data meets the fall condition, it is judged that the corresponding matching object has fallen.

[0053] Determine whether the matching object is within the detection area and update the detection area where it is located.

[0054] The "Number of Undetectable Times" of the matching object is updated to 0; that is, when the marked target (including the strongly marked target and the weakly marked target) matches the tracking target (getting the matching object), the "Number of Undetectable Times" of the corresponding tracking target is cleared to zero, indicating that the current frame is detected and the match is successful. If the tracking target cannot be detected for a long time and the "Number of Undetectable Times" reaches the "Maximum Number of Undetectable Times" in the parameters, the tracking target will be deleted.

[0055] v. Update the fall data of the matching object in the tracker; According to the variance, abnormal and large-deviation height data in a set number of consecutive frames of the matching object are eliminated, and the remaining heights are averaged to obtain the average height of the matching object as the target height of the corresponding matching object. At the same time, the acceleration direction of the target height change is calculated based on the target height difference and time difference between adjacent frames.

[0056] Update the height information, downward acceleration direction information, and whether the fall is eliminated in the fall data of the corresponding matching object (after the fall condition is met, the target height is greater than a certain threshold again, and it is considered that the target stands up again, and the judgment of the corresponding target falling is eliminated).

[0057] If the current target height is less than a certain threshold, the falling data of the matching object (including the target height of the corresponding target obtained through the previous frame and the acceleration direction of the corresponding target obtained according to the previous two adjacent frames) is used to analyze whether the downward acceleration array is continuously downward. If it is continuously downward and the target height reached in the current frame is less than the preset threshold or one-third of the target height, the falling algorithm analysis is performed on the target.

[0058] The fall algorithm analysis includes using the fall data of the matching object to judge that when the target meets all the following fall conditions at the same time, the target falls.

[0059] Fall conditions: 1) The acceleration direction of the target height change is downward for many consecutive times.

[0060] 2) The overall point cloud posture is tilted forward; specifically, the displacement of the center of mass of the circumscribed ellipse of the target corresponding point cloud on the x-axis and y-axis planes is greater than the set value.

[0061] 3) The target's last height reaches a certain threshold.

[0062] 4) During the downward process, the point cloud center of mass offsets the starting point of the fall by more than 0.6 meters.

[0063] 5) The target height does not rise above a certain range within 60 frames after it is considered to have fallen.

[0064] vi. Update of valid data in the tracker; The matching object meets the conditions for the target to disappear: Determine the number of times the matching object has only micro-movement points. If it is not a strongly marked target, when the number of times (number of frames) when there are only micro-movement points reaches 300 times (in this embodiment, when there are 5 frames in 1 minute, 300 times is 1 minute), the target status is marked as disappeared. And the corresponding strongly marked area list is cleared. If it is a strongly marked target, when the number of times when there are only micro-movement points reaches 1800 times (6 minutes), the matching object status is marked as deleted.

[0065] For targets including matched objects and unmatched general tracking targets: If the target is outside the detection area and cannot be detected for five consecutive frames (one second), it will be marked as deleted. Targets that are larger than one limit edge of the detection area will be marked as deleted to prevent targets that have passed through the detection area or have not entered the detection area from being tracked.

[0066] If two targets are within 0.5 meters of each other, only the target with a larger number of moving points is retained and initialized.

[0067] If the tracking target does not match the cluster target (the marked target after cluster target classification, including strong marked targets and weak marked targets), the tracking target is initialized. The initialization requires tracking for 3 consecutive frames (considering the possibility of frame loss, it can also be required to be tracked for 3 consecutive times), and the number of dynamic point clouds detected each time is greater than 30, and there are no other tracking targets within 0.6 meters.

[0068] After initialization, the "number of undetectable times" of the tracking target is reset to zero. It will not be deleted immediately, and cascade matching will continue in each frame.

[0069] Traverse the tracker and delete the targets whose status is deleted. Keep the most recent feature information of each tracked object (including matched objects), including: location, tracking sequence number, number of all point clouds, number of dynamic point clouds, number of micro-motion point clouds, and centroid information after clustering.

[0070] c) Output the information in the area based on the updated tracker.

[0071] Traverse all matching objects (other tracking objects may also be included according to the needs of upper-layer applications), and return feature information and fall data based on whether the matching object is in the detection area, the time of the last tracking, and whether it is in the shielded area. Specifically: When not in the shielded area, if in the detection area, the matching object whose last moving point was detected within 200 frames (40 seconds) is added to the return information and is ready to be output to the upper-level application. If not in the detection area, the matching object whose last moving point was detected within 10 frames (2 seconds) is added to the return information and is ready to be output to the upper-level application.

[0072] 5. When the value of Kalman target detection is equal to 0 (that is, the target is not tracked in the current frame), the tracker is updated and the personnel detection information in the area is processed.

[0073] (1) Perform clustering, screening and aggregation analysis on the point cloud data obtained in the current frame, including: a) Analyze the number of moving points and micro-moving points in each track area; perform cluster analysis on the point cloud data according to the DBSCAN clustering algorithm, and remove clusters with less than 5 moving points and related noise to reduce the amount of calculation; b) Perform target correction based on the distance from the cluster centroid of the cluster target to the corresponding target detected by the tracker in the current frame, and modify the center of the target detected in the current frame to the centroid of the cluster; the target in the corresponding track is hereinafter referred to as the cluster target; c) Aggregate the micro-motion points and moving point information of the clustered target and the clustering information including the position, etc., to obtain the input information for target detection and classification, and the data type of the input information is a list. Each input information contains [abscissa x, abscissa y, height z, number of moving points of the target after clustering, total number of micro-motion points and moving points of the target after clustering, and the region]. The abscissa x, abscissa y, and height z are the positions of the corresponding clustered targets in the real coordinate system; the region is the position of the corresponding clustered target, that is, which specific region in the detection area or shielding area it is located in.

[0074] (2) Point cloud frame update based on DeepSort target tracking algorithm.

[0075] Since the tracking target is not tracked in the current frame, the clustered targets obtained by clustering the moving points and slightly moving points in the current frame are no longer matched with the tracking target, but the tracker is directly updated.

[0076] i. For each tracked target in the tracker, first update the “number of undetectable times”: add 1 to the “number of undetectable times” of all tracked targets.

[0077] ii. Determine whether the cluster target corresponding to each input information is a strongly marked object. The confirmation algorithm of the strongly marked object is: according to the position data of the latest 100 frames of the track corresponding to the cluster target, all position data is obtained for less than 100 frames. If the track enters the detection area from outside the detection area and the number of detections in the area reaches a certain value, and the point cloud data density on the target of each corresponding frame is greater than the preset value, then the cluster target is set as a strongly marked object. If the cluster target moves out of the area, and there is only one frame out of the area, delete the label of the strongly marked object.

[0078] In this point cloud frame, if there are cluster targets with 5 moving points or more than 13 micro-moving points near the tracking target (the tracking target is not tracked in this frame, but is within the set range near the tracking target that was tracked in the previous frame or before and has not been deleted), and the cluster target is a strong tracking object, the relevant feature data of the tracking target in the tracker is updated to the corresponding cluster target in this frame, that is, the cluster target in this frame is associated with the tracking target that meets the above conditions.

[0079] The location information and point cloud information of the successfully associated cluster target are added to the tracking object class corresponding to the successfully associated tracking target.

[0080] After the tracking target is associated with the cluster target obtained in this frame, this tracking target is referred to as an associated object. Subsequently, the fall data of the associated object will be updated in each frame, and when the fall data meets the fall condition, it is determined that the corresponding associated object has fallen.

[0081] Determine whether the associated object is within the detection area and update the detection area where it is located.

[0082] The "Number of Undetectable Times" of the associated object is updated to 0; that is, when the cluster target is associated with the tracking target (the associated object is obtained), the "Number of Undetectable Times" of the corresponding tracking target is cleared to zero, indicating that although the tracking target is not tracked in the current frame, it is successfully associated with the target (cluster target) detected in this frame, and the relevant analysis of fall detection continues for the target. If the tracking target cannot be detected for a long time and the "Number of Undetectable Times" reaches the "Maximum Number of Undetectable Times" in the parameters, the tracking target will be deleted.

[0083] iii. Update the fall data of the associated object in the tracker; The target height of the target corresponding to the associated object in the previous frame is used as the target height corresponding to this frame, and the downward acceleration of the associated object in this frame is set to 0.

[0084] Update the height information in the fall data of the corresponding matching object, the acceleration direction information of the target height change, and whether to eliminate the fall (after the fall condition is met, the target height is greater than a certain threshold again, and it is considered that the target stands up again, and the judgment of the corresponding target falling is eliminated).

[0085] If the current target height is less than a certain threshold, the fall data of the associated object (including the target height of the corresponding target obtained through the previous frame and the acceleration direction of the corresponding target obtained according to the previous two adjacent frames) is used to analyze whether the downward acceleration array is continuously downward. If it is continuously downward and the target height reached in the current frame is less than the preset threshold or one-third of the target height, the fall algorithm analysis is performed on the target.

[0086] The fall algorithm analysis includes using the fall data of the matching object to judge that when the target meets all the following fall conditions at the same time, the target falls.

[0087] Fall conditions: 1) The acceleration direction of the target height change is downward for many consecutive times.

[0088] 2) The overall point cloud posture is tilted forward; specifically, the displacement of the center of mass of the circumscribed ellipse of the target corresponding point cloud on the x-axis and y-axis planes is greater than the set value.

[0089] 3) The target's last height reaches a certain threshold.

[0090] 4) During the downward process, the point cloud center of mass offsets the starting point of the fall by more than 0.6 meters.

[0091] 5) The target height does not rise above a certain range within 60 frames after it is considered to have fallen.

[0092] iv. Update of valid data in the tracker; The associated object meets the conditions for the target to disappear: Determine the number of times the associated object has only micro-movement points. When the number of times there are only micro-movement points reaches 1800 times (6 minutes), the status of the associated object is marked as deleted.

[0093] For targets including linked objects and unlinked general tracking targets: If the target is outside the detection area and cannot be detected for five consecutive frames (one second), it will be marked as deleted. Targets that are larger than one limit edge of the detection area will be marked as deleted to prevent targets that have passed through the detection area or have not entered the detection area from being tracked.

[0094] If two targets are within 0.5 meters of each other, only the target with a larger number of moving points is retained and initialized.

[0095] If there are no clustered targets with more than 5 moving points or 13 micro-moving points near the tracking target (the tracking target is not tracked in this frame, but is within the set range near the tracking target that was tracked in the previous frame or before and has not been deleted), the data of the tracking target is updated only based on whether it needs to be deleted according to the "maximum number of undetectable times".

[0096] If the tracking target is not associated with the cluster target, the tracking target is initialized. The initialization requires tracking for 3 consecutive frames (considering the possibility of frame loss, it can also be required to be tracked for 3 consecutive times), and the number of dynamic point clouds detected each time is greater than 30, and there is no other tracking target within 0.6 meters.

[0097] The "number of undetectable times" of the initialized tracking target is reset to zero, and it will not be deleted immediately. It will continue to try to associate with the cluster target in each frame.

[0098] Traverse the tracker and delete the associated objects whose status is deleted. Keep the most recent feature information of each associated object, including: location, tracking sequence number, number of all point clouds, number of dynamic point clouds, number of micro-motion point clouds, and centroid information after clustering.

[0099] d) Output the information in the area based on the updated tracker.

[0100] Traverse all associated objects (other tracking objects may also be included depending on the needs of upper-layer applications), and return feature information and fall data based on whether the associated object is in the detection area, the time of the last tracking, and whether it is in the shielded area. Specifically: When not in the shielded area, if in the detection area, the last detected point cloud frame within 200 frames (40 seconds) of the associated object is added in the return information to be output to the upper application. If not in the detection area, the last detected detection frame within 10 frames (2 seconds) of the associated object is added in the return information to be output to the upper application.

[0101] 6. Add processing time and header related data to the returned information, and output it to the upper-level application in combination with the returned area information data.

[0102] Computer System Embodiment: A computer system of the present invention, such as Figure 2 As shown, it includes a processor and a memory, and the processor is used to execute computer-readable instructions stored in the memory to implement a fall detection method based on spatial point cloud posture analysis of the present invention introduced in the method embodiment.

Claims

1. A fall detection method based on spatial point cloud posture analysis, characterized in that: Human targets are detected based on point cloud videos and the targets are tracked. For each point cloud video frame, target detection is first performed based on the point cloud data of the point cloud frame image, dynamic point clouds that meet the conditions are clustered to form cluster targets, and the cluster targets are matched and associated with the tracking targets of the target tracking algorithm based on spatial positions. After the matching and association, the fall data for fall detection obtained based on the cluster targets is added to the class of the tracking targets on the corresponding matching association; thereby, it is determined whether the target has fallen based on the fall data of the corresponding targets in multiple consecutive frames.

2. The fall detection method based on spatial point cloud posture analysis according to claim 1 is characterized in that: If the current frame tracks the tracking target, the clustered targets are divided into strong detection targets and weak detection targets according to the number of dynamic point clouds on the clustered targets in the current frame, and the number of dynamic point clouds on the strong detection targets is greater than the number of dynamic point clouds on the weak detection targets; Compared with the spatial position condition for matching the weak detection target with the corresponding tracking target, the spatial position condition for matching the strong detection target with the corresponding tracking target has a closer distance requirement.

3. The fall detection method based on spatial point cloud posture analysis according to claim 1, characterized in that: If the tracking target is not tracked in the current frame, if there is a cluster target in the current frame whose number of dynamic point clouds meets the preset conditions near the position of the corresponding tracking target in the frame in the previous frame where the tracking target can be tracked, then the cluster target is associated with the tracking target, and the cluster target in the current frame is updated according to the feature data of the corresponding tracking target in the previous frame that can be tracked, and the fall data of the cluster target in the current frame whose number of dynamic point clouds meets the preset conditions is added to the class of the associated tracking target.

4. The fall detection method based on spatial point cloud posture analysis according to claim 1, characterized in that: Judging whether the target has fallen includes a pre-fall judgment process based on the change in target height and a fall posture analysis algorithm judgment process based on the target posture. If the fall data of the corresponding target meets the pre-fall judgment, the fall posture analysis algorithm judgment is performed. If the corresponding target further meets the fall posture analysis algorithm judgment, the target is confirmed to have fallen.

5. The fall detection method based on spatial point cloud posture analysis according to claim 4 is characterized in that: The fall data includes height data of a corresponding target and acceleration direction data of a height change of the target.

6. The fall detection method based on spatial point cloud posture analysis according to claim 5, characterized in that: The height data is the average value of the height of the corresponding target in the latest several frames.

7. The fall detection method based on spatial point cloud posture analysis according to claim 5, characterized in that: The pre-fall judgment includes, when the target height is less than a certain threshold or one third of the original target height, judging whether the acceleration direction of the target height change is continuously downward.

8. The fall detection method based on spatial point cloud posture analysis according to claim 5, characterized in that: The falling posture analysis algorithm determines that the overall point cloud posture tilts forward and downward, and the target point cloud center of mass deviates in the horizontal direction and the starting falling position is greater than a set distance.

9. The fall detection method based on spatial point cloud posture analysis according to claim 7, characterized in that: The fall posture analysis algorithm also includes determining that the target height is no longer higher than a certain range within a set number of frames after the fall is considered.

10. A computer system comprising a processor, characterized in that: The processor is used to execute the steps of implementing the fall detection method based on spatial point cloud posture analysis as described in any one of claims 1 to 9.

Images