Human fall detection method, device and equipment based on key point sequence analysis

Through key point sequence analysis and filtering processing, combined with timing analysis, the accuracy problem of fall detection in the existing technology is solved, and the highly accurate identification and distinction of human falls is achieved, and the accuracy and reliability of fall detection is improved.

CN118592943BActive Publication Date: 2024-11-12NINGBO SIMSHINE INTELLIGENT TECH CO LTD

Patent Information

Application Number
CN202411075372.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-12
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The existing human fall detection technology has problems of misjudgment and misjudgment when accurately distinguishing daily activities from real falls, and is sensitive to different environmental factors and individual biomechanical differences, affecting the detection accuracy.

Method used

Using a method based on key point sequence analysis, the frame processing of real-time video streams, human key point detection and filtering processing, combined with timing analysis, the relative position and change rate of human movement are obtained, and the fall detection conditions are set to determine the fall event.

Benefits of technology

It improves the accuracy and reliability of fall detection, and can effectively distinguish falls from other actions in complex action sequences, reduce noise and error, and provide a more stable and accurate data basis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118592943B_ABST
    Figure CN118592943B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of human fall detection, solves the problem of low accuracy of human fall detection in the prior art, and provides a human fall detection method, device and equipment based on key point sequence analysis. The method includes: acquiring a real-time video stream collected in a target object monitoring scene, decomposing the real-time video stream into multiple frames of real-time images; performing human key point detection and filtering processing on each frame of real-time image, and outputting the target position information of the human key points; performing motion analysis on the target object based on the target position information, and outputting the motion analysis results; based on the preset fall detection conditions and the motion analysis results, when it is determined that the target object has fallen, a safety reminder is issued. The present invention can highly accurately and sensitively identify and distinguish human falls from other actions, thereby improving the accuracy and reliability of fall detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of human fall detection, and in particular to a human fall detection method, device and equipment based on key point sequence analysis. Background Art

[0002] Human fall detection technology is a technology that has developed rapidly as the problem of population aging becomes increasingly serious and awareness of personal health management increases. It is mainly used to timely detect falls of the elderly, people living alone or people with weak physical conditions, so that rescue measures can be taken quickly in an emergency to reduce the injuries caused by falls. Falls not only cause immediate harm to the elderly, but may also lead to long-term health problems, including phobias, mobility difficulties and even death. Therefore, human fall detection technology has broad application prospects in the fields of medical health, elderly care and smart homes. The existing human fall detection technology solutions mainly include detection based on wearable devices, detection based on environmental sensors and detection based on intelligent video analysis. The above technologies judge fall events by monitoring the movement state of the human body, environmental changes or image information. Although the technology continues to advance, it still faces some technical problems, such as accuracy. The existing technology cannot accurately distinguish between daily activities and real human falls; at the same time, for wearable devices, the portability and comfort of the device will also affect the user experience.

[0003] The existing Chinese patent CN112163564A discloses a fall prediction method based on human key point behavior recognition and LSTM, including: using an RGB camera to collect RGB images of human behavior; converting the RGB images into infrared images; performing preliminary clarity processing on the infrared images; extracting active human images; based on the adjacent human key point grouping method, dividing the human body into three areas of head, torso and legs for behavior recognition; using LSTM, that is, long short-term memory neural network mechanism to realize the memory function of the collected video, thereby realizing the analysis and recognition function of human behavior changes, and finally classifying the recognition results into three categories: fall, non-fall and others. However, the above patent compares the current state of the guardian with the model library, uses the fall model library to analyze a series of video frames before, during and after the fall, and judges the fall of the human body according to the speed of the fall action and the positional relationship of the key points of the human body to realize the fall prediction function. This solution relies on a large, accurate and diverse model library. These model libraries need to contain various action sequences before, during and after the fall. If the data in the model library is not extensive enough or the annotations are not accurate, it will lead to misjudgment or missed judgment; at the same time, the fall prediction model is sensitive to different environmental factors (such as light changes, background interference, etc.), which will also affect the detection accuracy of key points, and thus affect the accuracy of fall judgment; secondly, the movement speed of the human body and the position of key points may vary from person to person. The biomechanical differences between different individuals cause the same key points and speed patterns to show different fall risks in different individuals.

[0004] Therefore, how to improve the accuracy of human fall detection is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a human fall detection method, device and equipment based on key point sequence analysis, so as to solve the problem of low accuracy of human fall detection in the prior art.

[0006] The technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention provides a human fall detection method based on key point sequence analysis, the method comprising:

[0008] Acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images;

[0009] Perform human body key point detection and filtering on each frame of real-time image, and output the target position information of human body key points;

[0010] Performing motion analysis on the target object according to the target position information, and outputting the motion analysis result;

[0011] According to the preset fall detection conditions and the motion analysis results, when it is determined that the target object has fallen, a safety reminder is issued.

[0012] Preferably, the performing of human body key point detection and filtering processing on each frame of real-time image and outputting target key point position information comprises:

[0013] Input each frame of real-time image into a pre-trained key point detection model, and output the first position information of the key points of the human body;

[0014] Performing spatial filtering on the first position information to determine second position information of the blocked key point;

[0015] Perform time series filtering on the first position information and the second position information, and output the target position information.

[0016] Preferably, performing spatial filtering on the first position information to determine the second position information of the blocked key point includes:

[0017] Detecting target key points connected to the blocked key points according to the first position information, and obtaining the number of target key points;

[0018] If the number of target key points is 1, or the number of target key points is 2 but there are target key points that have not been successfully detected, the position information corresponding to the blocked key points in the previous frame of real-time image is used as the second position information;

[0019] If the number of target key points is 2 and all target key points are successfully detected, the median coordinates of the two target key points are used as the second position information.

[0020] Preferably, performing time series filtering on the first position information and the second position information and outputting the target position information comprises:

[0021] Determine key point position sequence information in time sequence according to the first position information and the second position information;

[0022] According to a preset sliding window size, the key point position sequence information is subjected to sliding median filtering, and the target position information after filtering is output.

[0023] Preferably, performing motion analysis on the target object based on the target position information and outputting the motion analysis result includes:

[0024] According to the target position information, obtain reference target position information and analysis target position information, wherein the reference target includes at least ankle key points, and the analysis target includes at least head key points;

[0025] Determining relative position information between the reference target and the analysis target based on the reference target position information and the analysis target position information;

[0026] Based on the relative position information, relative motion parameters are determined as the motion analysis result, wherein the relative motion parameters at least include: relative displacement, relative motion speed and relative motion acceleration.

[0027] Preferably, when it is determined that the target object falls, issuing a safety reminder based on the preset fall detection condition and the motion analysis result includes:

[0028] Acquire preset assessment thresholds related to human body falls, wherein the assessment thresholds at least include a displacement threshold, a velocity threshold, and an acceleration threshold;

[0029] According to the motion analysis result and the evaluation threshold, when the motion analysis result meets the fall detection condition, it is determined that the target object has fallen.

[0030] Preferably, the step of determining that the target object has fallen when the motion analysis result meets the fall detection condition based on the motion analysis result and the evaluation threshold comprises:

[0031] Acquiring the motion analysis result, wherein the motion analysis result at least includes: relative displacement, relative motion speed and relative motion acceleration;

[0032] The relative displacement is compared with the displacement threshold, and if the relative displacement is less than the displacement threshold, the relative motion speed is compared with the speed threshold, or the relative motion acceleration is compared with the acceleration threshold;

[0033] If the relative movement speed is greater than the speed threshold or the relative movement acceleration is greater than the acceleration threshold, it is determined that the target object has fallen.

[0034] In a second aspect, the present invention provides a human fall detection device based on key point sequence analysis, the device comprising:

[0035] A real-time image acquisition module is used to acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images;

[0036] The key point detection and filtering processing module is used to detect and filter the key points of the human body on each frame of real-time image and output the target position information of the key points of the human body;

[0037] A motion analysis module, used to perform motion analysis on the target object according to the target position information, and output a motion analysis result;

[0038] The fall detection module is used to issue a safety reminder when it is determined that the target object has fallen based on the preset fall detection conditions and the motion analysis results.

[0039] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0040] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0041] In summary, the beneficial effects of the present invention are as follows:

[0042] The present invention provides a human fall detection method, device and equipment based on key point sequence analysis. The method comprises: obtaining a real-time video stream collected in a target object monitoring scene, decomposing the real-time video stream into multiple frames of real-time images; performing human key point detection and filtering processing on each frame of real-time image, and outputting target position information of the human key points; performing motion analysis on the target object based on the target position information, and outputting the motion analysis result; and issuing a safety reminder when it is determined that the target object has fallen based on preset fall detection conditions and the motion analysis result. The present invention not only focuses on the key point position information within a single frame, but also considers the changing trend of the key points over time. This comprehensive analysis method can capture the dynamic information of the human body more comprehensively, especially fall detection, which relies on the sensitive capture of subtle changes in the motion process. By performing human key point detection and filtering processing on each frame of real-time image, the rapid changes in the key point positions of the human body before and after the fall can be accurately identified, thereby greatly improving the accuracy of fall detection; the introduced filtering method effectively solves the problem of inaccurate key point identification caused by occlusion or other factors, and ensures the continuity and reliability of the data. This processing method not only improves the accuracy of single-frame data, but also smoothes the key point motion trajectory through time series analysis, reduces noise and errors, and provides a more stable and accurate data basis for subsequent fall analysis; by performing motion analysis on the target object, the specific movements and position changes of the human body during the fall process can be more sensitively captured. This judgment logic based on relative position changes and change rates can accurately determine fall events in many cases, especially in complex action sequences, and can effectively distinguish falls from other similar movements. Therefore, the present invention can highly accurately and sensitively identify and distinguish a human fall from other actions, thereby improving the accuracy and reliability of fall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0044] Figure 1 Schematic diagram of the overall working process of the human fall detection method based on key point sequence analysis in Example 1 of the present invention;

[0045] Figure 2 This is a schematic diagram of the process of performing human body key point detection and filtering processing on each frame of real-time image in Embodiment 1 of the present invention;

[0046] Figure 3 This is a schematic diagram of a process of performing spatial filtering on the first position information in Embodiment 1 of the present invention;

[0047] Figure 4 This is a schematic diagram of a process of performing time series filtering on the first position information and the second position information in Embodiment 1 of the present invention;

[0048] Figure 5 is a schematic diagram of a process of performing motion analysis on the target object in Embodiment 1 of the present invention;

[0049] Figure 6 is a schematic diagram of a process for determining that a target object has fallen according to a preset fall detection condition and the motion analysis result in Embodiment 1 of the present invention;

[0050] Figure 7 is a schematic diagram of a process for determining that a target object has fallen according to the motion analysis result and the evaluation threshold in Embodiment 1 of the present invention;

[0051] Figure 8 is a structural block diagram of a human fall detection device based on key point sequence analysis in Example 2 of the present invention;

[0052] Fig. 9 It is a schematic diagram of the structure of the electronic device in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the phrase "comprising..." do not exclude the existence of other identical elements in the process, method, article or device comprising the elements. If there is no conflict, the embodiments of the present invention and the various features in the embodiments can be combined with each other, all within the protection scope of the present invention.

[0054] Example 1

[0055] See also Figure 1 Embodiment 1 of the present invention discloses a human fall detection method based on key point sequence analysis, the method comprising:

[0056] Acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images;

[0057] Specifically, a monocular camera is used to capture real-time video streams in monitoring scenarios. The monocular camera is installed at an inclined angle or a top view angle, which can optimize the field of view coverage, thereby improving the monitoring effect on the environment and target objects; the inclined angle installation helps to capture more scene details and dynamic changes, while the top view angle helps to reduce line of sight occlusion and distortion, and decompose the real-time video stream into multiple frames of real-time images, so that each frame of real-time image can more clearly and completely display the scene and human body movements. By decomposing the real-time video stream frame by frame, each change of the movement can be continuously captured, providing a rich and detailed image data foundation for subsequent motion analysis, behavior recognition and emergency event detection, such as falls.

[0058] Perform human body key point detection and filtering on each frame of real-time image, and output the target position information of human body key points;

[0059] Specifically, human key point detection and filtering are performed on each frame of real-time image extracted from the real-time video stream. First, computer vision and machine learning techniques are used to identify the key structural points of the human body in the image, such as the head, shoulders, arms, legs, etc. After key point detection, in order to improve the accuracy and robustness of the data, the detected key point information will be filtered. The filtering processing method includes spatial filtering and temporal filtering to remove noise and smooth data to deal with errors caused by factors such as image quality, illumination changes or occlusion. After filtering, the target position information of the human key point is output, which provides a reliable foundation for subsequent dynamic analysis, such as motion tracking, behavior recognition and specific events, such as fall detection. In this way, key events in the monitoring scene can be monitored and responded to more sensitively and accurately.

[0060] In one embodiment, see Figure 2 , the human body key point detection and filtering process are performed on each frame of real-time image, and the target position information of the human body key point is outputted, including:

[0061] Input each frame of real-time image into a pre-trained key point detection model, and output the first position information of the key points of the human body;

[0062] Specifically, each frame of real-time image is input into a pre-trained YoloV8s-Pose key point detection model, which is trained based on the COCO pedestrian key point dataset. The COCO pedestrian key point dataset contains a large number of multi-frame human images in different postures, and marks 17 key points of each frame of human image, including: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle. These key points cover the main joints of the human body and provide a comprehensive foundation for motion analysis. By processing real-time images with the pre-trained YoloV8s-Pose key point detection model, the key point position information of the human body can be output quickly and accurately, thereby outputting the first position information of the key points of the human body; the YoloV8s-Pose model uses a deep learning algorithm to optimize the detection speed and accuracy, so that when it is used in a real-time video surveillance system, it can effectively process continuous video streams and maintain a high recognition accuracy even when pedestrians move quickly or the scenes are complex. In addition, since YoloV8s-Pose is optimized based on the YOLO architecture, it can maintain high detection accuracy while meeting the needs of real-time processing, which is very critical for real-time monitoring and timely response.

[0063] Performing spatial filtering on the first position information to determine second position information of the blocked key point;

[0064] Specifically, in the key point detection process, it is often encountered that some key points cannot be directly observed due to partial occlusion of the human body or camera angle problems. In order to solve this problem, the first position information detected initially, that is, the key point position directly identified from the image, is subjected to spatial filtering processing. This step uses a filtering algorithm, which includes Kalman filtering, particle filtering and spatial filtering, to estimate and correct the positions of key points that are occluded or cannot be accurately detected due to other reasons. The spatial filtering processing selected here not only relies on the data of a single frame, but also takes into account the movement and spatial relationship of the key points in continuous frames, thereby inferring the positions of these occluded key points as the second position information, thereby improving the accuracy and robustness of the overall key point detection. Such processing can more accurately capture and analyze human body dynamics, and can work effectively even when the key points are partially occluded.

[0065] In one embodiment, see Figure 3 , performing spatial filtering on the first position information to determine the second position information of the blocked key point includes:

[0066] Detecting target key points connected to the blocked key points according to the first position information, and obtaining the number of target key points;

[0067] Specifically, the first position information is spatially filtered. The spatial filtering scheme is intended to improve the accuracy and robustness of key point detection, especially when the key points cannot be directly detected due to occlusion or other visual obstacles. Based on the first position information, the target key points connected to the occluded key points are detected to obtain the number of target key points.

[0068] If the number of target key points is 1, or the number of target key points is 2 but there are target key points that have not been successfully detected, the position information corresponding to the blocked key points in the previous frame of real-time image is used as the second position information;

[0069] Specifically, first, based on the number of target key points, if an occluded key point is only connected to one target key point and the occluded key point is not detected in the current frame, the position of the occluded key point in the previous frame will be automatically used as the position of the current frame. This is because human body movements do not change much in a short period of time, and the data of the previous frame can be used as a reasonable estimate of this frame.

[0070] If the number of target key points is 2 and all target key points are successfully detected, the median coordinates of the two target key points are used as the second position information.

[0071] Specifically, if an occluded keypoint is connected to two target keypoints, and both target keypoints are effectively detected in the current frame, the position of the occluded keypoint will be replaced by the median coordinates of the two detected keypoints. This method takes advantage of the relative position relationship between human keypoints and provides a reasonable position estimate through geometric averaging. However, if at least one of the two target keypoints is not detected, the occluded keypoint will also fall back to using the position of the occluded keypoint in the previous frame. This filtering strategy continues to effectively track keypoints when the keypoints are partially occluded or temporarily disappear from sight. Through such processing, the impact of occlusion and detection failure on keypoint tracking accuracy is reduced, ensuring data continuity and reliability of overall analysis, which is crucial for dynamic behavior analysis and further applications such as fall detection.

[0072] In one embodiment, in the step of inputting each frame of real-time image into a pre-trained key point detection model, after outputting the first position information of the key point of the human body, the method further includes:

[0073] Determine the initial sequence of key point time-series positions according to the first position information corresponding to each frame of real-time image;

[0074] Specifically, according to the first position information (x, y coordinates) of the key points of the human body detected in each frame of real-time image, the detected first position information is arranged in chronological order to form a time series, namely, the initial sequence of the key point temporal positions.

[0075] Using Kalman filtering and Gaussian filtering to smooth the key point time series position sequence, and output a key point time series position smooth sequence;

[0076] Specifically, Kalman filtering and Gaussian filtering are used to smooth the initial sequence of key point temporal positions. The Kalman filter eliminates noise in the detection process and provides a smooth trajectory through iterative prediction and update steps. The Gaussian filter further eliminates sharp fluctuations by performing convolution processing on the key point temporal positions. The result of the smoothing process is a smooth sequence of key point temporal positions, which ensures the continuity and stability of key point position changes.

[0077] According to the smoothed sequence of the key point temporal positions, and according to the first position information corresponding to the real-time images of adjacent frames, the displacement of the key point of the human body is calculated;

[0078] Specifically, the displacement of each key point between adjacent frames is calculated based on the key point temporal position smoothing sequence and the first position information in the adjacent frame real-time image. The displacement can be obtained by calculating the position change of the same key point in two adjacent frames.

[0079] Calculate the speed of key points of the human body according to the time interval between adjacent frames of real-time images and the displacement;

[0080] Specifically, the speed of each key point is calculated according to the time interval (Δt) between adjacent frames of real-time images and the calculated displacement. Through this calculation, the movement speed of the key point in the time series is obtained.

[0081] Calculating the acceleration of key points of the human body according to the speed and the time interval;

[0082] Specifically, according to the speed of the key point, the speed difference between two adjacent frames is calculated, and the acceleration of the key point is calculated by combining the time interval between adjacent frames. Through this calculation, the motion acceleration of the key point in the time series is obtained.

[0083] Using a pre-trained motion pattern recognition algorithm, the displacement, velocity and acceleration are classified and labeled to determine the motion trajectory of key points of the human body;

[0084] Specifically, the displacement, velocity and acceleration of each key point are classified and labeled using pre-trained motion pattern recognition algorithms. These algorithms can identify different motion patterns (such as stillness, uniform motion, accelerated motion, etc.) and determine the motion trajectory of the key points of the human body based on these patterns. The identified motion trajectory provides the motion status and change trend of the key points in different time periods.

[0085] According to the motion trajectory, a key point position prediction model is established;

[0086] Specifically, a key point position prediction model is established based on the identified motion trajectory. The model can be trained using a machine learning algorithm (such as LSTM, RNN) or a physical model (such as a kinematics-based prediction model). The model inputs the historical motion data of the key points and outputs the position prediction of the future key points.

[0087] Using the key point prediction and analysis model, predict and analyze the position information of the occluded key points, and output multiple prediction and analysis results;

[0088] Specifically, the established key point position prediction model is used to predict and analyze the position information of the occluded key points. The model predicts the position of the key points during the occlusion period based on the motion trajectory before the occlusion, and generates multiple predicted position results through multiple prediction analyses.

[0089] A weighted average calculation is performed on each prediction analysis result to determine the second position information of the blocked key point.

[0090] Specifically, a weighted average calculation is performed on the results of each prediction analysis to determine the final position information of the occluded key point. The weighted average sets the weight by considering the confidence of the prediction result or the historical prediction error. The final determined second position information provides the position estimate of the key point during the occlusion, ensuring the continuity and accuracy of the motion trajectory.

[0091] Perform time series filtering on the first position information and the second position information, and output the target position information.

[0092] Specifically, the first position information and the second position information determined by spatial filtering will be input into the timing filter for further processing. The timing filtering process takes into account the movement and position change trend of the key points in continuous frames. By using a timing filtering algorithm, the timing filtering algorithm includes Kalman filtering and sliding window averaging. These position information are smoothed and corrected to reduce the position instability caused by noise, jitter or accidental errors. The stability and continuity of the key point position information can be improved through timing filtering, making subsequent motion analysis and behavior recognition more accurate and reliable.

[0093] In one embodiment, see Figure 4 , performing time series filtering on the first position information and the second position information, and outputting the target position information comprises:

[0094] Determine key point position sequence information in time sequence according to the first position information and the second position information;

[0095] Specifically, first, it is necessary to obtain a preset research cycle. For example, every 11 frames of real-time images are selected as a research cycle. For each key point, a key point sequence is formed. For example, K1 represents the sequence of the first key point, which contains the position information of this key point in 11 consecutive frames of images, that is, p1, p2,..., p11; for the 17 human key points detected above, the position information of each key point in 11 consecutive frames of images is formed: K1(p1, p2,..., p11), K2(p1, p2,..., p11),...K17(p1, p2,..., p11).

[0096] According to a preset sliding window size, the key point position sequence information is subjected to sliding median filtering, and the target position information after filtering is output.

[0097] Specifically, after selecting the research period, the preset sliding window size is obtained next. For example, the length of the preset sliding window is also 11, which is the same as the length of the research period. Then, the key point position sequence corresponding to each key point is subjected to median filtering. Median filtering is a common filtering technique. By taking the median of the data in the window to smooth the data, noise and outliers can be effectively removed. When new key point coordinates arrive in the sliding window, they will first be pre-processed by spatial filtering to process the occluded or lost key point information. Then, the new key point data after spatial filtering is added to the respective key point sequences, and the sequence data is updated to ensure that the sequence contains the latest 11 frames of key point information. The updated key point sequence is used to perform median filtering again. The purpose of this step is to smooth the mutations and noise of the key points during the movement process. In this way, the stability and continuity of the key point position information can be further improved, making the subsequent motion analysis and behavior recognition more accurate and reliable. In summary, this time series filtering process combines the selection of the study period, the application of median filtering, and the processing of new key point coordinates to ensure that the key point sequence contains the latest information, and smoothes the key point data through filtering processing, thereby improving the accuracy and stability of the key point position information.

[0098] Performing motion analysis on the target object according to the target position information, and outputting the motion analysis result;

[0099] Specifically, based on the obtained target position information, the target object is subjected to motion analysis, which includes analyzing the motion trajectory, speed, acceleration, etc. of the target object, and determining the motion direction, speed and motion trajectory of the target object by comparing the key point position changes between different frames, and then identifying various behaviors, such as walking, running, turning, etc. This motion analysis result can be provided to the monitoring system or other applications for further behavior recognition, event detection or evaluation of the state of the target object, thereby realizing real-time monitoring and analysis of the target object's behavior.

[0100] In one embodiment, see Figure 5 , performing motion analysis on the target object based on the target position information, and outputting the motion analysis result includes:

[0101] According to the target position information, obtain reference target position information and analysis target position information, wherein the reference target includes at least ankle key points, and the analysis target includes at least head key points;

[0102] Specifically, based on the target position information, when performing motion analysis on the target object, it is first necessary to determine the reference target and the analysis target, and select several key points at the bottom of the human body, such as the two ankle points, as reference targets. This is because the ankle is usually a relatively stable part of the human body during movement, and it can also remain relatively stable during a fall, and can provide a relatively fixed reference point. The position vectors of the two ankle points are combined into a reference target set R={r1,r2,...,rn}, where ri represents the position vector of the i-th reference target, which helps to establish a stable benchmark in subsequent analysis.

[0103] Determining relative position information between the reference target and the analysis target based on the reference target position information and the analysis target position information;

[0104] Specifically, other key points besides the reference target, such as head key points, are selected as analysis targets, and the head key points include at least nose, left eye, right eye, left ear and right ear key points. This is because the head key points change dramatically during a fall. The analysis target set is set to A={a1,a2,...,am}, where aj represents the position vector of the jth analysis target. The analysis target set A contains the position vectors of the head key points, and the movement of the target object is understood by analyzing the head key points. The relative position vector ΔPij between the reference target and the analysis target is calculated, where ΔPij=aj−ri is the relative position vector obtained by subtracting the position vector of the reference target from the position vector of the analysis target. The purpose of this is to obtain the position change information of the analysis target relative to the reference target, which is crucial for understanding the relative position and movement trend of the target object.

[0105] Based on the relative position information, relative motion parameters are determined as the motion analysis result, wherein the relative motion parameters at least include: relative displacement, relative motion speed and relative motion acceleration.

[0106] Specifically, by analyzing the changing trend of the relative position vector over time, the change rate Δvij of the relative position vector between consecutive frames is calculated. This change rate reflects the movement speed of the analysis target relative to the reference target, and then the relative movement acceleration is calculated based on the movement speed. By accurately quantifying the motion state of the target object, including the changes in its speed and acceleration, the motion behavior of the target object can be better understood. In general, by selecting appropriate reference targets and analysis targets, and analyzing the relative position and motion changes between them, the motion of the target object can be fully understood, providing an important data basis for subsequent fall behavior recognition and fall event detection.

[0107] According to the preset fall detection conditions and the motion analysis results, when it is determined that the target object has fallen, a safety reminder is issued.

[0108] Specifically, based on the preset fall detection conditions and motion analysis results, the target object is judged to have fallen, wherein the preset fall detection conditions include threshold settings for key point positions, speeds, and acceleration indicators, or recognition models for fall features; when it is detected that the key point positions, speeds, or acceleration indicators of the target object meet the fall conditions, a fall judgment will be triggered and a safety reminder will be issued. This reminder can be conveyed to the guardian or system administrator through a sound alarm, mobile phone notification, or other means, so that necessary rescue measures can be taken in a timely manner to ensure the safety of the target object. Such a fall detection system can provide timely warnings at critical moments, help reduce the risk of injury caused by falls, and improve safety in monitoring scenarios.

[0109] In one embodiment, see Figure 6 , when it is determined that the target object has fallen, issuing a safety reminder based on the preset fall detection condition and the motion analysis result includes:

[0110] Acquire preset assessment thresholds related to human body falls, wherein the assessment thresholds at least include a displacement threshold, a velocity threshold, and an acceleration threshold;

[0111] Specifically, a preset evaluation threshold value related to human body falling is obtained, wherein the evaluation threshold value includes at least a displacement threshold value, a speed threshold value and an acceleration threshold value. The evaluation threshold value is obtained based on the understanding and analysis of human body falling behavior and the study of kinematic characteristics. The displacement threshold value represents the displacement of the human body in the horizontal direction, the speed threshold value represents the movement speed of the human body in the vertical direction, and the acceleration threshold value represents the movement acceleration of the human body in the vertical direction. By setting these evaluation threshold values, the motion state of the human body is evaluated according to the motion data obtained in real time. In the threshold conditions for fall judgment, the minimum distance threshold value θd, the maximum motion speed threshold value θv and the maximum acceleration threshold value θa relative to the reference target are included. The minimum distance threshold value θd is used to ensure that the distance between the analysis target and the reference target is not too large, thereby eliminating the misjudgment effect of certain non-falling actions on the system. The maximum motion speed threshold value θv and the maximum acceleration threshold value θa are used to limit the motion speed and acceleration of the analysis target. Exceeding these thresholds indicates that the target object is in a high-speed motion state and there is a possibility of falling.

[0112] In one embodiment, before obtaining a preset assessment threshold value related to a human body fall, the method includes:

[0113] Obtaining preset initial threshold and monitoring object information;

[0114] Specifically, a set of preset initial thresholds is obtained. The initial thresholds are set based on a large amount of human fall behavior data collected. The initial thresholds include displacement thresholds, speed thresholds, and acceleration thresholds, which reflect the range of motion changes under normal circumstances. These preset thresholds are used as a basic standard to preliminarily evaluate the motion state of the monitored object. Next, basic information about the monitored object is collected, including age, weight, height, and health status, such as whether there are chronic diseases and whether the movement is inconvenient. This information is obtained through health questionnaires filled out by users or from medical records and stored in user files. In this way, the individual characteristics of each monitored object are fully understood, providing necessary data support for subsequent threshold adjustments.

[0115] According to the monitored object information, the initial threshold is adjusted for a second time to determine the evaluation threshold.

[0116] Specifically, after obtaining the basic information of the monitored object, the preset initial threshold is adjusted for the second time based on this information to determine a more personalized evaluation threshold. For example, for users with heavier weight, considering that they will produce larger changes in movement speed and acceleration when falling, the speed threshold and acceleration threshold will be increased accordingly, that is, the speed threshold and the acceleration threshold are positively correlated with the user's weight. For users with taller heights, the displacement threshold is adjusted to adapt to the larger displacement involved in their movement and falling, that is, the displacement threshold is positively correlated with the user's height. At the same time, if the user has health conditions such as mobility difficulties or chronic diseases, the threshold standard is lowered accordingly to more sensitively detect abnormal movement conditions, that is, the displacement threshold, speed threshold and acceleration threshold are all positively correlated with the user's health. The smaller the user's health, the more unhealthy the user, and the corresponding threshold is smaller. The larger the user's health, the healthier the user, and the corresponding threshold is larger. By comprehensively considering these factors, the initial threshold is adjusted to an evaluation threshold that is more suitable for each individual, and the evaluation threshold is finally determined. The evaluation threshold includes at least a displacement threshold, a velocity threshold, and an acceleration threshold. These thresholds are applied in the real-time monitoring process to accurately identify and evaluate the motion status of the monitored object and provide more reliable security protection and monitoring services.

[0117] According to the motion analysis result and the evaluation threshold, when the motion analysis result meets the fall detection condition, it is determined that the target object has fallen.

[0118] Specifically, the motion analysis results, including the relative position, movement speed and acceleration of the analysis target relative to the reference target, are used to compare these motion analysis results with pre-set fall detection conditions. If the motion analysis results meet the fall detection conditions, that is, its relative position, speed or acceleration and other indicators exceed the preset threshold range, then it is determined that the target object has fallen. Such a judgment is based on the understanding of kinematic characteristics and falling behavior. By combining motion analysis and threshold setting, the fall of the target object can be accurately detected in real-time monitoring, and corresponding safety measures can be taken in time to ensure the health and safety of the target object.

[0119] In one embodiment, see Figure 7 , the step of determining that the target object has fallen when the motion analysis result meets the fall detection condition based on the motion analysis result and the evaluation threshold comprises:

[0120] Acquiring the motion analysis result, wherein the motion analysis result at least includes: relative displacement, relative motion speed and relative motion acceleration;

[0121] The relative displacement is compared with the displacement threshold, and if the relative displacement is less than the displacement threshold, the relative motion speed is compared with the speed threshold, or the relative motion acceleration is compared with the acceleration threshold;

[0122] If the relative movement speed is greater than the speed threshold or the relative movement acceleration is greater than the acceleration threshold, it is determined that the target object has fallen.

[0123] Specifically, in any continuous frame, if there is at least one analysis target whose relative position relative to the reference target is less than the preset minimum distance threshold θd, this may mean that the analysis target is very close to or even close to the reference target, which usually occurs when a person falls. Then, the condition further stipulates that if the absolute value of the change rate ΔVij of the relative position exceeds the preset maximum motion speed threshold θv, or the acceleration exceeds the preset maximum acceleration threshold θa, then it is determined that a fall event has occurred. The condition that there is at least one analysis target whose relative position relative to the reference target is less than θd indicates that the relative position distance between the analysis target and the reference target is too close, which may mean that the analysis target is close to the ground, that is, the human body may be in a state of falling or falling; the condition that the absolute value of the change rate ΔVij exceeds θv or the acceleration exceeds θa further emphasizes the suddenness and drastic nature of the action. When the relative position of the analysis target changes drastically, the absolute value of its change rate ΔVij will increase, indicating a rapid change in motion speed, and the acceleration exceeding θa indicates that the acceleration of the motion is abnormally large, because the target object is experiencing a rapid and abnormal motion state, such as a sharp change in speed and acceleration during a fall. In general, this judgment condition combines relative position, movement speed, acceleration and other indicators to determine whether the target object has fallen in a comprehensive way. When these conditions are met at the same time, it will be judged as a fall event, and corresponding safety reminders or rescue measures can be triggered to ensure the safety of the target object.

[0124] Example 2

[0125] See also Figure 8 Embodiment 2 of the present invention further provides a human fall detection device based on key point sequence analysis, the device comprising:

[0126] A real-time image acquisition module is used to acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images;

[0127] The key point detection and filtering processing module is used to detect and filter the key points of the human body on each frame of real-time image and output the target position information of the key points of the human body;

[0128] A motion analysis module, used to perform motion analysis on the target object according to the target position information, and output a motion analysis result;

[0129] The fall detection module is used to issue a safety reminder when it is determined that the target object has fallen based on the preset fall detection conditions and the motion analysis results.

[0130] Specifically, a human fall detection device based on key point sequence analysis provided by an embodiment of the present invention is adopted, and the device includes: a real-time image acquisition module, which is used to acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images; a key point detection and filtering processing module, which is used to perform human key point detection and filtering processing on each frame of real-time image, and output target position information of the human key point; a motion analysis module, which is used to perform motion analysis on the target object according to the target position information, and output the motion analysis result; a fall detection module, which is used to issue a safety reminder when it is determined that the target object has fallen according to a preset fall detection condition and the motion analysis result. The device not only focuses on the key point position information within a single frame, but also considers the changing trend of the key points over time. This comprehensive analysis method can capture the dynamic information of the human body more comprehensively, especially fall detection, which relies on the sensitive capture of subtle changes in the motion process. By performing human key point detection and filtering processing on each frame of real-time image, the rapid changes in the key point positions of the human body before and after the fall can be accurately identified, thereby greatly improving the accuracy of fall detection; the introduced filtering method effectively solves the problem of inaccurate key point identification caused by occlusion or other factors, and ensures the continuity and reliability of the data. This processing method not only improves the accuracy of single-frame data, but also smoothes the key point motion trajectory through time series analysis, reduces noise and errors, and provides a more stable and accurate data basis for subsequent fall analysis; by performing motion analysis on the target object, the specific movements and position changes of the human body during the fall process can be more sensitively captured. This judgment logic based on relative position changes and change rates can accurately determine fall events in many cases, especially in complex action sequences, and can effectively distinguish falls from other similar movements. Therefore, the device can highly accurately and sensitively identify and distinguish human falls from other actions, thereby improving the accuracy and reliability of fall detection.

[0131] Example 3

[0132] In addition, combined Figure 1 The human fall detection method based on key point sequence analysis of Embodiment 1 of the present invention described above can be implemented by an electronic device. Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in Embodiment 3 of the present invention is shown.

[0133] The electronic device may include a processor and a memory storing computer program instructions.

[0134] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0135] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0136] The processor implements any one of the human fall detection methods based on key point sequence analysis in the above embodiments by reading and executing computer program instructions stored in the memory.

[0137] In one example, the electronic device may further include a communication interface and a bus. Fig. 9 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0138] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0139] The bus includes hardware, software or both, and the components of the device are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention contemplates any suitable bus or interconnection.

[0140] Example 4

[0141] In addition, in combination with the human fall detection method based on key point sequence analysis in the above embodiment 1, embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any human fall detection method based on key point sequence analysis in the above embodiments is implemented.

[0142] In summary, the embodiments of the present invention provide a method, device and apparatus for human fall detection based on key point sequence analysis.

[0143] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0144] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0145] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant location, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0146] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0147] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A human fall detection method based on key point sequence analysis, It is characterized in that The method comprises: Acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images; Perform human body key point detection and filtering on each frame of real-time image, and output the target position information of human body key points; Performing motion analysis on the target object according to the target position information, and outputting the motion analysis result; According to the preset fall detection conditions and the motion analysis results, when it is determined that the target object has fallen, a safety reminder is issued; The human body key point detection and filtering process are performed on each frame of real-time image, and the target position information of the human body key point is outputted, including: Input each frame of real-time image into a pre-trained key point detection model, and output the first position information of the key points of the human body; Performing spatial filtering on the first position information to determine second position information of the blocked key point; Performing time series filtering processing on the first position information and the second position information, and outputting the target position information; The performing spatial filtering on the first position information to determine the second position information of the blocked key point comprises: Detecting target key points connected to the blocked key points according to the first position information, and obtaining the number of target key points; If the number of target key points is 1, or the number of target key points is 2 but there are target key points that have not been successfully detected, the position information corresponding to the blocked key points in the previous frame of real-time image is used as the second position information; If the number of target key points is 2 and all target key points are successfully detected, the median coordinates of the two target key points are used as the second position information; The performing time series filtering on the first position information and the second position information and outputting the target position information comprises: Determining a key point position sequence in time sequence according to the first position information and the second position information; According to a preset sliding window size, the key point position sequence is subjected to sliding median filtering, and the target position information after filtering is output; according to the target position information, the target object is subjected to motion analysis, and the output of the motion analysis result includes: According to the target position information, obtain reference target position information and analysis target position information, wherein the reference target includes at least ankle key points, and the analysis target includes at least head key points; Determining relative position information between the reference target and the analysis target based on the reference target position information and the analysis target position information; Based on the relative position information, relative motion parameters are determined as the motion analysis result, wherein the relative motion parameters at least include: relative displacement, relative motion speed and relative motion acceleration.

2. The human fall detection method based on key point sequence analysis according to claim 1, It is characterized in that The step of issuing a safety reminder when it is determined that the target object has fallen according to the preset fall detection condition and the motion analysis result includes: Acquire preset assessment thresholds related to human body falls, wherein the assessment thresholds at least include a displacement threshold, a velocity threshold, and an acceleration threshold; According to the motion analysis result and the evaluation threshold, when the motion analysis result meets the fall detection condition, it is determined that the target object has fallen.

3. The human fall detection method based on key point sequence analysis according to claim 2, It is characterized in that The determining that the target object has fallen when the motion analysis result meets the fall detection condition based on the motion analysis result and the evaluation threshold comprises: Acquiring the motion analysis result, wherein the motion analysis result at least includes: relative displacement, relative motion speed and relative motion acceleration; The relative displacement is compared with the displacement threshold, and if the relative displacement is less than the displacement threshold, the relative motion speed is compared with the speed threshold, or the relative motion acceleration is compared with the acceleration threshold; If the relative movement speed is greater than the speed threshold or the relative movement acceleration is greater than the acceleration threshold, it is determined that the target object has fallen.

4. A human fall detection device based on key point sequence analysis, It is characterized in that The device comprises: A real-time image acquisition module is used to acquire a real-time video stream collected in a target object monitoring scene, and decompose the real-time video stream into multiple frames of real-time images; The key point detection and filtering processing module is used to detect and filter the key points of the human body on each frame of real-time image and output the target position information of the key points of the human body; A motion analysis module, used to perform motion analysis on the target object according to the target position information, and output a motion analysis result; A fall detection module, configured to issue a safety reminder when it is determined that the target object has fallen, based on preset fall detection conditions and the motion analysis results; The human body key point detection and filtering process are performed on each frame of real-time image, and the target position information of the human body key point is outputted, including: Input each frame of real-time image into a pre-trained key point detection model, and output the first position information of the key points of the human body; Performing spatial filtering on the first position information to determine second position information of the blocked key point; Performing time series filtering processing on the first position information and the second position information, and outputting the target position information; The performing spatial filtering on the first position information to determine the second position information of the blocked key point comprises: Detecting target key points connected to the blocked key points according to the first position information, and obtaining the number of target key points; If the number of target key points is 1, or the number of target key points is 2 but there are target key points that have not been successfully detected, the position information corresponding to the blocked key points in the previous frame of real-time image is used as the second position information; If the number of target key points is 2 and all target key points are successfully detected, the median coordinates of the two target key points are used as the second position information; The performing time series filtering on the first position information and the second position information and outputting the target position information comprises: Determining a key point position sequence in time sequence according to the first position information and the second position information; According to a preset sliding window size, the key point position sequence is subjected to sliding median filtering, and the target position information after filtering is output; The performing motion analysis on the target object according to the target position information and outputting the motion analysis result comprises: According to the target position information, obtain reference target position information and analysis target position information, wherein the reference target includes at least ankle key points, and the analysis target includes at least head key points; Determining relative position information between the reference target and the analysis target based on the reference target position information and the analysis target position information; Based on the relative position information, relative motion parameters are determined as the motion analysis result, wherein the relative motion parameters at least include: relative displacement, relative motion speed and relative motion acceleration.

5. An electronic device, It is characterized in that include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 3.

6. A storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Citation Information

Patent Citations

  • Tumble pre-judgment method based on human body key point behavior recognition and LSTM

    CN112163564A

  • Fall risk prediction method for old people

    CN114694252A

  • Human body posture tracking method based on millimeter wave radar

    CN115345906A

  • Falling action recognition method and system based on key points of human skeleton

    CN116721470A

  • Fall accurate monitoring method and system, electronic equipment and storage medium

    CN117831072A

Cited By

  • Fall detection method and related equipment

    CN122067316A

  • Fall detection method and related equipment

    CN122067317A