Fall detection and early warning system for the elderly based on multi-modal data fusion

Abstract

The present application relates to the technical field of intelligent fall detection, in particular to an old people fall detection and early warning system based on multi-modal data fusion, which comprises a dynamic weight fusion unit, a precursor triggering unit, a multi-level decision unit and a verification feedback unit, wherein the dynamic weight fusion unit collects three-axis acceleration, surface electromyography signal and millimeter wave radar data, realizes multi-source data correlation analysis through a bivariate decay model and dynamic threshold calibration, the precursor triggering unit activates electromyography high-frequency sampling and joint three-dimensional space angle monitoring when the exercise intensity is abnormal, captures the fall precursor in advance, the multi-level decision unit constructs a fall probability algorithm based on a random forest model, carries out three-level verification in combination with acceleration standard deviation, ground contact point density and historical pattern matching, realizes accurate discrimination of fall events, provides a highly reliable intelligent solution for old people safety protection, reduces fall injury risk and improves emergency response efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent fall detection, in particular to an elderly fall detection and early warning system based on multi-modal data fusion. BACKGROUND

[0002] Intelligent fall detection is an important technology. In the context of an aging population, elderly falls are one of the main risk factors leading to injury and death. Real-time monitoring of motion state through wearable sensors and environmental perception devices, as well as accurate identification of fall events, are of great significance in reducing fall injury rates and improving emergency response efficiency.

[0003] Prior art uses a single acceleration sensor or camera to detect falls, but is limited by the single dimension of the data, making it difficult to balance detection accuracy and practicality in complex scenarios. However, traditional fall detection systems lack depth in multi-modal data fusion. Existing solutions often rely on single-dimensional data to determine fall events, ignoring the internal relationships between motion intensity, joint angle, muscle activation state, and other multi-source data. When the elderly bend down to pick up an object or quickly sit down, the acceleration signal may be highly similar to that when falling. Relying solely on single data can easily lead to false positives. In dimly lit environments or when wearing mobility aids, lack of coordinated analysis of spatial displacement rate and muscle electrical activity can further reduce the recognition accuracy of abnormal movements. In addition, the detection threshold of traditional systems is usually fixed and cannot be dynamically adapted to different gait symmetry and activity intensity of the elderly, resulting in a lag in response to unexpected collisions or muscle fatigue-induced balance decline. This one-sidedness of data fusion and rigidity of model adaptation directly leads to the dual dilemma of high false positive rate in simple scenarios and high missed detection rate in complex scenarios for traditional systems. To solve this technical problem, we provide an elderly fall detection and early warning system based on multi-modal data fusion. SUMMARY

[0004] The purpose of the present application is to provide an elderly fall detection and early warning system based on multi-modal data fusion to solve the problems raised in the background.

[0005] 1. Because the traditional system has a single data dimension, it is prone to false positives due to similar actions. Therefore, the present case uses a dynamic weight fusion unit to collect three-axis acceleration and surface electromyography signals, calculates the limb movement intensity index, and fuses millimeter wave radar data to perform multi-modal data correlation analysis, reduce false positives, and improve detection accuracy.

[0006] 2. Since the threshold of the traditional system is fixed, it cannot adapt to individual differences and complex scenes. Therefore, the present case sets a dynamic threshold through a precursor triggering unit, triggers high-frequency sampling in combination with the derivative of vertical displacement rate and the three-dimensional space angle, can dynamically adjust according to individual activity characteristics, accurately captures the precursor of falling, and improves the detection reliability in complex scenes.

[0007] To achieve the above purpose, a fall detection and warning system for the elderly based on multi-modal data fusion is provided, comprising;

[0008] The dynamic weight fusion unit collects the three-axis acceleration characteristic value and surface electromyogram energy value of the wearable sensor, calculates the limb movement intensity index based on the three-axis acceleration characteristic value, triggers the dynamic adjustment of the environmental sensor weight when the limb movement intensity index is greater than the movement intensity index threshold, and calculates the environmental sensor weight using an exponential decay function. The environmental sensor data only contains the vertical displacement rate of the millimeter wave radar;

[0009] The precursor triggering unit activates the 200Hz high-frequency sampling mode of the electromyogram signal when the limb movement intensity index is greater than 1.5 times the preset threshold and the derivative of the vertical displacement rate is greater than 3m / s³, and captures the three-dimensional space angle of the hip joint and the knee joint through the millimeter wave radar;

[0010] The multi-level decision unit constructs a fall probability model according to the three-dimensional space angle and the surface electromyogram energy value, extracts the standard deviation of the acceleration vector in the first 0.5 second window when the fall probability output by the fall probability model is greater than the fall probability threshold, and if the standard deviation is greater than 15m / s² and the three-dimensional space angle is greater than 75° for two consecutive frames, a three-level warning signal is generated and sent to the verification feedback unit. Extract the millimeter wave radar point cloud data in the last 3 seconds before falling, and calculate the ground contact point density. When the ground contact point density is greater than 50 points / cm² and the duration exceeds 0.2 seconds, an effective fall event is confirmed, otherwise return to the dynamic weight fusion unit to recalibrate the sensor weight.

[0011] As a further improvement of the technical solution, the dynamic adjustment method of the environmental sensor weight in the dynamic weight fusion unit comprises:

[0012] A double-variable decay model based on the user's real-time activity intensity is established, wherein the decay rate is positively correlated with the limb movement intensity index and negatively correlated with the ambient light intensity;

[0013] When the smart walking aid device is detected to be worn by the user, the weight compensation mechanism is activated, so that the minimum value of the environmental sensor weight is not less than 0.4, and the historical activity mode matching algorithm is introduced. When the current movement intensity deviates from the user's regular activity mode by more than 30%, the weight decay coefficient is automatically reset.

[0014] As a further improvement of the technical solution, the dynamic calibration of the limb movement intensity index threshold in the dynamic weight fusion unit comprises:

[0015] The baseline threshold is generated by integrating the user's biological characteristic parameters, including the body mass index, gait symmetry, and sit-to-walk test time. The threshold is corrected in real time by the step frequency coefficient of variation collected by the inertial sensor, with a correction amplitude of 5% threshold increase per 0.1Hz variation. When three threshold breakthrough events are detected consecutively, the threshold learning mode is triggered, and the threshold curve is refitted based on the previous 24 hours of activity data.

[0016] As a further improvement of the technical solution, the processing method of the surface electromyography signal in the dynamic weight fusion unit is:

[0017] First stage: use complex wavelet transform to extract 80-200Hz frequency band energy, and dynamically suppress high-frequency noise components;

[0018] Second stage: establish a muscle synergy model to screen 4 groups of muscle group signals related to lower limb stability;

[0019] Third stage: dynamically compensate the energy value based on the muscle fatigue index, and the compensation coefficient increases linearly with the continuous activity time.

[0020] As a further improvement of the technical solution, the calculation method of the three-dimensional space included angle in the precursor triggering unit comprises:

[0021] A hip-knee-ankle joint chain motion model is constructed, and an inverse kinematics algorithm is used to solve the joint rotation plane. A gravity vector compensation mechanism is introduced. When the knee joint rotation angular velocity is detected to exceed 45 degrees per second, a 15-degree compensation angle is automatically added. A joint motion trajectory prediction model is established. When the actual measurement value deviates from the predicted path by more than 20%, data re-sampling is triggered.

[0022] As a further improvement of the technical solution, the parameter dynamic adjustment method of the fall probability model in the multi-level decision unit is:

[0023] The weight value of the three-dimensional space included angle parameter is dynamically generated by a random forest model. The input features include the current time, ambient light intensity, and user step frequency;

[0024] When it is detected that the user is in a rehabilitation training mode, the weight value of the surface electromyography signal energy value parameter is increased, and a forgetting factor is introduced to update the parameter online.

[0025] As a further improvement of the technical solution, the calculation method of the acceleration standard deviation in the multi-level decision unit comprises:

[0026] The improved robust standard deviation algorithm is used to calculate the acceleration standard deviation, and the abnormal frame data is dynamically removed, if the acceleration value of a frame exceeds 3 times the acceleration standard deviation, it is marked as invalid, when the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit triggers to collect data again.

[0027] As a further improvement of the technical solution, the determination method of the three-dimensional space included angle in the multi-level decision unit comprises:

[0028] After the millimeter wave radar point cloud is matched by the bone topology, a hip-knee-ankle joint motion chain model is constructed, and the joint included angle is calculated in real time based on the inverse kinematics algorithm, when it is detected that the bending direction of the knee joint is greater than 30° with the gravity vector, the included angle compensation mechanism is started, and the three-dimensional space included angle is obtained.

[0029] As a further improvement of the technical solution, the verification method of the ground contact point density in the multi-level decision unit comprises:

[0030] After the point cloud data is fitted by the ground plane, it is projected onto a two-dimensional grid map, and morphological opening operation is used to eliminate isolated noise points, and the point density in the effective contact area is counted, when it is detected that the contact area presents asymmetric distribution, it is determined as a non-fall scene, wherein the asymmetric distribution is that the aspect ratio is greater than 2.

[0031] As a further improvement of the technical solution, after generating the three-level early warning signal, the following enhanced verification is performed in the multi-level decision unit:

[0032] The historical fall database is called for dynamic time warping matching, the similarity between the current acceleration sequence and the typical fall mode is calculated, if the similarity is greater than 85% and the electromyographic silence period of the surface electromyogram is less than 50ms, the verification feedback unit is directly triggered to trigger the final alarm, and when the verification fails, the motion intensity index threshold of the dynamic weight fusion unit is automatically adjusted to 120% of the original value.

[0033] Compared with the prior art, the beneficial effects of the present application are:

[0034] The dynamic weight fusion unit realizes the deep correlation of multi-modal data and the individual threshold adaptation through the multi-source data acquisition and dynamic weight adjustment method, improves the data reliability and scene adaptability, solves the problem of single data and rigid threshold of the traditional system, the precursor triggering unit captures the precursor characteristics before falling through high-frequency electromyographic sampling and joint motion monitoring, shortens the detection delay, improves the abnormal action recognition accuracy in complex scenes, solves the problem of precursor early warning lag of the traditional system, the multi-level decision unit realizes intelligent discrimination and rapid response of the fall event through the construction of the probability model and the multi-level verification mechanism, reduces the false alarm rate and improves the processing efficiency in emergency scenes, solves the problems of low detection accuracy and slow response of the traditional system. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a whole block diagram of the present application.

[0036] The meanings of various labels in the figure are as follows:

[0037] 1, dynamic weight fusion unit; 2, precursor triggering unit; 3, multi-stage decision unit; 4, verification feedback unit. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0039] Under the background of an aging society, the falling events of the elderly occur frequently. The traditional falling detection system is difficult to meet the actual needs due to the problems of high false alarm rate and poor environmental adaptability. Taking a typical scene as an example: an elderly person with Parkinson's disease suddenly loses balance due to muscle stiffness when walking in the living room. At this time, the system needs to quickly distinguish between "normal sitting" and "accidental falling". The dynamic weight fusion unit 1 of the present application realizes the dynamic matching of environmental perception weight and body state through multi-modal data adaptive fusion and physiological parameter coupling decision for the first time. The implementation manner thereof is described in detail as follows:

[0040] The present application provides an elderly falling detection and early warning system based on multi-modal data fusion. Referring to FIG. 1, the system comprises: Figure 1

[0041] The dynamic weight fusion unit 1 collects three-axis acceleration characteristic values and surface electromyogram energy values of a wearable sensor. The processing method of the surface electromyogram is as follows:

[0042] First stage: a complex wavelet transform is used to extract the energy in the 80-200Hz frequency band, and the 500Hz high-frequency noise component is dynamically suppressed.

[0043] Second stage: a muscle synergy model is established, and the signals of 4 muscle groups related to lower limb stability are screened, which are rectus femoris, vastus lateralis, gluteus maximus and gastrocnemius.

[0044] Third stage: based on the muscle fatigue index The energy value is dynamically compensated, and the new surface electromyogram value is The compensation coefficient increases linearly with the continuous activity time.

[0045] ​Three-axis acceleration and surface electromyography signal are core data reflecting human motion intensity and muscle state, which need to be collected in real time through wearable devices. The three-axis accelerometer is integrated in the waist sensor, with a sampling rate of 100 Hz, a range of ±16 g, and a resolution of 0.001 g, which is used to capture the translational and rotational acceleration of the human body. The surface electromyography sensor is attached to the lower limb muscle groups of the quadriceps femoris and gastrocnemius, with a sampling rate of 1000 Hz, and uses a differential amplification circuit to suppress common-mode noise. The root mean square value of the electromyography signal is extracted as the surface electromyography signal energy value. The limb motion intensity index is calculated based on the three-axis acceleration characteristic value. The three-axis acceleration characteristic needs to be converted into a unified motion intensity index for cross-scene comparison. The calculation method is as follows:

[0046] Calculate the acceleration vector module ; wherein, is the three-axis acceleration value, and the acceleration mean value is calculated using a 1-second window , standard deviation , and a feature vector is constructed , then the limb motion intensity index formula is ; wherein, is the limb motion intensity index, and the weight is determined by historical fall data training. A fixed threshold value cannot adapt to individual differences and needs to be dynamically adjusted in combination with biological characteristics. The dynamic calibration of the limb motion intensity index threshold value includes:

[0047] A baseline threshold value is generated by integrating user biological characteristic parameters, including body mass index, gait symmetry, and sit-to-walk test time, i.e. ; wherein, is the body mass index, is the gait symmetry score, and the higher the score, the more robust the gait, is the sit-to-walk test time, which reflects the lower limb strength. The threshold value is corrected in real time by the step frequency coefficient of variation collected by the inertial sensor, with a correction range of 5% threshold value per 0.1 Hz variation. The real-time correction formula is ; wherein, is the step frequency coefficient of variation, which is the relative deviation of the current step frequency from the average step frequency. When three threshold value breakthrough events are detected in succession, the threshold value learning mode is triggered. Based on the previous 24 hours of activity data, the threshold value curve is refitted. When the limb motion intensity index is greater than the motion intensity index threshold value, the dynamic adjustment of the environmental sensor weight is triggered. The importance of environmental perception (such as millimeter wave radar) needs to be dynamically matched with the muscle state to avoid false positives in static scenes. An exponential decay function is used to calculate the weight of the environmental sensor. The environmental sensor data only includes the vertical displacement rate of the millimeter wave radar.

[0048] The dynamic adjustment method of the environmental sensor weight includes:

[0049] Higher exercise intensity makes muscle state data (such as acceleration and electromyography) more critical for fall detection, requiring a reduction in the weight of environmental sensors. Conversely, dimmer lighting makes environmental perception (such as millimeter-wave radar) more important, necessitating an increase in its weight to avoid missed detections. A bivariate attenuation model based on the user's real-time activity intensity is established, with the model formula defined as follows: ;in, for The weighting of environmental sensors at any given time; the environmental sensors are millimeter-wave radars. The baseline weight represents the fundamental importance of environmental sensors in static scenarios. The exercise intensity decay coefficient controls the rate at which the weights decay as exercise intensity increases. This is the light compensation coefficient, which reflects the effect of light intensity on the weighting. for Motion intensity index at any given time. for Ambient light intensity is acquired synchronously every 100ms. and Substitute into the formula to update the weights, when At that time, forced removal To prevent environmental data from being completely ignored in extreme scenarios, the attenuation rate is positively correlated with the limb movement intensity index and negatively correlated with ambient light intensity. When the system detects that a user is wearing a smart walking aid, a weight compensation mechanism is activated. This involves scanning for nearby devices via Bluetooth Low Energy and matching the walker's preset MAC address. If a device signal is detected after three consecutive scans with a 5-second interval between each scan, it is determined to be in "wearing mode," triggering the compensation mechanism to ensure that the minimum weight value of the environmental sensor is no less than 0.4. A historical activity pattern matching algorithm is also incorporated. When the current movement intensity deviates from the user's regular activity pattern by more than 30%, the weight attenuation coefficient is automatically reset. The system also collects the user's MSI data from the past 7 days to calculate the average movement intensity. and standard deviation And define the regular activity range as Perform real-time deviation calculation ;when If the duration exceeds 5 minutes, it is judged as an "abnormal mode," the exercise intensity decay coefficient is reset to the historical average (calculated based on data from the past 30 days, with a default value of 0.15), and a 2-hour mode learning cycle is initiated to recalculate the average exercise intensity. and standard deviation The highest priority is the wearing of walking aids (forced weight ≥ 0.4), and the second priority is abnormal movement mode (reset attenuation coefficient). The basic adjustment is a bivariate attenuation model. When the wearing of walking aids and abnormal movement mode occur at the same time, the weight forced compensation is performed first, and then the attenuation coefficient is reset, so as to reduce the logical error rate of weight decision.

[0050] A single motion intensity anomaly can be caused by daily activities such as climbing stairs, and the derivative of the vertical displacement rate (acceleration change rate) is needed to distinguish between intense actions and regular movements to reduce false triggering. The precursor triggering unit 2 activates the 200Hz high-frequency sampling mode of the electromyographic signal when the limb motion intensity index is greater than 1.5 times the preset threshold and the derivative of the vertical displacement rate is greater than 3m / s³, wherein the dynamic generation formula of the preset threshold is , is the baseline motion intensity threshold, and the derivative of the vertical displacement rate is calculated as ; wherein is the vertical displacement measured by the millimeter wave radar, is the vertical displacement rate at the moment, which is calculated by the time difference of the radar point cloud data, seconds, when and , it is determined as a potential fall precursor, triggering subsequent high-frequency sampling and joint monitoring, effectively distinguishing between sudden acceleration before falling and acceleration change of regular movement. Abnormal activation or instantaneous relaxation (electromyographic silent period) of muscles before falling may be missed by regular sampling rates (such as 100Hz), so the sampling rate needs to be increased to capture details. Immediately after triggering, switch to 200Hz high-frequency mode and continue to collect until the event ends (such as when the motion intensity returns to below the baseline threshold). Characteristic changes in joint angles occur during a fall (such as excessive flexion of the knee joint or rapid extension of the hip joint). The three-dimensional point cloud data of the millimeter wave radar can accurately solve the joint motion trajectory. The three-dimensional space angle between the hip joint and the knee joint is captured by the millimeter wave radar. The calculation method of the three-dimensional space angle includes:

[0051] A hip-knee-ankle joint chain motion model is constructed to decompose complex whole-body motion into joint-level angle changes for accurate calculation. The hip joint is defined as the origin of the world coordinate system , the knee joint is defined as , and the ankle joint is defined as . The thigh vector is defined as , and the lower leg vector is defined as . Assuming that the thigh and lower leg are rigid bodies, the slight influence of muscle deformation on the position of the joint center point is ignored to simplify the kinematics calculation. Forward kinematics derives the end position from the joint angle, while inverse kinematics can deduce the joint angle from the end trajectory, which is suitable for solving the joint angle from the radar point cloud data (including the position of the limb end). The inverse kinematics algorithm is used to solve the joint rotation plane, and the three-dimensional coordinate sequence of the ankle joint (limb end) is extracted from the millimeter wave radar point cloud data , then the inverse kinematics equation is , ; wherein is the hip joint rotation angle, The knee joint angle, The distance from the hip joint to the ankle joint is calculated by the vector modulus formula, and the Newton-Raphson iteration method is used to solve the nonlinear equation, with a convergence threshold of Radian, to ensure the accuracy of angle calculation. Gravity will have a static effect on joint movement (e.g. the knee naturally flexes under gravity), so the gravity component needs to be compensated to obtain the true movement angle. Introduce a gravity vector compensation mechanism, with a gravity acceleration of Decomposed into three axes of the joint coordinate system, the gravity component is When the knee joint rotation angular velocity is detected to be greater than 45 degrees per second, it is determined to be dynamic movement, and a 15-degree compensation angle is automatically added. A joint movement trajectory prediction model is established. Radar point cloud may have abnormal values due to occlusion or noise. Establishing a prediction model can check the rationality of the data in real time and avoid false calculations. A second-order linear predictor is used to predict the current angle based on the previous two frames of joint angles and When the actual measurement value deviates from the predicted path by more than 20%, the data is re-sampled, effectively reducing false positives caused by radar noise.

[0052] The multi-level decision unit 3 constructs a fall probability model based on the three-dimensional space angle and surface electromyography signal energy value. The three-dimensional space angle and electromyography energy value are the core features of falling, and the construction of the probability model can quantify the possibility of falling, avoiding the arbitrariness of a single threshold. Input features: three-dimensional joint angle (hip / knee joint), surface electromyography signal energy value, and acceleration vector modulus mean / standard deviation. Output label: fall event (1) / non-fall event (0). Based on 1000 fall and 2000 non-fall samples labeled clinically, a random forest model is used, containing 50 decision trees. The input feature importance ranking is: knee joint angle (35%) > electromyography (30%) > acceleration standard deviation (25%). The optimal threshold is determined by the Youden index to be 0.65, i.e. when the model output probability is > 0.65, it is determined to be a suspected fall, effectively distinguishing between falls and similar actions.

[0053] The determination method of the three-dimensional space angle includes:

[0054] After the millimeter wave radar point cloud is topologically matched, a hip-knee-ankle joint movement chain model is constructed. Radar point cloud needs to be converted into human skeletal structure to calculate joint angles. Topological matching is a key step in connecting sensor data and human models. Straight-through filtering is used to remove outliers, such as points with a distance greater than 5 meters. Voxel grid downsampling is used to reduce data density, with a voxel size of 1 cm³. Based on a pre-trained human pose estimation model, the key point coordinates of the hip joint, knee joint, and ankle joint are identified in the point cloud , and​ , meet the real-time detection requirements, improve the efficiency of the matching algorithm, thigh vector , shank vector is , then the joint angle geometric calculation is ; is the knee joint three-dimensional space angle, is the vector dot product, reflecting the consistency of the direction of the two vectors, is the vector length, representing the physical length of the thigh and shank, accurately reflecting the full range of motion of the knee joint from extension (180°) to flexion (30°), based on the inverse kinematics algorithm to calculate the joint angle in real time, given the position of the limb end (ankle joint), inverse kinematics can quickly back-propagate the joint angle, suitable for end trajectory data provided by radar point cloud, extract the real-time coordinate sequence of the ankle joint from the radar point cloud , as the input constraint of inverse kinematics, the pseudo-inverse Jacobian matrix method is used to iteratively update the hip and knee angles, the objective function is to minimize the end position error, ; wherein, is the pseudo-inverse of the Jacobian matrix, used to map the end error to the joint space, is the current frame ankle target coordinate, is the model predicted coordinate, compared with forward kinematics, the tracking error of the inverse calculation on the end trajectory is reduced, gravity will cause the knee joint to naturally bend (about 170° when standing), it is necessary to distinguish between passive bending caused by gravity and active severe bending when falling, by measuring the component of gravitational acceleration in the sensor coordinate system , construct the gravity vector , calculate the knee bending direction vector (pointing from the thigh to the shank) and the angle between the gravity vector ; wherein, is the angle between the bending direction and the gravity vector, ranging from 0° to 180°, to accurately identify the bending dominated by gravity and the abnormal bending when falling, when the knee bending direction and the gravity vector are detected to be greater than 30°, it indicates that there is a severe motion (such as rapid flexion when falling) caused by non-gravitational factors, then the joint angle compensation mechanism needs to be started, the compensation angle calculation is to eliminate the interference of gravity, and the three-dimensional space angle is obtained, when and the knee angular velocity , it is determined as a dynamic abnormal bending, and the compensation is started , wherein , is the compensated joint angle, is the compensation amount, the sign is determined by the consistency of the bending direction and the gravity direction, the compensation amount increases linearly with the increase of the angular velocity, the maximum compensation is 30°, after compensation, the detection error of the knee angle in the falling scene is reduced.

[0055] The parameter dynamic adjustment method of the fall probability model is:

[0056] The importance of joint angle to fall detection changes over time, environment and individual motion state, and the weight needs to be dynamically allocated to adapt to the scene difference. The weight value of the three-dimensional space angle parameter is dynamically generated through a random forest model. The input features include the current time, ambient light intensity and user step frequency. The random forest model architecture contains 100 decision trees, the input feature dimension is 3, and the output is the weight value of the three-dimensional space angle parameter. The training data is 200 fall and non-fall samples collected at different times, light and step frequency. The label is whether a fall occurs. Real-time weights are generated. The model updates the weights every 5 minutes to adapt to changes in light and fluctuations in step frequency. After the weight is dynamically adjusted, the night fall detection accuracy is improved.

[0057] Muscle activation patterns in rehabilitation training are different from daily activities, and electromyography can more sensitively reflect limb control ability. The weight of electromyography energy value parameter needs to be increased to avoid missing abnormal movements in training. When the user is detected in the rehabilitation training mode, the weight value of the surface electromyography energy value parameter is increased, and a forgetting factor is introduced for online updating of the parameter. The user's motor ability may change over time, and the forgetting factor needs to be used to reduce the influence of old data and improve the model's adaptability to the latest state. An incremental random forest algorithm is used, and the model is updated every time new data arrives. The training samples of the last 7 days are retained, and the older data is discarded. The forgetting factor is defined as ; wherein, is the timestamp of the data sample, controls the forgetting speed of old data, and model retraining is automatically performed every morning. The tree model is updated using weighted samples with a forgetting factor, focusing on the fall event features in recent data.

[0058] When the fall probability output by the fall probability model is greater than the fall probability threshold, the standard deviation of the acceleration vector in the previous 0.5 second window is extracted, wherein the calculation method of the standard deviation of the acceleration vector includes:

[0059] The acceleration standard deviation is calculated by using an improved robust standard deviation algorithm, and abnormal frame data is dynamically excluded. Single-frame abnormal acceleration may be caused by sensor false touch or environmental interference, which needs to be dynamically identified and excluded to avoid polluting the statistical results. The 50 frames of data in the window are compared one by one, and if the acceleration value of a certain frame exceeds 3 times the acceleration standard deviation, it is marked as invalid. When the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit 1 triggers to re-collect data, improves the data re-collection mechanism, and improves the reliability of the detection result. If the standard deviation of the acceleration vector is greater than 15 m / s² and the continuous 2 frames of three-dimensional space angle is greater than 75°, a three-level warning signal is generated and sent to the verification feedback unit 4. After generating a three-level warning signal in the multi-level decision unit 3, the following enhanced verification is performed:

[0060] More than 1000 clinically labeled fall event data are stored, including acceleration sequence and electromyographic silent period duration. Key features are extracted for each fall event, including acceleration peak, impact duration, and electromyographic silent period start time. The acceleration sequence is normalized to eliminate the influence of individual motion intensity differences. The historical fall database is called to perform dynamic time warping matching. The similarity between the current acceleration sequence and the typical fall pattern is calculated. The acceleration sequence of the last 2 seconds before the current three-level warning trigger is extracted. The reference sequence of the same type of fall pattern is selected in the historical database. The optimal time warping path is found by the dynamic time warping algorithm to minimize the cumulative distance. The cumulative distance minimum value is substituted into the similarity calculation formula to calculate the similarity. The envelope detection is performed on the surface electromyographic signal. The duration of continuous low threshold is calculated. The threshold is dynamically set to 20% of the current signal mean value to adapt to the electromyographic intensity differences of different individuals. If the similarity is >85% and the electromyographic silent period of the surface electromyographic signal is <50ms, it is determined as a high confidence fall event, then the final alarm is triggered directly by skipping the verification feedback unit 4. When the verification fails, the motion intensity index threshold of the dynamic weight fusion unit 1 is automatically adjusted to 120% of the original value. The threshold adjustment lasts for 30 minutes. If the warning is triggered again during this period, the original threshold is restored and the manual review process is triggered to reduce the false positive rate after verification failure.

[0061] The original point cloud contains noise and non-ground points, which need to be preprocessed to accurately extract the ground contact area. The millimeter wave radar point cloud data of the last 3 seconds before falling is extracted, and the ground contact point density is calculated. When the ground contact point density is greater than 50 points / cm² and the duration is more than 0.2 seconds, an effective fall event is confirmed. Otherwise, return to the dynamic weight fusion unit 1 to recalibrate the sensor weight.

[0062] The verification method of the ground contact point density in the multi-level decision unit 3 includes:

[0063] The point cloud data is projected to a two-dimensional grid map after ground plane fitting, and the morphological opening operation is used to eliminate isolated noise points. The three-dimensional point cloud is projected to a two-dimensional plane, which is convenient for statistical contact area geometric characteristics. The ground point cloud is projected to an XY plane and divided into 1cm*1cm grid units. The density value of each grid is the number of point clouds in the unit, forming a density matrix. A 3*3 structure element is used for opening operation to improve the smoothness of the contact area boundary. The point density in the effective contact area is counted. When the contact area presents an asymmetric distribution, it is determined as a non-fall scene. The asymmetric distribution is that the aspect ratio is greater than 2. The real fall large-area contact and the accidental collision small-area contact need to be distinguished. The contact strength is quantified by a density threshold. The matrix after morphological processing is labeled for connected regions. All connected regions are extracted. The small area with an area less than 10cm2 is excluded. The point density of each effective region is calculated. The point cloud data within 3 seconds is continuously monitored. If the ground contact point density of a region is greater than 50 points / cm2 and the duration exceeds 0.2 seconds, it is determined as an effective ground contact. The long axis and short axis of the minimum circumscribed rectangle of each effective region are calculated. When the aspect ratio of the long axis and the short axis is greater than 2, it is determined as an asymmetric distribution, which excludes the possibility of falling. Otherwise, it is retained as a suspected fall area. If a fall event is not confirmed, the front-end sensor data acquisition needs to be optimized through a feedback mechanism to avoid repeated misjudgment. When the verification fails, an instruction is sent to the dynamic weight fusion unit 1 to increase the millimeter wave radar weight by 0.2 (such as from 0.6 to 0.8) and reduce the motion intensity index threshold to 80% of the original value to improve sensitivity. The calibration lasts for 1 minute. If the contact area is detected again during this period, the symmetry analysis is skipped and the density verification is directly entered to shorten the response time.

[0064] In the present application, the dynamic weight fusion unit 1 collects three-axis acceleration, surface electromyography signal and millimeter wave radar data. The multi-source data correlation analysis is realized through the bivariate decay model and dynamic threshold calibration. The precursor trigger unit 2 activates the electromyography high-frequency sampling and joint three-dimensional space angle monitoring when the motion intensity is abnormal, to capture the fall precursor in advance. The multi-level decision unit 3 constructs a fall probability algorithm based on the random forest model, and performs three-level verification combined with acceleration standard deviation, ground contact point density and historical pattern matching, to realize accurate discrimination of fall events. The present application provides a highly reliable intelligent solution for the safety protection of the elderly, reduces the risk of fall injury, and improves the efficiency of emergency response.

[0065] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An elderly fall detection and warning system based on multi-modal data fusion, characterized in that, The application relates to a wearable fall detection system based on multi-sensor fusion technology, which comprises the following units: The dynamic weight fusion unit (1) collects three-axis acceleration characteristic values and surface electromyogram energy values of a wearable sensor, calculates a limb movement intensity index based on the three-axis acceleration characteristic values, triggers dynamic adjustment of the weight of an environmental sensor when the limb movement intensity index is greater than a movement intensity index threshold value, and calculates the weight of the environmental sensor by using an exponential decay function, wherein the environmental sensor data only contains a vertical displacement rate of a millimeter wave radar; The precursor triggering unit (2) activates a 200Hz high-frequency sampling mode of the electromyogram signal when the limb movement intensity index is greater than 1.5 times a preset threshold value and the derivative of the vertical displacement rate is greater than 3m / s3, and captures a three-dimensional space angle of a hip joint and a knee joint by using the millimeter wave radar; The multi-stage decision unit (3) constructs a fall probability model according to the three-dimensional space angle and the surface electromyogram energy value, extracts the standard deviation of an acceleration vector in a 0.5-second window when the fall probability output by the fall probability model is greater than a fall probability threshold value, generates a three-stage early warning signal if the standard deviation is greater than 15m / s2 and the three-dimensional space angle is greater than 75 degrees for two consecutive frames, and sends the three-stage early warning signal to the verification feedback unit (4), extracts millimeter wave radar point cloud data in the last 3 seconds before falling, and calculates a ground contact point density, wherein a valid fall event is confirmed when the ground contact point density is greater than 50 points / cm2 and the duration is longer than 0.2 seconds, and the sensor weight is recalibrated by the dynamic weight fusion unit (1) if the fall event is not valid.

2. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The dynamic adjustment method of the environmental sensor weight in the dynamic weight fusion unit (1) comprises the following steps: A double-variable decay model based on real-time activity intensity of a user is established, wherein the decay rate is positively correlated with the limb movement intensity index and is negatively correlated with the environmental light intensity; When the smart walking aid device worn by the user is detected, a weight compensation mechanism is activated, the minimum value of the environmental sensor weight is not less than 0.4, and a historical activity mode matching algorithm is introduced, so that the weight decay coefficient is automatically reset when the current movement intensity deviates from the regular activity mode of the user by more than 30%.

3. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The dynamic calibration of the limb movement intensity index threshold value in the dynamic weight fusion unit (1) comprises the following steps: A reference threshold value is generated by comprehensively considering biological characteristic parameters of a user, including a body mass index, gait symmetry and sitting-walking test time, the threshold value is corrected in real time by a step frequency variation coefficient collected by an inertial sensor, the correction amplitude is 5% threshold value per 0.1Hz variation, a threshold value learning mode is triggered when three threshold value breakthrough events are detected continuously, and a threshold value curve is refitted based on activity data in the last 24 hours.

4. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The processing method of the surface electromyogram signal in the dynamic weight fusion unit (1) comprises the following steps: In the first stage, a complex wavelet transform is used to extract energy in a 80-200Hz frequency band and dynamically suppress high-frequency noise components; In the second stage, a muscle synergy model is established, and signals of four muscle groups related to lower limb stability are screened; In the third stage, energy values are dynamically compensated based on a muscle fatigue index, and a compensation coefficient linearly increases with continuous activity time.

5. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The calculation method of the three-dimensional space angle in the precursor triggering unit (2) comprises the following steps: A hip-knee-ankle joint chain motion model is constructed, a reverse kinematics algorithm is used to solve the joint rotation plane, a gravity vector compensation mechanism is introduced, when the knee joint rotation angular velocity is detected to be greater than 45 degrees / second, a 15-degree compensation angle is automatically added, a joint motion trajectory prediction model is established, and when the actual measurement value deviates from the predicted path by more than 20%, data re-sampling is triggered.

6. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The parameter dynamic adjustment method of the fall probability model in the multi-level decision unit (3) is: The weight value of the three-dimensional space angle parameter is dynamically generated by a random forest model, and the input features include the current time, the ambient light intensity and the user's step frequency; When it is detected that the user is in a rehabilitation training mode, the weight value of the surface electromyogram energy value parameter is increased, and a forgetting factor is introduced to update the parameter online.

7. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The calculation method of the acceleration standard deviation in the multi-level decision unit (3) includes: An improved robust standard deviation algorithm is used to calculate the acceleration standard deviation, and abnormal frame data is dynamically excluded. If a certain frame acceleration value exceeds 3 times the acceleration standard deviation, it is marked as invalid. When the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit (1) is triggered to reacquire data.

8. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, The determination method of the three-dimensional space angle in the multi-level decision unit (3) includes: After the millimeter wave radar point cloud is matched by the skeleton topology, a hip-knee-ankle joint motion chain model is constructed, and the joint angle is calculated in real time based on the reverse kinematics algorithm. When it is detected that the knee joint bending direction and the gravity vector have an angle greater than 30°, the angle compensation mechanism is started, and the three-dimensional space angle is obtained. 9.The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1, wherein, The verification method of the ground contact point density in the multi-level decision unit (3) includes: After the point cloud data is fitted by the ground plane, it is projected onto a two-dimensional grid map, and morphological opening operation is used to eliminate isolated noise points. The point density in the effective contact area is counted. When it is detected that the contact area presents asymmetric distribution, it is determined as a non-fall scene, wherein the asymmetric distribution is that the aspect ratio is greater than 2.

10. The multi-modal data fusion based fall detection and warning system for the elderly as claimed in claim 1 wherein, After generating the three-level early warning signal, the following enhanced verification is performed in the multi-level decision unit (3): The historical fall database is called for dynamic time warping matching, the similarity between the current acceleration sequence and the typical fall mode is calculated, if the similarity is greater than 85% and the electromyogram silence period of the surface electromyogram is less than 50ms, the verification feedback unit (4) is directly triggered to trigger the final alarm, and when the verification fails, the motion intensity index threshold of the dynamic weight fusion unit (1) is automatically adjusted to 120% of the original value.

Images