Old people falling detection and early warning system based on multi-modal data fusion

Abstract

The invention relates to the technical field of intelligent fall detection, in particular to an old people fall detection and early warning system based on multi-modal data fusion, which comprises a dynamic weight fusion unit, a precursor trigger unit, a multi-level decision unit and a verification feedback unit, the dynamic weight fusion unit collects three-axis acceleration, surface electromyographic signals and millimeter wave radar data, multi-source data association analysis is achieved through a double-variable attenuation model and dynamic threshold calibration, and the precursor trigger unit activates electromyographic high-frequency sampling and joint three-dimensional space included angle monitoring when the exercise intensity is abnormal. And the multi-level decision-making unit constructs a tumble probability algorithm based on a random forest model, and performs three-level verification in combination with acceleration standard deviation, ground contact point density and historical mode matching, thereby realizing accurate discrimination of tumble events, providing a high-reliability intelligent solution for safety protection of old people, reducing the risk of tumble injury, and improving the safety of the old people. And the first-aid response efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fall detection, and in particular to a fall detection and early warning system for the elderly based on multimodal data fusion. Background Art

[0002] Intelligent fall detection is an important technology. Against the backdrop of an aging population, falls among the elderly are one of the main risk factors leading to injury and death. Using wearable sensors and environmental sensing devices to monitor movement status in real time and accurately identify fall events is of great significance in reducing the rate of fall injuries and improving the efficiency of emergency response.

[0003] Existing technologies use a single acceleration sensor or camera to detect falls, but are limited by the single data dimension, making it difficult to balance detection accuracy and practicality in complex scenarios. However, traditional fall detection systems have the problem of insufficient multimodal data fusion depth. Existing solutions often only judge fall events based on single-dimensional data, but ignore the inherent correlation between multi-source data such as movement intensity, joint angles, and muscle activation status. When the elderly bend over to pick up something or sit down quickly, the acceleration signal may be highly similar to the signal when they fall. Relying solely on a single data point can easily lead to false alarms, and in scenarios with dim ambient light or wearing walking aids, the lack of data can greatly reduce the risk of false alarms. The lack of coordinated analysis of spatial displacement rate and muscle electrical activity will further reduce the accuracy of abnormal movement recognition. In addition, the detection threshold of traditional systems is usually fixed and cannot be dynamically adapted to the different gait symmetry and activity intensity of the elderly, resulting in a delayed response to accidental collisions or decreased balance ability caused by muscle fatigue. This one-sidedness of data fusion and the rigidity of model adaptation directly lead to the traditional system facing the dual dilemma of "high false alarm rate in simple scenarios and high missed detection rate in complex scenarios" in practical applications. To solve this technical problem, we provide a fall detection and warning system for the elderly based on multimodal data fusion. Summary of the Invention

[0004] The purpose of the present invention is to provide a fall detection and warning system for the elderly based on multimodal data fusion to solve the problems raised in the above background technology.

[0005] 1. Because traditional systems have a single data dimension and are prone to false alarms due to movement similarity, this case uses a dynamic weight fusion unit to collect three-axis acceleration and surface electromyography signals, calculate the limb movement intensity index, and integrate it with millimeter-wave radar data. This enables multimodal data correlation analysis, reduces false alarm rates, and improves detection accuracy.

[0006] 2. Because traditional systems have fixed thresholds and cannot adapt to individual differences and complex scenarios, this case uses a precursor trigger unit to set a dynamic threshold. Combined with the vertical displacement rate derivative and the three-dimensional spatial angle to trigger high-frequency sampling, it can dynamically adjust according to individual activity characteristics, accurately capture precursors to falls, and improve detection reliability in complex scenarios.

[0007] To achieve the above objectives, a fall detection and warning system for the elderly based on multimodal data fusion is provided, including; The dynamic weight fusion unit collects the three-axis acceleration characteristic values ​​and surface electromyography signal energy values ​​of the wearable sensor and calculates the limb movement intensity index based on the three-axis acceleration characteristic values. When the limb movement intensity index exceeds the movement intensity index threshold, it triggers the dynamic adjustment of the environmental sensor weight and calculates the environmental sensor weight using an exponential decay function. The environmental sensor data only contains the vertical displacement rate of the millimeter wave radar. When the limb movement intensity index exceeds 1.5 times the preset threshold and the derivative of the vertical displacement rate exceeds 3m / s³, the precursor trigger unit activates the 200Hz high-frequency sampling mode of the electromyographic signal and uses millimeter-wave radar to capture the three-dimensional angle between the hip and knee joints. The multi-level decision-making unit constructs a fall probability model based on the three-dimensional spatial angle and the surface electromyography signal energy value. When the fall probability output by the fall probability model is greater than the fall probability threshold, the standard deviation of the acceleration vector in the previous 0.5-second window is extracted. If the standard deviation is greater than 15m / s² and the three-dimensional spatial angle of two consecutive frames is greater than 75°, a third-level warning signal is generated and sent to the verification feedback unit. The millimeter-wave radar point cloud data 3 seconds before the fall is extracted and the ground contact point density is calculated. When the ground contact point density is greater than 50 points / cm² and the duration exceeds 0.2 seconds, a valid fall event is confirmed. Otherwise, it returns to the dynamic weight fusion unit to recalibrate the sensor weight.

[0008] As a further improvement of the present technical solution, the method for dynamically adjusting the weights of the environmental sensors in the dynamic weight fusion unit includes: A bivariate decay model based on the user's real-time activity intensity was established, where the decay rate was positively correlated with the limb movement intensity index and negatively correlated with the ambient light intensity. When it is detected that the user is wearing an intelligent walking aid device, the weight compensation mechanism is activated to ensure that the minimum weight of the environmental sensor is not less than 0.4. The historical activity pattern matching algorithm is introduced. When the current exercise intensity deviates from the user's regular activity pattern by more than 30%, the weight attenuation coefficient is automatically reset.

[0009] As a further improvement of the present technical solution, the dynamic calibration of the limb movement intensity index threshold in the dynamic weight fusion unit includes: A baseline threshold is generated based on comprehensive user biometric parameters, including body mass index, gait symmetry, and sit-stand-walk test time. The threshold is corrected in real time using the coefficient of variation of cadence collected by the inertial sensor. The correction amplitude increases the threshold by 5% for every 0.1Hz variation. When three consecutive threshold breakthrough events are detected, the threshold learning mode is triggered and the threshold curve is refitted based on the previous 24 hours of activity data.

[0010] As a further improvement of the present technical solution, the surface electromyography signal processing method in the dynamic weight fusion unit is as follows: The first stage: complex wavelet transform is used to extract the energy of the 80-200 Hz frequency band and dynamically suppress the high-frequency noise component; Phase II: Establish a muscle synergy model and screen four muscle group signals related to lower limb stability; The third stage: Dynamically compensate the energy value based on the muscle fatigue index, and the compensation coefficient increases linearly with the continuous activity time.

[0011] As a further improvement of the present technical solution, the method for calculating the three-dimensional space angle in the precursor trigger unit includes: A hip-knee-ankle joint chain motion model was constructed, and the inverse kinematics algorithm was used to solve the joint rotation plane. A gravity vector compensation mechanism was introduced. When the knee joint rotation angular velocity was detected to exceed 45 degrees / second, a 15-degree compensation angle was automatically added. A joint motion trajectory prediction model was established, and data re-collection was triggered when the actual measurement value deviated from the predicted path by more than 20%.

[0012] As a further improvement of this technical solution, the method for dynamically adjusting the parameters of the fall probability model in the multi-level decision-making unit is as follows: The weights of the three-dimensional angle parameters are dynamically generated using a random forest model, with input features including the current time, ambient light intensity, and user cadence. When it is detected that the user is in rehabilitation training mode, the weight value of the surface electromyography signal energy value parameter is increased, and a forgetting factor is introduced to update the parameter online.

[0013] As a further improvement of this technical solution, the method for calculating the acceleration standard deviation in the multi-level decision-making unit includes: An improved robust standard deviation algorithm is used to calculate the acceleration standard deviation, and abnormal frame data is dynamically eliminated. If the acceleration value of a frame exceeds 3 times the acceleration standard deviation, it is marked as invalid. When the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit is triggered to re-collect data.

[0014] As a further improvement of the present technical solution, the method for determining the three-dimensional space angle in the multi-level decision unit includes: After the millimeter-wave radar point cloud is matched with the skeleton topology, a hip-knee-ankle joint motion chain model is constructed, and the joint angle is calculated in real time based on the inverse kinematics algorithm. When the angle between the knee joint bending direction and the gravity vector is detected to be greater than 30°, the angle compensation mechanism is activated to obtain the three-dimensional spatial angle.

[0015] As a further improvement of the present technical solution, the verification method of the ground contact point density in the multi-level decision-making unit includes: After fitting the point cloud data with the ground plane, it is projected onto a two-dimensional grid map. Morphological opening is used to eliminate isolated noise points, and the point density within the effective contact area is counted. When an asymmetric distribution of the contact area is detected, it is determined to be a non-fall scenario, where the asymmetric distribution means an aspect ratio greater than 2.

[0016] As a further improvement of this technical solution, the multi-level decision-making unit performs the following enhanced verification after generating the third-level warning signal: The historical fall database is called for dynamic time warping matching, and the similarity between the current acceleration sequence and the typical fall pattern is calculated. If the similarity is greater than 85% and the electromyographic silent period of the surface electromyographic signal is less than 50ms, the verification feedback unit is skipped and the final alarm is directly triggered. If the verification fails, the motion intensity index threshold of the dynamic weight fusion unit is automatically adjusted to 120% of the original value.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The dynamic weight fusion unit uses multi-source data collection and dynamic weight adjustment methods to achieve deep correlation of multimodal data and adaptation of individual thresholds, thereby improving data reliability and scenario adaptability, and solving the problems of single data and rigid thresholds in traditional systems. The precursor trigger unit uses high-frequency electromyography sampling and joint motion monitoring to capture fall precursor features in advance, shorten detection delays, and improve the accuracy of abnormal movement recognition in complex scenarios, solving the problem of delayed precursor warnings in traditional systems. The multi-level decision-making unit achieves intelligent identification and rapid response to fall events by constructing probability models and multi-level verification mechanisms, reducing false alarm rates and improving emergency scenario processing efficiency, solving the problems of low detection accuracy and slow response in traditional systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an overall block diagram of the present invention.

[0019] The meaning of each number in the figure is: 1. Dynamic weight fusion unit; 2. Precursor trigger unit; 3. Multi-level decision unit; 4. Verification feedback unit. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] In the context of an aging society, falls among the elderly are frequent. Traditional fall detection systems are unable to meet actual needs due to their high false alarm rate and poor environmental adaptability. Take a typical scenario as an example: an elderly person with Parkinson's disease suddenly loses balance due to muscle stiffness while walking in the living room. At this time, the system needs to quickly distinguish between "normal sitting down" and "accidental fall". The dynamic weight fusion unit 1 of the present invention realizes the dynamic matching of environmental perception weights and body status for the first time through adaptive fusion of multimodal data and coupled decision-making of physiological parameters. The following is a detailed description of its implementation method: The present invention provides a fall detection and warning system for the elderly based on multimodal data fusion, please refer to Figure 1 Shown, including: The dynamic weight fusion unit 1 collects the three-axis acceleration characteristic values ​​and surface electromyography signal energy values ​​of the wearable sensor, wherein the surface electromyography signal is processed as follows: The first stage: complex wavelet transform is used to extract the energy of the 80-200Hz frequency band and dynamically suppress the 500Hz high-frequency noise component; Phase II: Establish a muscle synergy model and screen four muscle group signals related to lower limb stability, namely the rectus femoris, vastus lateralis, gluteus maximus, and gastrocnemius; Phase 3: Based on Muscle Fatigue Index The energy value is dynamically compensated, and the new surface electromyography signal value is , the compensation coefficient increases linearly with the continuous activity time.

[0022] Three-axis acceleration and surface electromyography (EMG) signals are core data reflecting human motion intensity and muscle status. They must be collected in real time by wearable devices. The three-axis accelerometer is integrated into the waist sensor with a sampling rate of 100 Hz, a range of ±16 g, and a resolution of 0.001 g. It is used to capture the body's translational and rotational accelerations. The surface electromyography (EMG) sensors are attached to the quadriceps and gastrocnemius muscles of the lower limbs with a sampling rate of 1000 Hz. A differential amplifier circuit is used to suppress common-mode noise. The root mean square value of the electromyography signal is extracted as the surface electromyography signal energy value. The limb motion intensity index is calculated based on the three-axis acceleration characteristic values. The three-axis acceleration characteristics need to be converted into a unified motion intensity index to facilitate cross-scenario comparison. The calculation method is as follows: Calculate the acceleration vector modulus ;in, is the acceleration value of the three axes, and the acceleration mean is calculated using a 1-second window , standard deviation , construct the feature vector , then the formula for the limb movement intensity index is ;in, The limb movement intensity index is weighted and determined through training with historical fall data. Fixed thresholds cannot adapt to individual differences and need to be dynamically adjusted in combination with biometrics. The dynamic calibration of the limb movement intensity index threshold includes: The baseline threshold is generated by integrating the user's biometric parameters, including body mass index, gait symmetry and sit-stand-walk test time, i.e. ;in, is the body mass index, Scoring gait symmetry: the higher the score, the more stable the gait. The sit-stand-walk test time reflects the lower limb strength. The threshold is corrected in real time by the cadence variation coefficient collected by the inertial sensor. The correction amplitude is 5% of the threshold for every 0.1Hz variation. The real-time correction formula is: ;in, is the cadence variation coefficient, which is the relative deviation between the current cadence and the average cadence. When three threshold breakthrough events are detected consecutively, the threshold learning mode is triggered, and the threshold curve is refitted based on the activity data of the previous 24 hours. When the limb movement intensity index is greater than the movement intensity index threshold, the dynamic adjustment of the environmental sensor weight is triggered. The importance of environmental perception (such as millimeter-wave radar) needs to be dynamically matched with the body state to avoid false alarms in static scenes. The exponential decay function is used to calculate the environmental sensor weight. The environmental sensor data only contains the vertical displacement rate of the millimeter-wave radar.

[0023] The dynamic adjustment method of the environmental sensor weight includes: The higher the exercise intensity, the more critical the body state data (such as acceleration and electromyography) is for fall detection, and the weight of environmental sensors needs to be reduced. The darker the light, the more important environmental perception (such as millimeter wave radar) is, and its weight needs to be increased to avoid missed detection. A two-variable attenuation model based on the user's real-time activity intensity is established. The model formula is defined as ;in, for The weight of the environmental sensor at each moment. The environmental sensor is a millimeter-wave radar. is the baseline weight, which represents the basic importance of environmental sensors in static scenes. is the exercise intensity attenuation coefficient, which controls the attenuation rate of the weight as the exercise intensity increases. is the light compensation coefficient, which reflects the effect of light intensity on weight improvement. for Momentary limb movement intensity index, for Ambient light intensity at all times, acquired synchronously every 100ms and , substitute into the formula to update the weight, when When forced to take , to prevent environmental data from being completely ignored in extreme scenarios, where the attenuation rate is positively correlated with the limb movement intensity index and negatively correlated with the ambient light intensity. When it is detected that the user is wearing an intelligent walking aid, the weight compensation mechanism is activated, and the nearby devices are scanned through Bluetooth low energy to match the preset MAC address of the walker. When the device signal is detected for three consecutive scans with an interval of 5 seconds each time, it is determined to be "wearing state" and the compensation mechanism is triggered to make the minimum weight of the environmental sensor not less than 0.4. The historical activity pattern matching algorithm is introduced. When the current exercise intensity deviates from the user's regular activity pattern by more than 30%, the weight attenuation coefficient is automatically reset. The user's MSI data for the past 7 days is collected to calculate the average exercise intensity and standard deviation , and define the regular activity interval as , perform real-time deviation calculation ;when If the duration is greater than 5 minutes, it is considered an "abnormal mode" and the exercise intensity attenuation coefficient is reset to the historical average value (calculated based on the past 30 days' data, with a default value of 0.15). A 2-hour mode learning cycle is initiated to recalculate the average exercise intensity. and standard deviation The highest priority is wearing a walking aid (forced weight ≥ 0.4), the secondary priority is abnormal movement pattern (reset attenuation coefficient), basic adjustment: two-variable attenuation model, when wearing a walking aid and abnormal movement pattern occur at the same time, weight forced compensation is performed first, and then the attenuation coefficient is reset to reduce the logical error rate of weight decision.

[0024] Abnormal single movement intensity may be caused by daily activities (such as climbing stairs). It is necessary to distinguish between intense movements and regular movements in combination with the derivative of the vertical displacement rate (acceleration rate) to reduce false triggering. When the limb movement intensity index is greater than 1.5 times the preset threshold and the derivative of the vertical displacement rate is greater than 3m / s³, the precursor trigger unit 2 activates the 200Hz high-frequency sampling mode of the electromyographic signal. The preset threshold is dynamically generated by the formula: , is the baseline motion intensity threshold, and the derivative of the vertical displacement rate is calculated as ;in, is the vertical displacement measured by the millimeter-wave radar, for The vertical displacement rate at each moment is calculated by time difference of radar point cloud data. Seconds, when and When the sensor is detected, it is judged as a potential precursor to a fall, triggering subsequent high-frequency sampling and joint monitoring, effectively distinguishing sudden acceleration before a fall from acceleration changes in regular movement. Before a fall, muscles will be abnormally activated or instantly relaxed (myographic silence period). Conventional sampling rates (such as 100Hz) may miss microsecond-level myoelectric mutations, and the sampling rate needs to be increased to capture details. After triggering, it is immediately switched to 200Hz high-frequency mode and continues to collect data until the end of the event (such as when the exercise intensity returns to below the baseline threshold). During a fall, the joint angle will undergo characteristic changes (such as excessive flexion of the knee or rapid extension of the hip joint). The three-dimensional point cloud data of the millimeter-wave radar can accurately calculate the joint motion trajectory, and the millimeter-wave radar is used to capture the three-dimensional spatial angle between the hip and knee joints. The calculation method of the three-dimensional spatial angle includes: Constructing a hip-knee-ankle joint chain motion model can decompose complex whole-body motion into joint-level angle changes, facilitating accurate calculation, with the hip joint as the origin of the world coordinate system. , the knee joint is , the ankle joint is , define the thigh vector , the calf vector is , assuming that the thigh and calf are rigid bodies, ignoring the slight influence of muscle deformation on the position of the joint center point, simplifying the kinematic calculation, forward kinematics derives the end position from the joint angle, while inverse kinematics can reversely deduce the joint angle from the end trajectory. It is suitable for solving the joint angle through radar point cloud data (including the end position of the limb), using the inverse kinematics algorithm to solve the joint rotation plane, and extracting the three-dimensional coordinate sequence of the ankle joint (end of the limb) through millimeter wave radar point cloud data , then the inverse kinematics equation is , ;in, is the hip joint rotation angle, is the knee joint angle, is the distance from the hip joint to the ankle joint, which is calculated by the vector modulus formula. The Newton-Raphson iteration method is used to solve the nonlinear equation, and the convergence threshold is set to Radians ensure the accuracy of angle calculation. Gravity will have a static effect on joint movement (such as the knee joint naturally flexes under the action of gravity). The gravity component needs to be compensated to obtain the true movement angle. The gravity vector compensation mechanism is introduced, and the gravity acceleration Decompose into the three axes of the joint coordinate system to obtain the gravity component , when the knee joint rotation angular velocity is detected When it exceeds 45 degrees / second, it is determined to be dynamic motion and a 15-degree compensation angle is automatically added to establish a joint motion trajectory prediction model. The radar point cloud may have abnormal values ​​due to occlusion or noise. The establishment of a prediction model can verify the rationality of the data in real time to avoid miscalculation. A second-order linear predictor is used to calculate the joint angles of the previous two frames. and Predict the current angle: When the actual measurement value deviates from the predicted path by more than 20%, data re-collection is triggered, effectively reducing false alarms caused by radar noise.

[0025] The multi-level decision unit 3 constructs a fall probability model based on the three-dimensional spatial angle and the surface electromyographic signal energy value. The three-dimensional spatial angle and the electromyographic signal energy value are the core features of falling. The probability model can quantify the possibility of falling and avoid the arbitrariness of a single threshold. The input features are: three-dimensional joint angle (hip / knee joint), surface electromyographic signal energy value and acceleration vector modulus mean / standard deviation. The output label is: fall event (1) / non-fall event (0). It is trained based on 1000 clinically annotated fall samples and 2000 non-fall samples. The random forest model is used, which contains 50 decision trees. The input feature importance is ranked as follows: knee joint angle (35%) > electromyographic (30%) > acceleration standard deviation (25%). The optimal threshold is determined to be 0.65 by the Youden index. That is, when the model output probability is >0.65, it is judged as a suspected fall, which effectively distinguishes falls from similar actions.

[0026] Among them, the determination method of the three-dimensional space angle includes: After the millimeter-wave radar point cloud is matched with the skeleton topology, a hip-knee-ankle joint kinematic chain model is constructed. The radar point cloud must be converted into the human skeleton structure to solve the joint angle. Topology matching is a key step in connecting the sensor data with the human body model. Straight-through filtering is used to remove outliers. Outliers are points greater than 5 meters away. Data density is reduced by downsampling the voxel grid with a voxel size of 1 cm³. Based on the pre-trained human posture estimation model, the coordinates of the key points of the hip, knee, and ankle joints are identified in the point cloud. 、 as well as , to meet the real-time detection requirements and improve the matching algorithm efficiency, thigh vector , the calf vector is , then the joint angle geometry is calculated as ; is the three-dimensional angle of the knee joint, is the vector dot product, reflecting the directional consistency of the two vectors, It is the vector modulus, representing the physical length of the thigh and calf, accurately reflecting the full range of motion of the knee joint from extension (180°) to flexion (30°). It calculates the joint angle in real time based on the inverse kinematics algorithm. When the position of the limb end (ankle joint) is known, inverse kinematics can quickly infer the joint angle. It is applicable to the end trajectory data provided by the radar point cloud, and extracts the real-time coordinate sequence of the ankle joint from the radar point cloud. , as the input constraint of inverse kinematics, the pseudo-inverse Jacobian matrix method is used to iteratively update the hip and knee joint angles. The objective function is to minimize the end position error. ;in, is the pseudo-inverse of the Jacobian matrix, which is used to map the end error to the joint space. is the ankle joint target coordinate of the current frame, To predict the coordinates for the model, the tracking error of the end trajectory of the reverse solution is reduced compared to forward kinematics. Gravity will cause the knee joint to bend naturally (such as about 170° when standing). It is necessary to distinguish between passive bending caused by gravity and active and severe bending during falls. The component of gravity acceleration in the sensor coordinate system is measured by the accelerometer , construct the gravity vector , calculate the knee joint bending direction vector (the vector from the thigh to the calf) and the gravity vector Angle ;in, The angle between the bending direction and the gravity vector is in the range of 0°~180°, so as to accurately identify gravity-dominated bending and abnormal bending during falls. When the angle between the bending direction of the knee joint and the gravity vector is greater than 30°, it indicates that there is a violent movement caused by non-gravity factors (such as rapid flexion during a fall). In this case, the angle compensation mechanism needs to be activated to calculate the compensation angle to eliminate gravity interference and obtain the three-dimensional space angle. And knee joint angular velocity When it is determined to be a dynamic abnormal bending, compensation is started ,in , is the joint angle after compensation, is the compensation amount, the sign of which is determined by the consistency between the bending direction and the gravity direction. The compensation amount increases linearly with the increase of angular velocity, with a maximum compensation of 30°. After compensation, the detection error of the knee joint angle in the fall scenario is reduced.

[0027] Among them, the dynamic adjustment method of the parameters of the fall probability model is: The importance of joint angles to fall detection varies with time, environment, and individual motion state, requiring dynamic weighting to adapt to scenario differences. The weights for the three-dimensional angle parameters are dynamically generated using a random forest model. Input features include the current time, ambient light intensity, and user cadence. The random forest model architecture consists of 100 decision trees, with an input feature dimension of 3. The output is the weighted value for the three-dimensional angle parameter. The training data consists of 200 samples of falls and non-falls collected at different time periods, lighting conditions, and cadences. The labels indicate whether a fall occurred, generating real-time weights. The model updates the weights every 5 minutes to adapt to lighting changes and cadence fluctuations. Dynamic weight adjustment improves the accuracy of nighttime fall detection. The muscle activation pattern in rehabilitation training is different from that in daily activities. The electromyographic signal can more sensitively reflect the limb control ability, and its weight needs to be increased to avoid missing abnormal movements during training. When it is detected that the user is in rehabilitation training mode, the weight value of the surface electromyographic signal energy value parameter is increased, and a forgetting factor is introduced to update the parameter online. The user's motor ability may change over time, and the forgetting factor needs to be used to reduce the impact of old data and improve the model's adaptability to the latest state. The incremental random forest algorithm is used to update the model every time new data arrives, retaining the training samples of the last 7 days and discarding older data. The forgetting factor is defined as ;in, is the timestamp of the data sample, In order to control the forgetting speed of old data, the model is automatically retrained every morning, and the tree model is updated using weighted samples with forgetting factors, focusing on the characteristics of fall events in recent data.

[0028] When the fall probability output by the fall probability model is greater than the fall probability threshold, the standard deviation of the acceleration vector within the previous 0.5 second window is extracted. The standard deviation of the acceleration vector is calculated using the following method: An improved robust standard deviation algorithm is used to calculate the acceleration standard deviation and dynamically remove abnormal frame data. Single-frame abnormal acceleration may be caused by sensor mistouch or environmental interference and needs to be dynamically identified and removed to avoid contaminating statistical results. The 50 frames of data in the window are compared one by one. If the acceleration value of a frame exceeds 3 times the acceleration standard deviation, it is marked as invalid. When the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit 1 is triggered to re-collect data, improving the data re-collection mechanism and making the detection results more reliable. If the standard deviation of the acceleration vector is greater than 15m / s² and the angle between two consecutive frames in three-dimensional space is greater than 75°, a third-level warning signal is generated and sent to the verification feedback unit 4. After generating the third-level warning signal, the multi-level decision unit 3 performs the following enhanced verification: It stores more than 1,000 clinically annotated fall event data, including acceleration sequences and electromyographic silent period duration, extracts key features for each fall event, including acceleration peak, impact duration, and electromyographic silent period start time, normalizes the acceleration sequence to eliminate the influence of individual movement intensity differences, calls the historical fall database for dynamic time warping matching, calculates the similarity between the current acceleration sequence and the typical fall pattern, extracts the acceleration sequence 2 seconds before the current third-level warning is triggered, selects reference sequences of the same type of fall pattern in the historical database, and uses the dynamic time warping algorithm to find the optimal time bending path to minimize the cumulative distance. The value is substituted into the similarity calculation formula to calculate the similarity, and the surface electromyographic signal is envelope detected to calculate the duration of time continuously below the threshold. The threshold is dynamically set to 20% of the current signal mean to adapt to the differences in electromyographic strength of different individuals. If the similarity is greater than 85% and the electromyographic silence period of the surface electromyographic signal is less than 50ms, it is determined to be a high-confidence fall event, and the verification feedback unit 4 is skipped to directly trigger the final alarm. When the verification fails, the motion intensity index threshold of the dynamic weight fusion unit 1 is automatically adjusted to 120% of the original value. The threshold adjustment lasts for 30 minutes. If the warning is triggered again during this period, the original threshold is restored and the manual review process is triggered to reduce the false positive rate after verification failure.

[0029] The original point cloud contains noise and non-ground points and requires preprocessing to accurately extract the ground contact area. The millimeter-wave radar point cloud data for 3 seconds before the fall is extracted, and the ground contact point density is calculated. When the ground contact point density is greater than 50 points / cm² and the duration exceeds 0.2 seconds, a valid fall event is confirmed. Otherwise, the data returns to the dynamic weight fusion unit 1 to recalibrate the sensor weights.

[0030] The verification method of the ground contact point density in the multi-level decision-making unit 3 includes: After fitting the ground plane, the point cloud data is projected onto a two-dimensional grid map, and morphological opening operation is used to eliminate isolated noise points. The three-dimensional point cloud is projected onto a two-dimensional plane to facilitate the statistical analysis of the geometric features of the contact area. The ground point cloud is projected onto the XY plane and divided into 1cm×1cm grid units. The density value of each grid is the number of point clouds in the unit to form a density matrix. The 3×3 structural element is used for opening operation to improve the smoothness of the contact area boundary. The point density in the effective contact area is counted. When the contact area is detected to be asymmetric, it is determined to be a non-fall scene, where the asymmetric distribution means the aspect ratio is greater than 2. It is necessary to distinguish between large-area contacts of real falls and small-area contacts of accidental collisions. The contact strength is quantified by the density threshold, and the connected areas of the morphologically processed matrix are marked. All connected areas are extracted, and small areas with an area of ​​<10cm² are excluded. The point density is calculated for each valid area, and the point cloud data is continuously monitored within 3 seconds. If the ground contact point density in a certain area is greater than 50 points / cm² and the duration exceeds 0.2 seconds, it is determined to be effective ground contact. The major and minor axes of the minimum circumscribed rectangle are calculated for each valid area. When the aspect ratio of the major and minor axes is greater than 2, it is determined to be an asymmetric distribution, and the possibility of a fall is ruled out; otherwise, it is retained as a suspected fall area. If the fall event is not confirmed, the front-end sensor data collection needs to be optimized through the feedback mechanism to avoid repeated misjudgments. When the verification fails, an instruction is sent to the dynamic weight fusion unit 1 to increase the millimeter-wave radar weight by 0.2 (for example, from 0.6 to 0.8) and reduce the motion intensity index threshold to 80% of the original value to improve sensitivity. The calibration lasts for 1 minute. During this period, if the contact area is detected again, the symmetry analysis is skipped and the density verification is directly entered to shorten the response time.

[0031] In the present invention, the dynamic weight fusion unit 1 collects three-axis acceleration, surface electromyography signals and millimeter-wave radar data, and realizes multi-source data correlation analysis through a two-variable attenuation model and dynamic threshold calibration. The precursor trigger unit 2 activates electromyography high-frequency sampling and joint three-dimensional spatial angle monitoring when the exercise intensity is abnormal, and captures the precursors of falls in advance. The multi-level decision unit 3 constructs a fall probability algorithm based on the random forest model, and performs three-level verification based on the acceleration standard deviation, ground contact point density and historical pattern matching to achieve accurate judgment of fall events, providing a highly reliable intelligent solution for the safety protection of the elderly, reducing the risk of fall injuries, and improving the efficiency of emergency response.

[0032] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. Elderly fall detection and warning system based on multimodal data fusion, characterized by: include: The dynamic weight fusion unit (1) collects the three-axis acceleration characteristic values ​​and surface electromyographic signal energy values ​​of the wearable sensor, calculates the limb movement intensity index based on the three-axis acceleration characteristic values, and triggers the dynamic adjustment of the environmental sensor weight when the limb movement intensity index is greater than the movement intensity index threshold. The environmental sensor weight is calculated using an exponential decay function. The environmental sensor data only includes the vertical displacement rate of the millimeter wave radar; When the limb movement intensity index is greater than 1.5 times the preset threshold and the derivative of the vertical displacement rate is greater than 3 m / s³, the precursor trigger unit (2) activates the 200 Hz high-frequency sampling mode of the electromyographic signal and captures the three-dimensional spatial angle between the hip joint and the knee joint through the millimeter wave radar; The multi-level decision-making unit (3) constructs a fall probability model based on the three-dimensional spatial angle and the surface electromyography signal energy value. When the fall probability output by the fall probability model is greater than the fall probability threshold, the standard deviation of the acceleration vector in the first 0.5 second window is extracted. If the standard deviation is greater than 15m / s² and the three-dimensional spatial angle of two consecutive frames is greater than 75°, a third-level warning signal is generated and sent to the verification feedback unit (4). The millimeter-wave radar point cloud data 3 seconds before the fall is extracted and the ground contact point density is calculated. When the ground contact point density is greater than 50 points / cm² and the duration exceeds 0.2 seconds, a valid fall event is confirmed. Otherwise, the signal returns to the dynamic weight fusion unit (1) to recalibrate the sensor weight.

2. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for dynamically adjusting the weights of environmental sensors in the dynamic weight fusion unit (1) includes: A bivariate decay model based on the user's real-time activity intensity was established, where the decay rate was positively correlated with the limb movement intensity index and negatively correlated with the ambient light intensity. When it is detected that the user is wearing an intelligent walking aid device, the weight compensation mechanism is activated to ensure that the minimum weight of the environmental sensor is not less than 0.

4. The historical activity pattern matching algorithm is introduced. When the current exercise intensity deviates from the user's regular activity pattern by more than 30%, the weight attenuation coefficient is automatically reset.

3. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The dynamic calibration of the limb movement intensity index threshold in the dynamic weight fusion unit (1) includes: A baseline threshold is generated based on comprehensive user biometric parameters, including body mass index, gait symmetry, and sit-stand-walk test time. The threshold is corrected in real time using the coefficient of variation of cadence collected by the inertial sensor. The correction amplitude increases the threshold by 5% for every 0.1Hz variation. When three consecutive threshold breakthrough events are detected, the threshold learning mode is triggered and the threshold curve is refitted based on the previous 24 hours of activity data.

4. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for processing the surface electromyography signal in the dynamic weight fusion unit (1) is as follows: The first stage: complex wavelet transform is used to extract the energy of the 80-200 Hz frequency band and dynamically suppress the high-frequency noise component; Phase II: Establish a muscle synergy model and screen four muscle group signals related to lower limb stability; The third stage: Dynamically compensate the energy value based on the muscle fatigue index, and the compensation coefficient increases linearly with the continuous activity time.

5. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for calculating the three-dimensional space angle in the precursor trigger unit (2) includes: A hip-knee-ankle joint chain motion model was constructed, and the inverse kinematics algorithm was used to solve the joint rotation plane. A gravity vector compensation mechanism was introduced. When the knee joint rotation angular velocity was detected to exceed 45 degrees / second, a 15-degree compensation angle was automatically added. A joint motion trajectory prediction model was established, and data re-collection was triggered when the actual measurement value deviated from the predicted path by more than 20%.

6. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for dynamically adjusting the parameters of the fall probability model in the multi-level decision-making unit (3) is as follows: The weights of the three-dimensional angle parameters are dynamically generated using a random forest model, with input features including the current time, ambient light intensity, and user cadence. When it is detected that the user is in rehabilitation training mode, the weight value of the surface electromyography signal energy value parameter is increased, and a forgetting factor is introduced to update the parameter online.

7. The elderly fall detection and warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for calculating the acceleration standard deviation in the multi-level decision unit (3) includes: The improved robust standard deviation algorithm is used to calculate the acceleration standard deviation and dynamically remove abnormal frame data. If the acceleration value of a frame exceeds 3 times the acceleration standard deviation, it is marked as invalid. When the proportion of valid frames in the window is less than 70%, the dynamic weight fusion unit (1) is triggered to re-collect data.

8. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The method for determining the three-dimensional space angle in the multi-level decision unit (3) includes: After the millimeter-wave radar point cloud is matched with the skeleton topology, a hip-knee-ankle joint motion chain model is constructed, and the joint angle is calculated in real time based on the inverse kinematics algorithm. When the angle between the knee joint bending direction and the gravity vector is detected to be greater than 30°, the angle compensation mechanism is activated to obtain the three-dimensional spatial angle.

9. The elderly fall detection and early warning system based on multimodal data fusion according to claim 1 is characterized in that: The verification method of the ground contact point density in the multi-level decision unit (3) includes: After fitting the point cloud data with the ground plane, it is projected onto a two-dimensional grid map. Morphological opening is used to eliminate isolated noise points, and the point density within the effective contact area is counted. When an asymmetric distribution of the contact area is detected, it is determined to be a non-fall scenario, where the asymmetric distribution means an aspect ratio greater than 2.

10. The elderly fall detection and warning system based on multimodal data fusion according to claim 1 is characterized in that: After generating the third-level warning signal, the multi-level decision-making unit (3) performs the following enhanced verification: The historical fall database is called for dynamic time warping matching, and the similarity between the current acceleration sequence and the typical fall pattern is calculated. If the similarity is greater than 85% and the electromyographic silent period of the surface electromyographic signal is less than 50ms, the verification feedback unit (4) is skipped and the final alarm is directly triggered. If the verification fails, the motion intensity index threshold of the dynamic weight fusion unit (1) is automatically adjusted to 120% of the original value.

Images