Intelligent fall detection method and system based on multi-modal feature fusion

Abstract

The invention discloses an intelligent tumble detection method and system based on multi-modal feature fusion, and the method comprises the steps: collecting data through a waist three-axis acceleration sensor, separating a gravity component through low-pass filtering, and calculating a resultant acceleration; extracting time domain, frequency domain and attitude composite features by adopting a sliding window; and a bidirectional LSTM model is constructed to analyze time sequence relevance, and multi-level judgment is realized in combination with dynamic threshold adjustment and feature template matching. The system is composed of a flexibly packaged MPU6050 sensing module, an STM32F407 processor, a dual-mode communication unit and a sound-light alarm module, and supports GPS / Beidou positioning and encryption transmission. According to the invention, a three-level linkage detection architecture is constructed, and dynamic threshold determination, deep learning time sequence modeling and adaptive parameter calibration technologies are combined, so that high-precision and low-power-consumption real-time fall monitoring is realized.

Description

Technical Field

[0001] The present invention belongs to the field of wearable health monitoring and intelligent sensing technology, and in particular relates to an intelligent fall detection method and system based on multimodal feature fusion. Background Art

[0002] Fall detection technology, as an important safeguard against accidental injuries, is in urgent demand in areas such as elderly health monitoring and industrial safety protection. Current mainstream detection solutions are mostly based on inertial sensor data analysis, which detects falls by analyzing the acceleration changes during human motion. Traditional methods generally use a fixed threshold judgment strategy. Although this method can achieve real-time response in milliseconds, it faces severe challenges in practical applications: a single acceleration threshold is difficult to adapt to the differences in individual movement patterns, especially when dealing with complex movements such as rapid squats and sharp turns. The misjudgment rate increases significantly, and experimental data shows that the false alarm rate generally exceeds 15%. At the same time, existing algorithms fail to capture the dynamic temporal characteristics of the fall process, resulting in a recognition rate of less than 70% for special scenarios such as lateral falls and gradual falls, which seriously restricts the progress of the technology's practical application.

[0003] While the recent rise of machine learning methods has improved pattern recognition capabilities to a certain extent, they still have significant flaws. Neural network-based detection models often require thousands of sets of labeled data for training, resulting in high data acquisition and annotation costs. Static feature extraction strategies can easily lose the continuous spatiotemporal correlation of motion signals, resulting in missed detection of key vital signs. While multi-sensor fusion solutions improve detection accuracy by introducing modules such as gyroscopes and barometers, they cause device power consumption to surge by over 50% and exacerbate wearer discomfort. More critically, existing systems lack dynamic parameter calibration mechanisms, making it impossible to adaptively adjust detection sensitivity based on user activity intensity. The overall detection rate in mixed scenarios with elderly and young adults is less than 80%. Industry tests have shown that the average response delay of current technical solutions on standard data sets is 320 milliseconds, making it difficult to meet the demand for immediate alerts in high-risk scenarios, becoming a core bottleneck restricting the implementation of the technology. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent fall detection method and system based on multimodal feature fusion, which realizes high-precision, low-power real-time fall monitoring by constructing a three-level linkage detection architecture and combining dynamic threshold judgment, deep learning time series modeling and adaptive parameter calibration technology.

[0005] To achieve the purpose of the present invention, on the one hand, the present invention provides an intelligent fall detection method based on multimodal feature fusion, comprising the following steps:

[0006] Step 1: By collecting the three-axis acceleration signal of the human waist in real time, first obtain the original acceleration component a at a sampling frequency of 100 Hzx 、a y 、a z Then, the original signal is filtered using a fourth-order Butterworth low-pass filter with a cutoff frequency of 0.2 Hz to separate the static gravity component and calculate the gravitational acceleration modulus. where g x 、g y 、g z is the filtered gravity component, and finally through the formula Determine the net acceleration after removing the effect of gravity;

[0007] Step 2: Perform sliding window processing on the combined acceleration, with a window length of 1 second corresponding to 100 sampling points and an overlap rate of 50%, to extract the time domain feature set Frequency domain feature set F f =[E b , E 3w , H p and posture features Δθ;

[0008] Among them, μ t is the window acceleration mean, is the variance, is the improved zero-crossing rate; E b is the energy ratio of 3-8Hz frequency band, E w3 is the energy of the third layer detail coefficient of wavelet decomposition, H p is the power spectrum entropy value; Δθ is the Euler angle deviation between the current posture and the initial posture solved by the quaternion method;

[0009] Step 3: Using the time domain feature set F t , frequency domain feature set F f , posture feature Δθ, construct a composite feature vector containing the posture deviation angle:

[0010]

[0011] in represents a real number;

[0012] Step 4: According to the combined acceleration a sum and the composite feature vector FV current Perform multi-level joint judgment, first calculate the extreme value sequence of the combined acceleration in the current sliding window in real time When the combined acceleration a is detected sum When the five consecutive sampling points are lower than 0.7g, mark t0 as the start time of weightlessness; then within the 300ms time window after t0, if there is the said combined acceleration a sum >1.5g and the composite feature vector FV currentIf the attitude angle deviation Δθ>40°, the impact time t1 is recorded; finally, the acceleration stability is continuously monitored within 800ms after t1. When max(|a sum -g|)<0.15g, the output contains a feature vector sequence of time tags [t0-500ms, t1+800ms] As a result of dynamic detection;

[0013] Step 5: The feature vector sequence The sequence is framed and aligned according to the time mark, and then input into a bidirectional LSTM neural network model containing 64 hidden units. The model extracts the temporal correlation between the acceleration dynamic pattern and the posture evolution characteristics through a time sliding window, and finally outputs the fall probability value P of the current period through a sigmoid activation function. fall ∈[0, 1], where the normalization of the input sequence satisfies Constraints;

[0014] Step 6: Based on the total acceleration a sum The data stream is used to calculate the historical acceleration mean μ in real time using the exponentially weighted moving average method. hist and standard deviation σ hist ; Dynamically adjust the detection sensitivity based on the statistics to generate a dynamic threshold T adj :

[0015] T adj =T base ·(1+0.3·σ hist / μ hist );

[0016] Among them, the basic threshold T base =0.82 was determined by statistical analysis of a typical fall data set; the dynamic threshold T adj With the fall probability value P fall Perform timing alignment cache;

[0017] Step 7: Based on the fall probability value P fall , the dynamic threshold T adj , the composite feature vector FV current , determine the preset typical fall feature template FV by the following formula template Similarity:

[0018]

[0019] When P is satisfied fall >T adj and S>0.7, it was determined as a valid fall event;

[0020] Step 8: When step 7 determines that a fall event is valid, an encrypted alarm signal containing the weightlessness start timestamp t0 marked in step 4, the user identification code and the GPS / Beidou dual-mode positioning data is transmitted concurrently through the wireless communication module. The wireless communication module supports a hybrid transmission mode of cellular network and short-range communication protocol; the local sound and light alarm device is synchronously activated to start a composite alarm mode of pulse sound waves and red and blue strobes, and its duration is based on the dynamic statistic σ calculated in step 6. hist / μ hist Adaptive adjustment: When σ hist / μ hist When the value is greater than 0.5, the alarm cycle is shortened to 30 seconds, otherwise it maintains the standard cycle of 60 seconds. The positioning data is generated in real time by the positioning unit integrated in the sensor terminal, and the encrypted transmission uses the AES-128 standard protocol to encapsulate the data frame.

[0021] The time domain feature set F in step 2 t Improved zero-crossing rate Specifically:

[0022]

[0023] Among them, γ = 0.15 is the adaptive noise suppression coefficient, is the indicator function;

[0024] And the frequency domain feature set F f The extraction method includes: performing Hanning window weighted fast Fourier transform on the window data to obtain the power spectrum density distribution, and calculating the normalized energy ratio E in the 3-8 Hz frequency band. b The third layer detail coefficient energy E is extracted by Daubechies4 wavelet three-layer decomposition w3 , and finally combined with the power spectrum entropy value H p Construct the frequency domain feature vector.

[0025] The training method of the bidirectional LSTM model in step 5 is specifically as follows:

[0026] First, the SisFall dataset is subjected to data augmentation processing with temporal noise addition and ±1.5% random scaling to generate an extended dataset containing dynamic noise and scale changes. Then, based on the extended dataset, the initial model training is performed using the Adam optimizer with an input sequence length of 20 frames as the benchmark. The obtained pre-trained model parameters are solidified and deployed to the detection terminal for real-time time series feature extraction. For the prediction probability output by the pre-trained model, the focal loss function is used to solve the sample imbalance problem, which is expressed as Where α=0.75、γ=2,p i is the model prediction probability, y iis the true label, and N is the number of batch samples. Further, temporal association regularization is implemented on the basis of the pre-trained model. During forward propagation, the LSTM hidden layer neurons are randomly discarded with a probability of 0.3. During backpropagation, the gradient continuity between time steps is retained, and the fully connected network in the output layer maintains complete neuron connections. Finally, a two-stage transfer learning strategy is adopted. The parameters are roughly adjusted based on the extended dataset at a learning rate of 0.001. Then, the output layer is fine-tuned at a learning rate of 0.0002 using 200 sets of fall event data collected locally, and the optimized model weights are synchronously updated to the real-time inference module.

[0027] The statistical calculation of the historical acceleration data in step 6 adopts the exponentially weighted moving average method:

[0028] μ hist (t) = βμ hist (t-1)+(1-β)a sum (t)

[0029]

[0030] The smoothing factor β is 0.95, and the update period is 5 minutes.

[0031] The feature template FV in step 7 template The construction method is as follows:

[0032] First, the acceleration and posture data of three typical scenarios, including forward falls, backward falls, and sideways falls, are collected. For each type, no less than 200 complete fall sequences are collected, covering the dynamic characteristics of weightlessness, impact, and static stages. Then, the three types of data are dynamically time-warped and aligned, and individual motion speed differences are eliminated by nonlinear time scaling. Based on the weightlessness start time t0 and impact time t1 marked in step 4, the three key time windows of weightlessness stage [t0-500ms, t0+300ms], impact stage [t1-100ms, t1+200ms], and static stage [t1+500ms, t1+1500ms] are accurately divided. The mean acceleration μ is further extracted for each time window. t , 3-8Hz frequency band energy ratio E b The statistical distribution characteristics of the posture angle deviation Δθ are used to construct a composite feature space with the dimension of [3 stages × 3 features × 3 types]. The K-means clustering algorithm is used to iteratively optimize the feature mean vectors of the three types of falls, namely forward, backward, and sideways, as the initial cluster center, and output the classification feature template. and Finally, the weight coefficient ω is set according to the frequency of various falls in actual scenarios. k , through the weighted fusion formula Generate a global feature template where the weights satisfy ∑ωk = 1 to achieve a unified representation of multi-type fall features.

[0033] The present invention also provides a system for intelligent fall detection method based on multimodal feature fusion, comprising the following modules:

[0034] The sensor module, which uses an MPU6050 inertial measurement unit in a flexible PCB package, is fixed to the belt buckle on the human waist and is used to collect three-axis acceleration and angular velocity signals at an adjustable sampling frequency of 50-200Hz;

[0035] The data processing module is built on an STM32F407 microcontroller and has a built-in execution program for the method, which is used to implement signal filtering, multimodal feature fusion, and dynamic threshold calculation. It is directly connected to the alarm output module via the SPI bus;

[0036] Communication module, integrating NB-IoT cellular network and BLE5.0 dual-mode unit, for performing encrypted alarm signal transmission;

[0037] The alarm output module includes an audible and visual alarm unit and a positioning data triggering unit, wherein the audible and visual alarm unit is composed of a 96dB pulse buzzer and a red and blue dual-color LED array. In response to the judgment signal in step 7, it activates a 1250Hz intermittent sound wave and a 2Hz strobe light. At the same time, the integrated high-precision positioning chip generates GPS / Beidou dual-mode geographic coordinates in real time, and binds the positioning data to the alarm timestamp and injects it into the communication data stream. The working cycle of the alarm output module is controlled by the dynamic statistics σhist / t / μhist in step 6.

[0038] Furthermore, the sensor module is packaged in a flexible PCB and has a built-in low-power mode switching circuit, which automatically switches to a 10Hz sampling frequency in a static state, and the dynamic power consumption is less than 0.8mW.

[0039] Furthermore, the NB-IoT alarm channel of the communication module uses the AES-128 protocol to encapsulate the data frame containing the user identity code, GPS / Beidou dual-mode positioning coordinates and the timestamp marked in step 4, and the BLE channel is used to push warning information to mobile terminals within a range of 10 meters.

[0040] Furthermore, when σ hist / t / μ hist>0.5, the alarm output module switches the sound and light alarm cycle from the 60-second standard mode to the 30-second enhanced mode through the hardware PWM controller, and at the same time increases the data upload frequency of the NB-IoT module to every 5 seconds.

[0041] Compared with the existing technology, the significant progress of the present invention lies in: (1) Three-level detection architecture design: The present invention achieves millisecond-level response through dynamic threshold initial screening, combines LSTM time series analysis to capture the dynamic characteristics of falls, and finally eliminates the influence of individual differences through adaptive decision-making to form a step-by-step filtering mechanism; (2) Multimodal feature fusion: The present invention integrates three-dimensional information of time domain statistics, frequency domain energy distribution and posture deviation angle to construct a 12-dimensional feature vector, which improves the accuracy of traditional single-dimensional feature detection by 23.6%; (3) Low-power optimization technology: The present invention adopts an event-triggered sampling mechanism and automatically switches to 10Hz low-power mode in a static state; optimizes the LSTM model parameter size to 58KB, and the inference time is less than 15ms; the flexible PCB packaging makes the sensor module thickness less than 3mm and weighs only 7.8g.

[0042] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] Combine Figure 1 , an intelligent fall detection method based on multimodal feature fusion of the present invention comprises the following steps:

[0047] Step 1: By collecting the three-axis acceleration signal of the human waist in real time, first obtain the original acceleration component a at a sampling frequency of 100 Hz x 、a y 、a z Then, the original signal is filtered using a fourth-order Butterworth low-pass filter with a cutoff frequency of 0.2 Hz to separate the static gravity component and calculate the gravitational acceleration modulus. where g x 、g y 、g z is the filtered gravity component, and finally through the formula Determine the net acceleration after removing the effect of gravity;

[0048] Step 2: Perform sliding window processing on the combined acceleration, with a window length of 1 second corresponding to 100 sampling points and an overlap rate of 50%, and extract the time domain feature set. Frequency domain feature set F f =[E b , E 3w , H p and posture features Δθ;

[0049] Among them, μ t is the window acceleration mean, is the variance, is the improved zero-crossing rate; E b is the energy ratio of 3-8Hz frequency band, E w3 is the energy of the third layer detail coefficient of wavelet decomposition, H p is the power spectrum entropy value; Δθ is the Euler angle deviation between the current posture and the initial posture solved by the quaternion method;

[0050] Step 3: Using the time domain feature set F t , frequency domain feature set F f , posture feature Δθ, construct a composite feature vector containing the posture deviation angle:

[0051]

[0052] in represents a real number;

[0053] Step 4: According to the combined acceleration a sum and the composite feature vector FV current Perform multi-level joint judgment, first calculate the extreme value sequence of the combined acceleration in the current sliding window in real time When the combined acceleration a is detected sum When the five consecutive sampling points are lower than 0.7g, mark t0 as the start time of weightlessness; then within the 300ms time window after t0, if there is the said combined acceleration a sum >1.5g and the composite feature vector FV current If the attitude angle deviation Δθ>40°, the impact time t1 is recorded; finally, the acceleration stability is continuously monitored within 800ms after t1. When max(|a sum -g|)<0.15g, the output contains a feature vector sequence of time tags [t0-500ms, t1+800ms] As a result of dynamic detection;

[0054] Step 5: Sequence the feature vector The sequence is framed and aligned according to the time mark, and then input into a bidirectional LSTM neural network model containing 64 hidden units. The model extracts the temporal correlation between the acceleration dynamic pattern and the posture evolution characteristics through a time sliding window, and finally outputs the fall probability value P of the current period through a sigmoid activation function. fall ∈[0, 1], where the normalization of the input sequence satisfies Constraints;

[0055] Step 6: Based on the combined acceleration a sum The data stream is used to calculate the historical acceleration mean μ in real time using the exponentially weighted moving average method. hist and standard deviation σ hist ; Dynamically adjust the detection sensitivity based on the statistics to generate a dynamic threshold T adj :

[0056] T adj =T base ·(1+0.3·σ hist / μ hist );

[0057] Among them, the basic threshold T base =0.82 was determined by statistical analysis of a typical fall data set; the dynamic threshold T adj With the fall probability value P fall Perform timing alignment caching to provide a dynamic benchmark for subsequent decisions;

[0058] Step 7: Based on the fall probability value P fall , the dynamic threshold T adj , the composite feature vector FV current , determine the preset typical fall feature template FV by the following formula template Similarity:

[0059]

[0060] When P is satisfied fall >T adj and S>0.7, it was determined as a valid fall event;

[0061] Step 8: When step 7 determines that a fall event is valid, an encrypted alarm signal containing the weightlessness start timestamp t0 marked in step 4, the user identification code, and the GPS / Beidou dual-mode positioning data is transmitted concurrently through the wireless communication module. The wireless communication module supports a hybrid transmission mode of cellular network and short-range communication protocol; the local sound and light alarm device is synchronously activated to start a composite alarm mode of pulse sound waves and red and blue strobes, and its duration is based on the dynamic statistic σ calculated in step 6. hist / μhist Adaptive adjustment: When σ hist / μ hist When the value is greater than 0.5, the alarm cycle is shortened to 30 seconds, otherwise it maintains the standard cycle of 60 seconds. The positioning data is generated in real time by the positioning unit integrated in the sensor terminal, and the encrypted transmission uses the AES-128 standard protocol to encapsulate the data frame.

[0062] The time domain feature set F in step 2 t Improved zero-crossing rate Specifically:

[0063]

[0064] Among them, γ = 0.15 is the adaptive noise suppression coefficient, is the indicator function;

[0065] The method for extracting the frequency domain feature set F includes: performing a Hanning window weighted fast Fourier transform on the window data to obtain a power spectrum density distribution, and calculating the normalized energy ratio E in the 3-8 Hz frequency band. b The third layer detail coefficient energy E is extracted by Daubechies4 wavelet three-layer decomposition w3 , and finally combined with the power spectrum entropy value H p Construct the frequency domain feature vector.

[0066] The training method of the bidirectional LSTM model in step 5 is specifically as follows:

[0067] First, the SisFall dataset is subjected to data augmentation processing with temporal noise addition and ±15% random scaling to generate an extended dataset containing dynamic noise and scale changes. Then, based on the extended dataset, the initial model training is performed using the Adam optimizer with an input sequence length of 20 frames as the benchmark. The obtained pre-trained model parameters are solidified and deployed to the detection terminal for real-time time series feature extraction. For the predicted probability output by the pre-trained model, the focal loss function is used to solve the sample imbalance problem, and its expression is: Where α=0.75、γ=2,p i is the model prediction probability, y i is the true label, and N is the number of batch samples. Further, temporal association regularization is implemented on the basis of the pre-trained model. During forward propagation, the LSTM hidden layer neurons are randomly discarded with a probability of 0.3. During backpropagation, the gradient continuity between time steps is retained, and the fully connected network in the output layer maintains complete neuron connections. Finally, a two-stage transfer learning strategy is adopted. The parameters are roughly adjusted based on the extended dataset at a learning rate of 0.001. Then, the output layer is fine-tuned at a learning rate of 0.0002 using 200 sets of fall event data collected locally, and the optimized model weights are synchronously updated to the real-time inference module.

[0068] The statistical calculation of the historical acceleration data in step 6 adopts the exponentially weighted moving average method:

[0069] μ hist (t) = βμ hist (t-1)+(1-β)a sum (t)

[0070]

[0071] The smoothing factor β is 0.95, and the update period is 5 minutes.

[0072] The feature template FV in step 7 template The construction method is as follows:

[0073] First, the acceleration and posture data of three typical scenarios, including forward falls, backward falls, and sideways falls, are collected. For each type, no less than 200 complete fall sequences are collected, covering the dynamic characteristics of weightlessness, impact, and static stages. Then, the three types of data are aligned by dynamic time warping (DTW). The individual motion speed differences are eliminated by nonlinear time scaling. Based on the weightlessness start time t0 and impact time t1 marked in step 4, the three key time windows are accurately divided into the weightlessness stage [t0-500ms, t0+300ms], the impact stage [t1-100ms, t1+200ms], and the static stage [t1+500ms, t1+1500ms]. The mean acceleration μ is further extracted for each time window. t , 3-8Hz frequency band energy ratio E b The statistical distribution characteristics of the posture angle deviation Δθ are used to construct a composite feature space with the dimension of [3 stages × 3 features × 3 types]. The K-means clustering algorithm is used to iteratively optimize the feature mean vectors of the three types of falls, namely forward, backward, and sideways, as the initial cluster center, and output the classification feature template. and Finally, the weight coefficient ω is set according to the frequency of various falls in actual scenarios. k , through the weighted fusion formula Generate a global feature template where the weights satisfy ∑ω k = 1 to achieve a unified representation of multi-type fall features.

[0074] The present invention provides a system for intelligent fall detection based on multimodal feature fusion, comprising the following modules:

[0075] The sensor module, which uses an MPU6050 inertial measurement unit in a flexible PCB package, is fixed to the belt buckle on the human waist and is used to collect three-axis acceleration and angular velocity signals at an adjustable sampling frequency of 50-200Hz;

[0076] The data processing module is built on an STM32F407 microcontroller and has a built-in execution program for the method, which is used to implement signal filtering, multimodal feature fusion, and dynamic threshold calculation. It is directly connected to the alarm output module via the SPI bus;

[0077] Communication module, integrating NB-IoT cellular network and BLE5.0 dual-mode unit, for performing encrypted alarm signal transmission;

[0078] The alarm output module includes an audible and visual alarm unit and a positioning data triggering unit, wherein the audible and visual alarm unit is composed of a 96dB pulse buzzer and a red and blue dual-color LED array. In response to the judgment signal in step 7, the 1250Hz intermittent sound wave and the 2Hz strobe light are activated. At the same time, the GPS / Beidou dual-mode geographic coordinates are generated in real time through the integrated high-precision positioning chip, and the positioning data is bound to the alarm timestamp and injected into the communication data stream. The working cycle of the alarm output module is controlled by the dynamic statistics σ_hist / μ_hist in step 6.

[0079] Furthermore, the sensor module is packaged in a flexible PCB and has a built-in low-power mode switching circuit, which automatically switches to a 10Hz sampling frequency in a static state, and the dynamic power consumption is less than 0.8mW.

[0080] Furthermore, the NB-IoT alarm channel of the communication module uses the AES-128 protocol to encapsulate the data frame containing the user identity code, GPS / Beidou dual-mode positioning coordinates and the timestamp marked in step 4, and the BLE channel is used to push warning information to mobile terminals within a range of 10 meters.

[0081] Furthermore, when σ_hist / μ_hist>0.5, the alarm output module switches the sound and light alarm cycle from the 60-second standard mode to the 30-second enhanced mode through the hardware PWM controller, and at the same time increases the data upload frequency of the NB-IoT module to every 5 seconds.

[0082] Example

[0083] To verify the performance of this solution, 30 subjects (aged 22-78, BMI 18.5-28.7) were selected to conduct multi-scenario testing; the test environment included:

[0084] 1. Daily activity scenarios: 10 types of actions including walking, going up and down stairs, sitting and standing, with a cumulative test time of 150 hours;

[0085] 2. Fall simulation scenario: forward, sideways, and backward falls, each repeated 50 times;

[0086] 3. Extreme condition test: sensor offset angle > 15°, clothing obstruction, and other interference conditions.

[0087] The detection rate of this solution on the SisFall standard dataset is 98.6% (4382 / 4450), the false alarm rate is 2.1% (93 / 4450), and the average response time is 182ms. In industrial safety scenarios, the recognition rate of lateral falls is 96.3% (144 / 150), and the detection rate of gradual falls is 89.7% (269 / 300). Power consumption tests show that the system consumes 1.2mW in continuous working state and 0.25mW in standby mode. With an 850mAh lithium battery, it can achieve continuous operation for 32 days. In terms of wearing comfort, the flexible packaged sensor module weighs 7.8g and is 2.9mm thick. The subject's comfort score is 4.6 / 5 points.

[0088] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0089] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent fall detection method based on multimodal feature fusion, characterized in that: The following steps are involved: Step 1: By collecting the three-axis acceleration signal of the human waist in real time, first obtain the original acceleration component a at a sampling frequency of 100 Hz x 、a y 、a z Then, the original signal is filtered using a fourth-order Butterworth low-pass filter with a cutoff frequency of 0.2 Hz to separate the static gravity component and calculate the gravitational acceleration modulus. where g x 、g y 、g z is the filtered gravity component, and finally through the formula Determine the net acceleration after removing the effect of gravity; Step 2: Perform sliding window processing on the combined acceleration, with a window length of 1 second corresponding to 100 sampling points and an overlap rate of 50%, and extract the time domain feature set. Frequency domain feature set F f =[E b , E 3w , H p and posture features Δθ; Among them, μ t is the window acceleration mean, is the variance, is the improved zero-crossing rate; E b is the energy ratio of 3-8Hz frequency band, E w3 is the energy of the third layer detail coefficient of wavelet decomposition, H p is the power spectrum entropy value; Δθ is the Euler angle deviation between the current posture and the initial posture solved by the quaternion method; Step 3: Using the time domain feature set F t , frequency domain feature set F f , posture feature Δθ, construct a composite feature vector containing the posture deviation angle: in represents a real number; Step 4: According to the combined acceleration a sum and the composite feature vector FV current Perform multi-level joint judgment, first calculate the extreme value sequence of the combined acceleration in the current sliding window in real time When the combined acceleration a is detected sum When the five consecutive sampling points are lower than 0.7g, mark t0 as the start time of weightlessness; then within the 300ms time window after t0, if there is the said combined acceleration a sum >1.5g and the composite feature vector FV current If the attitude angle deviation Δθ in the collision is greater than 40°, the collision time t1 is recorded; finally, the acceleration stability is continuously monitored within 800ms after t1. sum -g|)<0.15g, the output contains a feature vector sequence of time tags [t0-500ms, t1+800ms] As a result of dynamic detection; Step 5: Sequence the feature vector The sequence is framed and aligned according to the time mark, and then input into a bidirectional LSTM neural network model containing 64 hidden units. The model extracts the temporal correlation between the acceleration dynamic pattern and the posture evolution characteristics through a time sliding window, and finally outputs the fall probability value P of the current period through a sigmoid activation function. fall ∈[0, 1], where the normalization of the input sequence satisfies Constraints; Step 6: Based on the combined acceleration a sum The data stream is used to calculate the historical acceleration mean μ in real time using the exponentially weighted moving average method. hist and standard deviation σ hist ; Dynamically adjust the detection sensitivity based on the statistics to generate a dynamic threshold T adj : T adj =T base ·(1+0.3·s hist / m hist ); Among them, the basic threshold T base =0.82 was determined by statistical analysis of a typical fall data set; the dynamic threshold T adj With the fall probability value P fall Perform timing alignment cache; Step 7: Based on the fall probability value P fall , the dynamic threshold T adj , the composite feature vector FV current , determine the preset typical fall feature template FV by the following formula template Similarity: When P is satisfied fall >T adj and S>0.7, it was determined as a valid fall event; Step 8: When step 7 determines that a fall event is valid, an encrypted alarm signal containing the weightlessness start timestamp t0 marked in step 4, the user identification code, and the GPS / Beidou dual-mode positioning data is transmitted concurrently through the wireless communication module. The wireless communication module supports a hybrid transmission mode of cellular network and short-range communication protocol; the local sound and light alarm device is synchronously activated to start a composite alarm mode of pulse sound waves and red and blue strobes, and its duration is based on the dynamic statistic σ calculated in step 6. hist / μ hist Adaptive adjustment: When σ hist / μ hist When the value is greater than 0.5, the alarm cycle is shortened to 30 seconds, otherwise it maintains the standard cycle of 60 seconds. The positioning data is generated in real time by the positioning unit integrated in the sensor terminal, and the encrypted transmission uses the AES-128 standard protocol to encapsulate the data frame.

2. The intelligent fall detection method based on multimodal feature fusion according to claim 1 is characterized in that: The time domain feature set F in step 2 t Improved zero-crossing rate Specifically: Among them, γ = 0.15 is the adaptive noise suppression coefficient, is the indicator function; And the frequency domain feature set F f The extraction method includes: performing Hanning window weighted fast Fourier transform on the window data to obtain the power spectrum density distribution, and calculating the normalized energy ratio E in the 3-8 Hz frequency band. b The third layer detail coefficient energy E is extracted by Daubechies4 wavelet three-layer decomposition w3 , and finally combined with the power spectrum entropy value H p Construct the frequency domain feature vector.

3. The intelligent fall detection method based on multimodal feature fusion according to claim 1 is characterized in that: The training method of the bidirectional LSTM model in step 5 is specifically as follows: First, the SisFall dataset is augmented with temporal noise and ±15% random scaling to generate an extended dataset containing dynamic noise and scale variations. Based on this extended dataset, an initial model is trained using the Adam optimizer, with an input sequence length of 20 frames as the benchmark. The resulting pre-trained model parameters are solidified and deployed to the detection terminal for real-time time series feature extraction. For the prediction probability output by the pre-training model, the focal loss function is used to solve the sample imbalance problem, and its expression is Where α=0.75、γ=2,p i is the model prediction probability, y i is the true label, and N is the number of batch samples. Further, temporal association regularization is implemented on the basis of the pre-trained model. During forward propagation, the LSTM hidden layer neurons are randomly discarded with a probability of 0.

3. During backpropagation, the gradient continuity between time steps is retained, and the fully connected network in the output layer maintains complete neuron connections. Finally, a two-stage transfer learning strategy is adopted. The parameters are roughly adjusted based on the extended dataset at a learning rate of 0.

001. Then, the output layer is fine-tuned at a learning rate of 0.0002 using 200 sets of fall event data collected locally, and the optimized model weights are synchronously updated to the real-time inference module.

4. The intelligent fall detection method based on multimodal feature fusion according to claim 1, characterized in that: The statistical calculation of the historical acceleration data in step 6 adopts the exponentially weighted moving average method: μ hist (t)=βμ hist (t-1)+(1-β)a sum (t) The smoothing factor β is 0.95, and the update period is 5 minutes.

5. The intelligent fall detection method based on multimodal feature fusion according to claim 1 is characterized in that: The feature template FV in step 7 template The construction method is as follows: First, the acceleration and posture data of three typical scenarios, including forward falls, backward falls, and sideways falls, are collected. For each type, no less than 200 complete fall sequences are collected, covering the dynamic characteristics of weightlessness, impact, and static stages. Then, the three types of data are dynamically time-warped and aligned, and individual motion speed differences are eliminated by nonlinear time scaling. Based on the weightlessness start time t0 and impact time t1 marked in step 4, the three key time windows of weightlessness stage [t0-500ms, t0+300ms], impact stage [t1-100ms, t1+200ms], and static stage [t1+500ms, t1+1500ms] are accurately divided. The mean acceleration μ is further extracted for each time window. t , 3-8Hz frequency band energy ratio E b The statistical distribution characteristics of the posture angle deviation Δθ are used to construct a composite feature space with the dimension of [3 stages × 3 features × 3 types]. The K-means clustering algorithm is used to iteratively optimize the feature mean vectors of the three types of falls, namely forward, backward, and sideways, as the initial cluster center, and output the classification feature template. and Finally, the weight coefficient ω is set according to the frequency of various falls in actual scenarios. k , through the weighted fusion formula Generate a global feature template where the weights satisfy ∑ω k = 1 to achieve a unified representation of multi-type fall features.

6. The system of the intelligent fall detection method based on multimodal feature fusion according to claims 1-5, characterized in that: Includes the following modules: The sensor module, which uses an MPU6050 inertial measurement unit in a flexible PCB package, is fixed to the belt buckle on the human waist and is used to collect three-axis acceleration and angular velocity signals at an adjustable sampling frequency of 50-200Hz; The data processing module is built on an STM32F407 microcontroller and has a built-in execution program for the method, which is used to implement signal filtering, multimodal feature fusion, and dynamic threshold calculation. It is directly connected to the alarm output module via the SPI bus; Communication module, integrating NB-IoT cellular network and BLE5.0 dual-mode unit, for performing encrypted alarm signal transmission; The alarm output module includes an audible and visual alarm unit and a positioning data triggering unit, wherein the audible and visual alarm unit is composed of a 96dB pulse buzzer and a red and blue dual-color LED array. In response to the judgment signal in step 7, the 1250Hz intermittent sound wave and the 2Hz strobe light are activated. At the same time, the GPS / Beidou dual-mode geographic coordinates are generated in real time through the integrated high-precision positioning chip, and the positioning data is bound to the alarm timestamp and injected into the communication data stream. The working cycle of the alarm output module is controlled by the dynamic statistics σ_hist / μ_hist in step 6.

7. The system of the intelligent fall detection method based on multimodal feature fusion according to claim 6, characterized in that: The sensor module is packaged in a flexible PCB and has a built-in low-power mode switching circuit. It automatically switches to a 10Hz sampling frequency in a static state, and the dynamic power consumption is less than 0.8mW.

8. The system of the intelligent fall detection method based on multimodal feature fusion according to claim 6, characterized in that: The NB-IoT alarm channel of the communication module uses the AES-128 protocol to encapsulate the data frame containing the user identity code, GPS / Beidou dual-mode positioning coordinates and the timestamp marked in step 4. The BLE channel is used to push warning information to mobile terminals within a range of 10 meters.

9. The system of the intelligent fall detection method based on multimodal feature fusion according to claim 6, characterized in that: When σ_hist / μ_hist>0.5, the alarm output module switches the sound and light alarm cycle from the 60-second standard mode to the 30-second enhanced mode through the hardware PWM controller, and at the same time increases the data upload frequency of the NB-IoT module to every 5 seconds.

Images