Fall detection method and system based on wrist-based inertial sensors

Abstract

The application discloses a fall detection method and system based on a wrist inertia sensor, and the method comprises the following steps: acquiring motion data collected by an inertia sensor installed on a human wrist and performing pretreatment to obtain target motion data; constructing a fall detection model and a fall direction detection model based on one-dimensional convolution and a long short-term memory neural network; performing fall detection by using the fall detection model; if falls are detected for a continuous setting number of times, it is confirmed that a fall occurs, then fall direction detection is performed by using the fall direction detection model, and a fall direction detection result is output. The application combines the advantages of a convolution neural network and a long short-term memory neural network to construct the fall detection model and the fall direction detection model, not only realizes automatic recognition of a fall direction, but also improves the accuracy of fall detection; and a judgment method based on multiple continuous sliding window data can effectively avoid that an approximate fall behavior is wrongly judged as a fall.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fall detection, in particular to a fall detection method and system based on wrist inertial sensor, a terminal and a computer readable storage medium. BACKGROUND

[0002] Fall is an important issue of public health. Fall is more common among the elderly, in addition to bringing serious threat to the physiological level of the elderly, it also brings serious consequences to the psychological level of the elderly. From the perspective of using equipment, fall detection can be divided into two categories, one is non-wearable fall detection research, which needs to use fixed equipment such as cameras, radars and infrared sensors for monitoring and detection, which is easy to invade privacy, cause discomfort and have great space-time limitations; the other is wearable fall detection research, which uses wearable sensors such as accelerometers, gyroscopes, magnetometers or inertial measurement units for detection. Considering that the kinematic data of the center of gravity of the human body changes more dramatically when falling, the fall characteristics are more obvious, the waist or chest close to the center of gravity is usually used as the wearing part, however, the comfort of wearing sensors on the waist or chest area is poor. With the development of smart watches or smart bracelets, people began to consider wearing sensors on the wrist, but human limbs are more active than the trunk, and the sensor signals generated by some daily activities are close to the fall signals, which leads to false positives or false negatives of the existing fall detection algorithm based on wrist sensors, and the accuracy is not high. Moreover, the existing fall detection algorithm cannot automatically identify the fall direction.

[0003] In addition, there are some fall detection methods using machine learning or deep learning models in the prior art, but they often do not consider the "near fall" behavior such as successfully recovering balance after slipping or stumbling, so these models are prone to misjudging this kind of "near fall" behavior as a fall. SUMMARY

[0004] The main purpose of the present application is to provide a fall detection method, system, terminal and computer readable storage medium based on wrist inertial sensor, which aims to solve the problem that the existing fall detection method has low accuracy and cannot automatically identify the fall direction.

[0005] To achieve the above-mentioned purpose of the application, the present application provides a fall detection method based on wrist inertial sensor, which comprises:

[0006] Obtaining the motion data collected by the inertial sensor installed on the wrist of the human body, and preprocessing the motion data to obtain target motion data;

[0007] The fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model for fall detection multiple times, and multiple fall detection results are output;

[0008] If the fall detection results output by the fall detection model for a continuous setting number of times are all falls, it is confirmed that a fall occurs;

[0009] The fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network, if a fall occurs, the target motion data is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output.

[0010] Optionally, the motion data collected by the inertial sensor installed on the wrist of the human body is acquired, and the motion data is preprocessed to obtain target motion data, specifically including:

[0011] The motion data collected by the inertial sensor installed on the wrist of the human body is acquired in real time, and the inertial sensor includes a 9-axis inertial sensor chip;

[0012] The motion data is extracted by using a sliding window extraction method to obtain motion data corresponding to the sliding window;

[0013] The motion data corresponding to the sliding window is standardized by using a Z-value standardization method to obtain target motion data.

[0014] Optionally, the motion data corresponding to the sliding window is standardized by using a Z-value standardization method to obtain target motion data, specifically including:

[0015] The mean and standard deviation of the motion data corresponding to the sliding window are calculated;

[0016] The motion data corresponding to the sliding window is standardized according to the mean and the standard deviation to obtain target motion data conforming to a standard normal distribution, and the formula is as follows:

[0017]

[0018] Wherein, X * is target motion data; X is motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

[0019] Optionally, the fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model for fall detection multiple times, and multiple fall detection results are output, specifically including:

[0020] An initial fall detection model is constructed based on a structure of one-dimensional convolution and long short-term memory neural network;

[0021] The initial fall detection model is trained and tested by using a first historical data set collected, and a fall detection model is obtained, the first historical data set including fall sample data and daily behavior sample data, the daily behavior sample data including normal behavior sample data and approximate fall behavior sample data after recovering balance after slipping or tripping;

[0022] The target motion data conforming to a standard normal distribution is input into the fall detection model for fall detection, and a fall detection result is output, the fall detection result including fall and daily behavior, the daily behavior including normal behavior and approximate fall behavior after recovering balance after slipping or tripping.

[0023] Optionally, if the fall detection result output by the fall detection model for a continuous setting number of times is fall, it is confirmed that fall occurs, and specifically includes:

[0024] If the fall detection result output by the fall detection model for a continuous setting number of times is fall, it is confirmed that fall occurs.

[0025] If the fall detection result output by the fall detection model is not fall for a continuous setting number of times, it is confirmed that fall does not occur.

[0026] Optionally, a fall direction detection model is constructed based on a structure of one-dimensional convolution and long short-term memory neural network, and if fall occurs, the target motion data conforming to a standard normal distribution corresponding to the occurrence of fall is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output, and specifically includes:

[0027] An initial fall direction detection model is constructed based on a structure of one-dimensional convolution and long short-term memory neural network;

[0028] The initial fall direction detection model is trained and tested by using a second historical data set collected, and a fall direction detection model is obtained, the second historical data set including forward fall sample data, backward fall sample data, left fall sample data, right fall sample data and upright fall sample data;

[0029] If it is confirmed that fall occurs, the target motion data conforming to a standard normal distribution corresponding to the occurrence of fall is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output, the fall direction detection result including forward fall, backward fall, left fall, right fall and upright fall.

[0030] Optionally, the one-dimensional convolution and long short-term memory neural includes a convolution module, a long short-term memory neural network layer, a random inactivation layer, a regularization module, a full connection layer and a Softmax layer.

[0031] The convolution module includes a plurality of feature extraction blocks, and each feature extraction block includes a convolution layer, a ReLU layer and a max-pooling layer.

[0032] The regularization module includes a plurality of connection regularization blocks, and each connection regularization block includes a full connection layer, a ReLU layer and a random inactivation layer.

[0033] In addition, to achieve the above-mentioned purposes, the application further provides a fall detection system based on a wrist-mounted inertial sensor, which comprises:

[0034] A data acquisition and preprocessing module is configured to acquire motion data collected by the inertial sensor mounted on the wrist of a human body, and to preprocess the motion data to obtain target motion data.

[0035] A fall detection module is configured to construct a fall detection model based on a one-dimensional convolution and a long short-term memory neural network, to input the target motion data into the fall detection model multiple times for fall detection, and to output multiple fall detection results.

[0036] A fall occurrence judgment module is configured to confirm that a fall has occurred if the fall detection results output by the fall detection model for a continuous number of times are all falls.

[0037] A fall direction detection module is configured to construct a fall direction detection model based on a one-dimensional convolution and a long short-term memory neural network, to input the target motion data into the fall direction detection model for fall direction detection if a fall has occurred, and to output a fall direction detection result.

[0038] In addition, to achieve the above-mentioned purposes, the application further provides a terminal, which comprises a memory, a processor and a fall detection program based on a wrist-mounted inertial sensor stored in the memory and executable on the processor, and the fall detection program based on a wrist-mounted inertial sensor, when executed by the processor, implements the steps of the fall detection method based on a wrist-mounted inertial sensor.

[0039] Further, in order to achieve the above object, the application also provides a computer readable storage medium storing a wrist-based inertial sensor fall detection program, which, when executed by a processor, implements the steps of the above-mentioned wrist-based inertial sensor fall detection method.

[0040] In the application, motion data collected by an inertial sensor installed on a human wrist is acquired and preprocessed to obtain target motion data; a fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model for fall detection, and a fall detection result is output; if the fall detection result output by the fall detection model for a continuous number of times is all falls, it is determined that a fall has occurred; a fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network, if a fall has occurred, the target motion data is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output. The application uses the idea of step-by-step solution, detects a fall first and then identifies the fall direction, thereby reducing the difficulty of fall detection; the advantages of convolution neural network and long short-term memory neural network are combined to construct the fall detection model and the fall direction detection model, not only realizing automatic identification of the fall direction, but also improving the accuracy of fall detection; the judgment method based on multiple continuous sliding window data effectively avoids the error judgment of near-fall behaviors such as successfully recovering balance after sliding feet or stumbling as a fall, thereby further improving the accuracy and reliability of fall detection. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a flowchart of a preferred embodiment of the wrist-based inertial sensor fall detection method of the application;

[0042] Figure 2 is another flowchart of a preferred embodiment of the wrist-based inertial sensor fall detection method of the application;

[0043] Figure 3 is a structure diagram of one-dimensional convolution and long short-term memory neural network of the application;

[0044] Figure 4 is a curve diagram of the accuracy and loss value of the fall detection model of the application;

[0045] Figure 5 is a curve diagram of the accuracy and loss value of the fall direction detection model of the application;

[0046] Figure 6 is a schematic diagram of the fall direction identification result of the fall direction detection model of the application;

[0047] Figure 7is a structure diagram of a preferred embodiment of a wrist-based inertial sensor-based fall detection system of the present application;

[0048] Figure 8 is a structure diagram of a preferred embodiment of a terminal of the present application. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application.

[0050] Fall is an important issue of global public health. Fall is more common among the elderly, in addition to bringing serious threat to the physiological level of the elderly, but also can bring serious consequences in the psychological level of the elderly. Generally, the elderly fall will temporarily lose the ability to act or be in a coma, if not timely treatment, will cause "long time lying on the ground" phenomenon. This phenomenon often aggravates the injury of the elderly, and even cause death. Therefore, timely and effective detection of fall, and with the help of communication tools to send the corresponding alarm to its relatives, shorten the rescue time, is an important means to prevent the harm caused by "long time lying on the ground" after falling.

[0051] From the perspective of using equipment, fall detection can be divided into two categories. One is non-wearable fall detection research, which needs to monitor and detect with fixed equipment such as cameras, radars and infrared sensors. The other is wearable fall detection research, which will use wearable sensors such as accelerometers, gyroscopes, magnetometers or inertial measurement units. Using cameras to monitor the elderly in real time is an effective fall detection method. However, the presence of cameras can easily make the elderly feel that their behavior is monitored by others, and at the same time can easily infringe privacy. In addition, the fall detection based on vision has great time and space limitations, and it is difficult to detect in outdoor and other activity spaces.

[0052] Wearable fall detection can overcome the time and space limitations in the above-mentioned scheme. In the current wearable fall detection research, accelerometers, gyroscopes and inertial measurement units (IMU) are the most common sensor equipment. Compared with non-wearable fall detection, the biggest advantage of these sensors is low cost, non-invasive, and not constrained by fixed scenes. The principle of widely used wearable sensors in fall detection research is that when falling, the human body loses balance and will produce violent kinematic changes in a short time, especially the moment of collision between the body and the ground will produce an acceleration peak, and in this process, the fall detection algorithm recognizes the special difference of the signal through various means.

[0053] The sensor wearing position is a key factor affecting the accuracy of fall detection. Previous studies have found that the waist and chest are the best wearing positions. This is because the kinematic data of the center of gravity changes more dramatically during a fall, and the fall characteristics are more obvious. However, for most people, the waist and chest areas near the center of gravity are not comfortable. If the elderly are asked to wear them, they may resist. Placing wearable sensors on the wrist, integrated into a watch or smart bracelet, is more in line with the wearing habits of the general public and is more likely to be accepted by the elderly. With the popularity of smart bracelets, research on fall detection based on wrist sensors undoubtedly has high research value and application prospects.

[0054] However, research on fall detection based on wrist sensors is extremely challenging because human limbs are much more active than the torso, and some daily activities produce wrist sensor signals similar to those of a fall, such as pushing a door or clapping hands, which produce large acceleration values. Existing fall detection systems based on wrist sensors often rely on a single sensor or simple algorithms, resulting in insufficient accuracy in identifying different fall patterns and fall-like activities in complex environments, and prone to false positives or false negatives.

[0055] In addition, if the elderly fall into a coma after a fall, it is impossible to immediately identify these injuries that cannot be identified by the naked eye (fractures, concussions). Therefore, if the direction of the fall can be automatically identified, the injured part can be identified first to assist in medical treatment. Existing fall detection methods or devices based on wrist sensors can only detect fall events and cannot identify the direction of the fall. This limitation restricts the effectiveness of the assessment of the consequences of the fall and subsequent rescue measures.

[0056] In addition, current fall detection methods established using machine learning or deep learning models often do not consider "near-fall" behaviors such as successfully recovering balance after slipping or tripping, so these models are prone to misjudging such "near-fall" events as falls.

[0057] In order to solve the above technical problems, the application provides a wrist-based inertial sensor fall detection method, which comprises the following steps: acquiring motion data collected by an inertial sensor installed on the wrist of a human body and preprocessing the motion data to obtain target motion data; constructing a fall detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall detection model for fall detection, and outputting a fall detection result; if the fall detection result output by the fall detection model for a continuous setting number of times is all falls, it is determined that a fall occurs; constructing a fall direction detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall direction detection model for fall direction detection if a fall occurs, and outputting a fall direction detection result. The application uses the idea of step-by-step solution, detects a fall first and then identifies the fall direction, reduces the difficulty of fall detection, and combines the advantages of convolution neural network and long short-term memory neural network to construct a fall detection model and a fall direction detection model, which not only realizes automatic identification of the fall direction, facilitates subsequent rescue and medical intervention, but also improves the accuracy of fall detection, reduces false positives and false negatives of falls, and ensures accurate identification of fall events under various environmental and activity conditions. The judgment method based on multiple continuous sliding window data effectively avoids the error judgment of near-fall behaviors such as sliding feet or stumbling and successfully recovering balance as falls, and further improves the accuracy and reliability of fall detection.

[0058] The application content will be further described through the description of the embodiments in combination with the drawings.

[0059] The preferred embodiment of the wrist-based inertial sensor fall detection method of the application is specifically shown in Figure 1 and Figure 2 , and comprises the following steps.

[0060] S1, acquiring motion data collected by an inertial sensor installed on the wrist of a human body, and preprocessing the motion data to obtain target motion data.

[0061] In one implementation mode of the embodiment, the acquiring of the motion data collected by the inertial sensor installed on the wrist of the human body and the preprocessing of the motion data to obtain the target motion data specifically comprises the following steps.

[0062] S11, acquiring motion data collected by an inertial sensor installed on the wrist of a human body in real time, wherein the inertial sensor comprises a 9-axis inertial sensor chip;

[0063] S12, extracting the motion data by using a sliding window extraction method to obtain motion data corresponding to the sliding window;

[0064] S13, the Z value standardization method is used to standardize the motion data corresponding to the sliding window to obtain target motion data.

[0065] In one implementation of the embodiment, the Z value standardization method is used to standardize the motion data corresponding to the sliding window to obtain target motion data, specifically including:

[0066] S131, the mean and standard deviation of the motion data corresponding to the sliding window are calculated.

[0067] S132, the motion data corresponding to the sliding window is standardized according to the mean and the standard deviation to obtain target motion data conforming to the standard normal distribution, which is expressed as follows:

[0068]

[0069] Wherein, X * is target motion data; X is motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

[0070] Specifically, the wrist inertia sensor designed by the application is used to obtain the motion data of the wrist through the communication protocol. The inertia sensor is loaded with a 9-axis inertia sensing chip BNO055, and the inertia sensing chip and the terminal communicate data through the master control chip nRF51822 in the Bluetooth wireless communication protocol.

[0071] Further, after the terminal receives the inertia sensor data (i.e. the motion data collected by the inertia sensor), the data is first preprocessed, including extracting the sliding window data by the sliding window extraction method and arranging the inertia sensor data of different units by the Z value standardization method, so as to obtain the neural network model input standard data (i.e. target motion data). The sliding window is a fixed size overlapping sliding window, and the length is preferably 1.5 seconds, and the overlap rate of each sliding window is preferably 90%. The Z value standardization first calculates the mean μ and the standard deviation σ of the inertia sensor data X in the sliding window, and then calculates the data set by the formula, so that the original inertia sensor data becomes the neural network model input standard data X* conforming to the standard normal distribution of μ=0, σ=1.

[0072] Considering the possible judgment error of the deep learning model, the application judges the fall according to multiple continuous sliding window data, which helps to reduce the misjudgment of the fall event.

[0073] S2, a fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model multiple times for fall detection, and multiple fall detection results are output.

[0074] In one implementation form of the embodiment, the fall detection model is constructed based on a one-dimensional convolution and a long short-term memory neural network, the target motion data is input into the fall detection model multiple times for fall detection, and multiple fall detection results are output. Specifically, the method comprises the following steps.

[0075] S21, constructing an initial fall detection model based on a one-dimensional convolution and a long short-term memory neural network structure;

[0076] S22, training and testing the initial fall detection model by using a collected first historical data set to obtain a fall detection model, wherein the first historical data set comprises fall sample data and daily behavior sample data, and the daily behavior sample data comprises normal behavior sample data and approximate fall behavior sample data after slipping or tripping and recovering balance;

[0077] S23, inputting the target motion data conforming to a standard normal distribution into the fall detection model for fall detection, and outputting a fall detection result, wherein the fall detection result comprises fall and daily behavior, and the daily behavior comprises normal behavior and approximate fall behavior after slipping or tripping and recovering balance.

[0078] Specifically, by establishing a fall detection model, the above-mentioned neural network model input standard data is received in real time, and detection and identification of two cases of “fall” and “daily behavior” are performed in real time, wherein the “daily behavior” further comprises approximate fall behavior after slipping or tripping and recovering balance. The fall detection model adopts a one-dimensional convolution and a long short-term memory neural network structure. Considering that the accuracy of existing fall detection algorithms based on deep learning models is not high, the advantages of convolutional neural networks and long short-term memory neural networks are fused to construct a fall detection model, thereby improving the accuracy of fall detection. In the model training process, falls such as slipping and tripping that are closer to real fall scenarios are taken into account, and approximate fall behavior after successfully recovering balance after slipping and tripping is considered, so that this model can avoid misjudging “approximate fall” events as falls.

[0079] S3, if the fall detection results output by the fall detection model for a continuous setting number of times are all falls, it is confirmed that a fall occurs.

[0080] In one implementation form of the embodiment, if the fall detection results output by the fall detection model for a continuous setting number of times are all falls, it is confirmed that a fall occurs. Specifically, the method comprises the following steps.

[0081] S31, if the fall detection results output by the fall detection model for a continuous setting number of times are all falls, it is confirmed that a fall occurs.

[0082] S32, if the fall detection result output by the fall detection model is not a fall for a continuous set number of times, it is determined that a fall has not occurred.

[0083] Specifically, the output result of the fall detection model is traversed. When the model output result is "fall" for a continuous set number of times (such as seven times), it is determined that a fall has occurred at this time, and the result is fed back in real time. When the model output result is "fall", but does not meet the condition of seven times in a row, it is determined that this is a daily behavior.

[0084] S4, a fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output.

[0085] In one implementation of the embodiment, the fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output. Specifically, it includes:

[0086] S41, an initial fall direction detection model is constructed based on the structure of one-dimensional convolution and long short-term memory neural network;

[0087] S42, the initial fall direction detection model is trained and tested using a collected second historical data set to obtain a fall direction detection model. The second historical data set includes forward fall sample data, backward fall sample data, left fall sample data, right fall sample data, and upright fall sample data.

[0088] S43, if it is determined that a fall has occurred, the target motion data conforming to the standard normal distribution corresponding to the fall is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output. The fall direction detection result includes forward fall, backward fall, left fall, right fall, and upright fall.

[0089] Specifically, when the terminal receives the judgment result of the occurrence of the fall, the fall direction detection model is activated quickly, the fall direction detection model receives the standard data of the neural network model in real time, and the fall signal is further classified and recognized, and the classification result includes five kinds, namely, forward fall, backward fall, left fall, right fall and upright fall (dizziness). The structure of the fall direction detection model is similar to that of the fall detection model, but the two models differ in that the data sets used to train the two models are different. The data used to train the fall detection model adopts a data set of falls (including slips, stumbles, etc.) and daily behaviors (including normal behaviors and approximate fall behaviors). The data used to train the fall direction detection model adopts a data set of forward fall, backward fall, left fall, right fall and upright fall (dizziness), so the final hyperparameters of the two models are different.

[0090] The present application realizes automatic recognition of the fall direction through the fall direction detection model, can assist in medical treatment according to the possible injured part in the first time, and improves the effectiveness of the evaluation of the fall consequences and the subsequent rescue measures.

[0091] In view of the low accuracy of the existing fall detection method, the present application uses the idea of step-by-step solution, first performs two-classification judgment on "fall" and "daily behavior", and then performs multi-classification judgment on the fall direction in the "fall" event. This method reduces the difficulty of model recognition in each link, and solves the problems of complexity and diversification of fall actions, and individual differences of individuals when falling, by ensuring the accuracy of each step-by-step model.

[0092] In one implementation manner of the embodiment, the one-dimensional convolution and long short-term memory nerve includes a convolution module, a long short-term memory neural network layer, a random inactivation layer, a regularization module, a full connection layer and a Softmax layer.

[0093] The convolution module includes a plurality of feature extraction blocks, and each feature extraction block includes a convolution layer, a ReLU layer and a maximum pooling layer.

[0094] The regularization module includes a plurality of connection regularization blocks, and each connection regularization block includes a full connection layer, a ReLU layer and a random inactivation layer.

[0095] Specifically, the structures of the fall detection model and the fall direction detection model both adopt one-dimensional convolution and long short-term memory neural network, as shown in Figure 3As shown, the one-dimensional convolution and long short-term memory neural network comprises a convolution module 1, a long short-term memory neural network layer 2, a random inactivation layer 3, a regularization module 4, a full connection layer 5 and a Softmax layer 6, and the convolution module 1 is composed of two stacked feature extraction blocks 7 (FEB). Each feature extraction block 7 is composed of a convolution layer 8 (Conv), a ReLU layer 9 and a maximum pooling layer 10 (Max Pooling, MP). Among them, the convolution layer 8 is used to extract the local features of the input data, the ReLU layer 9 is used to introduce nonlinearity, and the maximum pooling layer 10 is used to reduce the dimension of the feature map, thereby reducing the computational complexity and preventing overfitting. The two stacked feature extraction blocks 7 can extract higher-level features layer by layer, so that the model can better capture the complex patterns in the input data, and in addition, the stacked structure can more effectively pass information, so that the subsequent layers can utilize the rich features extracted by the previous layers.

[0096] The regularization module 4 is composed of two connected regularization blocks 8 (CRB). Each connected regularization block 8 is composed of a full connection layer 5, a ReLU layer 10 and a random inactivation layer 3. Among them, the full connection layer 5 is used to map the features to the output space, the ReLU layer 10 introduces nonlinearity to enhance the expression ability of the model, and the random inactivation layer 3 helps to prevent overfitting by randomly discarding part of the neurons, thereby improving the generalization ability of the model. The design of the two stacked connected regularization blocks 8 helps the model to converge faster during training through regularization and nonlinear activation, improves the overall training efficiency, and avoids the problem of gradient disappearance. And the random inactivation layer 3 introduced therein can reduce the dependence of the model on training data and reduce the risk of overfitting.

[0097] In addition, the fall detection method based on the wrist inertial sensor further comprises data visualization, real-time monitoring of data generated by the wrist inertial sensor IMU, and detection and identification results of the fall detection model and the fall direction detection model, and programming in Python language to present in a graphical manner on the user interface to realize data visualization.

[0098] In order to verify the performance of the fall detection method based on the wrist inertial sensor of the application, after the model is established, model verification experiments are carried out, 5491 fall samples and 5118 daily behavior samples are obtained through experimental means, the sample quantity ratio is approximately 1:1, and the "class imbalance" phenomenon is effectively avoided. As shown in Figure 4 As shown in Figure 4 (a) in the figure is the accuracy curve of the fall detection model, Figure 4 (b) in the figure is the loss value curve of the fall detection model, and the fall detection model verification set accuracy curve gradually stabilizes after 20 iterations, and the loss value curve also indicates that the model training is successful. As shown in Figure 5 As shown in Figure 5(a) in FIG. 6 is an accuracy curve of the fall direction detection model, Figure 5 (b) in FIG. 6 is a loss value curve of the fall direction detection model, the fall direction detection model validation set accuracy curve keeps rising in the process of ups and downs, the loss value curve keeps a downward trend in the process of ups and downs, the change trend conforms to the expectation, according to engineering application experience, it is explained that the model training is successful.

[0099] Through cross-validation, the performance evaluation results of the fall detection model and the fall direction detection model are obtained. As shown in Table 1, the accuracy of the fall detection model for "fall" and "daily behavior" detection reaches 93.56%.

[0100] Table 1: Performance evaluation index of fall detection model

[0101] Evaluation metrics Accuracy Precision Recall Specificity F1 score Fall detection model 93.56％ 89.74％ 95.24％ 92.38％ 92.41％

[0102] The results of the fall direction detection model for fall direction recognition are shown in Table 2. Figure 6 The overall accuracy is 74.29%. The correct recognition rate of upright fall, i.e. dizziness, reaches 93.75%, and the correct recognition rates of the other four categories are between 52.38% and 85.37%.

[0103] In addition, based on the above-mentioned fall detection method based on wrist inertial sensor, the application also correspondingly provides a fall detection system based on wrist inertial sensor, and a preferred embodiment of the fall detection system based on wrist inertial sensor is shown in FIG. 7. Figure 7 As shown in FIG. 7, the fall detection system based on wrist inertial sensor comprises:

[0104] A data acquisition and preprocessing module 01 is configured to acquire motion data collected by the inertial sensor installed on the wrist of the human body, and preprocess the motion data to obtain target motion data.

[0105] A fall detection module 02 is configured to construct a fall detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall detection model multiple times for fall detection, and output multiple fall detection results.

[0106] A fall occurrence judgment module 03 is configured to confirm that a fall occurs if the fall detection results output by the fall detection model for a continuous set number of times are all falls.

[0107] A fall direction detection module 04 is configured to construct a fall direction detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall direction detection model for fall direction detection if a fall occurs, and output a fall direction detection result.

[0108] Further, based on the above-described fall detection method and system using a wrist-based inertial sensor, the present application also provides a terminal, a preferred embodiment of which, as shown in Figure 8 includes a processor 10, a memory 20, and a display 30. Figure 8 Only some components of the terminal are shown, but it should be understood that all the shown components are not required, and more or less components can be alternatively implemented.

[0109] The memory 20 can be an internal storage unit of the terminal, such as a hard disk or a memory, in some embodiments. The memory 20 can also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, and the like, in other embodiments. Further, the memory 20 can include both an internal storage unit and an external storage device of the terminal. The memory 20 is used to store application software and various data installed in the terminal, such as program codes of the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a fall detection program 40 using a wrist-based inertial sensor, which can be executed by the processor 10 to implement the steps of the fall detection method using a wrist-based inertial sensor described in the present application.

[0110] The processor 10 can be a Central Processing Unit (CPU), a microprocessor, or other data processing chip, which is used to run program codes or process data stored in the memory 20, such as the fall detection program 40 using a wrist-based inertial sensor, in some embodiments.

[0111] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, and the like, in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0112] In an embodiment, the steps of the fall detection method using a wrist-based inertial sensor described above are implemented when the processor 10 executes the fall detection program 40 using a wrist-based inertial sensor stored in the memory 20.

[0113] The application also provides a computer readable storage medium, wherein the computer readable storage medium stores a wrist-based inertial sensor fall detection program, and the wrist-based inertial sensor fall detection program, when executed by a processor, implements the steps of the wrist-based inertial sensor fall detection method.

[0114] It should be noted that, in this document, the terms "comprising" and "including" or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include inherent elements of such processes, methods, articles, or terminals. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or terminal including the element.

[0115] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a magnetic disc, an optical disc, etc.

[0116] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should be within the protection scope of the appended claims of the application.

Claims

1. A fall detection method based on an inertial sensor on the wrist, characterized in that, The fall detection method based on an inertial sensor at the wrist includes: The motion data collected by the inertial sensor installed on the wrist of the human body is acquired, and the motion data is preprocessed to obtain the target motion data; A fall detection model is constructed based on a one-dimensional convolutional and long short-term memory neural network. The target motion data is input into the fall detection model multiple times to perform fall detection and output multiple fall detection results. The fall detection model, constructed based on a one-dimensional convolutional and long short-term memory neural network, inputs the target motion data multiple times into the fall detection model to perform fall detection and outputs multiple fall detection results, specifically including: An initial fall detection model was constructed based on the structure of a one-dimensional convolutional and long short-term memory neural network. The initial fall detection model is trained and tested using the collected first historical dataset to obtain a fall detection model. The first historical dataset includes fall sample data and daily behavior sample data. The daily behavior sample data includes normal behavior sample data and approximate fall behavior sample data of regaining balance after slipping or tripping. The target motion data conforming to the standard normal distribution is input into the fall detection model for fall detection, and the fall detection results are output. The fall detection results include falls and daily behaviors. The daily behaviors include normal behaviors and approximate fall behaviors such as regaining balance after slipping or tripping. If the fall detection model outputs a fall result for a set number of consecutive times, then a fall is confirmed to have occurred. A fall direction detection model is constructed based on a one-dimensional convolutional and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to detect the fall direction and output the fall direction detection result. The fall direction detection model, constructed based on a one-dimensional convolutional and long short-term memory neural network, detects the fall direction if a fall occurs. Specifically, it inputs the target motion data into the fall direction detection model to perform fall direction detection and outputs the fall direction detection result. An initial fall direction detection model is constructed based on the structure of a one-dimensional convolutional and long short-term memory neural network. The initial fall direction detection model is trained and tested using the collected second historical dataset to obtain the fall direction detection model. The second historical dataset includes forward fall sample data, backward fall sample data, left fall sample data, right fall sample data, and upright fall sample data. If a fall is confirmed, the target motion data corresponding to the fall, which conforms to a standard normal distribution, is input into the fall direction detection model for fall direction detection, and the fall direction detection result is output. The fall direction detection result includes falling forward, falling backward, falling to the left, falling to the right, and falling upright.

2. The fall detection method based on an inertial sensor at the wrist according to claim 1, characterized in that, The process of acquiring motion data from an inertial sensor mounted on the wrist and preprocessing the motion data to obtain target motion data specifically includes: The system acquires motion data in real time from an inertial sensor installed on the wrist of the human body. The inertial sensor includes a 9-axis inertial sensing chip. The motion data is extracted using a sliding window extraction method to obtain the motion data corresponding to the sliding window; The motion data corresponding to the sliding window is standardized using the Z-value standardization method to obtain the target motion data.

3. The fall detection method based on an inertial sensor at the wrist according to claim 2, characterized in that, The step of standardizing the motion data corresponding to the sliding window using the Z-value standardization method to obtain the target motion data specifically includes: Calculate the mean and standard deviation of the motion data corresponding to the sliding window; Based on the mean and standard deviation, the motion data corresponding to the sliding window is standardized to obtain target motion data that conforms to a standard normal distribution, as expressed by the following formula: Among them, X * denoted as Target motion data; X represents the motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

4. The fall detection method based on an inertial sensor at the wrist according to claim 1, characterized in that, If the fall detection model outputs a fall detection result for a set number of consecutive times, then a fall is confirmed to have occurred, specifically including: If the fall detection model outputs a fall detection result for a set number of consecutive times, then a fall is confirmed to have occurred. If the fall detection results output by the fall detection model do not show a continuous number of falls, then it is confirmed that no fall has occurred.

5. The fall detection method based on an inertial sensor at the wrist according to claim 1, characterized in that, The one-dimensional convolutional and long short-term memory neural network includes a convolutional module, a long short-term memory neural network layer, a random deactivation layer, a regularization module, a fully connected layer, and a Softmax layer; The convolution module includes several feature extraction blocks, each of which includes a convolutional layer, a ReLU layer, and a max pooling layer. The convolutional layer is used to extract local features of the input data, the ReLU layer is used to introduce non-linearity, and the max pooling layer is used to reduce the dimensionality of the feature map. The regularization module includes several connection regularization blocks, each including a fully connected layer, a ReLU layer, and a random deactivation layer. The fully connected layer is used to map features to the output space, the ReLU layer is used to introduce nonlinearity, and the random deactivation layer is used to randomly discard some neurons to prevent overfitting.

6. A fall detection system based on a wrist-based inertial sensor, wherein the fall detection system based on the wrist-based inertial sensor is applied to the fall detection method based on a wrist-based inertial sensor as described in any one of claims 1-5, characterized in that, The fall detection system based on an inertial sensor on the wrist includes: Data acquisition and preprocessing module: used to acquire motion data collected by an inertial sensor installed on the wrist of the human body, and to preprocess the motion data to obtain target motion data; Fall detection module: used to build a fall detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall detection model multiple times to perform fall detection, and output multiple fall detection results; Fall detection module: If the fall detection model outputs a fall result for a set number of consecutive times, then the fall is confirmed to have occurred. Fall direction detection module: It is used to build a fall direction detection model based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to detect the fall direction and output the fall direction detection result.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a fall detection program based on a wrist-based inertial sensor stored in the memory and executable on the processor. When the fall detection program based on a wrist-based inertial sensor is executed by the processor, it implements the steps of the fall detection method based on a wrist-based inertial sensor as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a fall detection program based on a wrist-based inertial sensor, which, when executed by a processor, implements the steps of the fall detection method based on a wrist-based inertial sensor as described in any one of claims 1-5.

Images