Fall detection method and system based on inertial sensor of wrist part

Abstract

The invention discloses a fall detection method and system based on an inertial sensor of a wrist, and the method comprises the steps: obtaining motion data collected by the inertial sensor installed at the wrist of a human body, and carrying out the preprocessing, and obtaining target motion data; the method comprises the following steps: constructing a tumble detection model and a tumble direction detection model based on a one-dimensional convolution and long-short-term memory neural network, carrying out tumble detection by using the tumble detection model, determining that tumble occurs if tumble is detected continuously for set times, then carrying out tumble direction detection by using the tumble direction detection model, and outputting a tumble direction detection result. The tumble detection model and the tumble direction detection model are constructed by fusing the advantages of the convolutional neural network and the long-short-term memory neural network, so that the tumble direction is automatically identified, and the tumble detection accuracy is improved; the judgment method based on multiple continuous sliding window data effectively prevents the approximate tumble behavior from being wrongly judged as tumble.

Description

Technical Field

[0001] The present invention relates to the field of fall detection technology, and in particular to a fall detection method, system, terminal and computer-readable storage medium based on an inertial sensor on a wrist. Background Art

[0002] Falling is an important public health issue. Falling is more common among the elderly. In addition to posing a serious threat to the elderly at the physiological level, it also has serious consequences at the psychological level. From the perspective of the use of equipment, fall detection can be divided into two categories. One is non-wearable fall detection research, which requires the use of fixed equipment such as cameras, radars and infrared sensors for monitoring and detection, which is easy to invade privacy, cause discomfort, and has large temporal and spatial limitations; the other is wearable fall detection research, which uses wearable sensors such as accelerometers, gyroscopes, magnetometers or inertial measurement units for detection. Considering that the kinematic data of the center of gravity of the human body changes more dramatically when falling, the characteristics of falling are more obvious. Usually, the waist or chest close to the center of gravity is used as the wearing part. However, wearing the sensor in the waist or chest area is not comfortable. With the development of smart watches or smart bracelets, people began to consider wearing sensors on their wrists. However, human limbs are much more active than the torso, and the sensor signals generated by some daily movements are close to fall signals, which makes the existing fall detection algorithms based on wrist sensors prone to false positives or false negatives, and the accuracy is not high. In addition, the existing fall detection algorithms generally cannot automatically identify the direction of the fall.

[0003] In addition, there are some fall detection methods in the prior art that use machine learning or deep learning models, but they often do not take into account "near-fall" behaviors such as successfully regaining balance after slipping or tripping. Therefore, these models are prone to misjudge such "near-fall" behaviors as falls. Summary of the invention

[0004] The main purpose of the present invention is to provide a fall detection method, system, terminal and computer-readable storage medium based on an inertial sensor on the wrist, aiming to solve the problem that the existing fall detection method has low accuracy and cannot automatically identify the direction of fall.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a fall detection method based on an inertial sensor on a wrist, and the fall detection method based on an inertial sensor on a wrist comprises:

[0006] Acquire motion data collected by an inertial sensor installed on a human wrist, and pre-process the motion data to obtain target motion data;

[0007] Building a fall detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall detection model for multiple times to perform fall detection, and outputting multiple fall detection results;

[0008] If the fall detection results output by the fall detection model for a set number of consecutive times are all falls, it is confirmed that a fall has occurred;

[0009] A fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output.

[0010] Optionally, the acquiring motion data collected by an inertial sensor installed on a human wrist and preprocessing the motion data to obtain target motion data specifically includes:

[0011] Acquire motion data collected by an inertial sensor installed on a human wrist in real time, wherein the inertial sensor includes a 9-axis inertial sensor chip;

[0012] Extracting the motion data using a sliding window extraction method to obtain motion data corresponding to the sliding window;

[0013] The motion data corresponding to the sliding window is standardized by using a Z value standardization method to obtain target motion data.

[0014] Optionally, the adopting a Z value normalization method to perform normalization processing on the motion data corresponding to the sliding window to obtain the target motion data specifically includes:

[0015] Calculating the mean and standard deviation of the motion data corresponding to the sliding window;

[0016] According to the mean and the standard deviation, the motion data corresponding to the sliding window is standardized to obtain target motion data that conforms to the standard normal distribution. The formula is as follows:

[0017]

[0018] Among them, X * is the target motion data; X is the motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

[0019] Optionally, the building of a fall detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall detection model for multiple times to perform fall detection, and outputting multiple fall detection results specifically includes:

[0020] Based on the structure of one-dimensional convolution and long short-term memory neural network, an initial fall detection model is constructed;

[0021] The initial fall detection model is trained and tested using the collected first historical data set to obtain a fall detection model, wherein the first historical data set includes fall sample data and daily behavior sample data, wherein the daily behavior sample data includes normal behavior sample data and approximate fall behavior sample data of recovering balance after slipping or stumbling;

[0022] The target motion data conforming to the standard normal distribution is input into the fall detection model for fall detection, and a fall detection result is output. The fall detection result includes falls and daily behaviors. The daily behaviors include normal behaviors and approximate fall behaviors of recovering balance after slipping or stumbling.

[0023] Optionally, if the fall detection results output by the fall detection model for a set number of consecutive times are all falls, confirming that a fall has occurred specifically includes:

[0024] If the fall detection results output by the fall detection model are all falls for a set number of consecutive times, it is confirmed that a fall has occurred;

[0025] If the fall detection results output by the fall detection model do not show falls for a set number of consecutive times, it is confirmed that a fall has not occurred.

[0026] Optionally, the fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network, and if a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output, which specifically includes:

[0027] Based on the structure of one-dimensional convolution and long short-term memory neural network, an initial fall direction detection model is constructed;

[0028] The initial fall direction detection model is trained and tested using the collected second historical data set to obtain a fall direction detection model, wherein the second historical data set includes forward fall sample data, backward fall sample data, left fall sample data, right fall sample data, and upright fall sample data;

[0029] If a fall is confirmed to have occurred, the target motion data corresponding to the fall that conforms to the standard normal distribution is input into the fall direction detection model for fall direction detection, and the fall direction detection result is output. The fall direction detection result includes falling forward, falling backward, falling to the left, falling to the right and falling upright.

[0030] Optionally, the one-dimensional convolution and long short-term memory neural network includes a convolution module, a long short-term memory neural network layer, a random inactivation layer, a regularization module, a fully connected layer and a Softmax layer;

[0031] The convolution module includes several feature extraction blocks, each of which includes a convolution layer, a ReLU layer and a maximum pooling layer, wherein the convolution layer is used to extract local features of input data, the ReLU layer is used to introduce nonlinearity, and the maximum pooling layer is used to reduce the dimension of the feature map;

[0032] The regularization module includes several connection regularization blocks, and the connection regularization block includes a fully connected layer, a ReLU layer and a random dropout layer, wherein the fully connected layer is used to map features to the output space, the ReLU layer is used to introduce nonlinearity, and the random dropout layer is used to randomly discard some neurons to prevent overfitting.

[0033] In addition, to achieve the above-mentioned purpose of the invention, the present invention also provides a fall detection system based on an inertial sensor on a wrist, and the fall detection system based on an inertial sensor on a wrist comprises:

[0034] Data acquisition and preprocessing module: used to acquire motion data collected by the inertial sensor installed on the human wrist, and preprocess the motion data to obtain target motion data;

[0035] Fall detection module: used to build a fall detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall detection model for multiple times to perform fall detection, and output multiple fall detection results;

[0036] A fall occurrence judgment module: used to confirm that a fall has occurred if the fall detection results output by the fall detection model for a set number of consecutive times are all falls;

[0037] Fall direction detection module: used to build a fall direction detection model based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection and output the fall direction detection result.

[0038] In addition, to achieve the above-mentioned purpose of the invention, the present invention also provides a terminal, which includes: a memory, a processor, and a fall detection program based on an inertial sensor on the wrist stored in the memory and executable on the processor, wherein the fall detection program based on the inertial sensor on the wrist, when executed by the processor, implements the steps of the fall detection method based on the inertial sensor on the wrist as described above.

[0039] In addition, to achieve the above-mentioned purpose of the invention, the present invention also provides a computer-readable storage medium, which stores a fall detection program based on an inertial sensor on the wrist. When the fall detection program based on the inertial sensor on the wrist is executed by a processor, the steps of the fall detection method based on the inertial sensor on the wrist as described above are implemented.

[0040] In the present invention, motion data collected by an inertial sensor installed on a human wrist is obtained and the motion data is preprocessed to obtain target motion data; a fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model for fall detection, and a fall detection result is output; if the fall detection results output by the fall detection model for a set number of consecutive times are all falls, it is confirmed that a fall has occurred; a fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network, and if a fall occurs, the target motion data is input into the fall direction detection model for fall direction detection, and a fall direction detection result is output. The present invention utilizes the idea of ​​solving the problem in steps, first detecting a fall and then identifying the direction of the fall, thereby reducing the difficulty of fall detection; integrating the advantages of a convolutional neural network and a long short-term memory neural network to construct a fall detection model and a fall direction detection model, not only realizing automatic identification of the direction of the fall, but also improving the accuracy of fall detection; the judgment method based on multiple continuous sliding window data effectively avoids misjudging similar fall behaviors such as successfully recovering balance after slipping or stumbling as falls, thereby further improving the accuracy and reliability of fall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a preferred embodiment of a fall detection method based on an inertial sensor on a wrist of the present invention;

[0042] Figure 2 is another flow chart of a preferred embodiment of the fall detection method based on the wrist inertial sensor of the present invention;

[0043] Figure 3 It is a structural diagram of the one-dimensional convolution and long short-term memory neural network of the present invention;

[0044] Figure 4 is a curve diagram of the accuracy and loss value of the fall detection model of the present invention;

[0045] Figure 5 is a curve diagram of the accuracy and loss value of the fall direction detection model of the present invention;

[0046] Figure 6 is a schematic diagram of a fall direction recognition result of a fall direction detection model of the present invention;

[0047] Figure 7is a structural diagram of a preferred embodiment of a fall detection system based on an inertial sensor on a wrist of the present invention;

[0048] Figure 8 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] Falls are a major global public health issue. Falls are more common among the elderly, and in addition to posing a serious threat to the elderly on a physiological level, they also have serious consequences on the psychological level. Usually, after a fall, the elderly will temporarily lose their ability to move or fall into a coma. If they are not treated in time, they will "lie on the ground for a long time". This phenomenon often aggravates the injuries of the elderly and even causes death. Therefore, timely and effective detection of falls, and the use of communication tools to send out corresponding alarms to notify their relatives and shorten the rescue time are important means to prevent injuries caused by "lying on the ground for a long time" after a fall.

[0051] From the perspective of the equipment used, fall detection can be divided into two categories. One is non-wearable fall detection research, which requires the use of fixed equipment such as cameras, radars and infrared sensors for monitoring and detection. The other is wearable fall detection research, which will use wearable sensors such as accelerometers, gyroscopes, magnetometers or inertial measurement units. Using cameras to monitor the elderly in real time is an effective way to detect falls. However, the presence of cameras can easily make the elderly feel that their behavior is being monitored by others, and it is easy to violate their privacy. In addition, vision-based fall detection has large temporal and spatial limitations, and it is difficult to detect in activity spaces such as outdoors.

[0052] Wearable fall detection can overcome the time and space limitations of the above solutions. In the current wearable fall detection research, accelerometers, gyroscopes and inertial measurement units (IMUs) are the most common sensor devices. Compared with non-wearable fall detection, the biggest advantage of these sensors is that they are low-cost, non-invasive, and not constrained by fixed scenes. The principle that wearable sensors are widely used in fall detection research is that when falling, the human body loses balance, which will produce drastic changes in kinematics in a short period of time, especially the moment when the body collides with the ground, an acceleration peak will be generated. In this process, the fall detection algorithm recognizes the special differences in the signal through various means.

[0053] The location where the sensor is worn is a key factor affecting the accuracy of fall detection. Previous studies have found that the waist and chest are the best places to wear the sensor. This is because the kinematic data of the center of gravity of the human body changes dramatically when falling, and the characteristics of the fall are more obvious. However, for most people, the areas close to the center of gravity of the human body, such as the waist and chest, are less comfortable. If the elderly are asked to wear it, I am afraid they will resist. Placing wearable sensors on the wrist and integrating them into watches or smart bracelets is more in line with the public's wearing habits and is easier for the elderly to accept. With the popularity of smart bracelets, fall detection research based on wrist sensors undoubtedly has high research value and application prospects.

[0054] However, the research on fall detection based on wrist sensors is extremely challenging because human limbs are much more active than the torso, and the wrist sensor signals generated by some daily actions are close to fall signals, such as pushing a door and clapping, which generate large acceleration values. Existing fall detection systems based on wrist sensors often rely on a single sensor or simple algorithms, resulting in insufficient accuracy in identifying different fall modes and fall-like activities in complex environments, and are prone to false positives or false negatives of falls.

[0055] In addition, if an elderly person is in a coma after a fall, it is impossible to immediately confirm these injuries (fractures, concussions) that cannot be identified by the naked eye. Therefore, if the direction of the fall can be automatically identified, medical assistance can be provided according to the possible injured parts in the first place. Existing fall detection methods or devices based on wrist sensors can only detect fall events, but cannot identify the direction of the fall. This defect limits the effectiveness of the assessment of the consequences of the fall and subsequent rescue measures.

[0056] In addition, current fall detection methods built using machine learning or deep learning models often do not take into account "near-fall" behaviors such as successfully regaining balance after slipping or tripping. Therefore, these models are prone to misjudging such "near-fall" events as falls.

[0057] In order to solve the above technical problems, the present invention provides a fall detection method based on an inertial sensor on a wrist, and the fall detection method based on an inertial sensor on a wrist comprises: acquiring motion data collected by an inertial sensor installed on a human wrist and preprocessing the motion data to obtain target motion data; constructing a fall detection model based on one-dimensional convolution and a long short-term memory neural network, inputting the target motion data into the fall detection model for fall detection, and outputting a fall detection result; if the fall detection results output by the fall detection model for a set number of consecutive times are all falls, it is confirmed that a fall has occurred; constructing a fall direction detection model based on one-dimensional convolution and a long short-term memory neural network, and if a fall occurs, inputting the target motion data into the fall direction detection model for fall direction detection, and outputting a fall direction detection result. The present invention utilizes the idea of ​​solving the problem in steps, first detecting a fall and then identifying the direction of the fall, thereby reducing the difficulty of fall detection; integrating the advantages of a convolutional neural network and a long short-term memory neural network to construct a fall detection model and a fall direction detection model, not only realizing automatic identification of the direction of the fall, facilitating subsequent rescue and medical intervention, but also improving the accuracy of fall detection, reducing false positives and false negatives of falls, and ensuring that fall events can be accurately identified in various environments and activity conditions; the judgment method based on multiple continuous sliding window data effectively avoids misjudging similar fall behaviors such as successfully recovering balance after slipping or stumbling as falls, thereby further improving the accuracy and reliability of fall detection.

[0058] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0059] A preferred embodiment of the fall detection method based on the inertial sensor on the wrist of the present invention is as follows: Figure 1 and Figure 2 As shown, specifically including:

[0060] S1. Acquire motion data collected by an inertial sensor installed on a human wrist, and pre-process the motion data to obtain target motion data.

[0061] In an implementation of this embodiment, acquiring motion data collected by an inertial sensor installed on a human wrist and preprocessing the motion data to obtain target motion data specifically includes:

[0062] S11, acquiring in real time motion data collected by an inertial sensor installed on a human wrist, wherein the inertial sensor includes a 9-axis inertial sensor chip;

[0063] S12, extracting the motion data using a sliding window extraction method to obtain motion data corresponding to the sliding window;

[0064] S13, using a Z value standardization method to standardize the motion data corresponding to the sliding window to obtain target motion data.

[0065] In an implementation of this embodiment, the Z-value normalization method is used to normalize the motion data corresponding to the sliding window to obtain the target motion data, specifically including:

[0066] S131, calculating the mean and standard deviation of the motion data corresponding to the sliding window;

[0067] S132, performing standardization processing on the motion data corresponding to the sliding window according to the mean and the standard deviation to obtain target motion data conforming to a standard normal distribution, which is expressed as follows:

[0068]

[0069] Among them, X * is the target motion data; X is the motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

[0070] Specifically, the present invention adopts a self-designed wrist inertial sensor to obtain wrist motion data through a communication protocol. The inertial sensor is equipped with a 9-axis inertial sensor chip BNO055, and the inertial sensor chip communicates with the terminal through the main control chip nRF51822 using the Bluetooth wireless communication protocol.

[0071] Furthermore, after receiving the inertial sensor data (i.e., the motion data collected by the inertial sensor), the terminal first pre-processes the data, including extracting the sliding window data through the sliding window extraction method and using the Z value standardization method to sort out the inertial sensor data of different units, thereby obtaining the neural network model input standard data (i.e., the target motion data). Among them, the sliding window is an overlapping sliding window of a fixed size, the length is preferably 1.5 seconds, and the overlap rate of each sliding window is preferably 90%. The Z value standardization first calculates the mean μ and standard deviation σ of the inertial sensor data X in the sliding window, and then performs a standardized processing calculation on the data set through a formula, so that the original inertial sensor data becomes the neural network model input standard data X* that conforms to the standard normal distribution of μ=0, σ=1.

[0072] Taking into account the possible judgment errors of the deep learning model, the present invention judges falls based on multiple continuous sliding window data, which helps to reduce the misjudgment of fall events.

[0073] S2. Building a fall detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall detection model for multiple times to perform fall detection, and outputting multiple fall detection results.

[0074] In an implementation of this embodiment, the fall detection model is constructed based on the one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model multiple times to perform fall detection, and multiple fall detection results are output, specifically including:

[0075] S21. Construct an initial fall detection model based on the structure of one-dimensional convolution and long short-term memory neural network;

[0076] S22, training and testing the initial fall detection model using the collected first historical data set to obtain a fall detection model, wherein the first historical data set includes fall sample data and daily behavior sample data, wherein the daily behavior sample data includes normal behavior sample data and approximate fall behavior sample data of recovering balance after slipping or stumbling;

[0077] S23. Input the target motion data conforming to the standard normal distribution into the fall detection model to perform fall detection, and output a fall detection result, wherein the fall detection result includes falls and daily behaviors, wherein the daily behaviors include normal behaviors and approximate fall behaviors of recovering balance after slipping or stumbling.

[0078] Specifically, by establishing a fall detection model, the above-mentioned neural network model input standard data is received in real time, and detection and identification of the two situations of "fall" and "daily behavior" are performed in real time, wherein "daily behavior" also includes approximate fall behaviors of recovering balance after slipping or stumbling. The fall detection model adopts a one-dimensional convolution and long short-term memory neural network structure. Considering that the accuracy of the existing fall detection algorithm based on the deep learning model is not high, the present invention integrates the advantages of convolutional neural network and long short-term memory neural network to construct a fall detection model and improve the accuracy of fall detection. In the model training process, falls such as slipping and stumbling, which are closer to real fall scenes, are included, and approximate fall behaviors of successfully recovering balance after slipping and stumbling are considered, so that this model can avoid misjudging "near fall" events as falls.

[0079] S3. If the fall detection results output by the fall detection model for a set number of consecutive times are all falls, it is confirmed that a fall has occurred.

[0080] In an implementation of this embodiment, if the fall detection results output by the fall detection model for a set number of consecutive times are all falls, then confirming that a fall has occurred specifically includes:

[0081] S31, if the fall detection results output by the fall detection model are all falls for a set number of consecutive times, confirming that a fall has occurred;

[0082] S32: If the fall detection results output by the fall detection model do not show falls for a set number of consecutive times, it is confirmed that a fall has not occurred.

[0083] Specifically, the output results of the fall detection model are traversed and stored. When the model outputs "fall" for a set number of consecutive times (for example, seven times), it is determined that a fall has occurred and the result is fed back in real time. When the model outputs "fall" but does not meet the condition of seven consecutive times, it is determined that this is a daily behavior.

[0084] S4. Construct a fall direction detection model based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, input the target motion data into the fall direction detection model to perform fall direction detection, and output the fall direction detection result.

[0085] In one implementation of this embodiment, the fall direction detection model is constructed based on the one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output, which specifically includes:

[0086] S41. Based on the structure of one-dimensional convolution and long short-term memory neural network, an initial fall direction detection model is constructed;

[0087] S42, training and testing the initial fall direction detection model using the collected second historical data set to obtain a fall direction detection model, wherein the second historical data set includes forward fall sample data, backward fall sample data, left fall sample data, right fall sample data, and upright fall sample data;

[0088] S43. If a fall is confirmed to have occurred, the target motion data corresponding to the fall and conforming to the standard normal distribution is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output. The fall direction detection result includes falling forward, falling backward, falling to the left, falling to the right and falling upright.

[0089] Specifically, when the terminal receives the judgment result of a fall, the fall direction detection model is quickly activated, the activated fall direction detection model receives the above-mentioned neural network model input standard data in real time, and further classifies and identifies the fall signal. The classification results include five types, namely, falling forward, falling backward, falling to the left, falling to the right, and falling upright (vertigo). The structure of the fall direction detection model is similar to that of the fall detection model, but the difference between the two models is that the data sets used to train the two models are different. The data for training the fall detection model uses data sets such as falls (including slips, trips, etc.) and daily behaviors (including normal behaviors and near-fall behaviors). The data for training the fall direction detection model uses data sets such as falling forward, falling backward, falling to the left, falling to the right, and falling upright (vertigo), so the final hyperparameters of the two models are different.

[0090] The present invention realizes automatic recognition of the falling direction through a falling direction detection model, can assist in seeking medical treatment according to the possible injured parts in the first time, and improves the evaluation of the consequences of the fall and the effectiveness of subsequent rescue measures.

[0091] In order to solve the problem of low accuracy of existing fall detection methods, the present invention uses the idea of ​​solving the problem step by step. First, a binary classification judgment is made on "fall" and "daily behavior", and then a multi-classification judgment is made to identify the direction of the fall in the "fall" event. This method reduces the difficulty of model recognition in each link, and solves the problems faced by the present invention, such as the complexity and diversity of fall actions, and the individual differences of individuals when falling, by ensuring the accuracy of each step-by-step model.

[0092] In one implementation of this embodiment, the one-dimensional convolution and long short-term memory neural network includes a convolution module, a long short-term memory neural network layer, a random inactivation layer, a regularization module, a fully connected layer and a Softmax layer;

[0093] The convolution module includes several feature extraction blocks, each of which includes a convolution layer, a ReLU layer and a maximum pooling layer, wherein the convolution layer is used to extract local features of input data, the ReLU layer is used to introduce nonlinearity, and the maximum pooling layer is used to reduce the dimension of the feature map;

[0094] The regularization module includes several connection regularization blocks, and the connection regularization block includes a fully connected layer, a ReLU layer and a random dropout layer, wherein the fully connected layer is used to map features to the output space, the ReLU layer is used to introduce nonlinearity, and the random dropout layer is used to randomly discard some neurons to prevent overfitting.

[0095] Specifically, the structures of the fall detection model and the fall direction detection model both use one-dimensional convolution and long short-term memory neural network, such as Figure 3As shown, the one-dimensional convolution and long short-term memory neural network includes a convolution module 1, a long short-term memory neural network layer 2, a random dropout layer 3, a regularization module 4, a fully connected layer 5 and a Softmax layer 6. The convolution module 1 consists of two superimposed feature extraction blocks 7 (FEB). Each feature extraction block 7 consists of a convolution layer 8 (Conv), a ReLU layer 9 and a maximum pooling layer 10 (Max Pooling, MP). Among them, the convolution layer 8 is used to extract local features of the input data, the ReLU layer 9 is used to introduce nonlinearity, and the maximum pooling layer 10 is used to reduce the dimension of the feature map, thereby reducing the computational complexity and preventing overfitting. The two superimposed feature extraction blocks 7 extract higher-level features layer by layer, so that the model can better capture the complex patterns in the input data. In addition, the superimposed structure can transmit information more effectively, so that the subsequent layers can use the rich features extracted by the previous layers.

[0096] The regularization module 4 consists of two connection regularization blocks 8 (CRBs). Each connection regularization block 8 consists of a fully connected layer 5, a ReLU layer 10 and a random dropout layer 3. Among them, the fully connected layer 5 is used to map features to the output space, the ReLU layer 10 introduces nonlinearity to enhance the expressiveness of the model, and the random dropout layer 3 helps prevent overfitting by randomly discarding some neurons, thereby improving the generalization ability of the model. The design of these two superimposed connection regularization blocks 8 helps to help the model converge faster during training through regularization and nonlinear activation, improve overall training efficiency, and avoid the problem of gradient disappearance. The random dropout layer 3 introduced can reduce the model's dependence on training data and reduce the risk of overfitting.

[0097] In addition, the fall detection method based on the wrist inertial sensor also includes: data visualization, real-time monitoring of the data generated by the wrist inertial sensor IMU, and the detection and recognition results of the fall detection model and the fall direction detection model, programming in Python language, and presenting them in a graphical way on the user interface to achieve data visualization.

[0098] In order to verify the performance of the fall detection method based on the wrist inertial sensor of the present invention, a model verification experiment was carried out after the model was established. 5491 fall samples and 5118 daily behavior samples were obtained through experimental means. The sample number ratio was approximately 1:1, which effectively avoided the "class imbalance" phenomenon. Figure 4 As shown, Figure 4 (a) in the figure is the accuracy curve of the fall detection model. Figure 4 (b) is the loss value curve of the fall detection model. The accuracy curve of the validation set of the fall detection model gradually stabilizes after twenty iterations. The loss value curve also shows that the model training is successful. Figure 5 As shown, Figure 5(a) in the figure is the accuracy curve of the fall direction detection model. Figure 5 (b) is the loss value curve of the fall direction detection model. The accuracy curve of the validation set of the fall direction detection model keeps rising during the oscillation process, and the loss value curve keeps a downward trend during the oscillation process. The change trend is in line with expectations. According to engineering application experience, it shows that the model training is successful.

[0099] Through cross-validation, the performance evaluation results of the fall detection model and the fall direction detection model are obtained. As shown in Table 1, the accuracy of the fall detection model for "fall" and "daily behavior" detection reaches 93.56%.

[0100] Table 1: Fall detection model performance evaluation indicators

[0101] Evaluation Metrics Accuracy Accuracy Recall Specificity F1 value Fall Detection Model 93.56％ 89.74％ 95.24％ 92.38％ 92.41％

[0102] The results of the fall direction detection model for fall direction recognition are as follows Figure 6 As shown in the figure, the overall accuracy is 74.29%. The correct recognition rate of upright fall, i.e. vertigo, is 93.75%, and the correct recognition rates of the other four categories range from 52.38% to 85.37%.

[0103] In addition, based on the above-mentioned fall detection method based on the inertial sensor on the wrist, the present invention also provides a fall detection system based on the inertial sensor on the wrist. The preferred embodiment of the fall detection system based on the inertial sensor on the wrist is as follows: Figure 7 As shown, including:

[0104] Data acquisition and preprocessing module 01: used to acquire motion data collected by an inertial sensor installed on a human wrist, and preprocess the motion data to obtain target motion data;

[0105] Fall detection module 02: used to construct a fall detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall detection model for multiple times to perform fall detection, and output multiple fall detection results;

[0106] Fall occurrence judgment module 03: used to confirm that a fall has occurred if the fall detection results output by the fall detection model for a set number of consecutive times are all falls;

[0107] Fall direction detection module 04: used to build a fall direction detection model based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection and output the fall direction detection result.

[0108] In addition, based on the above-mentioned fall detection method and system based on the wrist inertial sensor, the present invention also provides a terminal accordingly. A preferred embodiment of the terminal is as follows: Figure 8 As shown, it specifically includes a processor 10, a memory 20 and a display 30. Figure 8 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0109] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card) equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as program codes of the storage terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a fall detection program 40 based on an inertial sensor of the wrist is stored on the memory 20, and the fall detection program 40 based on the inertial sensor of the wrist can be executed by the processor 10, thereby implementing the steps of the fall detection method based on the inertial sensor of the wrist in the present application.

[0110] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 20, such as executing a fall detection program 40 based on an inertial sensor on the wrist.

[0111] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0112] In one embodiment, when the processor 10 executes the fall detection program 40 based on the wrist inertial sensor in the memory 20 , the steps of the fall detection method based on the wrist inertial sensor as described above are implemented.

[0113] The present invention also provides a computer-readable storage medium accordingly, wherein the computer-readable storage medium stores a fall detection program based on an inertial sensor on the wrist, and when the fall detection program based on the inertial sensor on the wrist is executed by a processor, the steps of the fall detection method based on the inertial sensor on the wrist are implemented as described above.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or terminal including the element.

[0115] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0116] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A fall detection method based on an inertial sensor on a wrist, characterized in that: The fall detection method based on the wrist inertial sensor includes: Acquire motion data collected by an inertial sensor installed on a human wrist, and pre-process the motion data to obtain target motion data; Building a fall detection model based on one-dimensional convolution and long short-term memory neural network, inputting the target motion data into the fall detection model for multiple times to perform fall detection, and outputting multiple fall detection results; If the fall detection results output by the fall detection model for a set number of consecutive times are all falls, it is confirmed that a fall has occurred; A fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output.

2. The fall detection method based on the wrist inertial sensor according to claim 1, characterized in that: The acquiring of motion data collected by an inertial sensor installed on a human wrist and preprocessing of the motion data to obtain target motion data specifically includes: Acquire motion data collected by an inertial sensor installed on a human wrist in real time, wherein the inertial sensor includes a 9-axis inertial sensor chip; Extracting the motion data using a sliding window extraction method to obtain motion data corresponding to the sliding window; The motion data corresponding to the sliding window is standardized by using a Z value standardization method to obtain target motion data.

3. The fall detection method based on the wrist inertial sensor according to claim 2, characterized in that: The Z value standardization method is used to standardize the motion data corresponding to the sliding window to obtain the target motion data, specifically including: Calculating the mean and standard deviation of the motion data corresponding to the sliding window; According to the mean and the standard deviation, the motion data corresponding to the sliding window is standardized to obtain target motion data that conforms to the standard normal distribution. The formula is as follows: Among them, X * is the target motion data; X is the motion data corresponding to the sliding window; μ is the mean; σ is the standard deviation.

4. The fall detection method based on the wrist inertial sensor according to claim 3, characterized in that: The fall detection model is constructed based on one-dimensional convolution and long short-term memory neural network, the target motion data is input into the fall detection model for multiple times to perform fall detection, and multiple fall detection results are output, specifically including: Based on the structure of one-dimensional convolution and long short-term memory neural network, an initial fall detection model is constructed; The initial fall detection model is trained and tested using the collected first historical data set to obtain a fall detection model, wherein the first historical data set includes fall sample data and daily behavior sample data, wherein the daily behavior sample data includes normal behavior sample data and approximate fall behavior sample data of recovering balance after slipping or stumbling; The target motion data conforming to the standard normal distribution is input into the fall detection model for fall detection, and a fall detection result is output. The fall detection result includes falls and daily behaviors. The daily behaviors include normal behaviors and approximate fall behaviors of recovering balance after slipping or stumbling.

5. The fall detection method based on the wrist inertial sensor according to claim 1, characterized in that: If the fall detection results output by the fall detection model for a set number of consecutive times are all falls, then confirming that a fall has occurred, specifically includes: If the fall detection results output by the fall detection model are all falls for a set number of consecutive times, it is confirmed that a fall has occurred; If the fall detection results output by the fall detection model do not show falls for a set number of consecutive times, it is confirmed that a fall has not occurred.

6. The fall detection method based on the wrist inertial sensor according to claim 3, characterized in that: The fall direction detection model is constructed based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection, and the fall direction detection result is output, which specifically includes: Based on the structure of one-dimensional convolution and long short-term memory neural network, an initial fall direction detection model is constructed; The initial fall direction detection model is trained and tested using the collected second historical data set to obtain a fall direction detection model, wherein the second historical data set includes forward fall sample data, backward fall sample data, left fall sample data, right fall sample data, and upright fall sample data; If a fall is confirmed to have occurred, the target motion data corresponding to the fall that conforms to the standard normal distribution is input into the fall direction detection model for fall direction detection, and the fall direction detection result is output. The fall direction detection result includes falling forward, falling backward, falling to the left, falling to the right and falling upright.

7. The fall detection method based on the wrist inertial sensor according to claim 4 or 6, characterized in that: The one-dimensional convolution and long short-term memory neural network includes a convolution module, a long short-term memory neural network layer, a random inactivation layer, a regularization module, a fully connected layer and a Softmax layer; The convolution module includes several feature extraction blocks, each of which includes a convolution layer, a ReLU layer and a maximum pooling layer, wherein the convolution layer is used to extract local features of input data, the ReLU layer is used to introduce nonlinearity, and the maximum pooling layer is used to reduce the dimension of the feature map; The regularization module includes several connection regularization blocks, and the connection regularization block includes a fully connected layer, a ReLU layer and a random dropout layer, wherein the fully connected layer is used to map features to the output space, the ReLU layer is used to introduce nonlinearity, and the random dropout layer is used to randomly discard some neurons to prevent overfitting.

8. A fall detection system based on an inertial sensor on a wrist, characterized in that: The wrist-based inertial sensor fall detection system includes: Data acquisition and preprocessing module: used to acquire motion data collected by the inertial sensor installed on the human wrist, and preprocess the motion data to obtain target motion data; Fall detection module: used to build a fall detection model based on one-dimensional convolution and long short-term memory neural network, input the target motion data into the fall detection model for multiple times to perform fall detection, and output multiple fall detection results; A fall occurrence judgment module: used to confirm that a fall has occurred if the fall detection results output by the fall detection model for a set number of consecutive times are all falls; Fall direction detection module: used to build a fall direction detection model based on one-dimensional convolution and long short-term memory neural network. If a fall occurs, the target motion data is input into the fall direction detection model to perform fall direction detection and output the fall direction detection result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a fall detection program based on an inertial sensor on a wrist stored in the memory and executable on the processor. When the fall detection program based on an inertial sensor on the wrist is executed by the processor, the steps of the fall detection method based on an inertial sensor on the wrist are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a fall detection program based on an inertial sensor on a wrist, and when the fall detection program based on an inertial sensor on a wrist is executed by a processor, the steps of the fall detection method based on an inertial sensor on a wrist are implemented.

Images