A gait detection method based on end-side LSTM

Abstract

This invention discloses a gait detection method based on edge-side LSTM, belonging to the field of inertial pedestrian navigation technology. The method first fixes a device equipped with inertial sensors to the feet of the test subject to collect gyroscope and accelerometer data in real time. Then, the collected data is divided into training and validation sets, and converted into several samples suitable for deep learning training. An LSTM neural network is constructed, the LSTM model is trained using the training set, and the model's classification ability is verified using the validation set. Model parameters and training parameters are adjusted until the model's classification results meet experimental expectations, and the LSTM model is saved. Finally, the LSTM model is appropriately pruned and converted into a general neural network model, which is then deployed in a wearable device to classify the real-time foot inertial data to obtain gait detection results. This invention can provide pedestrians with real-time edge-side gait detection capabilities, providing a basis for edge-side navigation calculation and pedestrian state judgment.

Description

Technical Field

[0001] This invention relates to the field of inertial pedestrian navigation technology, and in particular to a gait detection method based on end-side LSTM. Background Technology

[0002] Gait is a term used to describe the way pedestrians walk. While gait varies from person to person, overall, movement itself is continuous and periodic, and therefore gait follows general patterns. Pedestrian gait detection primarily uses these general patterns to detect and count steps. The process of walking is a periodic motion; the pedestrian's center of gravity maintains a roughly constant speed horizontally, while exhibiting regular, variable acceleration vertically. Therefore, by sensing the pedestrian's acceleration using a triaxial accelerometer, gait analysis can be performed. Pedestrian gait detection technology has wide applications in indoor navigation, motion monitoring, and other fields.

[0003] Currently, with the development of sensor technology, artificial intelligence, and other fields, pedestrian adaptive zero-velocity correction technology has also developed rapidly. However, current research still faces problems such as high algorithm complexity and low motion detection and positioning accuracy, which cannot meet application requirements. Moreover, due to the complexity of pedestrian motion, it has a significant impact on inertial-based real-time positioning, making it difficult for inertial-based pedestrian positioning methods to be widely used. Summary of the Invention

[0004] In view of this, the present invention proposes a gait detection method based on edge LSTM. This method can provide pedestrians with real-time gait detection capabilities at the edge, avoiding the step of further uploading data, and can provide a good basis for edge navigation calculation and pedestrian status judgment.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A gait detection method based on end-side LSTM includes the following steps:

[0007] (1) The equipment equipped with inertial sensors was fixed to the feet of multiple test personnel respectively, and the coordinate system followed the right-hand coordinate system rule of "right-forward-up"; after the equipment was fixed, multiple test personnel performed slow walking, running, lateral movement, backward movement, climbing stairs and descending stairs respectively, and collected gyroscope and accelerometer data in real time, and stored the experimental data into different files respectively.

[0008] (2) Divide the data collected in step (1) into training set and validation set, and preprocess the data to obtain samples for deep learning training; construct LSTM neural network, train LSTM neural network using training set, and verify the classification ability of LSTM neural network using validation set, adjust network parameters and training parameters until the classification result of LSTM neural network meets the requirements, and save the trained LSTM neural network.

[0009] (3) The trained LSTM neural network is pruned and converted into a general neural network model. The model is then deployed in a wearable device to classify the real-time foot inertial data and obtain gait detection results.

[0010] Furthermore, the specific method of step (1) is as follows:

[0011] Test subjects of various body types and genders were selected, and the device containing MEMS inertial sensors was worn on the instep of the test subjects. The "XYZ axes" of the inertial sensors correspond to the "right-front-up" directions of the human body, respectively.

[0012] The experimenters performed slow walking, running, lateral movement, backward movement, climbing stairs, and descending stairs. The inertial sensor data during the movement was collected in real time at a frequency of 100Hz and low-pass filtered to remove some high-frequency noise. The inertial sensor data consisted of 6-axis inertial data.

[0013] Furthermore, the specific method of step (2) is as follows:

[0014] The data collected in step (1) is preprocessed to form a sample set, which is then divided into a training set and a validation set.

[0015] The data was segmented using a sliding window method, with a window size of 150, a duration of 1.5 seconds, and each sample size of 150×6; samples were labeled according to their file names.

[0016] Build an LSTM neural network model, and determine the model's input and output based on the sample size and the number of gait types;

[0017] The model is trained using the training set, and its accuracy is validated using the validation set. The model parameters and training parameters are adjusted until the model's classification results meet the experimental expectations. The model parameters are then saved.

[0018] Furthermore, the MEMS inertial sensor device uses an STM32 microcontroller as the main control chip;

[0019] The specific method for step (3) is as follows:

[0020] The number of layers and neurons in the model is continuously reduced until the classification accuracy of the model is reduced to the required level, thereby achieving the pruning operation of the neural network model;

[0021] The pruned neural network model is deployed into the STM32 microcontroller of the MEMS inertial sensor device to classify the real-time foot inertial data.

[0022] The beneficial effects of this invention compared to the prior art are as follows:

[0023] 1) This invention uses an LSTM neural network to classify inertial data sequences, which can efficiently and accurately obtain gait detection results;

[0024] 2) This invention optimizes the feature extraction method, which can reduce the impact of different wearing methods on gait classification results;

[0025] 3) This invention deploys an LSTM classification network to a wearable terminal, realizing deep learning gait detection on the device side. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the principle in an embodiment of the present invention.

[0027] Figure 2 This is a data sample diagram from an embodiment of the present invention.

[0028] Figure 3 This is a comparison diagram of filtering in an embodiment of the present invention.

[0029] Figure 4 This is a classification result diagram from an embodiment of the present invention.

[0030] Figure 5 This is a graph showing the loss function and accuracy in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0032] like Figure 1-5 As shown, a gait detection method based on end-side LSTM includes the following steps:

[0033] (1) Fix the device equipped with inertial sensors to the feet of the test personnel. The fixing method can be by strapping or pasting. The coordinate system follows the right-hand coordinate system rule of "right-forward-up". After the device is fixed, multiple test personnel perform slow walking, running, lateral movement, backward movement, climbing stairs, and descending stairs. The gyroscope and accelerometer data are collected in real time, and the experimental data are stored in different files.

[0034] (2) Divide the data collected in step (1) into a training set and a validation set, and preprocess the data to convert it into several samples suitable for deep learning training. Construct an LSTM neural network, train the constructed LSTM model using the experimental data in the training set, and use the validation set to verify the classification ability of the model. Adjust the model parameters and training parameters until the classification results of the model meet the experimental expectations, and save the LSTM model.

[0035] (3) The LSTM model saved in step (2) is appropriately trimmed and converted into a general neural network model. The model is then deployed to a wearable device to classify the real-time foot inertial data and obtain gait detection results.

[0036] Furthermore, the specific method of step (1) is as follows:

[0037] The device containing a MEMS inertial sensor was fixed to the instep of the test subject, with the XYZ axes corresponding to the right-forward-upward directions of the human body. To avoid interference from device movement, it was secured with straps or adhesive tape to prevent slippage. Several test subjects of different body types and genders were selected, and the inertial sensor device was worn on their feet. They then performed slow walking, running, lateral movement, backward walking, climbing stairs, and descending stairs. The inertial sensor data during these movements was collected in real time at a frequency of 100Hz. A low-pass filter was used to remove some high-frequency noise, preparing for subsequent gait classification.

[0038] Furthermore, the specific method of step (2) is as follows:

[0039] First, the data collected in step (1) is preprocessed to form a sample set for model training. The sample set is divided into a training set and a validation set. In this patent, the data used is 6-axis inertial data. The sliding window method is used to segment the sequence data, with a window size of 150 (duration 1.5s). Therefore, each sample is 150×6 in size. Then, a label is added to the sample according to the file name. Next, an LSTM neural network model is built. LSTM stands for Long Short-Term Neural Network. Building a neural network model requires determining the sample size and the number of gait types to determine the model's input and output. Finally, the model is trained using the constructed training set, and the model accuracy is verified using the validation set. The model parameters and training parameters are adjusted until the model's classification results meet the experimental expectations. The LSTM model is then saved.

[0040] In this method, the MEMS inertial sensor device uses an STM32 microcontroller as the main control chip. Therefore, the neural network model trained in step (2) needs to be deployed to the STM32 microcontroller. Since the model in step (2) is large and limited by the computing power of the STM32 microcontroller, the model needs to be "pruned" to simplify it. This will inevitably affect the classification accuracy of the model. Therefore, a balance needs to be struck between model complexity and computing power. The specific operation of model pruning is to continuously reduce the number of layers and neurons in the model until the classification accuracy of the model is reduced to the required level. Then, this model is used as the final model deployed to the terminal.

[0041] Here is a more specific example:

[0042] A gait detection method based on end-side LSTM is proposed. First, a device containing a MEMS inertial sensor is fixed to the instep of the test subject. The XYZ axes of the inertial sensor correspond to the right-forward-upward directions of the human body, respectively. To avoid interference from device movement, it is secured using straps or adhesive tape to prevent slippage. Several test subjects of different body types and genders are selected. The inertial sensor device is worn on the feet of the test subjects, who then perform slow walking, running, lateral movement, backward movement, climbing stairs, and descending stairs. The inertial sensor data during the movement is collected in real time at a frequency of 100Hz. A low-pass filter is used to remove some high-frequency noise, preparing for subsequent gait classification.

[0043] The following steps include five parts: dataset construction, data filtering, model building, model training, and result validation.

[0044] Step 1: The raw data collected is time-continuous 6-axis inertial sensor data. Data for each gait is stored in different files. Therefore, it is necessary to remove useless data, segment the data, set sample labels, standardize the data, and build the dataset before subsequent model training and result verification can be carried out.

[0045] Step 2: Since the raw data contains a lot of noise and fluctuations that are not meaningful for classification (the variation of each axis sensor has a greater impact on gait detection), it is necessary to perform low-pass filtering on the data samples to speed up model convergence and classification accuracy; model building is a crucial step in the whole process, and the quality of the model directly affects whether classification can be performed and the classification accuracy.

[0046] Step 3: After constructing the dataset (divided into training and validation sets) and the model, training can begin. This involves selecting a loss function and optimizer, and setting training parameters to accelerate the training process. To avoid data leakage, the data is pre-divided into training and validation sets for the training and classification phases, respectively. After training, the model's classification ability needs to be validated to monitor the training process, derive the classification accuracy curve, and adjust the training parameters promptly during training.

[0047] The collected raw data consisted of time-continuous 6-axis inertial sensor data, with data for each gait stored in separate files. Therefore, data processing required steps such as removing useless data, data segmentation, sample labeling, data standardization, and dataset construction before subsequent model training and result validation. While efforts were made to ensure that each file contained data for the same gait, plotting revealed numerous invalid data points (such as start and end wait times and pause times). Including these invalid data in the sample set would significantly impact training; therefore, invalid data needed to be removed, retaining only valid training data before sampling.

[0048] LSTM stands for Long Short-Term Neural Network, first proposed by Sepp Hochreiter and Jürgen Schmidhuber in 1997. It is a specific form of Recurrent Neural Network (RNN), which is a general term for a series of neural networks capable of processing sequential data. Generally, RNNs possess the following three characteristics:

[0049] a) Recurrent neural networks can produce an output at each time point, and the connections between hidden units are cyclic;

[0050] b) A recurrent neural network can produce an output at each time point, and the output at that time point is only cyclically connected to the hidden units of the next time point.

[0051] c) Recurrent neural networks contain hidden units with recurrent connections and are able to process sequential data and output a single prediction.

[0052] In this method, the MEMS inertial sensor device uses an STM32 microcontroller as the main control chip. Therefore, the trained neural network model needs to be deployed on the STM32 microcontroller. Since the generated model is large and limited by the computing power of the STM32 microcontroller, it needs to be "pruned" to simplify it. This inevitably affects the model's classification accuracy, so a balance needs to be struck between model complexity and computing power. The specific pruning operation involves continuously reducing the number of layers and neurons in the model until the classification accuracy is reduced to the required level. This model is then used as the final model deployed to the terminal.

[0053] This invention introduces a deep LSTM model to process inertial sensor sequence data, realizing an LSTM-based algorithm for gait detection and personalized stride length estimation. From... Figures 2-5 As can be seen, during training, the misclassification rate and false positive rate for classifying multiple different gait states were both low, and the accuracy for each gait state was high and balanced. This indicates that the model built in the experiment can learn the features of different gait states well, thus achieving good classification results for different gait states. Overall, the comprehensive accuracy of gait detection based on deep learning reached 99.05%.

[0054] Meanwhile, during gait classification experiments, it was found that the features of the sample data have a significant impact on the accuracy of gait detection. When performing feature engineering on the data, it is necessary to ensure that the samples retain the unique features of each gait while filtering out the influence of individual abnormal movements. A proper balance between these two aspects is required to achieve the best classification results. In this invention, a sliding sampling method is used for data sampling. That is, the window length is fixed to the sample length, and samples are continuously slid across the data sequence by a certain distance. This ensures a sufficiently large sample size and avoids data waste. Experiments showed that low-pass filtering of the samples can effectively reduce invalid features and increase the accuracy of gait detection.

Claims

1. A gait detection method based on end-side LSTM, characterized in that, Includes the following steps: (1) The equipment equipped with inertial sensors was fixed to the feet of multiple test personnel. The coordinate system followed the right-hand coordinate system rule of "right-forward-up". After the equipment was fixed, multiple test personnel performed slow walking, running, lateral movement, backward movement, climbing stairs and descending stairs. Data from gyroscopes and accelerometers were collected in real time, and the experimental data were stored in different files. (2) Divide the data collected in step (1) into training set and validation set, and preprocess the data to obtain samples for deep learning training; construct LSTM neural network, train LSTM neural network using training set, and verify the classification ability of LSTM neural network using validation set, adjust network parameters and training parameters until the classification result of LSTM neural network meets the requirements, and save the trained LSTM neural network. (3) The trained LSTM neural network is pruned and converted into a general neural network model. The model is then deployed in a wearable device to classify the real-time foot inertial data and obtain gait detection results.

2. The gait detection method based on end-side LSTM according to claim 1, characterized in that, The specific method for step (1) is as follows: Test subjects of various body types and genders were selected, and the device containing MEMS inertial sensors was worn on the instep of the test subjects. The "XYZ axes" of the inertial sensors correspond to the "right-front-up" directions of the human body, respectively. The experimenters performed slow walking, running, lateral movement, backward movement, climbing stairs, and descending stairs. The inertial sensor data during the movement was collected in real time at a frequency of 100Hz and low-pass filtered to remove some high-frequency noise. The inertial sensor data consisted of 6-axis inertial data.

3. The gait detection method based on end-side LSTM according to claim 2, characterized in that, The specific method for step (2) is as follows: The data collected in step (1) is preprocessed to form a sample set, which is then divided into a training set and a validation set. The data was segmented using a sliding window method, with a window size of 150, a duration of 1.5 seconds, and each sample size of 150×6; samples were labeled according to their file names. Build an LSTM neural network model, and determine the model's input and output based on the sample size and the number of gait types; The model is trained using the training set, and its accuracy is validated using the validation set. The model parameters and training parameters are adjusted until the model's classification results meet the experimental expectations. The model parameters are then saved.

4. The gait detection method based on end-side LSTM according to claim 3, characterized in that, The MEMS inertial sensor device uses an STM32 microcontroller as the main control chip. The specific method for step (3) is as follows: The number of layers and neurons in the model is continuously reduced until the classification accuracy of the model is reduced to the required level, thereby achieving the pruning operation of the neural network model; The pruned neural network model is deployed into the STM32 microcontroller of the MEMS inertial sensor device to classify the real-time foot inertial data.

Images