Gait recognition and navigation method based on continuous wavelet transform and convolutional neural network

Abstract

The invention discloses a gait recognition and navigation method based on continuous wavelet transform and a convolutional neural network. Compared with the problem that a traditional image recognition method depends on an external environment and the like, the method has the advantages that gait data is collected through the MEMS, continuous wavelet transformation is carried out, so that a one-dimensional signal is converted into a two-dimensional time-frequency image, then the segmented image is used as a data set to carry out convolutional neural network training, and the gait classification precision is effectively improved. According to the method, time domain and frequency domain information can be fused into a time-frequency domain, the generated two-dimensional time-frequency image can more comprehensively reflect the characteristics of gait data, a more accurate zero-speed interval is obtained according to different gait self-adaptive sliding windows and an adjusted zero-speed judgment threshold, and finally position errors are compensated and updated through extended Kalman filtering. Only one inertial measurement unit is needed, effective positioning can be achieved without other sensors, and the detection method is simple and flexible.

Description

1. Technical Field

[0001] The present invention relates to a gait recognition and navigation method based on continuous wavelet transform and convolutional neural network, in particular, by converting the original gait signal into a time-frequency diagram, and then performing recognition and classification, and operating in an environment with weak satellite signals such as indoors and jungles to provide a navigation solution for individuals, belonging to the field of pedestrian walking navigation technology. 2. Background technology

[0002] Human activity recognition based on gait features is the mainstream research direction in the field of gait recognition. It refers to identifying the user's movement state through gait features. It is widely used in pedestrian navigation, kinematic analysis, action recognition and other fields. With the increasing demand for pedestrian navigation, how to achieve real-time positioning of pedestrians in the case of weak GPS signals is a key issue. The positioning and navigation algorithm based on visual information uses photos captured by the camera as input data and realizes positioning and navigation functions by analyzing the gait. This algorithm analyzes the gait features such as human posture, stride, and step frequency in the photo to infer the user's current position and direction of travel. This method has strong limitations, relies on equipment layout, and lacks real-time and flexibility. Pedestrian inertial navigation technology using inertial measurement units does not require the installation of signal devices in the scene, and has the advantages of low cost and good independence. However, the pedestrian navigation technology based on this method increases its cumulative error as the walking time increases. At the same time, due to the low performance of the low-cost inertial components used, the pedestrian gait change causes the position error to diverge rapidly, seriously affecting the positioning accuracy of the pedestrian navigation system. Therefore, it is particularly important to seek a gait recognition method that can maintain high accuracy during long-term use.

[0003] The gait recognition method based on continuous wavelet transform and convolutional neural network (CNN) can solve this problem. Continuous wavelet transform can capture the characteristics of gait signals in the time and frequency domain, while CNN has powerful feature extraction and pattern recognition capabilities. By combining these two technologies, continuous wavelet transform can be used to extract rich time and frequency features from gait signals, and then CNN can be used to learn and classify these features to achieve efficient and accurate gait recognition. The zero-speed judgment threshold is adaptively adjusted according to five different gaits, including walking, running, going upstairs, going downstairs, and standing, and finally the position error is compensated and updated within the zero-speed range. 3. Summary of the invention

[0004] In view of the deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a gait recognition and navigation method based on continuous wavelet transform and convolutional neural network for inertial navigation. The method fully considers the characteristics of different gait signals in the time and frequency domains, performs pattern recognition through neural network, and better adapts to the changes of different gait signals; comprehensively considers the adjustment and optimization of the zero-speed judgment threshold, reduces the phenomenon of missed and misjudgment in the zero-speed interval, and makes positioning more accurate.

[0005] The technical solution adopted by the present invention is a gait recognition and navigation method based on continuous wavelet transform and convolutional neural network, and the implementation method comprises the following steps:

[0006] Step 1: Analyze the data collected by the inertial measurement unit, select the Morlet wavelet basis function, and perform the following wavelet transform:

[0007]

[0008] Among them, a is the scale parameter, which controls the expansion and contraction of the wavelet function, and its value range is set to [0, 30], and π is the translation amount.

[0009] Step 2: In order to make the classification result more accurate, the y-axis gyroscope data is subjected to continuous wavelet transform, and then the frequency domain and time domain features are extracted. The wavelet coefficient image with the horizontal axis as time and the vertical axis as scale is drawn, that is, the frequency graph.

[0010] Step 3: Cut the wavelet coefficient image, divide the data set, and use the data set to perform classification training on the convolutional neural network.

[0011] Step 4, obtain the gait data of the pedestrian to be identified, perform wavelet transformation on the gait data of the pedestrian to be identified, cut it according to the initial window size, obtain image W1, input it into the trained convolutional neural network, and obtain the gait type of the pedestrian.

[0012] Step 5: For the i-th window image W i , input it into the trained convolutional neural network, and output the gait type. gait，i =G gait，i-1 , the window size remains unchanged, otherwise the window size is adjusted according to the gait type of the current window.

[0013] Step 6: Adaptively adjust the threshold ε according to the gait type f , when the corresponding conditions are met, it can be considered that the pedestrian's speed at this moment is zero, and the judgment conditions are:

[0014] T(k)<ε f

[0015] Step 7: Use extended Kalman filtering to perform zero-speed correction. For nonlinear systems, the discretized time state space model can be set as:

[0016] X k =f(X k-1 )+Γ k-1 W k-1

[0017] Z k =h(X k )+V k

[0018] Among them, f(X k )=[f1(X k )f2(X k )...f n (X k )] T is the n-dimensional nonlinear state function, X k With Z k are the state vector and observation vector of the system, h(X k )=[h1(X k )h2(X k )...h n (X k )] T is an m-dimensional nonlinear measurement function.

[0019] In step 4, the wavelet coefficient image is cut into windows of size 500 along the time axis, and a dictionary is created to store the association between image segments and labels. Set it as n groups of data, 80% is divided into training sets, 10% is divided into validation sets, and 10% is divided into test sets. The convolutional neural network is trained using the data.

[0020] In step 6, ε f The zero-speed judgment threshold is adaptively adjusted according to the gait and is obtained through calibration experiments. The specific process is as follows:

[0021] 1. Collect gait data of several pedestrians walking, running, going up and down stairs, and standing;

[0022] 2. Obtain the generalized likelihood ratio test value under different gait categories. The calculation formula is:

[0023]

[0024] Among them, w is the generalized likelihood detection window size, which is set to 3 here, and g is the local gravity acceleration. is the variance of acceleration and angular velocity noise, determined according to sensor performance. m (j) is the mean acceleration within the generalized likelihood ratio window.

[0025] The zero speed interval determination basis can be set as follows:

[0026]

[0027] ε f is an adaptive threshold based on the gait category, which is related to the gait category.

[0028] Beneficial effects of the present invention:

[0029] The present invention designs a gait recognition and navigation method based on continuous wavelet transform and convolutional neural network, which can realize the inference of pedestrian navigation information in indoor environment; the original IMU data is converted into a time-frequency diagram through wavelet transform to better capture the time and frequency characteristics of the data, and then the convolutional neural network is used for classification, which improves the accuracy of classification; when the pedestrian gait category produces random changes, the window size is adaptively adjusted to better adapt to the gait change characteristics, and the flexibility of signal analysis is improved; at the same time, the generalized likelihood ratio detection amount of the acceleration signal is zero-speed detected according to the threshold value adjusted by the gait, which reduces the judgment conditions and enhances the sensitivity of the detection algorithm. Compared with the positioning and navigation algorithm of visual information, this method only requires an inertial measurement unit, and does not require other sensors such as cameras, which reduces the equipment cost and dependence. This inertial navigation technology can be used in single-soldier combat, disaster relief and other fields, so the implementation method has broad application prospects. 4. Description of the drawings

[0030] Figure 1 It is an algorithm flow chart of gait recognition and navigation method based on continuous wavelet transform and convolutional neural network;

[0031] Figure 2 Get algorithm flow chart for adaptive window based threshold

[0032] Figure 3 is the Morlet wavelet basis function graph;

[0033] Figure 4 This is the time-frequency diagram of the y-axis gyroscope signal collected by IMU while walking; 5. Specific implementation methods

[0034] The present invention provides a gait recognition and navigation method based on continuous wavelet transform and convolutional neural network, which is implemented by the following steps:

[0035] Step 1: Install a tubular measurement unit on the pedestrian's shoes to collect the angular rate data output by the gyroscope and the acceleration data output by the accelerometer during movement. The origin of the inertial measurement unit is located in the center of the unit, and the x, y, and z axes point to the forward direction, the right direction, and the vertical downward direction, respectively.

[0036] Step 2, according to the wavelet transform formula is as follows:

[0037]

[0038] Among them, a is the scale parameter, π is the translation, and the scale a controls the expansion and contraction of the wavelet function, corresponding to the frequency. Here, the value range is [0, 30]. The translation is not set and traverses the entire signal. The Haar wavelet, Gabor wavelet and Morlet wavelet are used to process the signal respectively, and their performance in time-frequency resolution and feature extraction ability is comprehensively compared. The results show that the Morlet wavelet has the best effect when processing periodic features.

[0039] Step 3: Extract the y-axis gyroscope data, which is a one-dimensional signal with one row and n columns, i.e., y gyro =[a1, a2, …, a n ], and processed it with continuous wavelet transform to obtain a two-dimensional wavelet coefficient image with time as the horizontal axis and scale as the vertical axis.

[0040] Step 4: cut the wavelet coefficient image, divide the data set, and use the data set to perform classification training on the convolutional neural network.

[0041] Step 5, obtaining the gait data of the pedestrian to be identified, performing wavelet transformation on the gait data of the pedestrian to be identified, cutting it according to the initial window size, obtaining image W1, inputting it into the trained convolutional neural network, and obtaining the gait type of the pedestrian.

[0042] Step 6: For the i-th window image W i , input it into the trained convolutional neural network, and output the gait type. gait，i =G gait,i-1 , the window size remains unchanged, otherwise the window size is adjusted according to the gait type of the current window.

[0043] Step 7: Adaptively adjust the threshold ε according to the gait type f , when the corresponding conditions are met, it can be considered that the pedestrian's speed at this moment is zero, and the judgment conditions are:

[0044] T(k)<ε f

[0045] Step 8: Use extended Kalman filtering for zero-speed correction, use Taylor series to perform first-order Taylor expansion on nonlinear functions, locally linearize nonlinear system equations and observation equations, and then apply linear Kalman filtering to update state and covariance estimates. The core formula is as follows:

[0046]

[0047] Among them, X k|k-1 is the optimal estimated value X of the system state at time k-1 k-1 The prediction of the system state at time k, P k|k-1 Predict the system state Xk|k-1 X k|k-1 The corresponding error covariance matrix. The last three formulas are used for state update.

[0048] The construction process of the convolutional neural network described in step 4: first add a convolutional layer (Conv2D) to extract local features in the image and construct a feature map, then add a pooling layer (MaxPooling2D) to reduce the size of the feature map and retain the main features, then add a Dropout layer to reduce the risk of overfitting, and then add a Flatten layer to flatten the two-dimensional feature map into a one-dimensional vector as the input of the fully connected layer (Dense). The fully connected layer introduces nonlinear relationships and maps features to categories. The last layer uses the softmax activation function to output the probability distribution of each category.

[0049] Step 7 f The zero-speed judgment threshold is obtained by the calibration experiment based on the adaptive adjustment. The specific process is as follows:

[0050] 1. Collect gait data of several pedestrians walking, running, going up and down stairs, and standing;

[0051] 2. Obtain the generalized likelihood ratio test value under different gait categories. The calculation formula is:

[0052]

[0053] Among them, w is the generalized likelihood detection window size, which is set to 3 here, and g is the local gravity acceleration. is the variance of acceleration and angular velocity noise, determined according to sensor performance. m (j) is the mean acceleration within the generalized likelihood ratio window.

[0054] The zero speed interval determination basis can be set as follows:

[0055]

[0056] ε f is an adaptive threshold based on the gait category, which is related to the gait category.

Claims

1. A gait recognition and navigation method based on continuous wavelet transform and convolutional neural network, characterized in that: The steps include: Step 1: Analyze the data collected by the inertial measurement unit, select the Morlet wavelet basis function, and perform the following wavelet transform: Among them, a is the scale parameter, which controls the expansion and contraction of the wavelet function, and its value range is set to [0, 30], and π is the translation amount. Step 2: In order to make the classification result more accurate, the y-axis gyroscope data is subjected to continuous wavelet transform, and then the frequency domain and time domain features are extracted. The wavelet coefficient image with the horizontal axis as time and the vertical axis as scale is drawn, that is, the frequency graph. Step 3: Cut the wavelet coefficient image, divide the data set, and use the data set to perform classification training on the convolutional neural network. Step 4, obtain the gait data of the pedestrian to be identified, perform wavelet transformation on the gait data of the pedestrian to be identified, cut it according to the initial window size, obtain image W1, input it into the trained convolutional neural network, and obtain the gait type of the pedestrian. Step 5: for i window images W i , input it into the trained convolutional neural network, and output the gait type. gait，i =G gait，i-1 , the window size remains unchanged, otherwise the window size is adjusted according to the gait type of the current window. Step 6: Adaptively adjust the threshold ε according to the gait type f , when the corresponding conditions are met, it can be considered that the pedestrian's speed at this moment is zero, and the judgment conditions are: T(k)<ε f Step 7: Use extended Kalman filtering to perform zero-speed correction. For nonlinear systems, the discretized time state space model can be set as: X k =f(X k-1 )+Γ k-1 W k-1 Z k =h(X k )+V k Among them, f(X k )=[f1(X k )f2(X k )...f n (X k )] T is the n-dimensional nonlinear state function, X k With Z k are the state vector and observation vector of the system, h(X k )=[h1(X k )h2(X k )...h n (X k )] T is an m-dimensional nonlinear measurement function.

2. The gait recognition and navigation method based on continuous wavelet transform and convolutional neural network according to claim 1 is characterized in that: The step 2 uses wavelet transform to traverse the entire signal based on the determined wavelet basis and scale to obtain a series of wavelet coefficients, which describe the characteristics of the signal at different scales and positions, and then converts them into a time-frequency image.

3. The gait recognition and navigation method based on continuous wavelet transform and convolutional neural network according to claim 1 is characterized in that: In step 6, different gait features have different zero-speed detection thresholds. The generalized likelihood ratio detection amount of the calculated data is used for adaptive threshold detection. When the corresponding conditions are met, it can be considered that the pedestrian's movement speed at this moment is zero. The judgment conditions are: T(k)<ε f Among them, ε f It is the zero-speed judgment threshold that is adaptively adjusted according to different gaits and is obtained through calibration experiments.

Images