An adaptive step frequency detection method based on CNN-LSTM motion pattern recognition

Abstract

The application relates to a self-adaptive step frequency detection method based on a CNN-LSTM motion mode recognition, and the detection method comprises the following steps: obtaining sensor collection data when a user walks; inputting the collection data into a CNN-LSTM model for motion mode recognition classification; inputting the collection data and the classification result into a step frequency detection algorithm adopting a self-adaptive threshold peak detection to obtain a step frequency detection result; the step of adopting the CNN-LSTM model for motion mode recognition classification comprises the following steps: making two-dimensional slice data from the collection data, and inputting the slice data into a CNN; extracting motion features of the slice data by using a convolution layer; performing dimension reduction and down sampling through a pooling layer; expanding the data into a one-dimensional array by using a flattening layer and inputting the one-dimensional array into an LSTM layer; classifying different motion modes of the data by using the LSTM layer; and outputting a classification result through a full connection layer. Compared with the prior art, the application has the advantages of self-adaptation to different motion modes of pedestrians, no need of manually extracting features, high recognition accuracy, and elimination of false wave peaks.

Description

Technical Field

[0001] This invention relates to the field of step frequency detection technology, and in particular to an adaptive step frequency detection method based on CNN-LSTM motion pattern recognition. Background Technology

[0002] With the continuous development of information technology, indoor location services such as shopping navigation, parking navigation, and industrial positioning have become an indispensable part of people's daily lives and work. Benefiting from its independence from pre-deployed positioning devices and its applicability to unknown environments, the Pedestrian Dead Reckoning (PDR) indoor positioning method based on smartphones has stood out among many indoor location service technologies, attracting widespread attention and research. The PDR method uses readings from the inertial sensors built into most modern smartphones to calculate the user's stride frequency, stride length, and heading in order to estimate their position in real time.

[0003] A crucial aspect of current PDR (Pedestrian Direct Response) positioning technology is achieving robust and high-precision cadence detection. Typically, users don't carry their phones in a single, fixed position while walking; common carrying positions include holding it to their chest, swinging their arm, making a call, and putting it in their pocket. Furthermore, user actions while not walking can also generate false walking patterns, leading to over-counting. Therefore, research into cadence detection that identifies pedestrian movement patterns and adapts thresholds to different patterns is of great significance.

[0004] Chinese patent CN202110521228.5 discloses a trajectory estimation and positioning method based on pedestrian motion state recognition. The method includes: constructing a pedestrian motion state recognition classification model: collecting triaxial acceleration data of pedestrians in five motion states—walking, jogging, left stepping, right stepping, and backward—to construct the pedestrian motion state recognition classification model; identifying pedestrian motion states: collecting triaxial acceleration data and triaxial gyroscope data of pedestrians, and using the pedestrian motion state recognition classification model to identify the pedestrian motion state; performing step frequency detection to obtain single-step step frequency: performing step frequency detection on the collected triaxial acceleration data to obtain the single-step step frequency; and estimating step length: if the identified pedestrian motion state is walking, left stepping, right stepping, or backward, a linear step length model combined with the single-step step frequency is used to estimate the single-step step length; if the identified pedestrian motion state is jogging, a Weinberg nonlinear step length model combined with the single-step step frequency is used to estimate the single-step step length. Heading estimation: The acquired three-axis gyroscope data is integrated using angular velocity based on quaternion coordinate system transformation to calculate the heading angle, and a heuristic offset elimination algorithm is used to correct the heading angle. Track estimation: The initial position coordinates and initial heading angle of the pedestrian are set, and the pedestrian's position is updated according to the pedestrian's motion state, single step length, and corrected heading angle under five motion states.

[0005] The existing technology has the following shortcomings:

[0006] Traditional motion pattern recognition methods typically employ machine learning techniques such as decision trees, random forests, support vector machines, and artificial neural networks. While these traditional classifiers can identify different motion activities, their accuracy heavily relies on various types of features manually extracted from the source sensor data, such as statistical features (e.g., mean, variance, energy), time-domain features (e.g., zero-crossing ratio), and frequency-domain features (e.g., fast Fourier transform). Furthermore, different types of features have varying sensitivities to different motion patterns, and selecting suitable features requires extensive experience, making it difficult to generalize to other models.

[0007] Because a user's center of gravity fluctuates vertically while walking, the combined acceleration data exhibits periodic oscillations with peaks and troughs. Traditional peak detection determines the number of steps based on the number of peaks in the acceleration data. Although current technologies can eliminate most of the spurious peaks caused by factors such as movement patterns and body shaking through filtering, some spurious peaks still exist, leading to excessive step counting. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an adaptive step frequency detection method based on CNN-LSTM motion pattern recognition.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] As one aspect of the present invention, an adaptive cadence detection method based on CNN-LSTM motion pattern recognition is provided. The detection method includes the following steps: obtaining sensor data collected by a user walking; inputting the collected data into a CNN-LSTM model for motion pattern recognition and classification; inputting the collected data and classification results into a cadence detection algorithm using adaptive threshold peak detection to obtain cadence detection results.

[0011] The steps for motion pattern recognition and classification using the CNN-LSTM model include:

[0012] The collected data is processed into two-dimensional slices, and then the slices are input into a CNN.

[0013] Motion features of slice data are extracted using convolutional layers;

[0014] Dimensionality reduction and downsampling are performed after pooling layer;

[0015] The data is unfolded into a one-dimensional array by a flattening layer and then input into the LSTM layer.

[0016] LSTM layers are used to classify different motion patterns of the data;

[0017] The classification results are output through a fully connected layer, and the classification results include motion state and mobile phone posture.

[0018] As a preferred technical solution, the sensor collects data including triaxial acceleration and triaxial angular velocity.

[0019] As a preferred technical solution, the step of making the collected data into two-dimensional slice data involves sequentially making every 30 data points into two-dimensional slice data with a size of 30*6.

[0020] As a preferred technical solution, the activation function of the fully connected layer is Soft-max.

[0021] As a preferred technical solution, the movement states include normal walking and brisk walking, and the mobile phone postures include holding, talking, swinging arm, and pocket.

[0022] As a preferred technical solution, the step frequency detection algorithm includes the following steps:

[0023] Use a filter to filter the acquired data;

[0024] The filtered data is then subjected to minimum peak distance filtering using the following formula to obtain the peak values ​​that meet the criteria. t ;

[0025] Peak t =(a t ≥(a t-k ∶a t-1 )&&a t ≤(a t+1 ∶a t+k ))

[0026] Among them, a t The resultant acceleration value is given at sample t, and k is the threshold for the number of samples on both sides of t. k is set according to the motion state classification results of the collected data.

[0027] The dynamic threshold window size is calculated using the following formula.

[0028] W = (T s *F s )-1

[0029] Among them, T s For each step, F s This is the current sampling frequency of the mobile phone;

[0030] Then calculate the dynamic threshold TH in the current window according to the following formula. peak ;

[0031]

[0032] Among them, W max W is the maximum value of the current window. min The minimum value in the current window is d, and d represents the degree of approach to the peak value.

[0033] The peak value after minimum peak distance filtering is compared with the dynamic threshold within the window. If the current peak value is greater than or equal to the dynamic threshold within this window, it is counted as a true step.

[0034] As a preferred technical solution, the filter includes a low-pass filter and a convolutional smoothing filter, wherein the cutoff frequency of the low-pass filter is 3Hz.

[0035] As a preferred technical solution, the sample quantity threshold k is set to 20 in normal walking mode and 10 in fast walking mode.

[0036] As a preferred technical solution, the time T for each step s In sports mode, the time is set to 0.507 seconds in normal walking mode and 0.469 seconds in fast walking mode.

[0037] As a preferred technical solution, the degree of approaching the peak value d is set to 1.05 when the phone is in a handheld or telephone position, and to 0.9 when it is in a pocket or arm swing position.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1) This invention uses a CNN-LSTM network model and employs CNN for feature extraction. This avoids the need for traditional classification methods to manually extract different types of features from source sensor data twice. Then, LSTM is used to mine the correlation between data features, thereby improving the accuracy of motion pattern classification.

[0040] 2) This invention proposes an adaptive threshold peak detection algorithm for the traditional peak detection problem. By recognizing the user's movement pattern, it uses combined filtering to reduce data noise and eliminate obvious false peaks. It uses the selection of minimum peak distance and dynamic threshold method to adaptively detect whether the current step count meets the conditions. Compared with the traditional peak detection method, it not only has a higher step frequency detection accuracy, but is also applicable to step frequency detection of pedestrians in different movement patterns. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the process of the present invention;

[0042] Figure 2 This is a schematic diagram of the CNN-LSTM network model structure of the present invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0044] As one embodiment of the present invention, a CNN-LSTM deep network framework model is employed that eliminates the need for secondary feature extraction from the source data. This model first utilizes CNN to automatically extract and reduce sensor data features, then inputs these features into an LSTM network to identify correlations between features, ultimately achieving more accurate classification results for motion patterns. Minimum peak spacing selection and dynamic thresholding adaptive detection methods are used to filter peaks, resulting in higher accuracy in gait frequency detection and greater robustness for gait frequency detection across different motion patterns.

[0045] like Figure 1 As shown, the gait frequency of pedestrians in normal walking (normal walking, brisk walking) is usually below 3Hz. The triaxial acceleration (acc_x, acc_y, acc_z) and triaxial angular velocity (ang_x, ang_y, ang_z) data from a 50Hz sensor on a mobile phone are processed into 30*6 two-dimensional slices every 30 data points (approximately 2-3 steps). This allows the model to learn more comprehensively the data information containing temporal sequence information and complete gait. After inputting the slice data into a CNN, convolutional layers extract motion features, followed by dimensionality reduction and downsampling using a pooling layer of size 2 to select features with higher weights and remove noise interference, avoiding the problem of traditional classifiers' classification accuracy being affected by manually extracted features beforehand. The data is then flattened into a one-dimensional array and input into an LSTM layer. LSTM's ability to analyze the correlation of time-series data is utilized to analyze different motion patterns used in pedestrian walking, improving the model's classification accuracy. Finally, the classification result is output through a fully connected layer with a soft-max activation function.

[0046] When only raw acceleration and angular velocity data are input without manual secondary feature extraction, the CNN-LSTM network model achieves a significantly higher recognition accuracy than traditional decision trees and random forests. Compared to using only a single CNN and LSTM model, the CNN-LSTM network model improves accuracy by 2.77% and 2.57%, respectively.

[0047] Because the inexpensive inertial sensors used inside mobile phones have low measurement accuracy, and the sensors are affected by the user's body shaking during data acquisition, the raw data contains a lot of noise, which also causes spurious peaks, affecting the step counting accuracy of cadence detection. Therefore, a combination of two filters is set before cadence detection: low-pass filtering and convolutional smoothing filtering. Low-pass filtering first reduces noise in the raw data and eliminates obvious spurious peaks; then convolutional smoothing filtering smooths the data a second time, yielding a motion pattern more conducive to cadence detection. A low-pass filter with a cutoff frequency of 3Hz is first used to filter noise and eliminate a large number of spurious peaks. The low-pass filtering formula is as follows:

[0048]

[0049] Then, convolutional smoothing filtering is used, that is, by fitting a continuous subset of adjacent data points to a low-order polynomial using the linear least squares method, the data accuracy is improved without changing the signal trend, and the smoothed data yields a motion pattern more conducive to step frequency detection. The formula is as follows:

[0050]

[0051] Where Y j C is the filtered value. i is the convolution coefficient, and m is the order of the polynomial fitting.

[0052] Even after data preprocessing filters out a significant amount of noise, some spurious peaks still exist between two peak values. Traditional peak detection methods would include these spurious peaks in the normal step count, increasing the error. Therefore, based on the assumption that the motion cycle is consistent under the same motion pattern and the peak value is a local maximum in the current step, the minimum peak distance is used to remove the number of other peaks within adjacent peaks, and the filtered peak value (Peak) is calculated. t The formula is as follows:

[0053] Peak t =(a t ≥(a t-k ∶a t-1 )&&a t ≤(a t+1 ∶a t+k ))

[0054] Among them, a t The resultant acceleration value is given by sample t, and k is the threshold number of samples to the left and right of t. Since the frequency of a user's movement cycle differs between normal walking and brisk walking, an adaptive threshold algorithm is designed to detect the current movement state based on the classification results and adaptively adjust the minimum peak distance k. In normal walking mode, k = 20, and in brisk walking mode, k = 10.

[0055] Sensor data generated by users in different phone postures during walking exhibits varying periodic characteristics. In handheld mode, each actual step begins with several small peaks followed by a larger peak. These larger peaks correspond to actual steps, while the smaller peaks affecting step counting need to be eliminated as spurious peaks. In pocket mode, each actual step begins with a large peak, followed by several smaller peaks, and then a medium peak, exhibiting a large-to-medium pattern for the peaks that need to be counted as actual steps. In traditional fixed threshold setting methods, setting a small fixed threshold introduces spurious peaks into step counting, while setting a large fixed threshold excludes the medium peak in pocket mode as a spurious peak. Therefore, using an adaptive dynamic threshold method can effectively improve step counting accuracy in different movement modes.

[0056] The dynamic thresholding method is based on a dynamic threshold constructed from the maximum and minimum acceleration values ​​within a window. The window size depends on the pedestrian's motion state and the sampling frequency; different window sizes also affect the threshold accuracy. The threshold is determined based on the time T for each step under different motion states. s and the current mobile phone sampling frequency F s The formula for setting the window size is as follows:

[0057] W = (T s *F s )-1

[0058] Time T per step in normal walking and brisk walking states s The times were 0.507s and 0.469 seconds, respectively.

[0059] Dynamic threshold TH in the current window peak The calculation formula is as follows:

[0060]

[0061] Among them, W max W is the maximum value of the current window. min The minimum value within the current window is denoted by 'd', which indicates the degree of approach to the peak value. Since the peak amplitude is roughly the same for handheld and telephone postures, while the peaks in pocket and arm-swinging postures exhibit varying peak sizes, 'd' is set to 1.05 for handheld and telephone postures and 0.9 for pocket and arm-swinging postures. The peak value filtered by the minimum peak distance is then compared with the dynamic threshold within the window. If the current peak value is greater than or equal to the dynamic threshold TH within this window... peak Then it is counted as a real step.

[0062] Compared with traditional peak detection methods, the adaptive detection method using minimum peak spacing selection and dynamic threshold not only has a higher accuracy in cadence detection, but is also applicable to cadence detection of pedestrians walking in different motion modes.

[0063] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An adaptive step frequency detection method based on CNN-LSTM motion pattern recognition, characterized in that, The detection method includes the following steps: obtaining sensor data collected during user walking; inputting the collected data into a CNN-LSTM model for motion pattern recognition and classification; inputting the collected data and classification results into a gait frequency detection algorithm using adaptive threshold peak detection to obtain gait frequency detection results, including the following steps: Use filters to filter the acquired data; The filtered data is then filtered for minimum peak distance using the following formula to obtain the peak values ​​that meet the criteria. t ; Peak t =(a t ≥(a t-k ∶a t-1 )&&a t ≥(a t+1 ∶a t+k )) Among them, a t The resultant acceleration value is given at sample t, and k is the threshold for the number of samples on both sides of t. k is set according to the motion state classification results of the collected data. The size of the dynamic threshold window W is calculated using the following formula: W=(T s *F s )-1 Among them, T s For each step, F s This is the current sampling frequency of the mobile phone; Then calculate the dynamic threshold TH in the current window according to the following formula. peak ; Among them, W max W is the maximum value of the current window. min The minimum value in the current window is d, and d represents the degree of approach to the peak value. The peak value after minimum peak distance filtering is compared with the dynamic threshold within the window. If the current peak value is greater than or equal to the dynamic threshold within this window, it is counted as a true step. The steps for motion pattern recognition and classification using the CNN-LSTM model include: The collected data is processed into two-dimensional slices, and then the slices are input into a CNN. Motion features of slice data are extracted using convolutional layers; Dimensionality reduction and downsampling are performed after pooling layer; The data is unfolded into a one-dimensional array by a flattening layer and then input into the LSTM layer. LSTM layers are used to classify different motion patterns of the data; The classification results are output through a fully connected layer, and the classification results include motion state and mobile phone posture.

2. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The sensor collects data including triaxial acceleration and triaxial angular velocity.

3. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The process of creating two-dimensional slices from the collected data involves sequentially creating two-dimensional slices of size 30*6 from every 30 data points.

4. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The activation function of the fully connected layer is Soft-max.

5. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The movement states include normal walking and brisk walking, and the phone postures include holding, talking, swinging arm, and pocket.

6. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The filter includes a low-pass filter and a convolutional smoothing filter, and the low-pass filter has a cutoff frequency of 3Hz.

7. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The sample size threshold k is set to 20 in normal walking mode and 10 in fast walking mode.

8. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The time T for each step s In sports mode, the time is set to 0.507 seconds in normal walking mode and 0.469 seconds in fast walking mode.

9. The adaptive step frequency detection method based on CNN-LSTM motion pattern recognition according to claim 1, characterized in that, The degree of approaching the peak value d is set to 1.05 when the phone is in a handheld or telephone position, and to 0.9 when it is in a pocket or arm swing position.

Images