An indoor positioning method based on human activity recognition assisted pedestrian dead reckoning

Through the Wavelet-CNN deep learning network, human activity recognition model is constructed, assisting in calculating step size and heading angle, solving the problem of large errors in complex activities by traditional pedestrian dead calculating methods, and achieving high-precision and low-cost indoor positioning.

CN115200575BActive Publication Date: 2024-11-19THE 54TH RESEARCH INSTITUTE OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210818070.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-13
Publication Date
2024-11-19
Estimated Expiration
2042-07-13

AI Technical Summary

Technical Problem

The traditional pedestrian dead calculating method produces large errors in the pace frequency, step length and heading angle during complex activities, resulting in low indoor positioning accuracy.

Method used

Using the human activity recognition method based on Wavelet-CNN deep learning network, a human activity recognition model is constructed through offline data processing and online real-time data input, assisting in calculating step size and heading angle, reducing errors and improving positioning accuracy.

Benefits of technology

High-precision indoor positioning under complex activity conditions is achieved, positioning errors are reduced, and it is strongly robust and cost-effective.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115200575B_ABST
    Figure CN115200575B_ABST
Patent Text Reader

Abstract

The present invention discloses an indoor positioning method based on human activity recognition-assisted pedestrian dead reckoning, which belongs to the field of indoor positioning and navigation technology of smart phones. First, in the offline stage, a MEMS sensor in a terminal is used to perform multi-channel data sampling, and the sampled data is pre-processed such as missing filling, window sliding and filtering, and annotated to create an offline data set; then a Wavelet‑CNN deep learning network is constructed and used to train and test the pre-offline data set to obtain a human activity recognition model; finally, in the online real-time positioning stage, different motion states of users are identified based on the human activity recognition model, and pedestrian step detection and step length estimation algorithms are adaptively adjusted to achieve indoor positioning in the terminal. This method can accurately identify a variety of complex human motion states, and by combining human activity recognition technology, it reduces the cumulative error of traditional pedestrian dead reckoning methods, improves positioning accuracy, greatly reduces positioning errors, and has strong robustness and effectiveness.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of indoor positioning and navigation of smart phones, and in particular to an indoor positioning method for assisting pedestrian dead reckoning based on human activity recognition performed by a smart terminal. Background Art

[0002] At present, many indoor positioning technologies, such as WiFi-based positioning technology, radio frequency identification technology, ultra-wideband-based positioning technology, and wireless positioning methods such as Bluetooth, often rely on the installation of expensive hardware infrastructure and consume a lot of time for pre-training. With the development of low-cost inertial sensor technology, the pedestrian dead reckoning method based on smartphones has attracted more and more researchers' attention. It can calculate the cadence, step length and heading by using the information collected by inertial sensors and heading reference systems, thereby solving the pedestrian track.

[0003] However, the traditional pedestrian dead reckoning method still has the limitation of large error calculation. When pedestrians hold mobile phones to perform complex activities such as running, jumping, and going up and down stairs, large cumulative errors will occur in step frequency, step length, and heading angle, which will lead to low accuracy of positioning results. The sensors in smartphones can not only analyze and calculate the current location information of pedestrians, but also perceive the user's mobile behavior information. Therefore, using human activity recognition technology to assist pedestrian dead reckoning methods to correct trajectories is a new research direction, which can obtain stable, convenient, and continuous indoor location services at a low cost. Summary of the invention

[0004] In view of this, the present invention proposes an indoor positioning method based on human activity recognition assisted pedestrian dead reckoning, which reduces the cumulative error of traditional pedestrian dead reckoning methods, improves positioning accuracy, greatly reduces positioning error, and has strong robustness and effectiveness.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] An indoor positioning method based on human activity recognition assisted pedestrian dead reckoning includes the following steps:

[0007] (1) In the offline stage, the terminal's MEMS sensor is used for multi-channel sampling, and the sampled data is processed by gap filling, window sliding, and filtering. The processed sampled data is labeled to create an offline data set;

[0008] (2) Constructing a Wavelet-CNN deep learning network, decomposing and recombining the offline data set obtained in step (1) using Wavelet, and inputting it into the Wavelet-CNN deep learning network for multiple iterative training to obtain a trained human activity recognition model;

[0009] (3) In the online stage, the real-time data collected by the terminal’s built-in sensors is input into the trained human activity recognition model to obtain the current activity status of humans in real time;

[0010] (4) Based on the real-time data collected by the terminal's built-in sensor, the number of steps under different activity states is detected by sliding the window to detect the peak value;

[0011] (5) Calculate the step length of humans under different activity states through a step length estimation algorithm assisted by human activity recognition;

[0012] (6) The real-time data collected by the built-in sensor of the terminal is integrated to obtain the heading angle, and the heading angle, the step detection result obtained in step (4), and the step length estimation result obtained in step (5) are integrated to achieve indoor positioning based on human activity recognition-assisted pedestrian dead reckoning, and accurately estimate the pedestrian trajectory under complex human activity patterns.

[0013] Furthermore, the specific implementation of step (1) is as follows:

[0014] (101) In the offline stage, the sampling data of the three-axis accelerometer and the three-axis gyroscope built into the terminal are used to fill in the missing values ​​using a linear interpolation method;

[0015] (102) Use a short sliding window of fixed length to segment the sampled data after filling in missing values;

[0016] (103) A fourth-order Butterworth digital low-pass filter is used to filter the segmented sampling data to remove the high-frequency interference noise of the original signal;

[0017] (104) The filtered sampling data is labeled with a state to form an offline data set.

[0018] Furthermore, the specific implementation of step (2) is as follows:

[0019] A Wavelet-CNN deep learning network is constructed, and the offline dataset is decomposed into high-frequency and low-frequency components using Wavelet to achieve local transformation of time and frequency. The two are then recombined as input data of the Wavelet-CNN deep learning network. Parameters are tuned through multiple iterative training of the offline dataset to obtain the optimal human activity recognition model.

[0020] Furthermore, the Wavelet-CNN deep learning network described in step (2) includes a convolutional layer, a maximum pooling layer, a global average pooling layer, a batch normalization layer and a fully connected layer, wherein the activation function is ReLU, which is used to detect the identification features of the original data, and the output of the network model is the current activity state of the human being.

[0021] Furthermore, the specific implementation of step (4) is as follows:

[0022] (401) Set the sliding window length to K and set the given threshold δ under different activity states other , dynamic threshold diff threshold and sliding window time interval ΔT threshold ;

[0023] (402) Calculate the synthetic acceleration value a(t) of the triaxial accelerometer based on the current window sampling data avg , and judge a(t) avg ≥δ other Is it true? If so, let diff threshold =a(t) avg , execute step (403); otherwise, continue sliding the window;

[0024] (403) Determine whether the time interval between adjacent peaks and troughs of the current window sampling data is ≥ ΔT threshold Is it true? If so, execute step (404); otherwise, continue sliding the window and return to step (402);

[0025] (404) Detect the difference between the adjacent peaks and troughs of the current window sampling data, which is greater than diff threshold The time step number is increased by 1, the sliding window continues, and the process returns to step (402).

[0026] Furthermore, the specific implementation of step (5) is as follows:

[0027] The data collected by the sensor in real time is processed to obtain the maximum and minimum values ​​of the acceleration in the current activity state, as well as the pedestrian's activity frequency, standard deviation and variance of the acceleration. The step length parameters determined by the deep learning network training data under different human activity states are used to calculate the step length of humans under different activity states. L ; The calculation formula is:

[0028]

[0029] In the formula, a max and a min Represent the maximum and minimum acceleration of human’s current activity state, respectively, f s 、v s and w s They represent the pedestrian's activity frequency, standard deviation and variance of acceleration respectively. α, β, γ and η represent the step length parameters determined by the deep learning network training data under different human activity states.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) The present invention proposes an adaptive step detection method assisted by human activity recognition. By setting a window sliding detection peak value, the threshold under different activity states is adaptively adjusted. This method can solve the problem of low step detection accuracy caused by large fluctuations in sensor data under different human activity states.

[0032] (2) The present invention proposes a step length estimation algorithm that combines human activity status to adaptively adjust the step length under different human activities. The step length parameters under different human activity states are determined by deep learning model training data. This can solve the problem that when pedestrians hold terminals for complex activities, the step length will produce large cumulative errors, which in turn leads to poor positioning results.

[0033] (3) Compared with the traditional pedestrian dead reckoning method, the present invention can obtain stable, convenient and continuous indoor location services at a lower cost. By accurately identifying a variety of complex human motion states, the pedestrian dead reckoning method combined with human activity recognition reduces the cumulative error of the traditional method and can achieve higher positioning accuracy. Moreover, the present method uses a deep learning method for human activity recognition, the model converges faster, and has stronger robustness and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flow chart of a method for assisting pedestrian dead reckoning based on human activity recognition in an embodiment of the present invention.

[0035] Figure 2 Schematic diagram of filtering effect in an embodiment of the present invention.

[0036] Figure 3 This is a rendering of the dynamic threshold detection method in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0038] An indoor positioning method based on human activity recognition assisted pedestrian dead reckoning includes an offline data model building stage and an online real-time positioning stage. In this method, first, in the offline stage, the MEMS sensor in the smart terminal is used to perform multi-channel data sampling, and the sampled data is preprocessed by gap filling, window sliding and filtering, and annotated to create an offline data set; then, a Wavelet-CNN deep learning network is constructed and used to train and test the preprocessed sensor data to obtain a human activity recognition model; finally, in the online real-time positioning stage, the different motion states of the user are identified based on the human activity recognition model, and the pedestrian step detection and step length estimation algorithms are adaptively adjusted, and the indoor positioning method based on human activity recognition assisted pedestrian dead reckoning is implemented on the smart terminal. The method flow is as follows: Figure 1 shown.

[0039] The method specifically comprises the following steps:

[0040] Step 1: In the offline stage, the data collected by the multi-channel MEMS sensor built into the terminal is used to fill the missing values ​​using linear interpolation, and a short sliding window with a length of 64 is used to segment and obtain more test data, and a cutoff frequency of w is used. c =0.2π fourth-order Butterworth digital low-pass filter filters out the high-frequency interference noise of the original signal. The filtering effect is as follows Figure 2 As shown, the filtered sampling data is then labeled with states to form an offline data set.

[0041] Step 2: Build a Wavelet-CNN deep learning network, decompose the offline data set into high-frequency components and low-frequency components using Wavelet, realize local transformation of time and frequency, and recombine the two as input data of the Wavelet-CNN deep learning network, and optimize the parameters through multiple iterative training of the offline data set to obtain the optimal human activity recognition model. The Wavelet-CNN deep learning network includes convolutional layers, maximum pooling layers, global average pooling layers, batch normalization layers, and fully connected layers, where the activation function is ReLU, which is used to detect the identification features of the original data, and the output of the network model is the current activity status of humans.

[0042] Step 3: In the online stage, the real-time data collected by the terminal's built-in sensors is input into the trained human activity recognition model to obtain the current human activity status in real time;

[0043] Step 4: Based on the real-time data collected by the terminal's built-in sensor, the step count detection under different activity states is performed by sliding the window to detect the peak value. The step count detection is accurately performed by adaptively adjusting the threshold under different activity states. The dynamic threshold detection results are as follows: Figure 3 shown.

[0044] The specific implementation method is:

[0045] (401) Set the sliding window length to K and set the given threshold δ under different activity states other , dynamic threshold diff threshold and sliding window time interval ΔT threshold ;

[0046] (402) Calculate the synthetic acceleration value a(t) of the triaxial accelerometer based on the current window sampling data avg , and judge a(t) avg ≥δ other Is it true? If so, let diff threshold =a(t) avg , execute step (403); otherwise, continue sliding the window;

[0047] (403) Determine whether the time interval between adjacent peaks and troughs of the current window sampling data is ≥ ΔT threshold Is it true? If so, execute step (404); otherwise, continue sliding the window and return to step (402);

[0048] (404) Detect the difference between the adjacent peaks and troughs of the current window sampling data, which is greater than diff threshold The time step number is increased by 1, the sliding window continues, and the process returns to step (402).

[0049] Step 5: Calculate the step length of humans under different activity states through a step length estimation algorithm assisted by human activity recognition;

[0050] The specific implementation method is:

[0051] The data collected by the sensor in real time is processed to obtain the maximum and minimum values ​​of the acceleration in the current activity state, as well as the pedestrian's activity frequency, standard deviation and variance of the acceleration. The step length parameters determined by the deep learning network training data under different human activity states are used to calculate the step length of humans under different activity states. L ; The calculation formula is:

[0052]

[0053] In the formula, a max and a min Represent the maximum and minimum acceleration of human’s current activity state, respectively, f s 、v s and w s They represent the pedestrian's activity frequency, standard deviation and variance of acceleration respectively. α, β, γ and η represent the step length parameters determined by the deep learning network training data under different human activity states.

[0054] Step 6: The real-time data collected by the terminal's built-in sensor is integrated to obtain the heading angle, and the heading angle, the step detection result obtained in step (4), and the step length estimation result obtained in step (5) are integrated to achieve indoor positioning based on human activity recognition-assisted pedestrian dead reckoning, and accurately estimate pedestrian trajectories under complex human activity patterns.

[0055] This method uses MEMS sensors in smart terminals to perform multi-channel data sampling to construct a data set in the offline stage, uses the Wavelet-CNN deep learning network to obtain a human activity recognition model, and in the online stage, adaptively adjusts the step detection and step length estimation algorithms in the pedestrian dead reckoning method based on the human activity recognition model to achieve real-time indoor positioning on the smart terminal. The Wavelet-CNN model is a feature extraction method in the deep learning method, which learns the sampling data features of the built-in sensors of the smart terminal to construct a human activity recognition model. The adaptive step detection method assisted by the human activity recognition model adaptively adjusts the dynamic threshold diff under different human activity states. threshold Accurately detect the number of steps, and set the threshold δ for different human activity states other The human activity recognition-assisted adaptive step length estimation method adaptively adjusts the step length according to the step length parameters determined by the deep learning model training data under different activity states.

[0056] In summary, the present invention avoids the problem that the traditional pedestrian dead reckoning method will cause large cumulative errors in step frequency, step length and heading angle when pedestrians hold terminals to perform complex activities such as running, jumping, and going up and down stairs, which in turn leads to low positioning accuracy. An indoor positioning method based on human activity recognition-assisted pedestrian dead reckoning by intelligent terminals is proposed. In this method, the Wavelet-CNN deep learning model is used to train and classify the sampled data of the built-in sensor of the intelligent terminal to obtain a human activity recognition model. In the real-time positioning stage, an adaptive pedestrian dead reckoning positioning method assisted by the human activity recognition model is designed, and the traditional pedestrian dead reckoning method is optimized from the two aspects of step detection and step length estimation, respectively, to achieve accurate and real-time position estimation on the intelligent terminal. The method is verified by experiments, and the results show that the method of the present invention can accurately identify a variety of complex human motion states, and the pedestrian dead reckoning method combined with human activity recognition-assisted reduces the cumulative error of the traditional method, improves the positioning accuracy, reduces the maximum error, and has strong robustness and effectiveness.

Claims

1. An indoor positioning method based on human activity recognition assisted pedestrian dead reckoning, characterized in that: The following steps are involved: (1) In the offline stage, the terminal's MEMS sensor is used for multi-channel sampling, and the sampled data is processed by gap filling, window sliding, and filtering. The processed sampled data is labeled to create an offline data set; (2) Constructing a Wavelet-CNN deep learning network, decomposing and recombining the offline data set obtained in step (1) using Wavelet, and inputting it into the Wavelet-CNN deep learning network for multiple iterative training to obtain a trained human activity recognition model; (3) In the online stage, the real-time data collected by the terminal’s built-in sensors is input into the trained human activity recognition model to obtain the current activity status of humans in real time; (4) Based on the real-time data collected by the terminal's built-in sensor, the number of steps under different activity states is detected by sliding the window to detect the peak value; (5) Calculate the step length of humans under different activity states through a step length estimation algorithm assisted by human activity recognition; (6) The real-time data collected by the built-in sensor of the terminal is integrated to obtain the heading angle, and the heading angle, the step detection result obtained in step (4), and the step length estimation result obtained in step (5) are integrated to achieve indoor positioning based on human activity recognition-assisted pedestrian dead reckoning, and accurately estimate the pedestrian trajectory under complex human activity patterns; The specific implementation of step (4) is as follows: (401) Set the sliding window length to K and set the given threshold δ under different activity states other , dynamic threshold diff threshold and sliding window time interval ΔT threshold ; (402) Calculate the synthetic acceleration value a(t) of the triaxial accelerometer based on the current window sampling data avg , and judge a(t) avg ≥δ other Is it true? If so, let diff threshold =a(t) avg , execute step (403); otherwise, continue sliding the window; (403) Determine whether the time interval between adjacent peaks and troughs of the current window sampling data is ≥ ΔT threshold Is it true? If so, execute step (404); otherwise, continue sliding the window and return to step (402); (404) Detect the difference between the adjacent peaks and troughs of the current window sampling data, which is greater than diff threshold The time step number is increased by 1, the sliding window continues, and the process returns to step (402).

2. The indoor positioning method based on human activity recognition assisted pedestrian dead reckoning according to claim 1, characterized in that: The specific implementation of step (1) is as follows: (101) In the offline stage, the sampling data of the three-axis accelerometer and the three-axis gyroscope built into the terminal are used to fill in the missing values ​​using a linear interpolation method; (102) Use a short sliding window of fixed length to segment the sampled data after filling in missing values; (103) A fourth-order Butterworth digital low-pass filter is used to filter the segmented sampling data to remove the high-frequency interference noise of the original signal; (104) The filtered sampling data is labeled with a state to form an offline data set.

3. The indoor positioning method based on human activity recognition assisted pedestrian dead reckoning according to claim 1, characterized in that: The specific implementation of step (2) is as follows: A Wavelet-CNN deep learning network is constructed, and the offline dataset is decomposed into high-frequency and low-frequency components using Wavelet to achieve local transformation of time and frequency. The two are then recombined as input data of the Wavelet-CNN deep learning network. Parameters are tuned through multiple iterative training of the offline dataset to obtain the optimal human activity recognition model.

4. The indoor positioning method based on human activity recognition assisted pedestrian dead reckoning according to claim 3, characterized in that: The Wavelet-CNN deep learning network described in step (2) includes a convolutional layer, a maximum pooling layer, a global average pooling layer, a batch normalization layer, and a fully connected layer, wherein the activation function is ReLU, which is used to detect the identification features of the original data, and the output of the network model is the current activity status of the human being.

5. The indoor positioning method based on human activity recognition assisted pedestrian dead reckoning according to claim 1, characterized in that: The specific implementation of step (5) is as follows: The data collected by the sensor in real time is processed to obtain the maximum and minimum values ​​of the acceleration in the current activity state, as well as the pedestrian's activity frequency, standard deviation and variance of the acceleration. The step length parameters determined by the deep learning network training data under different human activity states are used to calculate the step length of humans under different activity states. L ; The calculation formula is: In the formula, a max and a min Represent the maximum and minimum acceleration of human’s current activity state, respectively, f s 、v s and w s They represent the pedestrian's activity frequency, standard deviation and variance of acceleration respectively. α, β, γ and η represent the step length parameters determined by the deep learning network training data under different human activity states.

Citation Information

Patent Citations

  • Indoor positioning method fusing multi-source data

    CN109769296A

  • Pedestrian navigation method for online step length calibration, motion deviation angle correction and self-adaptive energy management

    CN111189443A

  • Pedestrian trajectory calculation method based on Mems inertial sensor

    CN114562995A

  • Self-correcting dead reckoning system and method based on activity identification and fusion filtering

    CN114674317A

  • Pedestrian navigation device and method based on novel multi-sensor fusion technology

    WO2017215024A1