A neural network auxiliary-based indoor pedestrian positioning method

Abstract

The application discloses a kind of pedestrian indoor positioning methods based on neural network auxiliary, the method is: pedestrian holds the intelligent mobile phone including main navigation module, pedestrian foot sets and includes auxiliary navigation module of IMU unit;Accelerometer, gyroscope information of main navigation module is as input data, whether IMU unit is zero speed point as label data, establish the step point identification model based on CNN to IMU unit is trained offline, determine the step point when pedestrian hand-held mobile phone according to zero speed time;Inertial sensor measurement value is collected as training data from the intelligent mobile phone, simultaneously, the walking distance of pedestrian is collected as training label from auxiliary navigation module, and the step length estimation model is trained;The error of pedestrian navigation information is corrected by extending Kalman filter, and the final pedestrian navigation information is obtained.The application effectively suppresses the cumulative error of inertial navigation system navigation information, and realizes the accurate positioning of personnel in various environments.

Description

Technical Field

[0001] This invention relates to the field of pedestrian indoor positioning technology, and in particular to a pedestrian indoor positioning method based on neural network assistance. Background Technology

[0002] Pedestrian navigation systems can provide navigation and positioning information for pedestrians in unfamiliar environments. Research on outdoor pedestrian navigation primarily focuses on satellite navigation or a combination of satellite and inertial navigation, achieving significant results in recent years. However, research on indoor navigation faces challenges. In complex environments such as tall buildings and forests, satellite signals suffer severe degradation, failing to meet navigation requirements. Inertial Navigation+System (INS), with its advantages of high concealment and strong anti-interference capabilities, has become a major research direction for pedestrian inertial navigation systems.

[0003] However, inertial measurement units (IMUs) have inherent errors, and navigation errors accumulate rapidly over time, causing long-endurance positioning accuracy of INS to fall short of requirements. Low-cost mobile phone IMUs exhibit even greater errors. Researchers have employed various compensation methods to address this issue, commonly including those based on indoor UWB, ZigBee, wireless networks, WiFi, and Bluetooth. However, these methods require the prior deployment of navigation information sources such as WiFi routers and beacon base stations indoors, increasing costs and limiting their applicability to specific environmental conditions. Pedestrian dead reckoning (PDR) is a technology that uses motion information collected by various sensors embedded in a mobile phone for indoor pedestrian positioning. This technology uses the pedestrian's initial position, walking distance, and direction information to estimate the user's position at any given time, without requiring additional equipment. However, due to the randomness of people's movements, the diversity of how they hold their phones, and sensor measurement noise, smartphone-based positioning technologies suffer from problems such as error accumulation and energy consumption that need to be addressed. Summary of the Invention

[0004] The purpose of this invention is to provide a pedestrian indoor positioning method based on neural network assistance, thereby effectively suppressing the cumulative error of navigation information of inertial navigation system and realizing accurate positioning of people in various environments.

[0005] The technical solution to achieve the purpose of this invention is as follows: a pedestrian indoor positioning method based on neural network assistance, wherein the pedestrian holds a smartphone, the smartphone includes a main navigation module, the main navigation module includes a gyroscope, an accelerometer, a magnetometer and an altimeter, a guidance navigation module is installed at the pedestrian's feet, the guidance navigation module includes an IMU unit, and the positioning method includes the following steps:

[0006] Step 1: Offline training for zero-speed detection: Using the accelerometer and gyroscope information of the main navigation module as input data, and whether the IMU unit of the auxiliary navigation module is at a zero-speed point as label data, a CNN-based step point recognition model is established to train the IMU unit of the auxiliary navigation module offline, identify the zero-speed moment of walking, and determine the step point when the pedestrian is holding a mobile phone based on the zero-speed moment.

[0007] Step 2: Offline training of the stride length estimation model: The stride length estimation model includes a TCN network and a noise-reducing autoencoder. Inertial sensor measurements are collected from a smartphone as training data, and pedestrian walking distances are collected from a navigation module installed on the foot as training labels. The stride length estimation model is trained and used to estimate the stride length of pedestrians.

[0008] Step 3: Online prediction of navigation information: Using the inertial sensor information of the smartphone, combined with the trained step length estimation model to detect step points, the pedestrian navigation information is corrected by extended Kalman filtering to obtain the final pedestrian navigation information.

[0009] Compared with the prior art, the significant advantages of this invention are: (1) It uses neural networks to compensate for the accumulation of errors in the inertial navigation system. In this process, the neural network is used to determine the step point and establish the heading angle, velocity and position error model and Kalman filter equation to obtain pedestrian navigation information; (2) It uses the pedestrian mobile phone IMU to realize inertial positioning. Without increasing the complexity of the system, the accuracy of step detection is improved by neural networks, and the stability of the pedestrian navigation system is improved; (3) The pedestrian step length model is trained according to the IMU information to improve the positioning accuracy of the algorithm. Combined with the step detection and distance prediction of pedestrian walking, the extended Kalman filter equation is established to finally realize the pedestrian position estimation. Attached Figure Description

[0010] Figure 1 This is the overall flowchart of the present invention.

[0011] Figure 2 This is a structural diagram of the step detection method of the present invention.

[0012] Figure 3 This is a structural diagram of the step size estimation of the present invention.

[0013] Figure 4 This is a Kalman filter error correction diagram according to the present invention. Detailed Implementation

[0014] This invention relates to a neural network-assisted step detection and stride length algorithm. Since the random movements of pedestrians during walking affect the estimation of step count and stride length, a method for predicting pedestrian step detection and walking distance based on a neural network model is proposed. This algorithm utilizes the pedestrian's mobile phone IMU for inertial positioning. Without increasing system complexity, it improves the accuracy of step detection and enhances the stability of the pedestrian navigation system through neural networks. Furthermore, a pedestrian stride length model is trained based on IMU information to improve the algorithm's positioning accuracy. Combining pedestrian step detection and distance prediction, an extended Kalman filter equation is established to ultimately calculate the pedestrian's position.

[0015] This invention discloses a neural network-assisted indoor pedestrian positioning method. The pedestrian holds a smartphone, which includes a main navigation module comprising a gyroscope, an accelerometer, a magnetometer, and an altimeter. A secondary navigation module, including an IMU unit, is installed at the pedestrian's feet. The positioning method includes the following steps:

[0016] Step 1: Offline training for zero-speed detection: Using the accelerometer and gyroscope information of the main navigation module as input data, and whether the IMU unit of the auxiliary navigation module is at a zero-speed point as label data, a CNN-based step point recognition model is established to train the IMU unit of the auxiliary navigation module offline, identify the zero-speed moment of walking, and determine the step point when the pedestrian is holding a mobile phone based on the zero-speed moment.

[0017] Step 2: Offline training of the stride length estimation model: The stride length estimation model includes a TCN network and a noise-reducing autoencoder. Inertial sensor measurements are collected from a smartphone as training data, and pedestrian walking distances are collected from a navigation module installed on the foot as training labels. The stride length estimation model is trained and used to estimate the stride length of pedestrians.

[0018] Step 3: Online prediction of navigation information: Using the inertial sensor information of the smartphone, combined with the trained step length estimation model to detect step points, the pedestrian navigation information is corrected by extended Kalman filtering to obtain the final pedestrian navigation information.

[0019] As a specific example, in step 1, the label data is obtained through the following process:

[0020] The triaxial accelerometer of the IMU unit in the navigation module outputs acceleration signals that are vectorized on the three orthogonal axes of the vehicle. If the acceleration is at zero speed, it is the stage where the person and the ground should be in complete contact. The acceleration vector sum is the local gravitational acceleration.

[0021] Based on the set threshold, it is determined whether the sum of the acceleration vectors is within the threshold. The mathematical model expression is:

[0022]

[0023]

[0024] Among them, |Acc (k) | represents the sum of acceleration vectors at time k, a x,k a y,k a z,k These are the components of the acceleration value measured by the inertial element at time k on the three orthogonal axes: x, y, and z. Acc_min and Acc_max are the maximum and minimum values ​​of the set threshold, respectively. C1 is a logical judgment value, where 1 represents the detection of the zero velocity range and 0 represents the motion state.

[0025] When in a zero-speed state, the soles of the feet are fully in contact with the ground, and the gyroscope outputs an angular velocity signal and a magnitude close to zero. If outside the zero-speed range, the angular velocity signal fluctuates. The zero-speed range is detected by identifying the walking state and determining the angular velocity threshold. The formula for the angular velocity magnitude is:

[0026]

[0027]

[0028] Among them, |w (k) | represents the magnitude of the angular velocity, w x,k w y,k w z,k These represent the components of the angular velocity at time k along the three orthogonal axes: x, y, and z; C2(k) is the logical value for angular velocity threshold detection, w max This represents the threshold value of the angular velocity magnitude; since the angular velocity magnitude is zero in the zero velocity range, the threshold value of the angular velocity magnitude is set to 1.

[0029] According to the above logical expression, when both conditions are met simultaneously, it is considered to be a zero-velocity interval, expressed as:

[0030] F (k) =C1(k)&C2(k)

[0031] The zero-velocity interval detected by zero velocity is used as the label data.

[0032] As a specific example, in step 1, a CNN-based step point recognition model is established. The specific process is as follows:

[0033] Based on the principle of zero-speed detection, relevant variables are found from the output of the IMU unit as the input feature vector:

[0034] The velocity at time k in the n-system It is the attitude transition matrix gyroscope measuring angular velocity value The incremental output Δv of the accelerometer k The incremental output Δθ of the gyroscope k A function of these variables, where Δv k , Δθ k as follows:

[0035]

[0036]

[0037] Among them, f b These are the measurements from the accelerometer. It is the angular velocity value measured by the gyroscope, velocity. Influencing factors include IMU output values ​​and attitude transfer matrix; attitude transfer matrix The pitch, roll, and yaw angles in the carrier coordinate system at time k represent the current three-dimensional attitude vector Φ. k Obtained through calculation;

[0038] Assuming f represents a functional relationship, the velocity expression calculated by the inertial navigation system is:

[0039]

[0040] Among them, f b , Φ k These four variables were selected as the input feature vector;

[0041] During the training process, the step point recognition model first arranges the process variable data into a matrix, which consists of 12-dimensional process variables collected within the sampling time T, with the sampling time being 1 / 125s.

[0042] Let x(t) be the input set of the step point recognition model, and let the input feature vector include all the data in the matrix T. x(t) is used as the input of the step point recognition model to predict whether the current moment is the zero velocity point.

[0043] As a specific example, the CNN-based step point recognition model in step 1 includes an input layer, convolutional layer, pooling layer, fully connected layer, and output layer. The calculation process is as follows:

[0044]

[0045] Where l represents the l-th convolutional layer, i represents a value of the i-th convolutional output matrix, j represents the index of the corresponding output matrix from 0 to N from left to right, and N is the number of convolutional output matrices. b represents the memory information of the matrix numbered j in the i-th convolution output matrix of the l-th convolutional layer. j For bias, This indicates the weight of the input memory information from the previous time step. Indicates the current input value;

[0046] f represents a nonlinear function, using the sigmoid function, and the calculation formula is as follows:

[0047]

[0048] The pooling layer uses a local averaging method for dimensionality reduction, and the calculation formula is as follows:

[0049]

[0050] This represents a local output term after the pooling process, obtained by averaging the local small matrix of size n*n from the previous layer;

[0051] Fully connected layer: Maps the features learned by the network to the label space of the samples. The fully connected layer transforms the two-dimensional feature vector output by the convolution into a one-dimensional feature vector.

[0052] As a specific example, in step 1, the process of identifying the zero-velocity moment includes using a CNN-based step point recognition model to process the three-axis data of the IMU unit in the navigation module in parallel, as follows:

[0053] ① Input the raw data collected by the triaxial accelerometer and gyroscope through the input layer;

[0054] ② Data processing through convolutional layers: 128 convolutional filters are used for parallel learning to create data feature representations, where each filter is 1×12 in size, the stride of the convolution is 1, and the random inactivation rate is 0.15.

[0055] ③ Apply the sigmoid function to the generated 100 feature representations and use a 1×4 max pooling layer for 4x dimensionality reduction;

[0056] ④ The feature vectors are then passed to the fully connected layer for feature labeling and classification;

[0057] ⑤ Output whether it is the zero speed moment through the output layer.

[0058] As a specific example, the noise reduction autoencoder in step 2 is as follows:

[0059] A noise reduction autoencoder consists of two parts: an encoding network and a decoding network. When the number of neurons in the hidden layer is less than that in the input and output layers, the encoding network maps the input samples to a low-dimensional feature space to obtain the feature representation of the data. The decoding network then remaps this feature representation back to the original feature space and minimizes the root mean square error of the input and output to achieve data reconstruction.

[0060] The input is denoted as x = {x 1 ,x 2 ,x 3 ,···,x N}, where N represents the number of input samples, then the encoding process is as follows:

[0061] a n =f θ (x n ) = s(W (1) x n +b (1) )

[0062] Where, θ={W (1) ,b (1)} represents the weights and biases of the encoding network, a n The hidden layer represents the feature map obtained, s() represents the activation function, and the result obtained during the decoding process is the input x. n Reconstruction form as follows:

[0063]

[0064] Where, θ′={W (2) ,b (2)} represents the weights and biases of the decoding network;

[0065] The training process involves reconstructing the input data to reduce reconstruction error. The cost function J for calculating the reconstruction error is as follows:

[0066]

[0067] When the reconstruction error approaches 0 through training, i.e., less than 10... -4 If we only look at the encoding network, the output obtained by the hidden layer is a low-dimensional representation of the original features, and this process is the noise reduction process.

[0068] As a specific example, in step 2, the TCN network is as follows:

[0069] The TCN network includes an extended causal convolution module, which uses causal convolution to restrict the sliding window to achieve time constraint, expands the receptive field of the convolution to learn a wider range of information through dilated convolution, and adds residual connections to prevent the gradient from vanishing due to excessive network layers in order to obtain time series information features.

[0070] Causal convolution: Causal convolution gives TCN the causal property, meaning that information at the current time can only be predicted based on information from previous times, that is, it can only be predicted based on the current input x. t And the previous input x1,x2,...,x t-1 Calculations are performed, but the information cannot be used for prediction afterward. Features are extracted from left to right:

[0071]

[0072] Where p(x) represents the convolution function, T is the total time of the time series, and t is the time of the current time series;

[0073] Dilated convolution: The text sequence is sampled and calculated at predetermined dilation intervals, which is equivalent to inserting d zeros inside the convolution kernel, where d is the number of dilation intervals and f is the kernel size after dilation. k_d for:

[0074] f k_d =(d-1)*(f k -1)+f k

[0075] Among them, f k This represents the kernel size of the current layer.

[0076] Residual links: Adding residual link structures forms short-circuit links. By summing the input x and the nonlinearly transformed F(x), the temporal convolutional network avoids gradient vanishing caused by the increase in the number of network layers.

[0077] R = x + F(x)

[0078] Where R is the output of the residual link;

[0079] After each dilated causal convolution calculation (Conv()), the parameter hierarchy is normalized using Hi-norm(), and then the ReLU activation function is used for nonlinear calculation. The result is summed with the input to achieve residual linking. The calculation process is as follows:

[0080] T i =Conv(W i *F j +b i )

[0081] {T0,T1,···,T n} = Hi-norm({T0,T1,···,T) n})

[0082] {T0,T1,···,T n} = Relu({T0,T1,···,T n})

[0083] Among them, T i W is the state value obtained by convolution at time i; i F is the word matrix calculated by convolution at time i; j b is the convolution kernel of the j-th layer; i Let {T0,T1,...,T1} be the bias matrix; n} is the encoding of the sequence after one complete convolution calculation;

[0084] The TCN network expands the receptive field of convolution by stacking multiple dilated causal convolutional layers, thereby obtaining more complete time-series features and extracting deeper information features.

[0085] As a specific example, in step 2, the stride estimation model is trained and used to estimate the stride length of pedestrians, as follows:

[0086] The actual step size is used to train the regression layer of the step size estimation model, which learns the nonlinear relationship between the denoised feature vectors and the corresponding actual step size during each step size period;

[0087] The global objective of the regression layer is to minimize the error loss function J(D,G) between the actual step size and the estimated value:

[0088]

[0089] Among them, y i It corresponds to the actual stride length of the input D(·). G(·) is the step size estimate from the regression layer, and G(·) is the predicted value in the final step size model.

[0090] The error loss function J(D,G) is minimized through RMS (Root Mean Square) propagation; after training, the step size estimation model is used to predict the step size S of pedestrians. k :

[0091]

[0092]

[0093] The above formula is the recursive formula for the PDR (Pedestrian Dead Reckoning) algorithm, where E and N are the coordinates of the eastward and northward positions, respectively, k is the current time, k-1 is the previous time, and S... k θ k t k-1 to t k At any given moment, pedestrians are positioned from the upper front east and north (E) k-1 N k-1 ) to location (E k N k The walking distance and azimuth angle of E; k N represents the current eastward position. k E0 is the current northward position, E0 is the initial eastward position, N0 is the initial northward position, and S is the current northward position. i Let θ be the distance traveled at time i. i Let i be the azimuth angle at time i;

[0094] The PDR algorithm combines the eastward and northward position information with the altitude H obtained from the mobile phone barometric altimeter to obtain the position information in the northeast-northeast coordinate system.

[0095]

[0096] As a specific example, in step 3, the pedestrian navigation information is corrected for errors by extending the Kalman filter, as follows:

[0097] The 15-dimensional error state variables of the system at time k constitute the error state variables δx of the Kalman filter. k As shown in the following formula:

[0098]

[0099] in, δv represents the three attitude errors: roll angle, pitch angle, and yaw angle. k =[δv kx δv ky δv kz ] T For the speed errors in the east, north, and sky directions, For longitude, latitude, and altitude position errors, These are the components of the gyroscope error along the x, y, and z axes. The components of the accelerometer error along the x, y, and z axes;

[0100] The discrete linearized representation of the system error model is as follows:

[0101]

[0102] Where, δz k It is the measurement vector, H k It is a measurement matrix, w k It is process noise, v k It is measurement noise, Φ k,k-1 It is the state transition matrix; δx k,k-1 For state prediction, δx k-1,k-1 This refers to measurement and prediction errors;

[0103] State transition matrix Φ k,k-1 The expression is as follows:

[0104]

[0105] Where Δt is the sampling interval, It is a skew-symmetric cross-product operator matrix. It is the direction cosine matrix, I 3×3 It is a 3×3 identity matrix;

[0106] When a pedestrian's state is determined to be standing, their velocity is considered to be zero. The velocity observation equation for zero velocity is as follows:

[0107]

[0108] in, For inertial navigation to calculate the velocity vector δz in the loaded system v It is the measurement speed error vector;

[0109] When a user walks forward regularly, the lateral and vertical velocities in the body coordinate system are zero. Walking speed is estimated by stride length, and the virtual speed measurement is represented as follows:

[0110]

[0111]

[0112] in, This refers to the virtual speed of the phone within the system, where Δt is the time interval between each step, S is the step size, and δz is the virtual speed. v For speed measurement error, For the velocity in the inertial navigation-based northeast-sky coordinate system, It is a rotation matrix;

[0113] By combining the position information output by the PDR algorithm and the attitude information obtained by the magnetometer, a complete measurement error equation is established:

[0114]

[0115]

[0116]

[0117] Where, δz p It is the measurement position error vector. It is the measurement attitude error vector, δz k It is the navigation information measurement error vector. It is the heading angle measured by the magnetometer. The attitude is calculated by the inertial navigation system. It is the position calculated by the inertial navigation system. It is the position calculated by PDR;

[0118] When the observations are valid, update the state variables and their corresponding covariances using the following method:

[0119]

[0120]

[0121]

[0122] Among them, K k It is the undetermined correction coefficient matrix. It is an estimation error. It is the state prediction error, δz k It is the measurement error, H k It is a measurement matrix. It is the state prediction value, P k It is the mean square error matrix of the state estimation, P k,k-1 It is the mean square error matrix for one-step state prediction, R k It is the measurement noise covariance matrix.

[0123] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0124] Example

[0125] Combination Figures 1-4 This invention relates to a neural network-assisted indoor pedestrian positioning method, which is divided into offline training and online prediction stages. In the offline training stage, step points are first detected and trained, and then the step length estimation model is trained. In the online prediction stage, sensor measurement information is obtained from various sensors to detect step points. The heading angle, velocity, and position error of the Kalman filter equation are obtained through the sensor measurement information. Error correction is performed on the pedestrian navigation information to obtain the final navigation information.

[0126] This invention relates to a neural network-assisted pedestrian navigation method. The pedestrian mobile phone includes a navigation module, which consists of a gyroscope, an accelerometer, a magnetometer, and an altimeter. The offline training phase includes foot inertial navigation-assisted determination of training labels, and includes the following steps:

[0127] Step 1: Fix the IMU to the heel of the human foot to collect acceleration and angular velocity information during movement, while minimizing the impact of the human walking state on the IMU. Through offline training of the inertial sensor installed on the human foot, based on the analysis of the human gait phase, and combined with the plantar acceleration, angular velocity values, and noise characteristics at each discrete moment, set upper and lower thresholds for the stationary interval. Zero-velocity points within the thresholds are used as label data, and the acceleration and angular velocity values ​​of the mobile phone IMU are used as input data. A CNN network is trained to identify the zero-velocity moments of walking, and the stride points when the pedestrian is holding the mobile phone are determined based on the zero-velocity moments.

[0128] Step 2: To improve the accuracy of stride length estimation, a TCN-based stride length estimation method is proposed to address the PDR algorithm error caused by distance estimation in complex environments. In the offline training phase, accelerometer and gyroscope data provided by the mobile phone's inertial sensor are used as training data, and stride length data obtained using the foot-based inertial navigation ZUPT algorithm is used as stride length labels. In the online prediction phase, sensor data collected by the mobile phone's IMU and the pre-trained model are used to predict the stride length for one gait cycle, and position information is obtained through dead reckoning.

[0129] Step 3: When a step point is identified during walking, the observed values ​​for heading angle error, velocity error, and position error are updated in this state. The heading angle error is obtained by measuring the three-axis magnetic components using a magnetometer and then transforming the coordinates to obtain the pedestrian's magnetic heading information. The velocity error is the velocity information of the step point, which can be calculated based on the step length. The position error is calculated from the step length and heading using the PDR algorithm. Combining the state equations, a Kalman filter equation is established to correct the errors in the pedestrian navigation system, finally completing the output of the pedestrian navigation information.

[0130] The entire process of this invention will now be described in detail:

[0131] Step 1: Offline training for zero-velocity detection. The accelerometer and gyroscope information from the inertial sensor are used as input data, and whether the foot inertial navigation system is at a zero-velocity point is used as label data. A CNN model is built and trained.

[0132] (1.1) Obtaining Tag Data

[0133] Acceleration vector sum and threshold detection involves vectorizing the acceleration signals output by a triaxial accelerometer (an inertial sensor) along three orthogonal axes on the carrier. If the acceleration is at zero velocity, the person should be in complete contact with the ground, and theoretically, the sum of the acceleration vectors should equal the local gravitational acceleration. Based on a set threshold, it is determined whether the sum of the acceleration vectors falls within that threshold. The mathematical model expression is as follows:

[0134]

[0135]

[0136] Among them, |Acc (k) | represents the sum of acceleration vectors at time k, a x,k a y,k a z,k These are the acceleration values ​​measured by the inertial element at time k, respectively. Acc_min and Acc_max are the maximum and minimum values ​​of the set threshold, respectively. C1 is the logical judgment value, where "1" represents the detection of the zero velocity interval and "0" represents the motion state.

[0137] Angular velocity threshold detection works as follows: When in the zero-velocity range, the entire foot is in contact with the ground, and the gyroscope's output angular velocity signal and its magnitude of change are considered to be close to zero. If outside the zero-velocity range, the angular velocity signal will exhibit strong fluctuations. By identifying the walking state, an angular velocity threshold is determined to detect the zero-velocity range. The formula for the angular velocity magnitude is:

[0138]

[0139]

[0140] Among them, |w (k) | represents the magnitude of the angular velocity, C2(k) is the logical value for angular velocity threshold detection, and w max This represents the threshold value for the angular velocity magnitude. Since the angular velocity magnitude is zero in the zero velocity range, the threshold value for the angular velocity magnitude is set to 1.

[0141] According to the above logical expression, when both conditions are met simultaneously, it can be identified as a zero-velocity interval, which is expressed as:

[0142] F (k) =C1(k)&C2(k)

[0143] Based on the zero-speed detection method proposed above, and processed according to the zero-speed detection algorithm, the zero-speed interval detected above is used as the label data.

[0144] (1.2) Establishment of step point recognition model

[0145] Based on the principle of zero-speed detection, relevant variables can be found from the IMU output as input feature vectors:

[0146] The velocity at time k in the n-system yes Functions of variables such as Δv and Δθ, where Δv and Δθ are the incremental outputs of the accelerometer and gyroscope.

[0147]

[0148]

[0149] Where f b These are the measurements from the accelerometer. This is the angular velocity value measured by the gyroscope. (Velocity) The main influencing factors are the IMU output value and the attitude transfer matrix. It can be calculated from the pitch angle, roll angle, and yaw angle in the carrier coordinate system at time k, that is, the three-dimensional attitude vector Φ at the current time. k .

[0150] Therefore, assuming f represents a functional relationship, the velocity expression calculated by the inertial navigation system can be expressed as:

[0151]

[0152] These four variables were also selected as the network's input feature vectors.

[0153] During network model training, the process variable data are first arranged into a matrix, which consists of 12-dimensional process variables collected within a certain sampling time T. The sampling time is 1 / 125s. Let the input set of the model be x(t), and the input feature vector include all IMU and INS data within T. x(t) is used as the input of the network model to predict whether the current time step is a zero velocity point. To ensure the efficiency of the input data, the data is normalized.

[0154] The CNN model used in this invention comprises a five-layer structure: input layer, convolutional layer, pooling layer, fully connected layer, and output layer. The calculation process is as follows:

[0155]

[0156] Where l represents the l-th convolutional layer, i represents a value of the i-th convolutional output matrix, j represents the index of the corresponding output matrix from 0 to N from left to right, and N is the number of convolutional output matrices. f represents a nonlinear function; this paper uses the sigmoid function, which has a small impact on the kernel size. The formula for calculating the sigmoid function is as follows:

[0157]

[0158] The pooling layer uses a local averaging method for dimensionality reduction. The calculation formula is as follows:

[0159]

[0160] In the formula, represents a local output term after the pooling process, which is obtained by averaging the local small matrix of size n*n in the previous layer.

[0161] Fully connected layer: Maps the features learned by the network to the label space of the samples. The fully connected layer transforms the two-dimensional feature vector output by the convolution into a one-dimensional feature vector.

[0162] This invention constructs a CNN model structure based on the above steps to perform parallel processing of three-axis data. First, the raw data collected by the three-axis accelerometer and gyroscope is input through the input layer. Then, data processing is performed through convolutional layers: 128 convolutional filters are learned in parallel to create rich data feature representations, where each filter is 1×12 in size and the stride of the convolution is 1. The random inactivation rate is 0.15. The sigmoid function is applied to the generated 100 feature representations, and a 1×4 max-pooling layer is used for 4x dimensionality reduction. The feature vectors are then passed to fully connected layers for feature labeling and classification. Finally, the output layer outputs whether it is a zero-velocity moment. The CNN training process is as follows: Figure 3 As shown.

[0163] Step 2: Automatically estimate pedestrian stride length using a smartphone's low-cost inertial sensor. The model consists of a TCN network and a denoising autoencoder. Inertial sensor measurements are collected from the smartphone as training data, while pedestrian walking distances are collected from an inertial navigation system module mounted on the foot as training labels.

[0164] This invention employs a step size estimation method based on Autoencoder-TCN, combining the advantages of both Autoencoder and TCN. On one hand, an autoencoder is used to denoise the input data; on the other hand, the TCN network is used to mine information between different time steps in long sequences, obtaining the intrinsic relationship between features and step size.

[0165] (2.1) Noise-reducing automatic encoder

[0166] Due to the poor quality of sensors built into current smartphones, raw sensor readings from accelerometers and gyroscopes inevitably contain noise, necessitating a denoising autoencoder to reduce its impact. An autoencoder can be divided into two parts: an encoding network and a decoding network. When the number of neurons in the hidden layer is less than that in the input and output layers, the encoding network maps the input samples to a low-dimensional feature space, obtaining the feature representation of the data. The decoding network then remaps this feature representation back to the original feature space and minimizes the root mean square error of the input and output, achieving data reconstruction. The input is denoted as x = {x...} 1 ,x 2 ,x 3 ,···,x N}, where N represents the number of input samples, then the encoding process is as follows:

[0167] a n =f θ (x n ) = s(W (1) x n +b (1) )

[0168] θ={W (1) ,b (1)} represents the weights and biases of the encoding network, a n s represents the feature map obtained from the hidden layer, and s() represents the activation function. The result of the decoding process is the input x. n Reconstruction form as follows:

[0169]

[0170] θ′={W (2) ,b (2)} represents the weights and biases of the decoding network. The training process involves reconstructing the input data to reduce reconstruction error. The cost function for calculating the reconstruction error is shown below:

[0171]

[0172] When the reconstruction error is made very small through training, if we only look at the encoding network, the output of the hidden layer is a low-dimensional representation of the original features. This process is called denoising, which obtains features with lower dimensions without losing information and improves generalization ability.

[0173] (2.2) TCN Network

[0174] The TCN network layer uses causal convolution to restrict the sliding window to achieve strict temporal constraints, expands the receptive field of the dilated convolution structure to learn a wider range of information, and adds residual connections to prevent the network layer from being too high and causing gradient vanishing, thereby obtaining information features of the time series.

[0175] Causal convolution: Causal convolution gives TCN strong causal properties, where information at the current time step can only be predicted based on information from previous time steps, that is, it can only be predicted based on the current input x. t And the previous input x1,x2,...,x t-1 The calculation is performed, and the information cannot be used for prediction afterward. The calculation is performed from left to right to extract features.

[0176]

[0177] Dilated convolution: Since causal convolution's feature extraction is limited by the kernel size, adding dilated convolution can compensate for this limitation. Sampling and calculating the text sequence at regular intervals of a certain number of holes is equivalent to inserting d zeros (d being the number of holes) inside the kernel. The kernel size after adding dilated convolution is:

[0178] f k_d =(d-1)*(f k -1)+f k

[0179] Among them, f k This represents the kernel size of the current layer.

[0180] Residual Links: Adding dilated convolutions requires increasing the number of network layers and expanding the receptive field of each layer. However, when the number of network layers is too deep, the gradient vanishing phenomenon is likely to occur. Adding residual link structures can form short-circuit links. By summing the input x and F(x) after nonlinear transformation, the temporal convolutional network avoids the gradient vanishing phenomenon caused by the increase in the number of network layers.

[0181] R = x + F(x)

[0182] The dilated causal convolution module used in this invention normalizes the parameter hierarchy using Hi-norm() after each dilated causal convolution calculation (Conv()), then performs nonlinear calculations using the ReLU activation function, and sums the result with the input to achieve residual linking. The calculation process is as follows:

[0183] T i =Conv(W i *F j +b i )

[0184] {T0,T1,···,T n} = Hi-norm({T0,T1,···,T) n})

[0185] {T0,T1,···,Tn} = Relu({T0,T1,···,T n})

[0186] Among them, T i W is the state value obtained by convolution at time i; i F is the word matrix calculated by convolution at time i; j b is the convolution kernel of the j-th layer; i Let {T0,T1,...,T1} be the bias matrix; n} is the encoding of the sequence after one complete convolution calculation.

[0187] The TCN network layer expands the receptive field of convolution by stacking multiple dilated causal convolutional layers, thereby obtaining more complete time-series features and extracting deeper information features.

[0188] Regression of stride length estimation as follows Figure 3 As shown in the diagram. Once the DAE is established, the actual stride length is used to train a supervised regression layer, which learns the nonlinear relationship between the denoised feature vectors during each stride and the corresponding actual stride length. The global objective of the regression layer is to minimize the error loss function J(D,G) between the actual stride length and the estimated value.

[0189]

[0190] Where y i It corresponds to the actual stride length of the input D(·). is the step size estimate from the regression layer, and G(·) is the regression module in the final regression model. Minimization is achieved through RMS propagation, which is frequently used in the training of neural networks.

[0191] (2.3) Step size estimation

[0192] The algorithm described above outlines the complete process of the proposed stride length estimation method. The algorithm trains the network using a set of training samples with corresponding actual stride lengths as input. Accelerometer and gyroscope readings are divided into multiple parts according to stepping events. Fixed-length stride length data are fed into the DAE-TCN network, and the actual stride lengths are used to train the network's regression layers. Once training is complete, the DAE-TCN model will be used to predict the stride length S of a pedestrian. k .

[0193]

[0194]

[0195] The recursive formula for the PDR algorithm, where E and N are the coordinates of the east and north directions respectively, and S... k θ kt k-1 to t k Pedestrians at any time from position (E) k-1 N k-1 ) to location (E k N k The walking distance and azimuth angle.

[0196] The PDR algorithm combines the eastward and northward position information with the altitude H obtained from the mobile phone barometric altimeter to obtain the position information in the northeast-northeast coordinate system:

[0197]

[0198] Step 3: Using low-cost sensor information provided by smartphones and the prediction results of neural networks, the pedestrian navigation system is corrected for errors by extending Kalman filtering, and finally the pedestrian navigation information is output.

[0199] The position error of the INS mechanical orchestration algorithm increases quadratically over time, and navigation performance largely depends on the accuracy of the sensors. For smartphones with built-in inertial sensors, the estimated position using the mechanical method of inertial navigation systems will introduce errors exceeding 10 meters within seconds, far exceeding acceptable limits. However, pedestrian motion characteristics can be used to construct virtual observations, thereby slowing down the accumulation rate of position errors. An extended Kalman filter is used to fuse constraint information from pedestrian motion and inertial navigation.

[0200] The 15-dimensional error state variables of the system at time k constitute the error state variables δx of the Kalman filter. k As shown in the following formula:

[0201]

[0202] in, δv represents the three attitude errors: roll angle, pitch angle, and yaw angle. k =[δv kx δv ky δv kz ] T For the velocity errors in the east, north, and sky directions, δp k =[δp kx δp ky δp kz ] T For longitude, latitude, and altitude position errors, These are the components of the gyroscope error along the x, y, and z axes. The components of the accelerometer error along the x, y, and z axes;

[0203] The discrete linearization of the systematic error model can be expressed as follows:

[0204]

[0205] Where, δz k It is the measurement vector, H k It is a measurement matrix, w k It is process noise, v k It is measurement noise, Φ k,k-1 It is the state transition matrix.

[0206] State transition matrix Φ k,k-1 The expression is as follows:

[0207]

[0208] Where Δt is the sampling interval, It is a skew-symmetric cross-product operator matrix. It is a direction cosine matrix.

[0209] When the pedestrian's state is determined to be standing, the velocity can be reasonably considered to be zero. For zero velocity, the velocity observation equation is given by the following:

[0210]

[0211] in, For inertial navigation to calculate the velocity vector δz in the loaded system v It is the measurement speed error vector.

[0212] When a user walks forward regularly, the lateral and vertical velocities in the body coordinate system should be zero because pedestrians typically walk on a plane and rarely move laterally or backward. Walking speed can be estimated by stride length. Therefore, virtual speed measurement is represented as follows:

[0213]

[0214]

[0215] in, It is the virtual speed of the mobile phone under the system, and Δt is the time interval of each step.

[0216] Combining the position information output by the PDR and the attitude information obtained by the magnetometer, a complete measurement error equation is established:

[0217]

[0218]

[0219]

[0220] Where, δz pIt is the measurement position error vector. It is the measurement attitude error vector. It is the heading angle measured by the magnetometer. The attitude is calculated by the inertial navigation system. It is the position calculated by the inertial navigation system.

[0221] When the observations are valid, update the state variables and their corresponding covariances using the following method:

[0222]

[0223]

[0224]

[0225] In the prediction phase, the step points are identified through a neural network. Combined with the inertial navigation calculation, the step length and magnetic heading are estimated by the neural network model. Based on the error of the inertial navigation system, equations for attitude, velocity and position errors in the step interval are given. An extended Kalman filter equation is established to correct the error of the pedestrian navigation system. Finally, the pedestrian navigation information is output, which effectively suppresses the cumulative error of the inertial navigation system and achieves accurate positioning in indoor environments.

Claims

1. A method for indoor pedestrian positioning based on neural network assistance, characterized in that, The pedestrian holds a smartphone, which includes a main navigation module comprising a gyroscope, an accelerometer, a magnetometer, and an altimeter. A secondary navigation module, including an IMU unit, is installed at the pedestrian's feet. The positioning method includes the following steps: Step 1: Offline training for zero-speed detection: Using the accelerometer and gyroscope information of the main navigation module as input data, and whether the IMU unit of the auxiliary navigation module is a zero-speed point as label data, a CNN-based step point recognition model is established. The three-axis data in the auxiliary navigation module are processed in parallel to identify the zero-speed moment of walking, and the step point of the pedestrian holding the mobile phone is determined based on the zero-speed moment. Step 2: Offline training of the stride length estimation model: The stride length estimation model includes a TCN network and a noise-reducing autoencoder. Inertial sensor measurements are collected from a smartphone as training data, and pedestrian walking distances are collected from a navigation module installed on the foot as training labels. The stride length estimation model is trained and used to estimate the stride length of pedestrians. Step 3: Online prediction of navigation information: Using the inertial sensor information of the smartphone, combined with the trained step length estimation model to detect step points, and using extended Kalman filtering to correct the error of pedestrian navigation information, the final pedestrian navigation information is obtained. In step 1, the label data is obtained through the following process: The triaxial accelerometer of the IMU unit in the navigation module outputs acceleration signals that are vectorized on the three orthogonal axes of the vehicle. If the acceleration is at zero speed, it is the stage where the person and the ground should be in complete contact. The acceleration vector sum is the local gravitational acceleration. Based on the set threshold, it is determined whether the sum of the acceleration vectors is within the threshold. The mathematical model expression is: in, express The sum of acceleration vectors at time t, , , Measured for inertial elements respectively The components of the acceleration value at time t in the three orthogonal axes x, y, and z. and These are the minimum and maximum values ​​for setting the threshold, respectively. In the diagram, 1 represents the zero-speed range and 0 represents the state of motion. When in a zero-speed state, the soles of the feet are fully in contact with the ground, and the gyroscope outputs an angular velocity signal and a magnitude close to zero. If outside the zero-speed range, the angular velocity signal fluctuates. The zero-speed range is detected by identifying the walking state and determining the angular velocity threshold. The formula for the angular velocity magnitude is: in, The magnitude of angular velocity. They are respectively The components of the angular velocity at time t in the three orthogonal axes x, y, and z; The logical value for angular velocity threshold detection. This represents the threshold value of the angular velocity magnitude; since the angular velocity magnitude is zero in the zero velocity range, the threshold value of the angular velocity magnitude is set to 1. According to logical expressions , When two conditions are met simultaneously, it is considered a zero-velocity interval, expressed as: The zero-velocity interval detected by zero velocity will be used as the label data. In step 1, a CNN-based step point recognition model is established. The specific process is as follows: Based on the principle of zero-speed detection, relevant variables are found from the output of the IMU unit as the input feature vector: n-system Speed ​​of time It is the attitude transition matrix gyroscope measurement angular velocity value Incremental output of accelerometer Incremental output of gyroscope Functions of these variables, where , as follows: in, These are the measurements from the accelerometer. It is the angular velocity value measured by the gyroscope. Influencing factors include IMU output values ​​and attitude transfer matrix; attitude transfer matrix Depend on The pitch, roll, and yaw angles in the carrier coordinate system at any given moment represent the three-dimensional attitude vectors at that moment. Obtained through calculation; Assumption The representative functional relationship, the velocity expression calculated by the inertial navigation system, is as follows: in, These four variables were selected as the input feature vector; During the training process, the gait recognition model first arranges the process variable data into a matrix, which consists of the sampling time... The data consists of 12-dimensional process variables collected internally, with a sampling time of 1 / 125. ; Let the input set of the step point recognition model be... The input feature vector includes All data in the inner matrix; It serves as input to the step point recognition model to predict whether the current moment is a zero velocity point; The CNN-based step point recognition model in step 1 includes an input layer, convolutional layer, pooling layer, fully connected layer, and output layer. The calculation process is as follows: in, Indicates the first One convolutional layer, Indicates the first A value of a convolution output matrix; The numbers represent the corresponding output matrix numbers, from 0 to 10 from left to right. , The number of convolution output matrices. Indicates the sample in the th The first convolutional layer The numbered convolutional output matrices are... The memory information of the matrix, For bias, This indicates the weight of the input memory information from the previous moment. Indicates the current input value; To represent a nonlinear function, the sigmoid function is used, and its calculation formula is as follows: The pooling layer uses a local averaging method for dimensionality reduction, and the calculation formula is as follows: This represents a local output term after the pooling process, with a size of [value missing] from the previous layer. The mean of the local small matrices is obtained; Fully connected layer: Maps the features learned by the network to the label space of the samples. The fully connected layer transforms the two-dimensional feature vector output by the convolution into a one-dimensional feature vector. In step 1, the process of identifying the zero-speed moment includes using a CNN-based step point recognition model to process the three-axis data of the IMU unit in the navigation module in parallel, as follows: ① Input the raw data collected by the triaxial accelerometer and gyroscope through the input layer; ② Data processing through convolutional layers: 128 convolutional filters are used for parallel learning to create data feature representations, where each filter is 1×12 in size, the stride of the convolution is 1, and the random inactivation rate is 0.

15. ③ Apply the sigmoid function to the generated 100 feature representations and use a 1×4 max pooling layer for 4x dimensionality reduction; ④ The feature vectors are then passed to the fully connected layer for feature labeling and classification; ⑤ Output whether it is the zero speed moment through the output layer.

2. The pedestrian indoor positioning method based on neural network assistance according to claim 1, characterized in that, The noise reduction autoencoder in step 2 is as follows: A noise reduction autoencoder consists of two parts: an encoding network and a decoding network. When the number of neurons in the hidden layer is less than that in the input and output layers, the encoding network maps the input samples to a low-dimensional feature space to obtain the feature representation of the data. The decoding network then remaps this feature representation back to the original feature space and minimizes the root mean square error of the input and output to achieve data reconstruction. Input is denoted as , If the input sample size is denoted by , then the encoding process is as follows: in, Represents the weights and biases of the encoding network. This represents the feature mapping obtained from the hidden layer. This represents the activation function; the result of the decoding process is the input. Reconstruction form ,as follows: in, This represents the weights and biases of the decoding network; The training process involves reconstructing the input data to reduce reconstruction error and calculating the cost function of the reconstruction error. as follows: When the reconstruction error approaches 0 through training, i.e., less than 0, If we only look at the encoding network, the output obtained by the hidden layer is a low-dimensional representation of the original features, and this process is the noise reduction process.

3. The pedestrian indoor positioning method based on neural network assistance according to claim 2, characterized in that, In step 2, the TCN network is as follows: The TCN network includes an extended causal convolution module, which uses causal convolution to restrict the sliding window to achieve time constraint, expands the receptive field of the convolution to learn a wider range of information through dilated convolution, and adds residual connections to prevent the gradient from vanishing due to excessive network layers in order to obtain time series information features. Causal convolution: Causal convolution gives TCN causal properties, meaning that information at the current time step can only be predicted based on information from previous time steps, i.e., it can only be predicted using the input at the current time step. And previous input Calculations are performed, but the information cannot be used for prediction afterward. Features are extracted from left to right: in, Represents the convolution function. The total time of the time series. The current time series; Dilated convolution: Samples and calculates the text sequence at predetermined dilation intervals, essentially inserting dilation parameters inside the convolution kernel. One zero, The number of holes represents the kernel size after dilation. for: in, This represents the kernel size of the current layer. Residual Linking: Adding residual linking structures creates short-circuit links, which are achieved by modifying the input... and after nonlinear transformation Summation helps temporal convolutional networks avoid gradient vanishing caused by increasing the number of network layers. in, This is the output of the residual link; In each dilated causal convolution calculation Then the parameter hierarchy is normalized. Then adopt The activation function performs nonlinear calculations, and the result is summed with the input to achieve residual linking. The calculation process is as follows: in, yes The state value obtained by convolution at time step; for The matrix of words calculated by convolution at each time step; For the first Convolutional kernels of layers; This is the bias matrix; It is the encoding of the sequence after one complete convolution calculation; The TCN network expands the receptive field of convolution by stacking multiple dilated causal convolutional layers, thereby obtaining more complete time-series features and extracting deeper information features.

4. The pedestrian indoor positioning method based on neural network assistance according to claim 3, characterized in that, In step 2, the stride estimation model is trained and used to estimate the stride length of pedestrians, as follows: The actual step size is used to train the regression layer of the step size estimation model, which learns the nonlinear relationship between the denoised feature vectors and the corresponding actual step size during each step size period; The global objective of the regression layer is to minimize the error loss function between the actual step size and the estimated value. Minimize: in, It corresponds to the input The actual stride length It is a step size estimate from the regression layer. It is the predicted value in the final step size model; The error loss function is implemented through RMS propagation. minimize; Training complete, the stride estimation model is now used to predict pedestrian stride length: The above formula is the recursive formula for the PDR algorithm. and These are the coordinates for the east and north directions, respectively. , They are respectively arrive pedestrians from the upper front east and north positions Arrive at the location The walking distance and azimuth; This represents the current eastward position. This represents the current northward position. The initial eastward position, This is the initial northward position; The altitude is obtained by combining the eastward and northward position information from the PDR algorithm with the mobile phone's barometric altimeter. To obtain the position information in the northeast celestial coordinate system : 。 5. The pedestrian indoor positioning method based on neural network assistance according to claim 4, characterized in that, In step 3, the pedestrian navigation information is corrected for errors by extending the Kalman filter, as follows: The system in The 15-dimensional error state variables at time t constitute the error state variables of the Kalman filter. As shown in the following formula: in, These are the three attitude errors: roll angle, pitch angle, and yaw angle. For the speed errors in the east, north, and sky directions, For longitude, latitude, and altitude position errors, These are the components of the gyroscope error along the x, y, and z axes. The components of the accelerometer error along the x, y, and z axes; The discrete linearized representation of the system error model is as follows: in, Navigation information measurement error vector It is a measurement matrix. It's process noise. It is measurement noise. It is the state transition matrix; For state prediction, This refers to measurement and prediction errors; State transition matrix The expression is as follows: in, It is the sampling interval. It is a skew-symmetric cross-product operator matrix. It is the direction cosine matrix. yes The identity matrix; When a pedestrian's state is determined to be standing, their velocity is considered to be zero. The velocity observation equation for zero velocity is as follows: in, It is the measurement speed error vector; When a user walks forward regularly, the lateral and vertical velocities in the body coordinate system are zero. Walking speed is estimated by stride length, and the virtual speed measurement is represented as follows: in, It is the virtual speed of the phone under the system. The time interval for each step, Step size, For the velocity in the inertial navigation-based northeast-sky coordinate system, It is a rotation matrix; By combining the position information output by the PDR algorithm and the attitude information obtained by the magnetometer, a complete measurement error equation is established: in, It is the measurement position error vector. It is the measurement attitude error vector. It is the heading angle measured by the magnetometer. The attitude is calculated by the inertial navigation system. It is the position calculated by the inertial navigation system. It is the position calculated by PDR; When the observations are valid, update the state variables and their corresponding covariances using the following method: in, It is the undetermined correction coefficient matrix. It is an estimation error. It is the state prediction error. It is a measurement matrix. It is a state prediction value. It is the mean square error matrix of the state estimation. It is the mean square error matrix for one-step state prediction. It is the measurement noise covariance matrix.

Images