Smart phone PDR positioning method based on Bayesian optimization assisted CNN-LSTM

Abstract

The invention relates to the technical field of deep learning and pedestrian indoor positioning, in particular to a smart phone PDR positioning method based on Bayesian optimization assisted CNN-LSTM, and the method comprises the steps: S1, collecting sensor features and test data in a pedestrian movement process based on a built-in MEMS sensor of a smart phone; s2, performing smoothing processing on the test data through a moving average method and Kalman filtering; s3, based on the acquired sensor features, introducing a Bayesian optimization algorithm, obtaining an optimal hyper-parameter combination of the CNN-LSTM fusion model, and training the mobile phone posture recognition capability; and S4, based on the smoothed test data, identifying the posture of the mobile phone, setting a binarization threshold value, a sliding window size and a step length estimation coefficient, calculating gait, step length and course information of the pedestrian, and constructing a motion equation to solve coordinates of the pedestrian. According to the technical scheme, the adaptive parameters can be set according to the mobile phone posture, and the mobile phone posture adaptability and the pedestrian positioning precision are improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and indoor pedestrian positioning technology, specifically to a smartphone PDR positioning method based on Bayesian optimized assisted CNN-LSTM. Background Technology

[0002] With the iterative upgrades of systems and hardware, smartphones have become the core carrier for pedestrian positioning and navigation. In complex scenarios such as indoor environments where satellite signals are unavailable or where there are dense obstructions, indoor positioning technologies based on smartphones, such as Bluetooth and Wi-Fi, largely rely on signal fingerprint databases and specialized facilities, resulting in limitations such as high deployment costs and computational complexity. In contrast, Pedestrian Dead Reckoning (PDR) relies solely on the built-in MEMS sensors of smartphones to acquire pedestrian motion data, analyze gait periodic characteristics, calculate key information such as stride length and heading, and determine pedestrian coordinates. It requires no specialized facilities and has become the mainstream research direction for indoor pedestrian positioning.

[0003] However, the diversity of mobile phone postures during pedestrian movement leads to significant differences in sensor data. Existing gait detection and stride length estimation methods often rely on the periodic changes in sensor data and set fixed thresholds. Experiments conducted on flat-sided phones (in navigation mode) are generally insufficient to adapt to various phone postures. For mobile phone posture recognition, the performance of traditional methods depends on the time-domain, frequency-domain, and time-frequency-domain features of sensor data. Manually extracting features is costly in terms of manpower and resources, and it is difficult to select the optimal feature combination, resulting in poor posture recognition performance.

[0004] Therefore, developing a smartphone PDR positioning method that can set a pending threshold based on the phone's posture is of great significance for improving the phone's posture adaptability and pedestrian positioning accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a smartphone PDR localization method based on Bayesian optimization-assisted CNN-LSTM. By introducing a Bayesian optimization-assisted CNN-LSTM fusion model, the invention identifies the phone's posture, determines the binarization threshold, sliding window size, and step size estimation coefficient based on different phone postures, calculates pedestrian gait, step size, and heading information, and solves for pedestrian coordinates, effectively improving the phone's posture adaptability and pedestrian localization accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A smartphone PDR localization method based on Bayesian optimization-assisted CNN-LSTM includes the following steps:

[0008] Step S1: Based on the built-in MEMS sensor of the smartphone, collect sensor features and test data such as triaxial acceleration, angular velocity, and magnetic field strength during the pedestrian's movement.

[0009] Step S2: Smooth the test data using the moving average method and Kalman filtering;

[0010] Step S3: Based on the sensor features collected above, a Bayesian optimization algorithm is introduced to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model and train the mobile phone's pose recognition capability.

[0011] Step S4: Based on the smoothed test data, identify the phone's posture, set the binarization threshold, sliding window size and step length estimation coefficient, calculate the pedestrian's gait, step length and heading information, and construct motion equations to solve for the pedestrian's coordinates.

[0012] Preferably, the data acquisition process described in step S1 specifically includes:

[0013] Based on MEMS sensors built into smartphones such as accelerometers, gyroscopes, and magnetometers, sensor data features and test data are collected to train the phone's posture recognition capabilities and analyze positioning performance. Specifically, the three-axis acceleration, angular velocity, and magnetic field strength of pedestrians during their movement are collected.

[0014] Preferably, the data smoothing process described in step S2 specifically includes:

[0015] Step S21: Keeping the sensor characteristics unchanged, the test data is processed using a moving average method with a window size of 9, as shown in the following expression:

[0016]

[0017] In the formula, a i and A i Let i and represent the i-th resultant acceleration before and after the moving average, respectively;

[0018] Step S22: Based on the test data after moving average, perform optimal estimation of the system state. Let the system observation value be a, the system state variable be A, and the Kalman filter be established as follows:

[0019]

[0020] In the formula, K is the Kalman gain, P is the error covariance, and R and Q are the noise covariances.

[0021] Preferably, the mobile phone pose recognition capability training process described in step S3 specifically includes:

[0022] Step S31: Introduce the Bayesian optimization algorithm to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model and improve the efficiency of mobile phone pose recognition.

[0023] Determine the four-dimensional hyperparameter combination x = {x1, x2, x3, x4}, consisting of the number of convolutional kernels, the size of the convolutional kernels, the number of LSTM units, and the learning rate;

[0024] Suppose a one-dimensional function f(x) = {f(x1), f(x2), f(x3), f(x4)}, a mean function μ(x) = 0, and the covariance matrix is ​​calculated using the RBF kernel function:

[0025]

[0026]

[0027] In the formula, l = 1, k(x) i ,x j ) with |x i -x j |Increases while decreasing;

[0028] Construct a Gaussian process based on the observation points:

[0029]

[0030] In the formula, each observation point x i The corresponding value f(x) i All of them satisfy a Gaussian distribution f(x) i )~N(μ(x i ),k(x i ,x i ));

[0031] Using the conditional distribution property of Gaussian processes, f(x) * The posterior mean μ * and variance σ *2 The calculation is as follows:

[0032] μ * =k *T K -1 y

[0033] σ *2 =k(x * ,x * )-k *T K -1 k *

[0034] In the formula, k * =[k(x * ,x1),k(x *,x2),k(x * ,x3),k(x * [x4)] is the covariance vector between the new point and the observed point;

[0035] Select the next evaluation point based on the expected improvement function EI:

[0036]

[0037] In the formula, Φ(Z) is the cumulative distribution function of the standard normal distribution, and φ(Z) is the probability density function of the standard normal distribution;

[0038] If the number of iterations is reached or the function value changes very little, stop and return to the x corresponding to the best f(x) in the historical observation points; otherwise, update the model and use α. EI The x with the largest (x) value is taken as the next observation data x. n+1 Calculate the true function value f(x) n+1 Repeat the above steps to obtain the optimal combination of four-dimensional hyperparameters.

[0039] Step S32: Construct a CNN-LSTM fusion model consisting of a data input layer, convolutional layer, pooling layer, LSTM layer, fully connected layer and pose output layer, input the above optimal four-dimensional hyperparameter combination, and train the mobile phone pose recognition capability.

[0040] The data input layer stores triaxial acceleration [a] x a y a z ] T Angular velocity [ω x ω y ω z ] T and magnetic field strength [B x B y B z ] T The sensor feature X, which is composed of the following components, is expressed as:

[0041] X = [a x a y a z ω x ω y ω z B x B y B z ] T

[0042] The convolutional layer performs a convolution operation between the local region of the sensor feature and the convolutional kernel, and the sensor feature X is extracted as:

[0043] Y = σ(W × X + b)

[0044] In the formula, Y represents the sensor features extracted by the convolutional layer, W is the weight matrix, b is the bias vector, and σ is the sigmoid activation function, calculated as follows:

[0045]

[0046] Based on the sensor features output by the above convolutional layer, a nonlinear transformation is performed using the ReLU function;

[0047] The pooling layer resamples the sensor features based on the maximum value in a local region. The resampled sensor feature Y is as follows:

[0048]

[0049] An LSTM layer consists of cell states, hidden states, forget gates, input gates, and output gates;

[0050] Using the output h from the previous time step t-1 and the current input x t The forget gate outputs the information f that needs to be discarded. t The expression is:

[0051] f t =σ(W f [h t-1 ,x t ]+b f )

[0052] In the formula, W f Let b be the weight matrix. f It is the bias vector;

[0053] The input gate outputs information that needs to be updated, i. t The expression is:

[0054] i t =σ(W i [h t-1 ,x t ]+b i )

[0055] In the formula, W i Let b be the weight matrix. i It is the bias vector;

[0056] At this point, the element state is calculated as follows:

[0057]

[0058] In the formula, For dot product operation, c t-1 W represents the cell state at the previous time step. cLet b be the weight matrix. c Let be the bias vector, and tanh(·) be the tanh activation function, calculated as follows:

[0059]

[0060] The output gate outputs the current time information h. t The expression is:

[0061]

[0062] In the formula, W o Let b be the weight matrix. o It is the bias vector;

[0063] After several convolutional layers, pooling layers, and LSTM layers, the fully connected layer expands the sensor feature matrix row by row and connects them into a vector.

[0064] The attitude output layer classifies sensor features, identifies and outputs the phone's attitude.

[0065] Preferably, the pedestrian coordinate calculation process in step S4 specifically includes:

[0066] Step S41: Input the smoothed test data into the CNN-LSTM fusion model with the optimal hyperparameter combination, identify the current phone pose, and set the current binarization threshold A. f Sliding window size w, step size estimation coefficient K;

[0067] Step S42: Based on the above binarization threshold A f The resultant acceleration a k Perform a preliminary state assessment to obtain a 0 / 1 binary state array State. The assessment condition is as follows:

[0068]

[0069] Based on the aforementioned sliding window size w, a backward sliding window is established. The binarized state array State is then mean-sampled and rounded to obtain a new binarized state array State′, expressed as:

[0070] State′=round[(State i +State i-1 +State i-2 +…+State i-w+1 ) / w]

[0071] Set a temporary variable m to count the occurrences of the binary state "1" to eliminate spurious peaks. The expression is:

[0072]

[0073] When m > w, the current gait cycle is detected and the gait information at this moment is recorded, including the start / end time of the gait cycle and the resultant acceleration.

[0074] Step S43: Based on the above step size estimation coefficient K, and according to the nonlinear relationship between the step size and the resultant acceleration, the step size l of the k-th step is determined. k The calculation is as follows:

[0075]

[0076] In the formula, These are the maximum and minimum values ​​of the resultant acceleration of the pedestrian in the k-th step, respectively;

[0077] Step S44: The heading calculation does not distinguish between mobile phone attitudes. For all mobile phone attitudes, Mahony complementary filtering and quaternion method are used to calculate the pedestrian heading.

[0078] Using triaxial acceleration [a x a y a z ] T Establish Mahony complementary filter for triaxial angular velocity [ω] x ω y ω z ] T Apply constraints:

[0079]

[0080] In the formula, [·] j For relevant information at time j, [Ω x Ω y Ω z ] T The constrained triaxial angular velocity information is [q0q1 q2 q3]. T It is a quaternion, k f This is the proportionality coefficient;

[0081] The quaternion is updated to:

[0082]

[0083] In the formula, dt is the sampling period;

[0084] The expression for calculating a pedestrian's heading using quaternions is:

[0085]

[0086] Step S45: Based on the changes in heading, determine the pedestrian's walking status and make heading corrections according to the different situations;

[0087] By using three consecutive changes in heading, determine the pedestrian's straight-ahead status:

[0088]

[0089]

[0090] In the formula, For the current course, For the first two steps of heading, when Z1 = 0, the pedestrian walks in a straight line; when Z1 = 1, the pedestrian turns and walks in a different direction.

[0091] Map the average of the three consecutive heading steps to the interval [0, 45°], and calculate the angle between the average and the main heading. Determine the pedestrian's main heading walking status:

[0092]

[0093]

[0094] In the formula, when Z2 = 0, the pedestrian walks along the main direction; when Z2 = 1, the pedestrian walks along a non-main direction.

[0095] When Z1 = 0 or Z2 = 0, the pedestrian walks along a straight line or along the main heading. Set the heading correction amount i to correct the heading.

[0096] When Z1 = 1 or Z2 = 1, if a pedestrian turns or walks along a non-main course, the course correction amount i is reduced by half to correct the course.

[0097] When Z1 = 0 and Z1 = 1, the pedestrian changes from turning to walking in a straight line, and the heading correction is as follows:

[0098]

[0099] In the formula, Let p be the angle between the heading and the main heading, where p∈{0,1,2,…,end};

[0100] Step S46: Construct the equations of motion and solve for the pedestrian coordinates. The expression is:

[0101]

[0102] In the formula, (X k ,Y k ) represents the position at step k, L k , These represent the step size and heading at step k, respectively.

[0103] Therefore, the present invention employs the above-mentioned smartphone PDR localization method based on Bayesian optimized CNN-LSTM, which has the following technical effects:

[0104] (1) This invention constructs a CNN-LSTM fusion model, extracts the spatial local features of sensor features, and captures temporal correlation to accurately identify the mobile phone posture, thus solving the limitation of the traditional PDR method that is only applicable to flat-end mobile phones.

[0105] (2) This invention introduces the Bayesian optimization algorithm to obtain the best hyperparameter combination of the CNN-LSTM fusion model, which significantly improves the accuracy and efficiency of mobile phone posture recognition.

[0106] (3) Based on the mobile phone posture, the present invention determines the binarization threshold, sliding window size and step length estimation coefficient, calculates pedestrian gait, step length and heading information, and solves pedestrian coordinates, which significantly improves pedestrian positioning accuracy. Attached Figure Description

[0107] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0108] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0109] Figure 1 A flowchart illustrating a smartphone PDR localization method based on Bayesian optimization-assisted CNN-LSTM provided for an embodiment of the present invention;

[0110] Figure 2 This is a schematic diagram of the structure of the Bayesian optimization-assisted CNN-LSTM fusion model provided in an embodiment of the present invention;

[0111] Figure 3 This is a schematic diagram of the sensor data acquisition process provided in an embodiment of the present invention;

[0112] Figure 4 The minimum target value iteration curve provided in the embodiments of the present invention;

[0113] Figure 5 The data processing results provided in the embodiments of the present invention are as follows: (a) is the result of the moving average method; (b) is the result of the Kalman filter method.

[0114] Figure 6 The gait detection results provided in the embodiments of the present invention are as follows: (a) is the adaptive parameter detection result; (b) is the fixed parameter detection result.

[0115] Figure 7 The step size estimation results provided in the embodiments of the present invention;

[0116] Figure 8 The pedestrian positioning error provided in the embodiments of the present invention. Detailed Implementation

[0117] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0118] This invention provides a smartphone PDR localization method based on Bayesian optimized CNN-LSTM, such as... Figure 1 As shown, it includes the following steps:

[0119] Step S1: Based on the built-in MEMS sensors of smartphones such as accelerometers, gyroscopes, and magnetometers, collect sensor features and test data, which are used to train the phone's posture recognition capability and analyze positioning performance. Specifically, collect the three-axis acceleration, angular velocity and magnetic field strength during the pedestrian's movement.

[0120] Step S2: Smooth the test data using the moving average method and Kalman filtering. The specific process is as follows:

[0121] Step S21: Keeping the sensor characteristics unchanged, the test data is processed using a moving average method with a window size of 9, as shown in the following expression:

[0122]

[0123] In the formula, a i and A i Let i and represent the i-th resultant acceleration before and after the moving average, respectively;

[0124] Step S22: Based on the test data after moving average, perform optimal estimation of the system state. Let the system observation value be a, the system state variable be A, and the Kalman filter be established as follows:

[0125]

[0126] In the formula, K is the Kalman gain, P is the error covariance, and R and Q are the noise covariances.

[0127] Step S3: Based on the sensor features acquired in Step S1, a Bayesian optimization algorithm is introduced to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model, thereby training the phone's pose recognition capability; specifically, such as... Figure 2 As shown, the Bayesian optimization algorithm constructs a probabilistic model of the objective function to guide the next evaluation point, focusing computational resources across multiple iterations on high-potential parameter regions. It then uses the expectation improvement function to balance exploration and development capabilities, obtaining the optimal hyperparameter combination for the CNN-LSTM fusion model. Training set features are input to the CNN convolutional layer via the data input layer, where wide convolutional kernels adaptively extract data features. These features are then passed through a max-pooling layer to reduce the data dimensionality and retain key feature information. The dimensionality-reduced features are used as the feature input to the LSTM layer for training and automatic learning of the data features. The fully connected layer expands the sensor feature matrix row-wise and concatenates it into a vector. Finally, the pose output layer identifies and stores the phone's pose. The specific process is as follows:

[0128] Step S31: Introduce the Bayesian optimization algorithm to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model and improve the efficiency of mobile phone pose recognition.

[0129] Determine the four-dimensional hyperparameter combination x = {x1, x2, x3, x4}, consisting of the number of convolutional kernels, the size of the convolutional kernels, the number of LSTM units, and the learning rate;

[0130] Suppose a one-dimensional function f(x) = {f(x1), f(x2), f(x3), f(x4)}, a mean function μ(x) = 0, and the covariance matrix is ​​calculated using the RBF kernel function:

[0131]

[0132]

[0133] In the formula, l = 1, k(x) i ,x j ) with |x i -x j |Increases while decreasing;

[0134] Construct a Gaussian process based on the observation points:

[0135]

[0136] In the formula, each observation point x i The corresponding value f(x) i All of them satisfy a Gaussian distribution f(x) i )~N(μ(x i ),k(x i ,x i ));

[0137] Using the conditional distribution property of Gaussian processes, f(x) * The posterior mean μ * and variance σ *2 The calculation is as follows:

[0138] μ * =k *T K -1 y

[0139] σ *2 =k(x * ,x * )-k *T K -1 k *

[0140] In the formula, k * =[k(x * ,x1),k(x * ,x2),k(x * ,x3),k(x * [x4)] is the covariance vector between the new point and the observed point;

[0141] Select the next evaluation point based on the expected improvement function EI:

[0142]

[0143] In the formula, Φ(Z) is the cumulative distribution function of the standard normal distribution, and φ(Z) is the probability density function of the standard normal distribution;

[0144] If the number of iterations is reached or the function value changes very little, stop and return to the x corresponding to the best f(x) in the historical observation points; otherwise, update the model and use α. EI The x with the largest (x) value is taken as the next observation data x. n+1 Calculate the true function value f(x) n+1 Repeat the above steps to obtain the optimal combination of four-dimensional hyperparameters.

[0145] Step S32: Construct a CNN-LSTM fusion model consisting of a data input layer, convolutional layer, pooling layer, LSTM layer, fully connected layer and pose output layer, and input the optimal four-dimensional hyperparameter combination to train the mobile phone pose recognition capability.

[0146] The data input layer stores triaxial acceleration [a] x a y a z ] T Angular velocity [ω x ω y ω z ]T and magnetic field strength [B x B y B z ] T The sensor feature X, which is composed of the following components, is expressed as:

[0147] X = [a x a y a z ω x ω y ω z B x B y B z ] T

[0148] The convolutional layer performs a convolution operation between the local region of the sensor feature and the convolutional kernel, and the sensor feature X is extracted as:

[0149] Y = σ(W × X + b)

[0150] In the formula, Y represents the sensor features extracted by the convolutional layer, W is the weight matrix, b is the bias vector, and σ is the sigmoid activation function, calculated as follows:

[0151]

[0152] Based on the sensor features output by the above convolutional layer, a nonlinear transformation is performed using the ReLU function;

[0153] The pooling layer resamples the sensor features based on the maximum value in a local region. The resampled sensor feature Y is as follows:

[0154]

[0155] An LSTM layer consists of cell states, hidden states, forget gates, input gates, and output gates;

[0156] Using the output h from the previous time step t-1 and the current input x t The forget gate outputs the information f that needs to be discarded. t The expression is:

[0157] f t =σ(W f [h t-1 ,x t ]+b f )

[0158] In the formula, W f Let b be the weight matrix. f It is the bias vector;

[0159] The input gate outputs information that needs to be updated, i. t The expression is:

[0160] i t =σ(W i [h t-1 ,x t ]+b i )

[0161] In the formula, W i Let b be the weight matrix. i It is the bias vector;

[0162] At this point, the element state is calculated as follows:

[0163]

[0164] In the formula, For dot product operation, c t-1 W represents the cell state at the previous time step. c Let b be the weight matrix. c Let be the bias vector, and tanh(·) be the tanh activation function, calculated as follows:

[0165]

[0166] The output gate outputs the current time information h. t The expression is:

[0167]

[0168] In the formula, W o Let b be the weight matrix. o It is the bias vector;

[0169] After several convolutional layers, pooling layers, and LSTM layers, the fully connected layer expands the sensor feature matrix row by row and connects them into a vector.

[0170] The attitude output layer classifies sensor features, identifies and outputs the phone's attitude.

[0171] Step S4: Based on the smoothed test data, identify the phone's posture, set the binarization threshold, sliding window size, and step length estimation coefficient, calculate the pedestrian's gait, step length, and heading information, and construct motion equations to solve for the pedestrian's coordinates. The specific process is as follows;

[0172] Step S41: Input the smoothed test data into the CNN-LSTM fusion model with the optimal hyperparameter combination, identify the current phone pose, and set the current binarization threshold A. f Sliding window size w, step size estimation coefficient K;

[0173] Step S42: Based on the above binarization threshold A f The resultant acceleration a k Perform a preliminary state assessment to obtain a 0 / 1 binary state array State. The assessment condition is as follows:

[0174]

[0175] Based on the aforementioned sliding window size w, a backward sliding window is established. The binarized state array State is then mean-sampled and rounded to obtain a new binarized state array State′, expressed as:

[0176] State′=round[(State i +State i-1 +State i-2 +…+State i-w+1 ) / w]

[0177] Set a temporary variable m to count the occurrences of the binary state "1" to eliminate spurious peaks. The expression is:

[0178]

[0179] When m > w, the current gait cycle is detected and the gait information at this moment is recorded, including the start / end time of the gait cycle and the resultant acceleration.

[0180] Step S43: Based on the above step size estimation coefficient K, and according to the nonlinear relationship between the step size and the resultant acceleration, the step size l of the k-th step is determined. k The calculation is as follows:

[0181]

[0182] In the formula, These are the maximum and minimum values ​​of the resultant acceleration of the pedestrian in the k-th step, respectively;

[0183] Step S44: The heading calculation does not distinguish between mobile phone attitudes. For all mobile phone attitudes, Mahony complementary filtering and quaternion method are used to calculate the pedestrian heading.

[0184] Using triaxial acceleration [a x a y a z ] T Establish Mahony complementary filter for triaxial angular velocity [ω] x ω y ω z ] T Apply constraints:

[0185]

[0186] In the formula, [·] j For relevant information at time j, [Ω x Ω y Ω z ] T The constrained triaxial angular velocity information is [q0q1 q2 q3]. T It is a quaternion, k f This is the proportionality coefficient;

[0187] The quaternion is updated to:

[0188]

[0189] In the formula, dt is the sampling period;

[0190] The expression for calculating a pedestrian's heading using quaternions is:

[0191]

[0192] Step S45: Based on the changes in heading, determine the pedestrian's walking status and make heading corrections according to the different situations;

[0193] By using three consecutive changes in heading, determine the pedestrian's straight-ahead status:

[0194]

[0195]

[0196] In the formula, For the current course, For the first two steps of heading, when Z1 = 0, the pedestrian walks in a straight line; when Z1 = 1, the pedestrian turns and walks in a different direction.

[0197] Map the average of the three consecutive heading steps to the interval [0, 45°], and calculate the angle between the average and the main heading. Determine the pedestrian's main heading walking status:

[0198]

[0199]

[0200] In the formula, when Z2 = 0, the pedestrian walks along the main direction; when Z2 = 1, the pedestrian walks along a non-main direction.

[0201] When Z1 = 0 or Z2 = 0, the pedestrian walks along a straight line or along the main heading. Set the heading correction amount i to correct the heading.

[0202] When Z1 = 1 or Z2 = 1, if a pedestrian turns or walks along a non-main course, the course correction amount i is reduced by half to correct the course.

[0203] When Z1 = 0 and Z1 = 1, the pedestrian changes from turning to walking in a straight line, and the heading correction is as follows:

[0204]

[0205] In the formula, Let p be the angle between the heading and the main heading, where p∈{0,1,2,…,end};

[0206] Step S46: Construct the equations of motion and solve for the pedestrian coordinates. The expression is:

[0207]

[0208] In the formula, (X k ,Y k ) represents the position at step k, L k , These represent the step size and heading at step k, respectively.

[0209] Example

[0210] In this embodiment, the experimental sites were selected in the lobby of an innovation practice building and the corridor of a dormitory building. MATLAB Mobile software was used to collect triaxial acceleration, angular velocity, and magnetic field strength of pedestrians during their movement at a sampling rate of 100Hz. Specifically, as shown... Figure 3 As shown, to train the CNN-LSTM fusion model, one male and one female participant walked 60 steps with a Huawei Mate 60 Pro and a Xiaomi 10S, respectively, in four modes: navigation, arm swing, call, and pocket. The step length was random, and 16 sets of sensor features were collected. After data segmentation, 240 sets × 9 dimensions × 200 columns of data features for pose recognition were generated and randomly divided into training and validation sets in a 7:3 ratio. To analyze the performance of the PDR positioning method, one participant randomly used a phone pose, walking 60 steps with a Huawei Mate 60 Pro. Each step was approximately 0.6m long, and the actual trajectory length was 36m. Four sets of test data were collected. The test data were processed using moving average and Kalman filtering. Some hardware models of the two smartphones are shown in Table 1.

[0211] Table 1 Smartphone Hardware Models

[0212] hardware Huawei Mate 60 Pro Xiaomi 10S Processor Model Kirin 9000S - 12 cores - up to 2.62GHz Qualcomm Snapdragon 870 - 8 cores - up to 3.2GHz Accelerometer model rgm 3-axis Accelerometer lsm6dso Accelerometer Non-wakeup Gyroscope Model rgm 3-axis Gyroscope lsm6dso Gyroscope Non-wakeup Magnetometer Model rgm 3-axis magnetic field sensor Ak0991x Magnetometer Non-wakeup

[0213] This embodiment uses a training set to train the CNN-LSTM fusion model, such as Figure 4 As shown, the iterative curves for observing the minimum target value and estimating the minimum target value are presented.

[0214] observe Figure 4The optimization results show that Bayesian optimization is complete after 15 iterations, with a total time consumption of 490.30s and a total objective function computation time of 486.69s. At this point, the number of convolutional kernels is 12, the kernel size is 7, the number of LSTM units is 70, and the learning rate is 9.99e-3, indicating the optimal hyperparameter combination. When the validation set is input into the CNN-LSTM fusion model with the optimal hyperparameter combination, the validation set accuracy is 100%.

[0215] This embodiment uses test data to infer pedestrian gait, stride length, and heading information, and calculates pedestrian coordinates. To improve pedestrian positioning accuracy, this embodiment utilizes a moving average method with a window size of 9 and Kalman filtering to process the resultant acceleration in the test data. Figure 5 As shown in (a) and 5(b), the data processing results of the moving average method and Kalman filtering are presented respectively. The four sub-graphs, from top to bottom, are the results of navigation mode, swing arm mode, call mode and pocket mode. The following sub-graphs are arranged in the same way.

[0216] observe Figure 5 As shown in (a) and 5(b), the moving average method basically maintains the original data attributes of the resultant acceleration, but the smoothing effect is generally poor, and there are still many spurious peaks. Kalman filtering has a better smoothing effect, but it easily destroys data attributes, especially for the arm swing mode and pocket mode, where the two gait cycles of the left and right legs are only identified as one gait cycle, which obviously does not conform to the definition of gait cycle in this invention. Therefore, for the navigation mode and the call mode, this embodiment uses the moving average method and Kalman filtering to process the resultant acceleration; for the arm swing mode and the pocket mode, this embodiment uses the moving average method to process the resultant acceleration.

[0217] In this embodiment, the processed test data is fed into the CNN-LSTM fusion model with the optimal hyperparameter combination to identify the phone's pose and determine the binarization threshold, sliding window size, and step size estimation coefficients accordingly. The parameter settings are shown in Table 2.

[0218] Table 2 Parameter Settings

[0219] parameter Navigation mode Swing arm mode Call mode Pocket Mode Fixed threshold <![CDATA[Binarization threshold (m / s 2 )]]> 9.88 10.18 9.78 9.78 9.78 Sliding window size 40 50 50 40 40 Step size estimation coefficient 0.59 0.47 0.61 0.53 0.57

[0220] This embodiment analyzes the performance of gait detection, stride length estimation, and coordinate calculation in smartphone PDR positioning based on traditional fixed parameters and the adaptive parameters proposed in this invention.

[0221] To facilitate observation and analysis of gait detection results, this embodiment displays the first 2000 sampling points, such as... Figure 6 As shown in (a) and (b), the gait detection results under adaptive and fixed parameters are presented respectively. The actual number of steps in this embodiment is 60, and the deviation in the number of detected steps is shown in Table 3.

[0222] Table 3 Step Deviation (Unit: Steps)

[0223] parameter Navigation mode Swing arm mode Call mode Pocket Mode mean Adaptive parameters 0 0 0 2 0.5 Fixed parameters 1 29 0 6 9 mean 0.5 14.5 0 4 \

[0224] observe Figure 6 As shown in (a), 6(b) and Table 3, both navigation mode and call mode can accurately identify pedestrian gait, and the step points are stable without oscillation. The gait detection performance is better than that of arm swing mode and pocket mode. The average step deviation of adaptive parameters is 0.5 steps, and the gait detection performance is better than that of fixed parameters, with an overall improvement of 8.5 steps.

[0225] Figure 7 The step size estimation results under adaptive and fixed parameters are shown in Table 4. The step size deviation rate is calculated and compared with the actual step size of 0.6m.

[0226] Table 4 Step Size Deviation Rate (Unit: %)

[0227] parameter Navigation mode Swing arm mode Call mode Pocket Mode mean Adaptive parameters 2.6646 5.5451 2.182 6.2722 4.166 Fixed parameters 4.6235 17.167 7.3748 7.7111 9.2191 mean 3.6441 11.3561 4.7784 6.9944 \

[0228] observe Figure 7 As shown in Table 4, the step size estimation performance of navigation mode and call mode is generally better than that of swing arm mode and pocket mode. The average step size deviation rate of adaptive parameters is 4.166%, and the step size estimation performance is better than that of fixed parameters, with an overall reduction of 5.0531% in step size deviation rate.

[0229] This embodiment does not differentiate between phone attitudes when calculating headings. For all phone attitudes, it uses the quaternion method and Mahony complementary filtering to calculate the pedestrian heading, and corrects the heading based on the pedestrian's walking state to calculate the pedestrian coordinates. In this embodiment, the actual trajectory is a straight line 36m long. The positioning error for each step of the pedestrian is calculated, such as... Figure 8 As shown in Table 5, the positioning error curves under adaptive and fixed parameters are illustrated. The average positioning error is also shown in Table 5.

[0230] Table 5 Positioning Error (Unit: m)

[0231] parameter Navigation mode Swing arm mode Call mode Pocket Mode mean Adaptive parameters 0.1535 0.1828 0.1578 1.9396 0.6084 Fixed parameters 0.4651 12.0157 1.3276 2.6333 4.1104 mean 0.3093 6.0993 0.7427 2.2865 \

[0232] observe Figure 8 As shown in Table 5, the coordinate calculation performance of navigation mode and call mode is generally better than that of swing arm mode and pocket mode. The average pedestrian positioning error of adaptive parameters is 0.6084m, and the coordinate calculation performance is better than that of fixed parameters, with an overall reduction in positioning error of 3.502m.

[0233] Therefore, this invention adopts the above-mentioned smartphone PDR positioning method based on Bayesian optimization-assisted CNN-LSTM, introduces a Bayesian optimization-assisted CNN-LSTM fusion model to identify the phone's posture, and dynamically adjusts the binarization threshold of the combined acceleration, the size of the backward sliding window, and the step length estimation coefficient based on different phone postures to deduce pedestrian gait, step length, and heading information, and solve for pedestrian coordinates, effectively improving the phone posture adaptability and pedestrian positioning accuracy.

[0234] Finally, it should be noted that the above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A smartphone PDR localization method based on Bayesian optimized CNN-LSTM, characterized in that, Includes the following steps: Step S1: Based on the built-in MEMS sensor of the smartphone, collect sensor features and test data such as triaxial acceleration, angular velocity, and magnetic field strength during the pedestrian's movement. Step S2: Smooth the test data using the moving average method and Kalman filtering; Step S3: Based on the sensor features collected above, a Bayesian optimization algorithm is introduced to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model and train the mobile phone's pose recognition capability. Step S4: Based on the smoothed test data above, identify the phone's posture, set the binarization threshold, sliding window size and step length estimation coefficient, calculate the pedestrian's gait, step length and heading information, and construct motion equations to solve for the pedestrian's coordinates.

2. The smartphone PDR localization method based on Bayesian optimized CNN-LSTM according to claim 1, characterized in that, The data acquisition process described in step S1 specifically includes: Based on MEMS sensors built into smartphones such as accelerometers, gyroscopes, and magnetometers, sensor features and test data are collected to train the phone's posture recognition capabilities and analyze positioning performance. Specifically, the three-axis acceleration, angular velocity, and magnetic field strength are collected during the pedestrian's movement.

3. The smartphone PDR localization method based on Bayesian optimized CNN-LSTM according to claim 2, characterized in that, The data smoothing process described in step S2 specifically includes: Step S21: Keeping the sensor characteristics unchanged, the test data is processed using a moving average method with a window size of 9, as shown in the following expression: In the formula, a i and A i Let i and represent the i-th combined acceleration before and after the moving average, respectively; Step S22: Based on the test data after moving average, perform optimal estimation of the system state. Let the system observation value be a, the system state variable be A, and the Kalman filter be established as follows: In the formula, K is the Kalman gain, P is the error covariance, and R and Q are the noise covariances.

4. The smartphone PDR localization method based on Bayesian optimized CNN-LSTM according to claim 3, characterized in that, The mobile phone pose recognition capability training process described in step S3 specifically includes: Step S31: Introduce the Bayesian optimization algorithm to obtain the optimal hyperparameter combination of the CNN-LSTM fusion model and improve the efficiency of mobile phone pose recognition. Determine the four-dimensional hyperparameter combination x = {x1, x2, x3, x4}, consisting of the number of convolutional kernels, the size of the convolutional kernels, the number of LSTM units, and the learning rate; Suppose a one-dimensional function f(x) = {f(x1), f(x2), f(x3), f(x4)}, a mean function μ(x) = 0, and the covariance matrix is ​​calculated using the RBF kernel function: In the formula, l = 1, k(x) i ,x j ) with |x i -x j |Increases while decreasing; Construct a Gaussian process based on the observation points: In the formula, each observation point x i The corresponding value f(x) i All of them satisfy a Gaussian distribution f(x) i )~N(μ(x i ),k(x i ,x i )); Using the conditional distribution property of Gaussian processes, f(x) * The posterior mean μ * and variance σ *2 The calculation is as follows: μ * =k *T K -1 y σ *2 =k(x * ,x * )-k *T K -1 k * In the formula, k * =[k(x * ,x1),k(x * ,x2),k(x * ,x3),k(x * [x4)] is the covariance vector between the new point and the observed point; Select the next evaluation point based on the expected improvement function EI: In the formula, Φ(Z) is the cumulative distribution function of the standard normal distribution, and φ(Z) is the probability density function of the standard normal distribution; If the number of iterations is reached or the function value changes very little, stop and return to the x corresponding to the best f(x) in the historical observation points; otherwise, update the model and use α. EI The x with the largest (x) value is taken as the next observation data x. n+1 Calculate the true function value f(x) n+1 Repeat the above steps to obtain the optimal combination of four-dimensional hyperparameters. Step S32: Construct a CNN-LSTM fusion model consisting of a data input layer, convolutional layer, pooling layer, LSTM layer, fully connected layer and pose output layer, input the above optimal four-dimensional hyperparameter combination, and train the mobile phone pose recognition capability. The data input layer stores triaxial acceleration [a] x a y a z ] T Angular velocity [ω x ω y ω z ] T and magnetic field strength [B x B y B z ] T The sensor features X, which are composed of these features, are expressed as follows: X=[a x a y a z oh x oh y oh z B x B y B z ] T The convolutional layer performs a convolution operation between the local region of the sensor feature and the convolutional kernel, and the sensor feature X is extracted as: Y = σ(W × X + b) In the formula, Y represents the sensor features extracted by the convolutional layer, W is the weight matrix, b is the bias vector, and σ is the sigmoid activation function, calculated as follows: Based on the sensor features output by the above convolutional layer, a nonlinear transformation is performed using the ReLU function; The pooling layer resamples the sensor features based on the maximum value in a local region. The resampled sensor feature Y is as follows: An LSTM layer consists of cell states, hidden states, forget gates, input gates, and output gates; Using the output h from the previous time step t-1 and the current input x t The forget gate outputs the information f that needs to be discarded. t The expression is: f t =σ(W f [h t-1 ,x t ]+b f ) In the formula, W f Let b be the weight matrix. f It is the bias vector; The input gate outputs information that needs to be updated, i. t The expression is: i t =σ(W i [h t-1 ,x t ]+b i ) In the formula, W i Let b be the weight matrix. i It is the bias vector; At this point, the element state is calculated as follows: In the formula, For dot product operation, c t-1 W represents the cell state at the previous time step. c Let b be the weight matrix. c Let be the bias vector, and tanh(·) be the tanh activation function, calculated as follows: The output gate outputs the current time information h. t The expression is: In the formula, W o Let b be the weight matrix. o It is the bias vector; After several convolutional layers, pooling layers, and LSTM layers, the fully connected layer expands the sensor feature matrix row by row and connects them into a vector. The attitude output layer classifies sensor features, identifies and outputs the phone's attitude.

5. The smartphone PDR localization method based on Bayesian optimized CNN-LSTM according to claim 4, characterized in that, The pedestrian coordinate calculation process described in step S4 specifically includes: Step S41: Input the smoothed test data into the CNN-LSTM fusion model with the optimal hyperparameter combination, identify the current phone pose, and set the current binarization threshold A. f Sliding window size w, step size estimation coefficient K; Step S42: Based on the above binarization threshold A f The resultant acceleration a k Perform a preliminary state assessment to obtain a 0 / 1 binary state array State. The assessment condition is as follows: Based on the aforementioned sliding window size w, a backward sliding window is established. The binarized state array State is then mean-sampled and rounded to obtain a new binarized state array State′, expressed as: State′=round[(State i +State i-1 +State i-2 +…+State i-w+1 ) / w] Set a temporary variable m to count the occurrences of the binary state "1" to eliminate spurious peaks. The expression is: When m > w, the current gait cycle is detected and the gait information at this moment is recorded, including the start / end time of the gait cycle and the resultant acceleration. Step S43: Based on the above step size estimation coefficient K, and according to the nonlinear relationship between the step size and the resultant acceleration, the step size l of the k-th step is determined. k The calculation is as follows: In the formula, These are the maximum and minimum values ​​of the resultant acceleration of the pedestrian in the k-th step, respectively; Step S44: The heading calculation does not distinguish between mobile phone attitudes. For all mobile phone attitudes, Mahony complementary filtering and quaternion method are used to calculate the pedestrian heading. Using triaxial acceleration [a x a y a z ] T Establish Mahony complementary filter for triaxial angular velocity [ω] x ω y ω z ] T Apply constraints: In the formula, [·] j For relevant information at time j, [Ω x Ω y Ω z ] T The constrained triaxial angular velocity information is [q0q1q2q3]. T It is a quaternion, k f This is the proportionality coefficient; The quaternion is updated to: In the formula, dt is the sampling period; The expression for calculating a pedestrian's heading using quaternions is: Step S45: Based on the changes in heading, determine the pedestrian's walking status and make heading corrections according to the different situations; By using three consecutive changes in heading, determine the pedestrian's straight-ahead status: In the formula, For the current course, For the first two steps of heading, when Z1 = 0, the pedestrian walks in a straight line; when Z1 = 1, the pedestrian turns and walks in a different direction. Map the average of the three consecutive heading steps to the interval [0, 45°], and calculate the angle between the average and the main heading. Determine the pedestrian's main heading walking status: In the formula, when Z2 = 0, the pedestrian walks along the main direction; when Z2 = 1, the pedestrian walks along a non-main direction. When Z1 = 0 or Z2 = 0, the pedestrian walks along a straight line or along the main heading. Set the heading correction amount i to correct the heading. When Z1 = 1 or Z2 = 1, if a pedestrian turns or walks along a non-main course, the course correction amount i is reduced by half to correct the course. When Z1 = 0 and Z1 = 1, the pedestrian changes from turning to walking in a straight line, and the heading correction is as follows: In the formula, Let p be the angle between the heading and the main heading, where p∈{0,1,2,…,end}; Step S46: Construct the equations of motion and solve for the pedestrian coordinates. The expression is: In the formula, (X k ,Y k ) represents the position at step k, L k , These represent the step size and heading at the k-th step, respectively.

Images