Multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning

Abstract

A multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning, specifically a multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning suitable for indoor positioning, aims to solve the problem that traditional trajectory fusion methods result in low crowdsourcing trajectory fusion accuracy in real scenes, leading to low indoor positioning accuracy. It includes obtaining the PDR trajectory of the target to be measured indoors; defining indoor landmarks, which include WiFi fingerprints and geomagnetic fingerprints, determining the WiFi similarity between different PDR trajectories and the geomagnetic dynamic time-normalized distance of similar WiFi trajectories; establishing global correlation constraints on PDR trajectories for indoor landmarks, and modeling the process of obtaining the minimum global correlation constraints as a Markov decision process; using a deep reinforcement learning network to solve the Markov decision process, with the input of the deep reinforcement learning network being the PDR trajectory and the global correlation constraints, thereby controlling the rotation and translation of each PDR trajectory to achieve trajectory fusion. It belongs to the field of indoor positioning.

Description

Technical Field

[0001] The present invention relates to a crowdsourcing trajectory fusion method, and in particular to a multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning suitable for indoor positioning, belonging to the field of indoor positioning. Background Art

[0002] Spatial location information has become an essential component of the mobile internet and the Internet of Things (IoT). It is a core component in emerging fields such as smart cities, intelligent transportation, and spatial big data analysis, and is closely linked to all aspects of national economic and social development. Satellite positioning systems, such as GPS, provide global, all-weather outdoor positioning services. However, there is currently no unified, stable, and universal solution for indoor positioning. Traditional fingerprint positioning (Wi-Fi, geomagnetic methods, etc.), the most mature indoor positioning solution, requires dedicated on-site surveys, which consumes considerable time and effort, making it currently difficult to implement on a large scale.

[0003] Crowd-sourced positioning technology collects sensor information from ordinary users through crowdsourcing. Leveraging user mobility, it correlates fingerprint information at different spatial locations, thereby automatically constructing a signal fingerprint library without manual surveying. Crowd-sourced positioning has become a key technology for rapidly deploying ubiquitous indoor positioning across a wide range of applications. It will significantly advance the development of location-based services in emerging fields such as smart cities, intelligent transportation, and spatial big data analytics. However, due to the large number of trajectories collected by crowd-sourced positioning technology, their short length, large heading errors, and unknown absolute positions, coupled with the limitations of traditional indoor landmark sparsity, crowd-sourced positioning in real-world scenarios still faces the problem of low crowd-sourced trajectory fusion accuracy. This makes it impossible to effectively construct the fingerprint map required for indoor positioning, resulting in low indoor positioning accuracy. Summary of the Invention

[0004] In order to solve the problem that traditional trajectory fusion methods cannot effectively fuse crowdsourced trajectories that are large in number, short in length, have large heading errors, and whose absolute positions are unknown, resulting in low crowdsourced trajectory fusion accuracy in real scenes and low indoor positioning accuracy, a multi-information crowdsourced trajectory fusion method based on deep reinforcement learning is proposed in this paper.

[0005] It includes the following steps:

[0006] S1. Obtain the PDR trajectory of the target to be measured indoors;

[0007] S2. Define indoor landmarks, including WiFi fingerprints and geomagnetic fingerprints, and determine the WiFi similarity between different PDR trajectories and the geomagnetic dynamic time-normalized distance of similar WiFi trajectories;

[0008] S3, establish the global correlation constraint of indoor landmarks on PDR trajectory, and model the process of obtaining the minimum global correlation constraint as a Markov decision process;

[0009] Based on the PDR trajectory and Markov decision process, a quaternion is obtained, and the global association constraint of the current indoor landmark on the PDR trajectory is calculated. The quaternion and the global association constraint are used as the training set to train the deep learning network. The PDR trajectory and the global association constraint are input, and the fused PDR trajectory is output. The trained deep learning network is obtained until the global association constraint is minimized.

[0010] S4. After obtaining the PDR trajectory of the target to be measured in a certain indoor scene and the corresponding indoor landmark information, the global association constraint of the indoor landmark on the PDR trajectory is calculated, the PDR trajectory and the global association constraint are input into the deep learning network in S3, and the fused PDR trajectory is output.

[0011] Furthermore, the specific process of S1 is:

[0012] S11, defining a stable placement posture of the target mobile phone to be tested, obtaining the Euler angle of the target mobile phone to be tested, and using a random forest classifier to identify the stable placement posture of the target mobile phone to be tested based on the Euler angle, the light sensor and the distance sensor of the mobile phone;

[0013] S12. Estimate the heading of the target mobile phone according to the stable placement posture of the target mobile phone, and generate a PDR trajectory of the target indoors according to the heading, the step count and the step length of the target.

[0014] Furthermore, the specific process of S11 is as follows:

[0015] S111. Define four stable placement postures for a mobile phone: pants pocket placement, jacket pocket placement, backpack placement, and handheld placement.

[0016] S112. Define the data collection time for each time as 30s-60s, obtain the accelerometer data, gyroscope data, and magnetometer data of the target mobile phone to be tested within the single collection time, use Kalman filtering to fuse the accelerometer data, gyroscope data, and magnetometer data, and estimate the Euler angle of the mobile phone based on the fused accelerometer data, gyroscope data, and magnetometer data;

[0017] S113. Obtain the light sensor and distance sensor of the mobile phone within a single acquisition time, calculate the Euler angle, the feature vector corresponding to the light sensor and the distance sensor of the mobile phone, where the feature vector includes the maximum value, minimum value, median value, average value, and range, input the above feature vector into the random forest classifier, and output the stable placement posture of the target mobile phone to be tested.

[0018] Furthermore, the specific process of S112 is as follows:

[0019] ① Calculate the initial posture quaternion of the target mobile phone to be tested;

[0020] ② Initialize the quaternion covariance matrix, process noise covariance matrix, accelerometer observation noise covariance matrix and magnetometer observation noise covariance matrix;

[0021] ③Calculate the state transfer equation of quaternion;

[0022] ④ Calculate the measurement equations of the accelerometer and magnetometer respectively;

[0023] ⑤Calculate the Kalman gain of the current state;

[0024] ⑥Update the quaternion and the covariance matrix of the quaternion;

[0025] ⑦Convert the quaternion to Euler angle.

[0026] Furthermore, the specific process of S12 is as follows:

[0027] S121, obtaining the accelerometer data, gyroscope data, and magnetometer data of the target mobile phone under stable placement in S113, fusing the above data using a Kalman filter, obtaining a heading angle in the mobile phone sensor coordinate system based on the fused data, and determining the heading of the target according to the heading angle;

[0028] S122: Based on the characteristic that the accelerometer amplitude value of the target under test has periodic variation when walking, a peak detection algorithm is used to estimate the peak time of each cycle of the accelerometer amplitude value, and the peak time of each cycle is used as the time for completing the gait estimation of the target under test. The step count and cadence of the target under test during the said time are obtained, where the cadence is the inverse of the time difference between adjacent peak values.

[0029] S123, assuming that the cadence and stride length of the target to be measured have a linear relationship, then calculating the stride length based on the cadence;

[0030] S124: Calculate the displacement according to the step length and heading of the target to be measured to obtain the PDR trajectory of the target to be measured.

[0031] Furthermore, the specific process of S2 is:

[0032] The WiFi signal strength and geomagnetic data of the target mobile phone under test are collected in a stable position. The cosine distance between the WiFi signal strength vectors of different mobile phones, different times, and different trajectories is used as the WiFi fingerprint similarity. The similarity value is recorded when the sampling points of different trajectories have similar WiFi fingerprint landmarks.

[0033] For PDR trajectories whose WiFi fingerprint similarity exceeds a preset threshold, the modulus of the three-axis geomagnetic data of each PDR trajectory is taken to construct one-dimensional geomagnetic data. The dynamic time-normalized distance of the one-dimensional geomagnetic data between different PDR trajectories is calculated. The dynamic time-normalized distance is used to determine whether different PDR trajectories have similar geomagnetic fingerprints. If the dynamic time-normalized distance is less than the preset threshold, the trajectories are considered similar. If there are similar geomagnetic fingerprints, the dynamic time-normalized distance between the similar PDR trajectories is recorded; otherwise, no operation is performed.

[0034] Furthermore, the specific process of S3 is as follows:

[0035] S31. Global association constraints of indoor landmarks on PDR trajectories:

[0036] F sum =w W ·F W +w M F M

[0037] Among them, w W 、w M are the weights of WiFi fingerprint and geomagnetic fingerprint respectively, F W 、F M are the weighted physical distances of WiFi fingerprint and geomagnetic fingerprint respectively;

[0038] S32. The process of minimizing the global correlation constraint is modeled as a Markov decision process. The position and orientation of the PDR trajectory are used as the state of the Markov decision process, and the movement and rotation of the PDR trajectory are used as the action of the Markov decision process. The rewards are divided into two categories:

[0039] If the action makes F sum If it increases, the reward r is set to a negative value;

[0040] If the action makes F sum Decrease, then the reward r is set to F sum The reciprocal of

[0041] If the action makes F sum If it does not change, the return is 0;

[0042] S33. Obtain the PDR trajectories of several targets to be measured indoors. According to the PDR trajectories, obtain the quaternary (s, a, r, s′) based on the Markov decision process, where s′ is the next state after the state s performs action a, and r is the reward. Calculate the global association constraint of the current indoor landmark on the PDR trajectory, use the quaternary and the global association constraint as the training set to train the deep learning network, input the PDR trajectory and the global association constraint, and output the fused PDR trajectory until the global association constraint F sumWhen minimized, a trained deep learning network is obtained.

[0043] Furthermore, the deep reinforcement learning network in S33 is divided into a target network and an evaluation network in turn. The parameter structures of the target network and the evaluation network are the same, but the parameter values ​​are different, that is, the weights and biases of each layer are different.

[0044] Furthermore, when the deep reinforcement learning network in S33 is trained, the parameters of the target network are updated in each iteration. Fixed the parameters θ of the evaluation network i , and define the command after each C step

[0045] Furthermore, the specific process of S4 is as follows:

[0046] The PDR trajectory of the target to be measured indoors and the corresponding global correlation constraint are input into the trained deep learning network. For the known positioning area involved in the PDR trajectory, the center positions of all PDR trajectories are initialized at any point in the positioning area. A PDR trajectory is translated up or down or left or right by 1 meter, and the translated PDR trajectory cannot exceed the range of the positioning area. The PDR trajectory is then rotated 1 degree clockwise or counterclockwise, and the parameters of the global correlation constraint are updated until the global correlation constraint is minimized. This completes the trajectory fusion and obtains the fused PDR trajectory.

[0047] Beneficial effects:

[0048] The present invention defines four stable placement postures for the target mobile phone to be tested, sets commonly used mobile phone placement states, and gives a restricted range of mobile phone states. The heading of the target mobile phone to be tested is estimated according to the stable placement posture, and the PDR trajectory of the target to be tested indoors is generated according to the heading, the step count and the step length of the target to be tested; indoor landmarks are defined, and the indoor landmarks include WiFi fingerprints and geomagnetic fingerprints. The indoor landmarks corresponding to the PDR trajectory are obtained, and the WiFi similarity between different PDR trajectories and the geomagnetic dynamic time normalization distance of similar trajectories are judged for global correlation constraint calculation in trajectory fusion; the goal of trajectory fusion is defined as minimizing the global correlation constraint between similar or identical landmark sampling points on different PDR trajectories, so the global correlation constraint of indoor landmarks on PDR trajectories is established, and the process of obtaining the minimized global correlation constraint is modeled as a Markov decision process. The Markov decision process is solved using a deep reinforcement learning network. Deep reinforcement learning can efficiently fit the value function by constructing a deep neural network, thereby greatly improving the expression ability of the value function, and for a large amount of trajectory data, deep reinforcement learning has better effects. The deep reinforcement learning network takes as input the PDR trajectory and global association constraints, thereby controlling the rotation and translation of each PDR trajectory, achieving trajectory fusion, and outputting the fused PDR trajectory. This multi-information crowdsourcing trajectory fusion technology based on deep reinforcement learning uses crowdsourced data acquisition to collect sensor information from the target phone under test, obtaining more accurate crowdsourced trajectories. Based on this crowdsourced sensor data, the network implements the following functions: PDR trajectory generation in four stable postures, multi-class landmark recognition, and deep reinforcement learning-based trajectory fusion. This achieves more accurate trajectory fusion and improves indoor positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0050] Specific implementation method 1: Combination Figure 1 This embodiment describes a multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning, which includes the following steps:

[0051] S1. Obtain the PDR trajectory of the target to be measured indoors. The specific process is as follows:

[0052] A large number of existing smartphone sensors collect data from the phone's accelerometer, gyroscope, magnetometer, light sensor, distance sensor, and WiFi signal strength, and store and process the data on the phone's server. To address the uncertainty of the target phone's posture in crowdsourcing data collection scenarios, this paper divides indoor PDR trajectory generation into two parts: phone placement posture recognition and offline user global heading estimation.

[0053] S11. Define the stable placement posture of the target mobile phone to be tested, obtain the Euler angle of the target mobile phone to be tested, and use the random forest classifier to identify the stable placement posture of the target mobile phone to be tested based on the Euler angle, the light sensor and the distance sensor of the mobile phone. The specific process is as follows:

[0054] The present invention simplifies the generation of PDR trajectories under arbitrary placement postures of the target to be measured into generation of PDR trajectories under four stable placement postures, namely, the four most common daily placement postures of mobile phones, namely, trouser pockets, jacket pockets, backpacks, and handheld flat ends. If the placement position of the mobile phone does not change during the data acquisition process, it is called a stable posture. Since the Euler angles and light intensities of the mobile phone are different under different placement positions, the Euler angles and light sensors obtained by the mobile phone sensors are used to estimate the stable placement posture of the target mobile phone to be measured. According to factors such as energy consumption and stability of data collected by ordinary users under crowdsourcing conditions, the present invention sets the acquisition time for each time to 30s-60s. First, the accelerometer data, gyroscope data, and magnetometer data of the target mobile phone to be measured are obtained within a single acquisition time, and the accelerometer data, gyroscope data, and magnetometer data are fused using Kalman filtering. The Euler angles of the mobile phone are estimated based on the fused accelerometer data, gyroscope data, and magnetometer data. The specific process is as follows:

[0055] ①Calculate the initial posture quaternion of the target mobile phone to be tested

[0056] Initialize the quaternion using the accelerometer and magnetometer:

[0057] a=[a x ,a y ,a z ] T (1)

[0058] m=[m x ,m y ,m z ] T (2)

[0059]

[0060]

[0061] Among them, a x ,a y ,a z is the three-axis measurement value of the accelerometer, m x ,m y ,m z is the three-axis measurement value of the magnetometer, v x and v y They represent the direction vectors of the x-axis and y-axis of the world coordinate system in the mobile phone coordinate system respectively.

[0062] set up v=[1,0,0] T ,but q x =[u·w,u×w] T .

[0063] And because y=q x v y q′ x , let u = y, v = [0, 1, 0] T , we can obtain q by the same logic y :

[0064] q′ x =[-q x1 ,q x2 ,q x3 ,q x4 ] T (5)

[0065] q′ y =[-q y1 ,q y2 ,q y3 ,q y4 ] T (6)

[0066] q′=[q′ x ,q′ y ] (7)

[0067] q=[q′1,-q′2,-q′3,-q′4] T (8)

[0068] Among them, q x v x The rotation quaternion about the world coordinate system's x-axis, q y v y After q x The rotation quaternion of the vector to the world y-axis after rotation. The subscript 1 represents the real part of the quaternion, and 2, 3, and 4 represent the imaginary part. x and q′ y Respectively with q x and q y The rotation direction is opposite, and the sum of the rotation angles is equal to 360°. x and q′ y The obtained q is the initial attitude quaternion. If q′1 is less than zero, a negative sign must be placed before q.

[0069] ② Initialize the covariance matrix P of the current system attitude quaternion k , process noise covariance matrix Q k , the accelerometer observation noise covariance matrix R k1and the magnetometer observation noise covariance matrix R k2 .

[0070] ③Calculate the state transfer equation of the quaternion:

[0071] ω=[ω x ,ω y ,ω z ] (9)

[0072] Among them, ω x ,ω y ,ω z It is the three-axis measurement data of the gyroscope.

[0073] The derivative formula for the current posture:

[0074]

[0075] Among them, q k is the current attitude quaternion, q′ k is the time derivative of the quaternion, is quaternion multiplication. Integral formula:

[0076]

[0077] Where Δt is the data sampling time interval.

[0078]

[0079] State transition equation:

[0080]

[0081] Quaternions Normalize and calculate the covariance matrix:

[0082]

[0083] ④Calculate the measurement equations of the accelerometer and magnetometer:

[0084] Quaternion rotation matrix:

[0085]

[0086] Acceleration measurement model:

[0087]

[0088] Among them, g is the acceleration due to gravity, which is the measurement unit of the accelerometer, and the modulus is 1. k1 is the Jacobian matrix of the accelerometer measurement equation, which is obtained by differentiating the measurement equation with respect to the quaternion. is a fixed form of the formula.

[0089] Earth's magnetic field measurement model:

[0090]

[0091] Among them, v m is the direction vector of the Earth's magnetic field.

[0092]

[0093] ⑤Calculate the Kalman gain of the current state of the system:

[0094]

[0095] ⑥Update the posterior state:

[0096]

[0097] ⑦Convert quaternion to Euler angle:

[0098] Define ψ, θ, φ as the rotation angles around the Z axis, Y axis, and X axis respectively, then

[0099]

[0100] Furthermore, the method calculates the characteristic vectors corresponding to the Euler angles, the phone's light sensor, and the distance sensor. These characteristic vectors include maximum, minimum, median, average, and range values. These characteristic vectors are then input into a random forest classifier for training, which outputs the stable placement posture of the phone. This method uses a machine learning algorithm to identify the placement and posture of the phone, effectively improving the accuracy of recognition of the placement and posture of the phone.

[0101] S12. Estimate the heading of the target mobile phone according to the stable placement posture of the target mobile phone, and generate a PDR trajectory of the target indoors according to the heading, the step count and the step length of the target.

[0102] First, the target phone's heading is estimated primarily based on a fusion of the accelerometer, gyroscope, and magnetometer. Based on the stable placement of the target phone obtained above, the accelerometer, gyroscope, and magnetometer data in this stable placement are fused via Kalman filtering to obtain the phone's heading angle θ in the sensor coordinate system. The heading angle is then used to determine the target's heading. Secondly, based on the periodic variation of the accelerometer amplitude values ​​while the user (target) is walking, a peak detection algorithm is used to estimate the peak time of each cycle of the accelerometer amplitude values. This peak time is used as the time to complete the user's (target) gait estimation. The target's step count and cadence during this time are then obtained. Cadence is the inverse of the time difference between adjacent peaks. Then, assuming a linear relationship between the user's (target's) cadence and stride length, the stride length is further estimated using the cadence obtained from the step count: L = a·f+b, where L is the stride length, a and b are preset parameters, and f is the cadence. Finally, based on the step counting results, the user (target to be measured) calculates the displacement through the step length and heading every time he takes a step, thereby obtaining the PDR trajectory of the target to be measured.

[0103] S2. Define indoor landmarks. Indoor landmarks include WiFi fingerprints and geomagnetic fingerprints. Determine the WiFi similarity between different PDR trajectories and the geomagnetic dynamic time-normalized distance between similar WiFi trajectories.

[0104] The WiFi signal strength value (vector) and geomagnetic data of the target mobile phone to be tested under a stable placement posture are collected respectively. Due to the different mobile phone models of different users, the present invention uses the cosine distance between the WiFi signal strength vectors under different mobile phones, different times, and different user tracks as the WiFi fingerprint similarity. When different tracks are judged to be sampling points with similar WiFi fingerprint landmarks, the similarity values ​​(cosine distances) of the different tracks are recorded for global association constraint calculation in track fusion. In the PDR track where the WiFi fingerprint similarity exceeds the preset threshold value, its similarity is further judged by the geomagnetic fingerprint. In order to reduce the impact of the stable placement posture of the mobile phone on the geomagnetic fingerprint, the modulus value of the three-axis geomagnetic data is constructed into one-dimensional geomagnetic data, and the dynamic time normalization (DTW) distance of the one-dimensional geomagnetic data between different PDR tracks is calculated. According to the dynamic time normalization distance, it is judged whether there is a similar geomagnetic fingerprint between different PDR tracks. If the dynamic time normalization distance is less than the preset threshold, it is judged to be similar. If there is a similar geomagnetic fingerprint, the dynamic time normalization distance (similarity) between the similar PDR tracks is recorded for global association constraint calculation in track fusion. Otherwise, no operation is performed.

[0105] S3. Establish global association constraints on PDR trajectories of indoor landmarks, model the process of obtaining the minimum global association constraints as a Markov decision process in a high-dimensional state, and use a deep reinforcement learning network to solve the Markov decision process in a high-dimensional state. The input of the deep reinforcement learning network is the PDR trajectory and the global association constraints, so as to control the rotation and translation of each PDR trajectory to achieve trajectory fusion and output the fused PDR trajectory.

[0106] The goal of trajectory fusion in this paper is defined as minimizing the global correlation constraint between similar or identical landmark sampling points on different PDR trajectories. The global correlation constraint is defined as the sum of weighted physical distances:

[0107] F=∑ i,j d i,j / S i,j (25)

[0108] Among them, d i,j is the physical distance between any two trajectories with similar landmark sampling points, S i,j is the similarity between two landmark sampling points in two trajectories. Since two indoor landmarks are defined above, two weighted physical distances are defined for the two indoor landmarks (WiFi fingerprint and geomagnetic fingerprint): F W and F M Since the similarity definitions of the two weighted physical distances are different, the two similarities are normalized:

[0109] S′=(SE(S)) / σ(S) (26)

[0110] Where E(S) is the mean similarity and σ(S) is the standard deviation of similarity. Using the normalized similarity S′, two weighted physical distances F are calculated according to formula (1): W and F M , F W and F M Perform weighted summation to obtain the global association constraint for the fusion of the two landmarks:

[0111] F sum =w W ·F W +w M F M (27)

[0112] Among them, w W 、w M are the weights of WiFi fingerprint and geomagnetic fingerprint respectively. As the discrimination of geomagnetic fingerprint and WiFi fingerprint as landmarks decreases in turn, w W 、w M The value will also decrease accordingly.

[0113] The process of obtaining the minimum global correlation constraint is:

[0114] ①Control each PDR trajectory to translate and rotate to minimize the global correlation constraint F sum Considering each PDR track as a rigid body, the state of each PDR track can be determined by its midpoint position (X, Y) and heading angle θ. The number of environmental states in the present invention increases exponentially with the number of PDR tracks.

[0115] ② First, since the PDR trajectory is based on the relative coordinates of the initial point obtained by the mobile phone's inertial sensor, the absolute position of the PDR trajectory in the geographic coordinate system is unknown. Therefore, for the known positioning area involved in the PDR trajectory, the center position of all PDR trajectories is initialized to any point within the positioning area. Secondly, each action is performed to move and rotate a PDR trajectory. An action consists of three decomposition actions:

[0116] 1. Select one of the PDR tracks.

[0117] 2. Shift the PDR trajectory by 1m upward, downward, left, or right, and the PDR trajectory after the shift cannot exceed the positioning area. This is arbitrary in the early stages of the algorithm, but with the training and iteration of the algorithm, the executed action converges to the optimal one.

[0118] 3. Rotate the PDR trajectory 1 degree clockwise or counterclockwise. This is arbitrary in the early stages of the algorithm, but as the algorithm is trained and iterated, the executed action converges to the optimal one.

[0119] Repeat the above steps iteratively, and finally the global association constraint F sum When it is minimized, that is, after the translation and rotation of the trajectory are completed, the PDR trajectory fusion is completed and the algorithm stops.

[0120] The process of minimizing the global correlation constraint is modeled as a Markov decision process, and deep reinforcement learning is used to solve this problem. Specifically, the optimal action is selected under different states, and each PDR is controlled to rotate and translate, thereby minimizing the optimization objective.

[0121] In reinforcement learning, the algorithmic agent needs to conduct a series of interactions with the environment to maximize the reward. Currently, the most effective algorithm in reinforcement learning is the Q-learning algorithm. The traditional Q-learning algorithm needs to implement the value function by constructing a state and action utility table. In this state and action utility table, each row is the state of the environment, each column is the action, and the value in the table is the action utility value. The Q-learning algorithm constructs a table through a series of exploration and utilization (Explore and Exploit), and obtains the action of the algorithmic agent in this state by querying the action corresponding to the maximum action utility value of each state in the table. Therefore, the present invention takes the above ① as state s and ② as action a. The algorithmic agent in the present invention takes a series of actions (selecting PDR trajectories, translating and rotating PDR trajectories) to change the environment. The environmental state includes the position and orientation of all PDR trajectories. Then the reward r can be divided into two categories: if the action makes F sum Increase, the reward r is set to a negative value (-1); if the action makes F sum Decrease, then the reward r is set to F sum The reciprocal of F sum If the reward is unchanged, the reward is 0. The algorithm is terminated after the global association constraint is minimized. However, since the Q learning algorithm solves sequential decision problems with a small number of environmental states, as described in ①, the number of environmental states in the present invention increases exponentially with the number of PDR tracks. For example, for 20 PDR tracks, the positioning target area is 60m*60m, and the PDR track heading statistical error is 30 degrees, the number of environmental states is (60·60·30). 20 , the number of actions is (20·4·2=160). That is, the number of environmental states in the present invention is too large to solve the Markov decision process by constructing a state and action utility table to achieve trajectory fusion. Therefore, the present invention uses a deep reinforcement learning algorithm to achieve trajectory fusion.

[0122] Deep reinforcement learning can efficiently fit the value function by constructing a deep neural network, thereby greatly improving the expressive power of the value function. Several PDR trajectories of the target to be measured are obtained indoors. Based on the PDR trajectories, a batch of quadruple pairs (s, a, r, s′) are obtained based on the Markov decision process, where s′ is the next state after state s performs action a. The global association constraints of the PDR trajectories are calculated. The quadruple pair and the global association constraints are used as training sets to train the deep learning network (DNN). The PDR trajectories and global association constraints are input and the fused PDR trajectories are output until the global association constraints F sum When the function is minimized, a trained deep learning network is obtained. Thus, the state and action utility tables in traditional Q learning are replaced by the deep learning network.

[0123] The present invention adopts a traditional deep reinforcement learning network structure DQN, which is divided into a target network and an evaluation network in sequence.

[0124] In DQN, the loss function is

[0125] L i (θ i )=E s,a,r,s′ ~D(r+γmax a′ Q(s′,a′;θ i - )-Q(s,a;θ i )) 2 (28)

[0126] Where γ is a preset weight value less than 1. Target network parameters and evaluate network parameters θ i The structure of is exactly the same, but the parameters are different, that is, the weights and biases of each layer in the neural network are different. In each iteration, update θ i ,fixed And it is stipulated that after each C step,

[0127] S4. After obtaining the PDR trajectory of the target to be measured in a certain indoor scene and the corresponding indoor landmark information, the global association constraints of the indoor landmarks are calculated, the PDR trajectory and the global association constraints are input into the deep learning network in S3, and the fused PDR trajectory is output, thereby achieving high-efficiency and high-precision trajectory fusion.

Claims

1. A multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning, characterized by: It includes the following steps: S1. Obtain the PDR trajectory of the target to be measured indoors; S2. Define indoor landmarks, including WiFi fingerprints and geomagnetic fingerprints, and determine the WiFi similarity between different PDR trajectories and the geomagnetic dynamic time-normalized distance of similar WiFi trajectories; S3, establish the global correlation constraint of indoor landmarks on PDR trajectory, and model the process of obtaining the minimum global correlation constraint as a Markov decision process; Based on the PDR trajectory and Markov decision process, a quaternion is obtained, and the global association constraint of the current indoor landmark on the PDR trajectory is calculated. The quaternion and the global association constraint are used as training sets to train the deep learning network. The PDR trajectory and the global association constraint are input, and the fused PDR trajectory is output. When the global association constraint is minimized, a trained deep learning network is obtained. The specific process is as follows: S31. The global association constraint is defined as the sum of weighted physical distances: F=∑ i,j d i,j / S i,j Among them, d i,j is the physical distance between any two trajectories with similar landmark sampling points, S i,j is the similarity between two landmark sampling points between two trajectories, and two weighted physical distances are defined for two indoor landmarks: F W and F M , the two indoor landmarks are WiFi fingerprint and geomagnetic fingerprint, and the two similarities are normalized: S′=(SE(S)) / σ(S) Among them, E(S) is the mean similarity, σ(S) is the standard deviation of similarity; Using normalized similarity S′, according to a=[a x ,a y ,a z ] T Calculate two weighted physical distances F W and F M , a x ,a y ,a z is the three-axis measurement value of the accelerometer, and F W and F M Perform weighted summation to obtain the global association constraint of the two landmark fusions, that is, the global association constraint of indoor landmarks on PDR trajectories: F sum =w W ·F W +w M F M Among them, w W 、w M are the weights of WiFi fingerprint and geomagnetic fingerprint respectively, F W 、F M are the weighted physical distances of WiFi fingerprint and geomagnetic fingerprint respectively; S32. The process of minimizing the global correlation constraint is modeled as a Markov decision process. The position and orientation of the PDR trajectory are used as the state of the Markov decision process, and the movement and rotation of the PDR trajectory are used as the action of the Markov decision process. The rewards are divided into two categories: If the action makes F sum If it increases, the reward r is set to a negative value; If the action makes F sum Decrease, then the reward r is set to F sum The reciprocal of If the action makes F sum If it does not change, the return is 0; S33. Obtain the PDR trajectories of several targets to be measured indoors. According to the PDR trajectories, obtain the quaternary (s, a, r, s′) based on the Markov decision process, where s′ is the next state after the state s performs action a, and r is the reward. Calculate the global association constraint of the current indoor landmark on the PDR trajectory, use the quaternary and the global association constraint as the training set to train the deep learning network, input the PDR trajectory and the global association constraint, and output the fused PDR trajectory until the global association constraint F sum When minimized, a trained deep learning network is obtained; S4. After obtaining the PDR trajectory of the target to be measured in a certain indoor scene and the corresponding indoor landmark information, the global association constraint of the indoor landmark on the PDR trajectory is calculated, the PDR trajectory and the global association constraint are input into the deep learning network in S3, and the fused PDR trajectory is output.

2. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 1, characterized in that: The specific process of S1 is as follows: S11, defining a stable placement posture of the target mobile phone to be tested, obtaining the Euler angle of the target mobile phone to be tested, and using a random forest classifier to identify the stable placement posture of the target mobile phone to be tested based on the Euler angle, the light sensor and the distance sensor of the mobile phone; S12. Estimate the heading of the target mobile phone according to the stable placement posture of the target mobile phone, and generate a PDR trajectory of the target indoors according to the heading, the step count and the step length of the target.

3. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 2, characterized in that: The specific process of S11 is as follows: S111. Define four stable placement postures for a mobile phone: pants pocket placement, jacket pocket placement, backpack placement, and handheld placement. S112. Define the data collection time for each time as 30s-60s, obtain the accelerometer data, gyroscope data, and magnetometer data of the target mobile phone to be tested within the single collection time, use Kalman filtering to fuse the accelerometer data, gyroscope data, and magnetometer data, and estimate the Euler angle of the mobile phone based on the fused accelerometer data, gyroscope data, and magnetometer data; S113. Obtain the light sensor and distance sensor of the mobile phone within a single acquisition time, calculate the Euler angle, the feature vector corresponding to the light sensor and the distance sensor of the mobile phone, where the feature vector includes the maximum value, minimum value, median value, average value, and range, input the above feature vector into the random forest classifier, and output the stable placement posture of the target mobile phone to be tested.

4. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 3, characterized in that: The specific process of S112 is as follows: ① Calculate the initial posture quaternion of the target mobile phone to be tested; ② Initialize the quaternion covariance matrix, process noise covariance matrix, accelerometer observation noise covariance matrix and magnetometer observation noise covariance matrix; ③Calculate the state transfer equation of quaternion; ④ Calculate the measurement equations of the accelerometer and magnetometer respectively; ⑤Calculate the Kalman gain of the current state; ⑥Update the quaternion and the covariance matrix of the quaternion; ⑦Convert the quaternion to Euler angle.

5. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 4, characterized in that: The specific process of S12 is as follows: S121, obtaining the accelerometer data, gyroscope data, and magnetometer data of the target mobile phone under stable placement in S113, fusing the above data using a Kalman filter, obtaining a heading angle in the mobile phone sensor coordinate system based on the fused data, and determining the heading of the target according to the heading angle; S122: Based on the characteristic that the accelerometer amplitude value of the target under test has periodic variation when walking, a peak detection algorithm is used to estimate the peak time of each cycle of the accelerometer amplitude value, and the peak time of each cycle is used as the time for completing the gait estimation of the target under test. The step count and cadence of the target under test during the said time are obtained, where the cadence is the inverse of the time difference between adjacent peak values. S123, assuming that the cadence and stride length of the target to be measured have a linear relationship, then calculating the stride length based on the cadence; S124: Calculate the displacement according to the step length and heading of the target to be measured to obtain the PDR trajectory of the target to be measured.

6. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 5, characterized in that: The specific process of S2 is: The WiFi signal strength and geomagnetic data of the target mobile phone under test are collected in a stable position. The cosine distance between the WiFi signal strength vectors of different mobile phones, different times, and different trajectories is used as the WiFi fingerprint similarity. The similarity value is recorded when the sampling points of different trajectories have similar WiFi fingerprint landmarks. For PDR tracks whose WiFi fingerprint similarity exceeds a preset threshold, the modulus of the three-axis geomagnetic data of each PDR track is taken to construct one-dimensional geomagnetic data. The dynamic time-normalized distance of the one-dimensional geomagnetic data between different PDR tracks is calculated. The dynamic time-normalized distance is used to determine whether different PDR tracks have similar geomagnetic fingerprints. If the dynamic time-normalized distance is less than the preset threshold, the tracks are considered similar. If there are similar geomagnetic fingerprints, the dynamic time-normalized distance between the similar PDR tracks is recorded. Otherwise, no action is taken.

7. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 6, characterized in that: The deep reinforcement learning network in S33 is divided into a target network and an evaluation network in turn. The parameter structures of the target network and the evaluation network are the same, but the parameter values ​​are different, that is, the weights and biases of each layer are different.

8. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 7, characterized in that: When training the deep reinforcement learning network in S33, the parameters of the target network are updated in each iteration. Fixed the parameters θ of the evaluation network i , and define the command after each C step 9. The multi-information crowdsourcing trajectory fusion method based on deep reinforcement learning according to claim 8, characterized in that: The specific process of S4 is as follows: The PDR trajectory of the target to be measured indoors and the corresponding global correlation constraint are input into the trained deep learning network. For the known positioning area involved in the PDR trajectory, the center positions of all PDR trajectories are initialized at any point in the positioning area. A PDR trajectory is translated up or down or left or right by 1 meter, and the translated PDR trajectory cannot exceed the range of the positioning area. The PDR trajectory is then rotated 1 degree clockwise or counterclockwise, and the parameters of the global correlation constraint are updated until the global correlation constraint is minimized. This completes the trajectory fusion and obtains the fused PDR trajectory.

Images