Fusion positioning method based on reinforcement learning and particle filtering, medium and equipment

Abstract

The invention relates to the technical field of wireless positioning, in particular to a fusion positioning method based on reinforcement learning and particle filtering, and the method comprises the steps: S1, selecting two effective base stations as a first base station and a second base station; s2, initializing model parameters of UWB and pedestrian track plotting; s3, predicting to obtain a particle state at a t moment; s4, introducing a reinforcement learning algorithm to adaptively select a ranging standard deviation; s5, updating the particle weight wt at the moment t according to the ranging standard deviation to obtain a particle weight wt '; s6, determining a resampling threshold value, if the number of effective particles is smaller than the resampling threshold value, performing resampling to obtain a particle weight w ''t, and entering S7; and S7, outputting a positioning result at the moment t by adopting a particle filtering algorithm, enabling t to be equal to t + 1, screening two optimal base stations as a new base station I and a new base station II, and returning to the S2. The reinforcement learning algorithm is introduced, the ranging standard deviation and the two base stations are adaptively selected, and the precision and robustness of the fusion positioning system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wireless positioning technology, and in particular to a fusion positioning method based on reinforcement learning and particle filtering. Background Art

[0002] In indoor wireless positioning technology, UWB technology, with its high time resolution and large bandwidth, has shown great advantages over other positioning methods in terms of resistance to environmental noise signal interference, coverage and positioning accuracy. In recent years, UWB positioning solutions have been more widely used in large indoor scenes such as factories and mines, with obvious performance advantages.

[0003] UWB technology can improve positioning accuracy to a certain extent by relying on its own hardware characteristics, but in actual applications, it will be affected by multipath effects, non-line-of-sight errors, etc., resulting in increased positioning errors and reduced positioning stability. In order to improve the stability and accuracy of UWB positioning systems, existing technologies have proposed a variety of optimization schemes. First, the geometric configuration of base station layout is adjusted, and a sufficient number of base station nodes are added, so that the UWB positioning system can autonomously select the three base station nodes with the smallest ranging error to complete three-sided positioning, which greatly increases the cost of base station node deployment; second, on the basis of three-sided positioning, at least three UWB base station nodes are deployed, and the relevant algorithms for identifying and reducing non-line-of-sight measurement errors are used to optimize the final positioning results. When obstacles completely block the UWB transmission signal and the number of valid base stations received by the system is less than 3, three-sided positioning cannot be performed, and the UWB positioning system will fail. Therefore, pure UWB positioning technology is difficult to meet the actual indoor positioning requirements for low cost and high performance. Third, in order to further improve the applicability of the UWB positioning system to positioning scenarios, reduce the cost of densely deploying UWB base stations, and enhance the stability of the positioning system, a UWB fusion positioning method assisted by pedestrian dead reckoning (PDR) is proposed. It can perform fusion positioning through the particle filter algorithm with only the ranging information of ≤3 base stations. On the one hand, this method utilizes the high-precision ranging of UWB to correct the cumulative error of PDR in real time. On the other hand, it uses the passive positioning of pedestrian dead reckoning to solve the continuous positioning of the UWB positioning system under the condition that the number of necessary base stations is less than that of the necessary number. The positioning accuracy and adaptability to the environment of the fusion of the two are higher than those of a single positioning method, which effectively reduces the base station deployment cost of the UWB positioning system and expands the applicability of the UWB positioning system.

[0004] When using the ranging information of PDR and two UWB base stations to perform particle filtering fusion positioning, the ranging standard deviation between the UWB tag and the effective base station is a key parameter of the particle filtering algorithm. The traditional particle filter uses a prior experiment to calibrate this ranging standard deviation to a fixed value. However, in actual positioning, the ranging accuracy between the UWB tag and the UWB base station is affected by many factors, including but not limited to multipath effects, signal attenuation, and hardware performance fluctuations. The error in fixed ranging cannot adapt to changes in complex environments, which directly affects the selection of effective particles in the particle filtering algorithm, thereby affecting the positioning accuracy and reducing the robustness of the fusion positioning system. In addition, the fusion accuracy of the ranging information of any two base stations and PDR is different. How to adaptively select the base station data with the lowest ranging error can significantly improve the performance of the fusion positioning system. Therefore, how to achieve a highly scalable and stable fusion positioning method in complex environments such as electromagnetic interference and multipath effects is an urgent problem to be overcome. Summary of the invention

[0005] The purpose of the present invention is to provide a fusion positioning method that achieves high scalability and high stability in complex environments such as electromagnetic interference and multipath effects. The specific technical solution is as follows:

[0006] The present invention provides a fusion positioning method based on reinforcement learning and particle filtering, comprising the following steps:

[0007] S1: Arrange UWB base stations in a certain scene, integrate UWB modules and inertial sensor modules as UWB tags and set them on mobile objects; given the initial position of the UWB tags, select any two valid UWB base stations that communicate with the UWB tags as UWB base station one and UWB base station two;

[0008] S2: Initialize the particle filter fusion positioning model parameters of UWB and pedestrian dead reckoning PDR, where: the particle filter fusion positioning model parameters include particle initialization, particle state at time t, and ranging vectors between the UWB tag and two UWB base stations; the initial position of the UWB tag corresponds to the particle filter fusion positioning model parameters when the iteration number t is 0;

[0009] S3: Input the particle state at time t-1 into the particle filter system state prediction model to predict the particle state at time t;

[0010] S4: Based on the particle state at time t, the reinforcement learning algorithm is introduced to train the particle filter system state prediction model to obtain a post-training filter prediction model that can adaptively select the UWB ranging standard deviation;

[0011] S5: Adaptively select the UWB ranging standard deviation at time t based on the trained filter prediction model to update the particle weight w at time t t, get the updated particle weight w t ';

[0012] S6: Determine the resampling threshold of the particle filter. If the number of valid particles N eff Less than the resampling threshold N th , then the particles are resampled according to the updated particle weight distribution to obtain the resampled particle weight w t ", enter S7; otherwise, let w" t =w' t Direct access to S7;

[0013] S7: According to the particle weight w t " and the particle state at time t in S2 uses the particle filter algorithm to output the positioning result of the UWB tag at time t;

[0014] Let t=t+1, and select the best two UWB base station combinations as new UWB base station one and UWB base station two based on reinforcement learning and the principle of minimum effective particle number, and return to S2.

[0015] Optionally, in S2, initializing the particle filter fusion positioning model parameters based on the initial position of the UWB tag specifically includes:

[0016] S2.1, generating a uniformly distributed particle set around the initial position of the UWB tag and constructing a particle filter fusion positioning model;

[0017] S2.2, initializing the particle filter fusion positioning model parameters of pedestrian dead reckoning PDR;

[0018] In S3, the specific expression of the particle filter system state prediction model is:

[0019]

[0020] in: is the particle state at time t; H is the state transfer matrix of the particle filter system state prediction model; j is the particle number; is the particle state at time t-1; B is the input matrix of the particle filter system state prediction model.

[0021] Optionally, the S4 includes:

[0022] S4.1. Construct a reward function for reinforcement learning based on the total number of valid particles in the particle set. The specific expression is as follows:

[0023]

[0024] Where: R[s q ][a j] is the reward function; N is the total number of particles in the particle set; s q is the particle state, s q ={s1,s2,...s m}, the value of m is determined by the number of positioning times of model training; a p is the standard action set in the reinforcement learning algorithm, a p ={a1,a2,...a n}, the value of n depends on the tolerance setting in the collection; is the particle weight of the jth particle at time t;

[0025] S4.2. Use the reward function to calculate the reward of the action in each state, and take the action with the highest reward as the UWB ranging standard deviation at the selected time t.

[0026] Optionally, a standard action set a p The specific determination process is: determine the upper and lower limits of the ranging error between any two UWB base stations and UWB tags in a dynamic environment; select the appropriate tolerance according to the upper and lower limits and set it as an arithmetic sequence set, and use this arithmetic sequence set as the standard action set a in reinforcement learning p .

[0027] Optionally, the S5 includes:

[0028] S5.1. Based on the particle state at time t predicted in S2, calculate the distance vector O corresponding to the particle state at time t t The probability of

[0029] S5.2. The distance vector O corresponding to the particle state at time t t The probability of the jth particle at time t is calculated based on the probability of the jth particle at time t and the adaptively selected UWB ranging standard deviation.

[0030] S5.3. According to the ranging probability obtained in S5.2 and the particle weight at time t-1, the updated particle weight at time t is obtained.

[0031] Optionally, in S5.1, the distance vector O corresponding to the particle state at time t is t Probability The specific calculation formula is as follows:

[0032]

[0033] In S5.2, the distance measurement probability corresponding to the jth particle at time t is The specific calculation formula is as follows:

[0034]

[0035] The specific calculation formula of S5.3 is as follows:

[0036]

[0037] Where: i is the UWB base station number, i = 1, 2; is the distance value between the UWB tag Tag and the i-th UWB base station in the t-th step, σ r is the adaptively selected UWB ranging standard deviation; is the X coordinate of the i-th UWB base station, is the Y coordinate of the i-th UWB base station; t is the X coordinate of the UWB tag at time t, y t is the Y coordinate of the UWB tag at time t.

[0038] Optionally, in S7, the specific calculation formula of the positioning result at time t is as follows:

[0039]

[0040] in: is the output particle state at time t and the positioning result of the UWB tag at time t;

[0041] In S8, the optimal two UWB base station combinations are selected based on reinforcement learning as UWB base station one and UWB base station two, specifically:

[0042] According to the ranging value between the UWB tag and each UWB base station, the reward function corresponding to each UWB base station is calculated respectively, and the reward functions corresponding to each UWB base station are sorted, and the two UWB base stations corresponding to the two smallest reward functions are selected as the new base station one and base station two.

[0043] Optionally, in S6, determining a resampling threshold of a particle filter specifically includes:

[0044] Set the resampling threshold set, specifically:

[0045] N th ={0.2N,0.3N,0.4N,0.5N,0.6N,0.7N,0.8N,0.9N,N};

[0046] According to the experimental data of particle filtering, the value in the resampling threshold set corresponding to the state with the smallest average positioning error is selected as the resampling threshold N th , and set as the hyperparameters of the reinforcement learning algorithm.

[0047] The present invention also provides a readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the above-mentioned fusion positioning method based on reinforcement learning and particle filtering is implemented.

[0048] The present invention also provides an electronic device, comprising: at least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, the fusion positioning method based on reinforcement learning and particle filtering as described above is implemented.

[0049] The present invention establishes the relationship between the UWB ranging error and the number of effective particles in particle filter fusion, and then establishes the relationship between the UWB ranging error and the positioning accuracy of the fusion system. By introducing a reinforcement learning algorithm, the UWB ranging standard deviation that can make the number of effective particles closest to the actual conditions is adaptively selected as the key parameter of particle filter fusion during fusion positioning, thereby improving the accuracy and robustness of the fusion positioning system.

[0050] Based on reinforcement learning, the optimal combination of two UWB base stations is selected to achieve adaptive selection of any two base station combinations, thereby improving the accuracy of the fusion positioning system.

[0051] By determining the resampling threshold of the particle filter to decide whether to resample, the update of the particle weight can be balanced under the two extreme conditions of particle degradation and uniform distribution, thereby improving the optimization capability and robustness of the fusion positioning system, reducing the positioning deviation caused by environmental changes, and lowering the requirements for the geometric layout of UWB base stations, further reducing costs and expanding the scope of application of the fusion positioning system.

[0052] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0054] Figure 1 It is a technical roadmap of the fusion positioning method based on reinforcement learning and particle filtering in an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of the positions of a UWB base station and a UWB tag in an embodiment of the present invention;

[0056] Figure 3This is a positioning accuracy comparison chart comparing the fusion positioning method based on reinforcement learning and particle filtering (RL-PF) in an embodiment of the present invention with the nonlinear least squares method (NonL-LS), the extended Kalman filter (EKF), pure UWB positioning, pure PDR positioning results and the real trajectory, i.e., the ground truth data (GD). DETAILED DESCRIPTION

[0057] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0058] In one embodiment, if Figure 1 and Figure 2 As shown in the figure, it is planned to deploy several UWB base stations in any indoor scene. In the experimental verification of the present invention, it is planned to deploy 4 UWB base stations in a 6×11 indoor space, numbered A, B, C, and D respectively; the obstacles in the indoor space are not drawn in the scene schematic diagram, which does not affect the implementation of the algorithm described in the present invention. It is planned to start from any initial position and walk along the direction shown by the dotted line in the figure to locate. The base station deployment can ensure that the tag receives the ranging information of no less than 2 UWB base stations at any position, determine the appropriate resampling threshold, introduce reinforcement learning training to adaptively select the ranging standard deviation, and update the particle weight by optimizing the number of effective particles to improve the accuracy and robustness of the positioning system.

[0059] A fusion positioning method based on reinforcement learning and particle filtering includes the following steps:

[0060] (I) Initialize the particle filter fusion positioning model parameters, such as X0, h t ,s t , wait;

[0061] S1: Arrange UWB base stations in a certain scene, integrate UWB modules and inertial sensor modules as UWB tags and set them on mobile objects; given the initial position of the UWB tags, select any two valid UWB base stations that communicate with the UWB tags as UWB base station one and UWB base station two;

[0062] S2: Initialize the particle filter fusion positioning model parameters of UWB and pedestrian dead reckoning PDR, where: the particle filter fusion positioning model parameters include particle initialization, particle state at time t, and ranging vectors between the UWB tag and two UWB base stations; the initial position of the UWB tag corresponds to the particle filter fusion positioning model parameters when the iteration number t is 0;

[0063] In S2, the particle filter fusion positioning model parameters are initialized based on the initial position of the UWB tag, including:

[0064] S2.1, generating a uniformly distributed particle set around the initial position of the UWB tag and constructing a particle filter fusion positioning model;

[0065] S2.2. Initialize the particle filter fusion positioning model parameters of pedestrian dead reckoning PDR, specifically:

[0066] 1) Particle initialization: Generate a uniformly distributed particle set around the initial position of the tag (the tag here refers to the integrated module of UWB and micro-inertial navigation, the micro-inertial navigation is responsible for PDR positioning, and the UWB is responsible for ranging). The initial particle set is j=1,...,N, N is the total number of particles, and the initial weight of the jth particle is

[0067] 2) The state of the particle in the initial state is expressed as: X0 = [x0, y0, 0, 0] T , (x0, y0) represents the coordinate position of the UWB tag Tag in the initial state.

[0068] 3) Assume that the system state of the particle at time t is expressed as: X t =[x t ,y t ,h t ,s t ] T ; where (x t ,y t ) is the coordinate of the UWB tag at time t, h t and t is the heading and step length estimated by the pedestrian dead reckoning PDR on the UWB tag at the tth time. The particle set P at the tth time t for: j=1,...,N, where N is the total number of particles. is the system state vector, is the weight of the j-th particle at the t-th time.

[0069] 4) The ranging vector between the UWB tag Tag and the two UWB base station Anchors in the initial state They represent the distance values ​​between the UWB tag Tag and the two base stations at time t respectively;

[0070] (ii) Substituting the input parameters into the particle filter prediction model for state prediction;

[0071] S3: Input the particle state at time t-1 into the particle filter system state prediction model to predict the particle state at time t;

[0072] In S3, the specific expression of the particle filter system state prediction model is:

[0073]

[0074] in: is the particle state at time t; H is the state transfer matrix of the particle filter system state prediction model; j is the particle number; is the particle state at time t-1, and the state at step t-1 is (x t-1 ,y t-1 ) is the coordinate of the UWB tag Tag at step t-1, h t-1 and t-1 They represent the heading and step length at step t-1 respectively; B is the input matrix of the particle filter system state prediction model;

[0075]

[0076] Where: Δh represents the zero-mean Gaussian noise in heading estimation, Δs represents the zero-mean Gaussian noise in step length estimation; are interfered by zero-mean Gaussian noise Δh and Δs respectively; σ h , σ s Respectively represent the heading h t and step length s t The standard deviation of .

[0077] (III) Introduce reinforcement learning algorithm for model training, such as adaptively selecting σ r , effective particles N eff ; Get the set resampling threshold N th , perform particle resampling of particle filtering;

[0078] S4: Based on the particle state at time t, the reinforcement learning algorithm is introduced to train the particle filter system state prediction model to obtain a post-training filter prediction model that can adaptively select the UWB ranging standard deviation;

[0079] The Q-leaning-based reinforcement learning algorithm does not rely on the environment to build the model. The algorithm aims to select the action with the highest reward in each state. We use Q-learning to obtain the best particle weight distribution in each iteration in order to obtain better positioning accuracy.

[0080] S4 includes:

[0081] ① First, construct the reward function of Q-learning, which is a key point of reinforcing learning. From the previous section, we can see that the performance of the fusion positioning system is affected by the particle weight distribution, and then by the standard deviation of the observation vector σ r, namely UWB ranging, an intuitive assumption is to use the distribution of particle weights to evaluate the effect σ r Therefore, the total number of effective particles N eff Set as a reward collection.

[0082] S4.1. Construct a reward function for reinforcement learning based on the total number of valid particles in the particle set. The specific expression is as follows:

[0083]

[0084] Where: R[s q ][a j ] is the reward function; N is the total number of particles in the particle set; s q is the particle state, s q ={s1,s2,...s m}, the value of m is determined by the number of positioning times of model training; a p is the standard action set in the reinforcement learning algorithm, a p ={a1,a2,...a n}, the value of n depends on the tolerance setting in the collection; is the particle weight of the jth particle at time t;

[0085] ② In the above ①, the reward function R[s q ][a j ], reinforcement learning aims to select the action with the highest reward in each state. The idea of ​​introducing reinforcement learning algorithm in this paper is to r Action Set (action-special component of reinforcement learning algorithm)a p Select the state s with the best particle weight q .

[0086] ③a p Determination: The fusion positioning of the distance measurement information of the dual UWB base stations and the tags is carried out through pedestrian dead reckoning. The upper and lower limits of the distance measurement error between any two UWB base stations and the tags are determined in a dynamic environment (including real environments with various interferences such as line-of-sight and non-line-of-sight). According to the upper and lower limits, the appropriate tolerance is set as an arithmetic sequence set, and this arithmetic sequence set is used as the standard action set a for model-free Q-learning in reinforcement learning p ={a1,a2,...a n}, where the value of n is determined by the tolerance setting in the set. According to the positioning accuracy requirements, the smaller the tolerance, the more accurate it is. The specific process of determining the standard action set a is: determine the upper and lower limits of the ranging error between any two UWB base stations and UWB tags in a dynamic environment; select the appropriate tolerance according to the upper and lower limits and set it as an arithmetic sequence set, and use this arithmetic sequence set as the standard action set a in reinforcement learning p .

[0087] ④ Status q Determination: In the particle filter fusion algorithm, the error in the distance measurement that is closest to the real state is selected, and the more accurate the positioning result is. The reinforcement learning training model is introduced to allow the positioning system to adaptively select the action with the highest reward in each state, so the state s q Set to the set of positioning results for each step in training. The larger the training sample, the more accurate the result.

[0088] S4.2. Use the reward function to calculate the reward of the action in each state, and take the action with the highest reward as the UWB ranging standard deviation at the selected time t.

[0089] S5: Adaptively select the UWB ranging standard deviation at time t based on the trained filter prediction model to update the particle weight w at time t t , get the updated particle weight w t ';

[0090] S5 includes:

[0091] S5.1. Based on the particle state at time t predicted in S2, calculate the distance vector O corresponding to the particle state at time t t The probability of

[0092] The distance vector O corresponding to the particle state at time t t Probability The specific calculation formula is as follows:

[0093]

[0094] S5.2. The distance vector O corresponding to the particle state at time t t The probability of the jth particle at time t is calculated based on the probability of the jth particle at time t and the adaptively selected UWB ranging standard deviation.

[0095] The distance measurement probability corresponding to the jth particle at time t The specific calculation formula is as follows:

[0096]

[0097] S5.3, according to the ranging probability obtained in S5.2 and the particle weight at time t-1, the updated particle weight at time t is obtained.

[0098] The specific calculation formula is as follows:

[0099]

[0100] Where: i is the UWB base station number, i = 1, 2; is the distance value between the UWB tag Tag and the i-th UWB base station in the t-th step, σ r is the adaptively selected UWB ranging standard deviation; is the X coordinate of the i-th UWB base station, is the Y coordinate of the i-th UWB base station; t is the X coordinate of the UWB tag at time t, y t is the Y coordinate of the UWB tag at time t.

[0101] It can be seen that the weight of the particle in step t is The influencing factors are: So accurate The update of particle weights is very important, which is the standard deviation σ of UWB ranging r The UWB ranging accuracy will be seriously affected by the NLOS condition. Due to the interference of the ranging environment, it is difficult to accurately determine σ r , in the existing particle filter, σ r Usually set to a fixed value, the impact on the environment cannot be adaptive. This algorithm uses reinforcement learning to adaptively select σ r ,improves the localization performance of traditional particle filters and makes them more robust to the environment.

[0102] (IV) Obtain updated particles and weights, and use particle filter algorithm to output positioning results; select the best 2 UWB base station combinations based on reinforcement learning to obtain the latest ranging vector; return to S1 for iterative loop positioning.

[0103] S6: Determine the resampling threshold of the particle filter. If the number of valid particles is less than the resampling threshold, resample the particles according to the updated particle weight distribution to obtain the resampled particle weight w" t , enter S7; otherwise, let w" t =w' t Direct access to S7;

[0104] From the above ①, we can see that the premise for selecting the optimal number of effective particles is N eff <N th , that is, choosing a suitable resampling threshold can improve the performance of the system. If the weight of any particle in step t is uniformly distributed, that is, Then the effective number of particles N is eff is the maximum value N (the total number of initialized particles). If only one particle has a total weight of 1 and the other particles are 0, then the effective number of particles N eff is the minimum value and the value is 1, so in order to choose a suitable threshold N th to control particle resampling.

[0105] Determining the resampling threshold of the particle filter specifically includes:

[0106] Set the resampling threshold set, specifically:

[0107] N th ={0.2N,0.3N,0.4N,0.5N,0.6N,0.7N,0.8N,0.9N,N};

[0108] According to the experimental data of particle filtering, the value in the resampling threshold set corresponding to the state with the smallest average positioning error is selected as the resampling threshold N th , and set as the hyperparameters of the reinforcement learning algorithm.

[0109] The specific calculation process of the average positioning error is as follows:

[0110] Before Q-learning, the Q table is arbitrarily initialized to store the rewards of actions in each state. The learning process is based on the Bellman equation:

[0111]

[0112] Where: Q is the cumulative reward function for estimating the quality of a state-action combination, Q(s,a) represents the prior cumulative reward for a in state s, represents the posterior reward, which is the immediate reward of R(s,a), Q(s′,a′) is the cumulative reward of the next state s′ by performing actions based on the current (s,a) combination, the variable max(Q(s′,a′)) represents the selected maximum achievable Q(s′,a′), α is the learning rate, which determines the value of the current information pair, generally set to 0.5, and γ is the discount factor that balances current and future rewards, which is set to 0.9 on the system;

[0113] In addition, in order to make the reinforcement learning training model converge to the minimum error, PE (Positioning error) is defined in the training model and represented by the positioning error set PE[k], where the value of k is related to N th The number of values ​​in the set is consistent. represents the estimated coordinate value of RL-PF (reinforcement learning + particle filter) positioning, (x tp ,y tp) are the exact coordinates of the test point.

[0114] The accuracy is evaluated by the maximum permissible error MPE (Maximum Permissible Error) and the standard deviation of the error STDE (Standard Deviation of the Error), as shown in the following table. During the positioning process, the positioning accuracy evaluation table 1 obtained by training under different UWB base station combinations and different resampling thresholds is as follows. It can be obtained that when N th When 0.7N is used, the positioning accuracy is the best.

[0115] Table 1 Evaluation table of positioning accuracy obtained by different resampling thresholds

[0116]

[0117] S7: According to the particle weight w t " and the particle state at time t in S2 uses the particle filter algorithm to output the positioning result of the UWB tag at time t;

[0118] Let t=t+1, and select the best two UWB base station combinations as new UWB base station one and UWB base station two based on reinforcement learning and the principle of minimum effective particle number, and return to S2.

[0119] The specific calculation formula of the positioning result at time t is as follows:

[0120]

[0121] in: is the output particle state at time t and the positioning result of the UWB tag at time t;

[0122] In S8, the optimal two UWB base station combinations are selected based on reinforcement learning as UWB base station one and UWB base station two, specifically:

[0123] According to the ranging value between the UWB tag and each UWB base station, the reward function corresponding to each UWB base station is calculated respectively, and the reward functions corresponding to each UWB base station are sorted, and the two UWB base stations corresponding to the two smallest reward functions are selected as the new base station one and base station two.

[0124] Calculate and output valid particles

[0125]

[0126] For effective particles Sort and select the two smallest Corresponding to the 2 UWB base stations.

[0127] N is selected based on reinforcement learning th The two UWB base stations (base stations C&D) with the smallest ranging error are adaptively selected to participate in the fusion positioning method based on reinforcement learning and particle filtering (Reinforcement Learning-ParticleFilter, RL-PF) and compared with the nonlinear least squares method (NonL-LS), extended Kalman filter (EKF), pure UWB positioning, pure PDR positioning, and true trajectory, i.e., ground truth data (Groundtruth Data: GD). The positioning accuracy is shown in the figure below. Figure 3 As shown, the particle filter adaptive multi-source fusion algorithm based on reinforcement learning in this embodiment has the best accuracy and robustness.

[0128] This embodiment also provides a readable storage medium, characterized in that computer program instructions are stored thereon, and when the computer program instructions are executed by a processor, the above-mentioned fusion positioning method based on reinforcement learning and particle filtering is implemented.

[0129] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0130] This embodiment also includes an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the fusion positioning method based on reinforcement learning and particle filtering as described above is implemented.

[0131] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0132] The electronic device may be a computing device such as a mobile phone, a desktop computer, a notebook, a PDA, a cloud server, etc. The electronic device may include, but is not limited to, a processor and a memory. For example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0133] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0134] The memory can be used to store the computer program and / or module, and the processor implements the computer program by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0135] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fusion positioning method based on reinforcement learning and particle filtering, characterized in that: The steps include: S1: Arrange UWB base stations in a certain scene, integrate UWB modules and inertial sensor modules as UWB tags and set them on mobile objects; given the initial position of the UWB tags, select any two valid UWB base stations that communicate with the UWB tags as UWB base station one and UWB base station two; S2: Initialize the particle filter fusion positioning model parameters of UWB and pedestrian dead reckoning PDR, where: the particle filter fusion positioning model parameters include particle initialization, particle state at time t, and ranging vectors between the UWB tag and two UWB base stations; the initial position of the UWB tag corresponds to the particle filter fusion positioning model parameters when the iteration number t is 0; S3: Input the particle state at time t-1 into the particle filter system state prediction model to predict the particle state at time t; S4: Based on the particle state at time t, the reinforcement learning algorithm is introduced to train the particle filter system state prediction model to obtain a post-training filter prediction model that can adaptively select the UWB ranging standard deviation; S5: Adaptively select the UWB ranging standard deviation at time t based on the trained filter prediction model to update the particle weight w at time t t , get the updated particle weight w t '; S6: Determine the resampling threshold of the particle filter. If the number of valid particles N eff Less than the resampling threshold N th , then the particles are resampled according to the updated particle weight distribution to obtain the resampled particle weight w t ”, go to S7; otherwise, let w″ t =w t 'Go directly to S7; S7: According to the particle weight w t " and the particle state at time t in S2 uses the particle filter algorithm to output the positioning result of the UWB tag at time t; Let t=t+1, and select the best two UWB base station combinations as new UWB base station one and UWB base station two based on reinforcement learning and the principle of minimum effective particle number, and return to S2.

2. The fusion positioning method based on reinforcement learning and particle filtering according to claim 1 is characterized in that: In S2, the particle filter fusion positioning model parameters are initialized based on the initial position of the UWB tag, specifically including: S2.1, generating a uniformly distributed particle set around the initial position of the UWB tag and constructing a particle filter fusion positioning model; S2.2, initializing the particle filter fusion positioning model parameters of pedestrian dead reckoning PDR; In S3, the specific expression of the particle filter system state prediction model is: in: is the particle state at time t; H is the state transfer matrix of the particle filter system state prediction model; j is the particle number; is the particle state at time t-1; B is the input matrix of the particle filter system state prediction model.

3. The fusion positioning method based on reinforcement learning and particle filtering according to claim 2 is characterized in that: The S4 includes: S4.

1. Construct a reward function for reinforcement learning based on the total number of valid particles in the particle set. The specific expression is as follows: Where: R[s q ][a j ] is the reward function; N is the total number of particles in the particle set; s q is the particle state, s q ={s1,s2,...s m }, the value of m is determined by the number of positioning times of model training; a p is the standard action set in the reinforcement learning algorithm, a p ={a1,a2,...a n }, the value of n depends on the tolerance setting in the collection; is the particle weight of the jth particle at time t; S4.

2. Use the reward function to calculate the reward of the action in each state, and take the action with the highest reward as the UWB ranging standard deviation at the selected time t.

4. The fusion positioning method based on reinforcement learning and particle filtering according to claim 3 is characterized in that: Standard Action Set A p The specific determination process is: determine the upper and lower limits of the ranging error between any two UWB base stations and UWB tags in a dynamic environment; select the appropriate tolerance according to the upper and lower limits and set it as an arithmetic sequence set, and use this arithmetic sequence set as the standard action set a in reinforcement learning p .

5. The fusion positioning method based on reinforcement learning and particle filtering according to claim 4 is characterized in that: The S5 includes: S5.

1. Based on the particle state at time t predicted in S2, calculate the distance vector O corresponding to the particle state at time t t The probability of S5.

2. The distance vector O corresponding to the particle state at time t t The probability of the jth particle at time t is calculated based on the probability of the jth particle at time t and the adaptively selected UWB ranging standard deviation. S5.

3. According to the ranging probability obtained in S5.2 and the particle weight at time t-1, the updated particle weight at time t is obtained.

6. The fusion positioning method based on reinforcement learning and particle filtering according to claim 5 is characterized in that: In S5.1, the distance vector O corresponding to the particle state at time t t Probability The specific calculation formula is as follows: In S5.2, the distance measurement probability corresponding to the jth particle at time t is The specific calculation formula is as follows: The specific calculation formula of S5.3 is as follows: Where: i is the UWB base station number, i = 1, 2; is the distance value between the UWB tag Tag and the i-th UWB base station in the t-th step, σ r is the adaptively selected UWB ranging standard deviation; is the X coordinate of the i-th UWB base station, is the Y coordinate of the i-th UWB base station; t is the X coordinate of the UWB tag at time t, y t is the Y coordinate of the UWB tag at time t.

7. The fusion positioning method based on reinforcement learning and particle filtering according to claim 6 is characterized in that: In S7, the specific calculation formula of the positioning result at time t is as follows: in: is the output particle state at time t and the positioning result of the UWB tag at time t; In S8, the optimal two UWB base station combinations are selected based on reinforcement learning as UWB base station one and UWB base station two, specifically: According to the ranging value between the UWB tag and each UWB base station, the reward function corresponding to each UWB base station is calculated respectively, and the reward functions corresponding to each UWB base station are sorted, and the two UWB base stations corresponding to the two smallest reward functions are selected as the new base station one and base station two.

8. The fusion positioning method based on reinforcement learning and particle filtering according to any one of claims 1 to 7, characterized in that: In S6, determining the resampling threshold of the particle filter specifically includes: Set the resampling threshold set, specifically: <h2 style=";text-align:left;direction:ltr">N<h2 style=";text-align:left;direction:ltr"> th <h2 style=";text-align:left;direction:ltr"> (0.2N, 0.3N, 0.4N, 0.5N, 0.6N, 0.7N, 0.8N, 0.9N, N) According to the experimental data of particle filtering, the value in the resampling threshold set corresponding to the state with the smallest average positioning error is selected as the resampling threshold N th , and set as the hyperparameters of the reinforcement learning algorithm.

9. A readable storage medium, characterized in that: Computer program instructions are stored thereon, and when the computer program instructions are executed by a processor, a fusion positioning method based on reinforcement learning and particle filtering as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: At least one processor, at least one memory, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the fusion positioning method based on reinforcement learning and particle filtering as described in any one of claims 1 to 7 is provided.

Images