Beidou inertial navigation fusion positioning method and system capable of adaptively and intelligently adjusting parameters

Abstract

The invention discloses a Beidou inertial navigation fusion positioning method and system for adaptive intelligent parameter adjustment, and the method comprises the steps: constructing an adaptive denoising model based on a state estimator and an actor-criticer network; training the adaptive denoising model, and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value; performing adaptive adjustment on a noise covariance matrix of a Kalman filtering fusion positioning model according to the noise parameter correction value; and obtaining accurate position information according to the fusion positioning model after adaptive denoising. The system comprises a denoising model building module, a parameter learning module, a parameter adjusting module and a fusion positioning module. By using the method, dynamic noise which cannot be modeled in fusion positioning and self-adaptive adjustment parameters can be effectively removed, and the adaptability of a positioning system to a complex and dynamically changed urban environment is improved. The method can be widely applied to the field of satellite navigation.

Description

Technical Field

[0001] The present invention relates to the field of satellite navigation, and in particular to a Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment. Background Art

[0002] In recent years, the BeiDou Navigation Satellite System (BDS) has achieved global service and can achieve decimeter or even centimeter-level positioning accuracy in open areas. Today, the BeiDou system is widely used in transportation, agriculture, forestry, fishery, military applications, disaster relief and mitigation, etc. due to its wide coverage and high positioning accuracy. However, in obstructed environments, such as high-rise buildings, high-speed tunnels and overpasses, satellite signals and differential signals will not be able to reach the vehicle's receiver smoothly due to obstruction, resulting in the loss of vehicle location information and serious deviation of positioning results. In order to solve the problem that Beidou satellite signals are easily interrupted in high-obstruction environments, 5G-assisted satellite-inertial combined navigation has become an urgent need in today's vehicle navigation industry.

[0003] Satellite-inertial integrated navigation is a navigation method that combines the inertial navigation system with the satellite navigation system through loose coupling, tight coupling or deep coupling. The inertial navigation system (INS) performs positioning and the satellite navigation system performs error correction, thereby achieving high-precision positioning.

[0004] At present, most satellite-inertial navigation technologies use model-based methods for information fusion and position calculation. However, traditional model-based methods require accurate system dynamics models and noise statistics, which are difficult to adapt to complex environmental changes during vehicle driving. Summary of the invention

[0005] In view of this, in order to solve the technical problem that the existing positioning method relies on accurate modeling and accurate noise estimation, which leads to low applicability, the present invention proposes a Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment, and the method comprises the following steps:

[0006] Based on the state estimator and actor-critic network, an adaptive denoising model is constructed;

[0007] Among them, the state estimator is used to obtain information from the multi-input observation space to form the confidence state of the strategy; the actor-critic network, in which the actor network is used to select appropriate actions for the agent from the confidence state, and the critic network judges whether the estimated value is accurate from different confidence states.

[0008] Training the adaptive denoising model, and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value;

[0009] Adaptively adjusting the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value;

[0010] Accurate location information is obtained based on the fusion positioning model after adaptive denoising.

[0011] In this embodiment, a reinforcement learning adaptive Kalman filter denoising model is constructed based on a state estimator and an actor-critic network. The model receives data-processed satellite signals and inertial navigation signals, and together with the vehicle history sequence as observation input, and then outputs the correction value of the noise covariance matrix in the Kalman filter fusion positioning model according to the noise parameter correction strategy learned during training. Subsequently, the fusion positioning model after adaptive denoising fuses the position information obtained by the satellite navigation and the inertial navigation, estimates the error value of the position information obtained by the inertial navigation, and compensates the estimated error value to the position information obtained by the inertial navigation at that moment, and finally obtains the precise position information of the vehicle.

[0012] In some embodiments, in the step of constructing an adaptive denoising model based on a state estimator and an actor-critic network:

[0013] Constructing a reinforcement learning environment to train the correction model based on a partially observable Markov decision process;

[0014] The reinforcement learning environment includes an observation space, an action space and a reward function;

[0015] The observation space includes vehicle historical position observations, satellite feature matrices, and IMU feature observations;

[0016] The state estimator fuses the feature information in the actual data to estimate the confidence state;

[0017] The actor-critic network is used to estimate the corrective action and predict the estimated values ​​of different actions under different confidence states.

[0018] Among them, the satellite characteristic matrix includes the line-of-sight vector and pseudorange residual;

[0019] Among them, the IMU feature observation includes the velocity components of the vehicle in the three coordinate axis directions in the navigation coordinate system;

[0020] In some embodiments, in the step of training the adaptive denoising model and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value:

[0021] Initialize an experience replay pool, and the reinforcement learning agent continues to interact with the environment to collect experience until the maximum storage capacity of the experience pool is reached;

[0022] Using the experience in the experience replay pool to train the adaptive denoising model, to obtain a trained adaptive denoising model;

[0023] obtaining the actual data;

[0024] Inputting the actual data into the trained adaptive denoising model to obtain a noise parameter correction value;

[0025] In some embodiments, the step of adaptively adjusting the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value specifically includes:

[0026] Obtaining the noise parameter correction value;

[0027] Based on Kalman filtering, a Beidou inertial navigation fusion positioning model is constructed;

[0028] Adaptively adjusting the process noise covariance matrix and the measurement noise covariance matrix of the Kalman filter according to the correction value;

[0029] The state equation and the measurement equation in the Beidou inertial navigation fusion positioning model based on Kalman filtering are updated to obtain a fusion positioning model after noise parameter correction.

[0030] In some embodiments, the step of obtaining accurate location information according to the fused positioning model after adaptive denoising specifically includes:

[0031] estimating the position error value of the inertial navigation according to the fused positioning model after the noise parameter correction;

[0032] According to the position error value, the position information obtained by the inertial navigation is corrected to obtain the accurate position information of the vehicle.

[0033] In some embodiments, it also includes: using an Adam optimizer with an adaptive learning rate.

[0034] The present invention also proposes a Beidou inertial navigation fusion positioning system with adaptive intelligent parameter adjustment, the system comprising:

[0035] Denoising model construction, based on the state estimator and actor-critic network, an adaptive denoising model is constructed;

[0036] A parameter learning module is used to train the adaptive denoising model and input actual data into the trained adaptive denoising model to obtain a noise parameter correction value;

[0037] A parameter adjustment module is used to adaptively adjust the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value;

[0038] The fusion positioning module obtains accurate location information based on the fusion positioning model after adaptive denoising.

[0039] Based on the above scheme, the present invention provides a Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment, based on deep reinforcement learning, constructs an adaptive denoising model for dynamic noise suppression and parameter learning, which is used to learn the influence of dynamically changing noise that cannot be modeled on combined navigation positioning, correct the process noise covariance and measurement noise covariance of the Kalman filter, and realize dynamic noise filtering of the Beidou inertial navigation fusion positioning model based on Kalman filtering. In addition, the present invention also integrates the vehicle motion trend information of IMU characteristics on the basis of historical position sequence data that can represent temporal characteristics and satellite feature data that can represent current environmental information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the steps of an adaptive Beidou satellite positioning method for dynamic noise suppression and parameter learning of the present invention;

[0041] Figure 2 It is a schematic diagram of data flow of the correction model of the present invention;

[0042] Figure 3 It is a schematic diagram of the LSTM neural network structure in the state estimator of the present invention;

[0043] Figure 4 is a flow chart of the correction model training of the present invention;

[0044] Figure 5 It is a schematic diagram of the process of the positioning model proposed in the present invention in the training stage and the testing stage;

[0045] Figure 6 The invention discloses a structural block diagram of an adaptive Beidou satellite positioning system for dynamic noise suppression and parameter learning. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0048] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0049] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0050] In the description of the embodiments of the present application, "plurality" means two or more than two. The following terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0051] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0052] Reference Figure 1 , which is a flow chart of an optional example of an adaptive Beidou satellite positioning method with dynamic noise suppression and parameter learning proposed by the present invention. The method can be applied to a computer device. The adaptive positioning method proposed in this embodiment may include but is not limited to the following steps:

[0053] Step S1, constructing an adaptive denoising model based on a state estimator and an actor-critic network;

[0054] Step S2, training the adaptive denoising model, and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value;

[0055] Step S3, adaptively adjusting the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value;

[0056] Step S4: Obtain accurate location information based on the fused positioning model after adaptive denoising.

[0057] The present invention designs a Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment, which is an adaptive denoising Kalman filter positioning method that integrates Beidou and inertial navigation based on deep reinforcement learning of multi-input comprehensive observation environment and long short-term memory (LSTM) structure state estimator. The model uses the vehicle's historical position sequence, the speed sequence obtained by the IMU system, and the Beidou satellite feature vector composed of pseudo-range residuals and line-of-sight (LOS) vectors of different satellites as input, and uses reinforcement learning to model the dynamic and changeable environmental factors during vehicle driving. Through the actor-critic based deep reinforcement learning model, the PPO algorithm is used to constrain the update of strategy parameters and select actions to correct the process noise covariance matrix and the measurement noise covariance matrix in the Kalman filter positioning model, and remove the dynamic interference noise in the subsequent Kalman filter satellite positioning method.

[0058] In some feasible embodiments, the correction model in step S1 is a Kalman filter adaptive denoising model based on reinforcement learning, which mainly consists of two parts: (1) a state estimator, which is used to obtain information from a multi-input observation space to form a confidence state of the strategy; (2) an actor-critic network, wherein the actor network is used to select appropriate actions for the agent from the confidence state, and the critic network judges whether the estimated value is accurate from different confidence states.

[0059] The model uses the optimal estimated position obtained by the Kalman filter as input, and fuses multiple features to correct the noise covariance matrix. The data processing process of this model is referred to Figure 2 .

[0060] Specifically, in order to effectively fuse the feature information in satellite observations and IMU observations to estimate the confidence state of the POMDP model, we used the LSTM network module as a state estimator for state estimation. LSTM is a deep learning model commonly used to process sequence data. It can better capture and remember dependencies in long sequences. Its main structure includes input gate, forget gate, hidden unit and output gate.

[0061] The structure of the LSTM neural network is as follows Figure 3 As shown, the following will describe the process of processing time series samples.

[0062] (1) Since the coordinate positions of historical position observations in different regions vary greatly, in order to avoid learning difficulties, we need to first transform O pos Normalize. The normalized historical position observation O pos , Satellite characteristic observation gnss and IMU feature observation O imu Input to the LSTM neural network. Among them, the forget gate is used to control how much information is retained in the cell state of the previous time step. Specifically, the sigmoid function is used to determine how much output information h of the previous moment is retained. t-1 and the cell state C at the previous moment t-1 Can be in the current cell state C t The calculation formula is as follows:

[0063] f t =σ(W f ·g[h t-1 ,x t ]+b f )#(1)

[0064] Among them, f t is the forget gate, σ is the sigmoid function, W f represents the weight matrix, b f is the bias vector, h t-1 is the hidden state of the previous time step, x t is the input of the current network, and g represents vector concatenation.

[0065] (2) Next is the calculation of the input gate and candidate cell states. The input gate is used to control the impact of the input information of the current time step on the cell state. The model passes the effective environmental information features through the input gate and uses the processing information of the sigmoid function and the processing information of the tanh function, that is, the product of the candidate cell state, to represent the current input x t To pass to the current unit state C t The calculation formula of the input gate is as follows:

[0066]

[0067] Among them, i t is the input gate, is a candidate cell state.

[0068] (3) Use the results of the forget gate and the input gate to update the cell state. The update formula is as follows:

[0069]

[0070] Among them, C t is the updated cell state.

[0071] (4) Next is the calculation of the output gate. The output gate is used to control the influence of the cell state on the hidden state. We will use it to encode the processed state information into a fixed sequence output. The calculation formula is as follows:

[0072] o t =σ(W0·g[h t-1 ,xt]+b0)#(4)

[0073] (5) Finally, the product of the output gate output and the processed information of the tanh function is used to determine how much information in the current cell state can be transmitted to the current output h t The calculation formula is as follows:

[0074] h t =σ(W0·g[h t-1 ,x t ]+b0)·tanh(C t )#(5)

[0075] (6) The LSTM network is updated cyclically according to the length of the time series, and the output is finally obtained. Since LSTM has a strong long-term memory function, the output of the last time step contains historical information. Therefore, the present invention regards the output of LSTM at the last time step as the confidence state of the vehicle agent at the current moment, and uses the symbol b t express.

[0076] A network of actors and critics:

[0077] The system uses two deep neural network modules to construct an actor network and a critic network. The actor network is responsible for estimating the correction action and directly outputs the correction value of the noise covariance matrix from the continuous action space, which can be expressed as: in and is the covariance matrix Q of the process and measurement noise at time t t-1 and R t-1 The adjustment is used to adaptively learn the noise parameters in the Kalman filter satellite positioning model; and is an estimate of the process and measurement noise at time t, which is used to adaptively remove the noise in the state equation and measurement equation in the Kalman filter fusion positioning model. At the same time, the critic network predicts the estimated values ​​of different actions under different confidence states based on the long-term cumulative rewards, thereby guiding the training of the actor network. In order to update the value function of the critic network, we use the TD (Temporal Difference) error to calculate the difference between the current state value and the next state value, that is, δ = r t +γV(b t+1 )-V(b t ), where r t represents the reward at the current moment, γ is the decay factor, V(b t+1 ) is the confidence state value at the next moment, V(b t ) is the confidence state value at the current moment. By calculating each state b t The square of the TD error δ is used to measure the current value function V(b t ) and use this error to update the parameters of the critic network.

[0078] In addition, in terms of the choice of optimizer, we use the Adam optimizer with adaptive learning rate. The Adam optimizer uses the first-order moment estimation and second-order moment estimation of the gradient to dynamically adjust the learning rate of each parameter. After bias correction, the learning rate of each iteration is kept within a certain range, making the parameter update more stable and reliable.

[0079] Setup of reinforcement learning model;

[0080] In reinforcement learning models, the environment is usually modeled by a Markov decision process (MDP). The agent interacts with the environment, obtains state values ​​and chooses actions to obtain the maximum reward from the environment. In the adaptive filtering denoising and positioning problem, the vehicle is generally used as an agent. Since the state of the environment cannot be fully observed during the driving process of the vehicle, the adaptive filtering denoising and positioning problem is usually expressed as a partially observable Markov decision process (POMDP), which infers the state of the environment through the observed partial information. POMDP can be defined as a septenary (S, A, T, R, O, Θ, γ), where S = {s i , i=1,…,t+1} is a set of states, A={a i ,i=1,…,t} is the action set of the agent, T:S×A×S→[0,1] is a set of conditional transition probabilities between states, is the reward function, O is a set of observations O = {o i,i=1,…,t+1}, Θ is the conditional observation probability, which is used to describe a certain relationship between the observation and the true state, that is, o~Θ(s), and γ∈[0,1] is the discount factor. At each moment, the vehicle agent is in a certain state s∈S. After that, the agent extracts and executes action a∈A in A according to the strategy, and the probability of state transition is T(s′|s,a). While entering the new state, the agent receives the observation o∈O, which depends on the new state of the environment, and the probability relationship can be expressed by Θ(o|s′,a). Finally, the agent receives the reward R(s,a) and repeats the above steps. The goal of reinforcement learning is to select appropriate actions at each time step to maximize the expected return:

[0081]

[0082] In the POMDP problem, the agent will use the belief state, which is a degree of confidence in the current state of the system based on past observations and model assumptions, as the state of the environment. Therefore, we can use a quinary number (B, A, τ, R, γ) to define the belief state Markov decision process. Among them, B is the set of belief states based on the original POMDP state, A is the same finite action set as the original POMDP, τ is the belief state transition function, is the reward function for the belief state, and γ is the same discount factor as "γ" in the original POMDP.

[0083] In this quinary number, R is different from that defined in the original POMDP model and needs to be derived from it, which can be expressed as:

[0084]

[0085] After taking action a and obtaining observation o, the agent needs to update the confidence b. Since the state is Markov, we only need to understand the belief state at the last moment, the action taken, and the current observation result to get the current confidence state, which can be expressed as b′=τ(b,a,o).

[0086] After obtaining the state s', the agent obtains the observation from the observation set o∈O according to the probability Θ(o|s',a). If b is regarded as a probability distribution based on the state space S, then b(s) represents the probability that the environment is in state s. Given b(s), the update of the confidence can be iteratively expressed as:

[0087]

[0088] in is the standardization constant, P r(o|b,a) =∑ s′∈S O(o|s′)∑ s∈ST(s′|s,a)b(s).

[0089] The transition formula of the confidence state Markov decision process can be expressed as:

[0090]

[0091] in,

[0092] Observation space settings;

[0093] Previous reinforcement learning positioning methods used past vehicle positions as observations and simultaneously introduced Beidou satellite measurements to learn the rules of errors caused by multipath effects in complex urban environments. However, existing methods are greatly affected by the quality of satellite signals, and the positioning accuracy drops sharply when the satellite signal is weak or interrupted. In order to make the observations represent the current state of the agent as much as possible, we introduced IMU features to form a multi-input observation space with the original Beidou satellite features and the vehicle's historical positions. The observations include satellite feature observations, vehicle historical position observations, and IMU feature observations, which can be expressed as: t = {O bds,t ,Q pos,t ,O imu,t}; The following will introduce the composition of the observation space:

[0094] 1. Vehicle historical position observation pos,t : It should be noted that the historical position sequence here only includes the position points when the satellite signal is received. Unlike the model-based method, the observation value of the vehicle position in our method is not limited to the last position received by the satellite, but a stacked vector consisting of the last position point and the historical predicted position. These historical predicted positions are the corrected outputs of the RL model in the past time steps, which helps the RL model understand the movement state of the vehicle. At the same time, we use the standard single-point positioning method based on the model to obtain the initial estimated position It is expressed as follows:

[0095]

[0096] in and Represents the initial estimated position of the point to be corrected on different coordinate axes in the Earth-Centered Earth-Fixed (ECEF) coordinate system. The construction of the vehicle's historical position sequence is carried out through a sliding window. When new position information is obtained, the sliding window moves forward one step to form a new observation vector of size N (the window size can be set), and so on. The vehicle's historical position observation at time t can be expressed as:

[0097]

[0098] Among them, pos pred =pos init +Δpos represents the historical position point after high-precision correction.

[0099] 2. Satellite characteristic matrix O bds,t :The satellite characteristic matrix consists of the line-of-sight vector los and the pseudorange residual res, which aims to better model the multipath error caused by different environments. The satellite characteristic observation at time t can be expressed as:

[0100]

[0101] Where n is the maximum number of visible satellites in the vehicle's trajectory. The expressions for los and res are as follows:

[0102]

[0103] in, is the estimated satellite position, is the measured pseudorange.

[0104] Since the number of visible satellites at different moments in the trajectory is usually inconsistent, we fill the feature information of invisible satellites with 0 during the process.

[0105] 3. IMU feature observation imu,t : In order to better learn the movement trend of the vehicle, we introduce the three-direction speed of the vehicle at the time to be corrected as the observation input, which can be expressed as:

[0106]

[0107] Then the IMU feature observation O imu,t It can be defined as:

[0108]

[0109] Action space settings:

[0110] In this embodiment, the action is defined as a correction action for the noise covariance matrix in the Kalman filter fusion positioning model. In order to avoid the difficulty of learning the action space caused by a large number of discrete actions, we define the action as a 24-dimensional continuous action setting, that is, Action a t Each element a in i,t Output the corresponding mean μ at time t through the actor network i,t and variance σ i,t The Gaussian distribution is used for sampling, that is, Furthermore, action a t Can be divided into four actions in and is the covariance matrix Q of the process and measurement noise at time t t-1 and R t-1 The adjustment of

[0111]

[0112]

[0113] and is the estimate of process and measurement noise at time t, which is used for the initial control quantity of KF and measurement The correction is:

[0114]

[0115]

[0116] in, represents the adjustment scale factor of the noise covariance matrix,

[0117] Reward function settings:

[0118] In order to accurately evaluate the effectiveness of the noise covariance correction action, we use the correction advantage error as the reward function. Compared with using the mean square error (MSE) to calculate the accuracy of positioning as a reward function, using the correction advantage error can better reflect the advantage of a good correction strategy over the baseline positioning method. The correction advantage error at the moment can be defined as:

[0119]

[0120] represents the initial estimated position, t represents the time, Indicates the corrected historical position point.

[0121] In some feasible embodiments, in step S2, the adaptive denoising model is trained using reinforcement learning, which specifically includes:

[0122] The general process of model training is as follows:

[0123] At the beginning of training, we first initialize an experience replay pool, and the reinforcement learning agent continues to interact with the environment to collect experience until the maximum storage capacity of the experience pool is reached. Then, the experience in the experience replay pool is used to train the adaptive denoising model, where the model parameters corresponding to the agent can be iteratively updated using the stochastic gradient descent algorithm to minimize the objective function. After that, when the maximum number of training iterations is reached, the parameters of the trained reinforcement learning model are fixed. The overall training flow chart of this model is as follows: Figure 4 The specific process of model training is as follows:

[0124] The trajectory used for training includes the Beidou inertial navigation fusion positioning estimate obtained by the Kalman filter-based method. First, the reinforcement learning positioning environment and reinforcement learning model are constructed. For example, at time t, the observation O of the environment is constructed through satellite measurement data and historical positioning information. t = {O bds,t ,O pos,t ,O imu,t}, which is characterized by satellite observation O bds,t , vehicle historical position sequence O pos,t , IMU feature observation O imu,t In addition, the reinforcement learning model with multiple observation inputs is initialized to construct feature extractors, actor networks, and evaluator networks. The evaluator network is used to estimate the value function of the current observation state, and the output action of the actor network is used to estimate the correction value of the noise covariance matrix to correct the fused noise covariance matrix; secondly, an experience replay pool is initialized, and the agent continuously interacts with the environment to collect experience. At the observation O at time t t Next, the feature extractor extracts the potential representation of the observed features and inputs it into the actor network to obtain action a t , through action a t Determine the corrected position result And use and Calculate the current reward R t , and reaches the observation O at time t+1 t+1 , and then collect the empirical data (O t ,a t ,R t ,O t+1 ) is stored in the experience replay pool until the maximum storage capacity of the experience pool is reached; finally, the experience in the experience replay pool is used to train the reinforcement learning model. The model parameters θ of the agent can be iteratively updated by minimizing the objective function using the stochastic gradient descent algorithm.

[0125] In some feasible embodiments, in step S3, the adaptive denoising model is trained using reinforcement learning, which specifically includes:

[0126] The general process of model testing is as follows:

[0127] The trained model is deployed on the satellite positioning cloud, and the positioning results are output to the user terminal in real time end-to-end, which is used to output positioning results in complex urban environments. First, satellite measurement data and inertial navigation measurement data are collected to construct the observation O at time t. t = {O bds,t ,O pos,t ,O imu,t The satellite measurement data uses the 5G-assisted RTK positioning and speed determination method to obtain the rough position coordinates p bds,t With the rough speed v bds,t ; Secondly, observe O t Input into the agent's feature extractor and actor network to get the output action Combined with the action to estimate the denoising of the measurement / process noise and the corresponding covariance adjustment; finally, p bds,t 、v bds,t The position coordinate p obtained by the inertial navigation imu,t and speed v imu,t The difference is used as the system observation input to the Kalman filter after the noise covariance matrix is ​​adaptively adjusted, and the satellite positioning result is solved in real time through the adaptive denoising Kalman filter satellite positioning method based on reinforcement learning.

[0128] The process diagram of the positioning model proposed in the present invention in the training phase and the testing phase is as follows: Figure 5 shown.

[0129] In some feasible embodiments, the fusion positioning model in step S4 is a Beidou inertial navigation fusion positioning model based on a Kalman filter, which mainly consists of three parts: (1) system state, which is used to describe the current state of the system to predict and estimate the position. The state transition model describes the state change of the system from one time point to the next time point; (2) system measurement, which is used to describe the current measurement of the system, and the measurement model describes how to generate observation data based on the state of the system; (3) system noise, which is used to describe the uncertainty of the system model and affects the accuracy of the state transition and measurement update process.

[0130] Specifically, in order to effectively integrate the information of satellite navigation and inertial navigation, we used a loosely coupled method to establish a Kalman filter model. The model uses the difference between the position information obtained by satellite navigation and the position information obtained by inertial navigation as the system measurement, and the inertial navigation prior error obtained through experience as the system state. After iterative updates through Kalman filtering, it finally outputs the estimated position error and velocity error and uses it to correct the position information obtained by inertial navigation, and obtains the corrected inertial navigation position solution value as the fusion positioning result.

[0131] First, the state vector includes the position error, velocity error, attitude error, accelerometer bias, and gyroscope bias components of the vehicle on the coordinate axis. Let δp x,t ,δp y,t ,δp z,t is the position error of the vehicle; δv x,t ,δp y,t ,δv z,t is the speed error of the vehicle; δθ x,t ,δθ y,t ,δθ z,t is the vehicle’s attitude error; ε x,t ,ε y,t ,ε z,t is the bias of the accelerometer in three directions; x,t ,ζ y,t ,ζ z,t is the bias of the angular velocity meter in three directions. Then the state vector can be expressed as:

[0132] x t =[δp x,t ,δp y,t ,δp z,t ,δv x,t ,δv y,t ,δv z,t ,δθ x,t ,δθ y,t ,δθ z,t ,ε x,t ,ε y,t ,ε z,t ,ζ x,t ,ζ y,t ,ζ z,t ] T #(twenty one)

[0133] Then define the state equation and measurement equation of the vehicle positioning of the filter at time t. Let x t and x t+1 is the predicted state of the Kalman filter at time t and time t-1, that is, the prediction error of the vehicle; is the state transfer matrix, defined by the kinematic equations; represents random measurement noise, which is assumed to follow a Gaussian distribution w t ~N(0,R), where R represents the measurement noise covariance matrix; represents random process noise, assuming it follows a Gaussian distribution v t ~N(0,Q), Q represents the process noise covariance matrix; z t is the measured value at time t, that is, the difference between the position and velocity calculated by the joint positioning model and the position and velocity calculated by the inertial navigation system; represents the state-to-measurement transition matrix, which is defined as H = [I 6×6 0 6×9 ]. Then the state equation and measurement equation of vehicle positioning can be expressed as:

[0134] x t =Fx t-1 +w t #(twenty two)

[0135] z t =Hx t +v t #(twenty three)

[0136] After initializing the state vector x0 and the state estimation covariance matrix P0, the filter begins to perform prediction optimization, and the process is as follows:

[0137] Prediction stage: The current position is predicted based on the previous position through the dynamic model, i.e. the state equation. The control quantity is corrected by the process noise estimated by the reinforcement learning model, and the process noise covariance matrix is ​​adjusted by the output action of the reinforcement learning model. Then the a priori estimated covariance matrix is ​​predicted: x t =Fx t-1 ,

[0138] Update phase: After receiving the measured value z t After that, the KF state is updated, and the measurement noise covariance matrix is ​​adjusted through the output action of the reinforcement learning model. Then the covariance matrix of the prior estimate is predicted, and the Kalman gain K is calculated to measure the weight of prediction and measurement: Calculate the optimal estimated state of the system The Kalman gain K is used to measure the weight and combine the predicted state and the measured data to calculate the optimal state estimate at the current moment: Covariance matrix P t Update: The prediction phase and the update phase are iterated alternately, and finally the optimal estimate of the fusion positioning is obtained.

[0139] Based on the above scheme, the present invention adds IMU feature data on the basis of historical position sequence data that can represent temporal characteristics and satellite feature data that can represent current environmental information, and uses this as a model input to represent temporal characteristics, environmental characteristics and motion trends; the present invention constructs a reinforcement learning framework, which includes an actor network and a critic network. The actor network selects appropriate actions according to the current state, while the critic network guides the update of the actor network according to the advantage function. In order to ensure the stability of parameter updates, we adopt a structured proximal strategy optimization method; the present invention proposes to interact with the real-time changing positioning environment through reinforcement learning, learn the influence of dynamic changes in measurement and process random noise on positioning, and adaptively learn the corresponding noise covariance matrix to improve the robustness of the positioning model to noise and the adaptability to the dynamically changing environment; the present invention considers the fusion of satellite data and IMU data for the first time in the field of reinforcement learning positioning; the present invention constructs a Beidou inertial navigation fusion positioning model with adaptive intelligent parameter adjustment, which is used to filter out dynamic noise that cannot be modeled and correct the process noise covariance matrix and measurement noise covariance matrix in the Kalman filter fusion positioning model.

[0140] like Figure 6 As shown, an adaptive Beidou satellite positioning system with dynamic noise suppression and parameter learning includes:

[0141] Denoising model construction, based on the state estimator and actor-critic network, an adaptive denoising model is constructed;

[0142] A parameter learning module is used to train the adaptive denoising model and input actual data into the trained adaptive denoising model to obtain a noise parameter correction value;

[0143] A parameter adjustment module constructs a training data set to train the correction model to obtain a trained correction model;

[0144] The fusion positioning module obtains accurate location information based on the fusion positioning model after adaptive denoising.

[0145] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0146] An adaptive Beidou satellite positioning device with dynamic noise suppression and parameter learning:

[0147] at least one processor;

[0148] at least one memory for storing at least one program;

[0149] When the at least one program is executed by the at least one processor, the at least one processor implements the Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment as described above.

[0150] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0151] A storage medium stores instructions executable by a processor, wherein the instructions executable by the processor are used to implement a Beidou inertial navigation fusion positioning method and system with adaptive intelligent parameter adjustment as described above when executed by the processor.

[0152] The contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A Beidou inertial navigation fusion positioning method with adaptive intelligent parameter adjustment, characterized in that: The following steps are involved: Based on the state estimator and actor-critic network, an adaptive denoising model is constructed; Training the adaptive denoising model, and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value; Adaptively adjusting the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value; Accurate location information is obtained based on the fusion positioning model after adaptive denoising.

2. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 1, it is characterized in that: The step of building an adaptive denoising model based on the state estimator and the actor-critic network includes: Constructing a reinforcement learning environment to train the correction model based on a partially observable Markov decision process; The reinforcement learning environment includes an observation space, an action space and a reward function; The observation space includes vehicle historical position observations, satellite feature matrices, and IMU feature observations; The state estimator fuses the feature information in the actual data to estimate the confidence state; The actor-critic network is used to estimate the corrective action and predict the estimated values ​​of different actions under different confidence states.

3. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 2, it is characterized in that: The step of training the adaptive denoising model and inputting actual data into the trained adaptive denoising model to obtain a noise parameter correction value specifically includes: Initialize an experience replay pool, and the reinforcement learning agent continues to interact with the environment to collect experience until the maximum storage capacity of the experience replay pool is reached; Using the experience in the experience replay pool to train the adaptive denoising model, to obtain a trained adaptive denoising model; obtaining the actual data; The actual data is input into the trained adaptive denoising model to obtain a noise parameter correction value.

4. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 3, it is characterized in that: The step of adaptively adjusting the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value specifically includes: Obtaining the noise parameter correction value; Based on Kalman filtering, a Beidou inertial navigation fusion positioning model is constructed; Adaptively adjusting the process noise covariance matrix and the measurement noise covariance matrix of the Kalman filter according to the correction value; The state equation and the measurement equation in the Beidou inertial navigation fusion positioning model based on Kalman filtering are updated to obtain a fusion positioning model after noise parameter correction.

5. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 4, it is characterized in that: The step of obtaining accurate location information based on the fusion positioning model after adaptive denoising specifically includes: estimating the position error value of the inertial navigation according to the fused positioning model after the noise parameter correction; According to the position error value, the position information obtained by the inertial navigation is corrected to obtain the accurate position information of the vehicle.

6. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 5, it is characterized in that: Also includes: The actual data are normalized.

7. According to claim 6, a Beidou inertial navigation fusion positioning method with adaptive intelligent parameter adjustment is characterized in that: The satellite characteristic matrix includes a line-of-sight vector and a pseudorange residual.

8. According to the adaptive intelligent parameter adjustment Beidou inertial navigation fusion positioning method of claim 7, it is characterized in that: The formula of the reward function is as follows: in, represents the initial estimated position, t represents the time, represents the reference position for model learning, Indicates the corrected historical position point.

9. A Beidou inertial navigation fusion positioning system with adaptive intelligent parameter adjustment, characterized in that: include: Denoising model construction, based on the state estimator and actor-critic network, an adaptive denoising model is constructed; A parameter learning module is used to train the adaptive denoising model and input actual data into the trained adaptive denoising model to obtain a noise parameter correction value; A parameter adjustment module is used to adaptively adjust the noise covariance matrix of the Kalman filter fusion positioning model according to the noise parameter correction value; The fusion positioning module obtains accurate location information based on the fusion positioning model after adaptive denoising.

Images