Kalman filtering network Beidou satellite positioning method and system based on environment interaction

Abstract

The invention discloses a Kalman filter network Beidou satellite positioning method and system based on environment interaction, and the method comprises the steps: carrying out the parameter initialization of a Kalman filter based on a mathematical model, carrying out the satellite positioning through the combination of Kalman gain optimization, and obtaining the positioning result of an unmanned plane at the current moment; a multi-dimensional Kalman gain satellite positioning environment is constructed, a Kalman gain correction action space is set, a reward function is set, and a deep reinforcement learning Kalman filtering network is constructed in combination with a positioning result of the unmanned aerial vehicle at the current moment; and training the deep reinforcement learning Kalman filtering network through a PPO algorithm to obtain an optimal filtering strategy in the dynamic complex urban environment. According to the method, the current optimal state fusion strategy can be directly learned and obtained in a complex urban environment with serious interference, and the positioning precision of Beidou satellite positioning is improved. The Kalman filtering network Beidou satellite positioning method and system based on environment interaction can be widely applied to the technical field of Beidou satellite positioning.

Description

Technical Field

[0001] This invention relates to the field of BeiDou satellite positioning technology, and in particular to a BeiDou satellite positioning method and system based on environmental interaction using a Kalman filter network. Background Technology

[0002] With the rapid development of technology, unmanned aerial vehicles (UAVs) have demonstrated enormous application potential in numerous fields such as logistics, maritime inspection, bridge surveying, and emergency rescue, leading to a continuous rise in market interest. In these applications, high-precision and reliable positioning of UAVs is a crucial foundation for ensuring mission success and flight safety. The BeiDou Navigation Satellite System (BDS), as one of the world's mainstream high-precision positioning technologies, can achieve decimeter- to centimeter-level positioning in areas with good signal coverage and open fields of view through high-precision positioning calculation technology, providing strong support for UAVs.

[0003] However, high-precision positioning of drones faces severe challenges when performing missions. When drones perform urban logistics delivery, bridge surveying, or complex inspection tasks, they need to enter environments such as urban canyons, densely populated areas, under overpasses, at the bottom of bridges, near pylons, and inside complex steel structures. When performing maritime inspection tasks, the specular reflection of the sea surface can also introduce serious interference. In these complex and variable scenarios, BeiDou satellite signals are highly susceptible to interference from building obstruction, multipath effects, or specular reflection, causing the state-space model (SSM) of the positioning system to exhibit strong non-stationarity and non-Gaussian characteristics. Secondly, the challenges stem not only from the environment but also from the hardware itself: the GNSS modules used in ordinary consumer-grade drones typically have a single-point positioning accuracy between 1 and 3 meters, which cannot meet the monitoring requirements at the centimeter or even millimeter level.

[0004] Current mainstream positioning algorithms, such as Weighted Least Squares (WLS) and Kalman Filter (KF), are based on a model-driven deterministic framework. Their performance is highly dependent on accurate modeling of system dynamics and ideal assumptions such as Gaussian noise and stationarity. When encountering severe SSM mismatch caused by the aforementioned complex environment, the theoretical optimality of Kalman Filter is destroyed, the calculation of the core parameter Kalman gain (KG) becomes inaccurate, and the fusion of prediction and observation information cannot be reasonably balanced. Ultimately, this leads to a significant deterioration (errors can reach tens of meters) and instability in the positioning accuracy of UAVs, seriously threatening flight safety and mission execution.

[0005] To address the issue of unstable positioning errors by drones in complex urban environments, data-driven methods based on deep learning (DL) have been explored for error correction, demonstrating some potential. However, existing DL correction models face key challenges in learning prior parameters: firstly, to ensure the optimality and stability of the Kalman filter, the error covariance matrix learned by the model must satisfy symmetric positive definiteness, requiring the introduction of additional parameterization strategies or projection constraints into the neural network output; secondly, the learned noise correction term is merely a fitting estimate based on historical data, making it difficult to guarantee optimal solutions and real-time performance in the current environment. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention aims to provide a BeiDou satellite positioning method and system based on environmental interaction using a Kalman filter network. This method can directly learn and obtain the current optimal state fusion strategy in complex urban environments with severe interference and low signal-to-noise ratio, thereby improving the positioning accuracy of BeiDou satellite positioning.

[0007] The first technical solution adopted in this invention is: a BeiDou satellite positioning method based on environmental interaction using a Kalman filter network, comprising the following steps: The parameters of the Kalman filter based on the mathematical model are initialized, and satellite positioning is performed by combining Kalman gain optimization to obtain the current positioning result of the UAV. A multi-dimensional Kalman gain satellite positioning environment is constructed, a Kalman gain correction action space is set, a reward function is set, and a deep reinforcement learning Kalman filter network is constructed by combining the current positioning results of the UAV. By training a deep reinforcement learning Kalman filter network using the PPO algorithm, the optimal filtering strategy in a dynamic and complex urban environment is obtained, enabling the Kalman filter network to achieve BeiDou satellite positioning.

[0008] Furthermore, the step of initializing the parameters of the Kalman filter based on the mathematical model and performing satellite positioning in conjunction with Kalman gain optimization to obtain the current positioning result of the UAV specifically includes: Based on the pseudorange measurement of the BeiDou satellite navigation system, a pseudorange measurement equation is constructed, and the pseudorange measurement equation is solved by the weighted least squares method to obtain the coarse positioning result of the UAV, which is then used as the observation value for Kalman filtering to construct the measurement equation. The vehicle speed is obtained by constructing a pseudo-distance rate measurement equation and used as a control variable to construct a state equation. By combining the measurement equation and the state equation, a Kalman filter based on a mathematical model is constructed. The parameters of the Kalman filter based on the mathematical model are initialized. The initialization process includes the noise covariance matrix, observation noise covariance matrix, prediction error covariance matrix, and innovation error covariance matrix, resulting in the initialized Kalman filter. Based on the initialized Kalman filter and the measurement equation of the Kalman filter, the weight allocation of prediction and measurement is measured, and the Kalman gain is calculated. The Kalman gain is corrected by a reinforcement learning model, and the state is corrected based on the corrected Kalman gain. The optimal state estimate is calculated, the covariance matrix is ​​updated to reflect the uncertainty of the state estimate, and the final positioning solution is output as the UAV's current positioning result.

[0009] Furthermore, the step of constructing a multi-dimensional Kalman gain satellite positioning environment, setting the Kalman gain correction action space, setting the reward function, and constructing a deep reinforcement learning Kalman filter network by combining the current positioning results of the UAV specifically includes: The pseudorange residuals, three-dimensional line-of-sight vectors, carrier-to-noise ratios, and elevation angles of different satellites are used as satellite observation features. The pseudorange residuals represent the prediction error of the initial position estimation, the three-dimensional line-of-sight vectors represent the coarse directional information from the satellite to the receiver, the carrier-to-noise ratios represent the current signal quality, and the elevation angles represent the angles between the receiver and the satellite in the vertical direction. Based on the historical position constraints of the UAV positioning system, a historical position sequence is defined, and a robust Kalman gain reinforcement learning observation space is constructed by combining satellite observation characteristics. By combining the actions of adjusting historical positions with the actions of adjusting the Kalman gain matrix, a Kalman gain correction action space is constructed. Define a reward function to correct the advantage error and construct a reinforcement learning reward function to correct the Kalman gain; By combining the current location results of the UAV, the robust Kalman gain reinforcement learning observation space, the Kalman gain correction action space, and the reinforcement learning reward function of the corrected Kalman gain, a deep reinforcement learning Kalman filter network is constructed.

[0010] Furthermore, the expression for the Kalman gain correction action space is as follows:

[0011]

[0012] In the above formula, express The position of the agent after correction at any given moment. express The approximate location of the moment, express The action of constantly adjusting the historical position. express The Kalman gain matrix adjusted by the agent at each time step. express The Kalman gain matrix calculated from the covariance matrix at each time step. express The action of adjusting the Kalman gain matrix at any time.

[0013] Furthermore, the expression for the corrected advantage error reward function is as follows:

[0014] In the above formula, This represents the reward function for correcting the advantage error. Indicates the position of the Kalman filter output of the benchmark method. Indicates the actual location being referenced. This indicates the optimal position of the Kalman filter output after the reinforcement learning agent generates corrective actions through real-time interaction with the environment and adaptively adjusts the Kalman gain.

[0015] Furthermore, the step of training a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, thereby realizing the BeiDou satellite positioning via the Kalman filter network, specifically includes: Constructing a deep reinforcement learning agent model; The deep reinforcement learning agent model interacts with the deep reinforcement learning Kalman filter network to obtain an experience tuple consisting of the observation state, the action to be performed, the immediate reward, and the transition state. Based on the empirical tuple, by randomly sampling batches of empirical data and using the temporal difference error as the core training signal, the policy function of the actor network and the value function of the evaluator network in the deep reinforcement learning agent model are optimized to obtain the optimized evaluator network and the optimized actor network. The agent is trained using the PPO algorithm. The optimized evaluator network estimates the value of the correction action, and the optimized actor network optimizes the Kalman gain correction strategy through the PPO-Clip objective function. This yields the optimal filtering strategy in a dynamic and complex urban environment, enabling the Kalman filter network to achieve BeiDou satellite positioning.

[0016] Furthermore, the process of training the agent using the PPO algorithm, optimizing the evaluator network to estimate the value of the corrective action, and then optimizing the agent network using the PPO-Clip objective function to refine the Kalman gain correction strategy, yields the optimal filtering strategy in a dynamic and complex urban environment, thus realizing the Kalman filter network for BeiDou satellite positioning. This step specifically includes: The system collects raw satellite observation data through a receiver, and calculates the reference positioning results based on the weighted least squares method and traditional Kalman filtering. Simultaneously, it uses a high-precision INS / GNSS combined system to obtain the true position value for reward calculation. Under observation, the agent generates corrective actions through an optimized actor network, outputs the predicted position through an adaptive Kalman filter, calculates the immediate reward, and initializes the experience replay pool. Once the experience replay pool reaches its capacity threshold, the PPO algorithm is executed for updating. The strategy advantage is estimated through generalized dominance to obtain the optimal filtering strategy in a dynamic and complex urban environment, thus realizing the BeiDou satellite positioning of the Kalman filter network.

[0017] The second technical solution adopted in this invention is: a BeiDou satellite positioning system based on an environment-interactive Kalman filter network, comprising: The first module is used to initialize the parameters of the Kalman filter based on the mathematical model and combine it with Kalman gain optimization for satellite positioning to obtain the current positioning result of the UAV. The second module is used to construct a multi-dimensional Kalman gain satellite positioning environment, set the Kalman gain correction action space, set the reward function, and combine the current positioning results of the UAV to construct a deep reinforcement learning Kalman filter network. The third module is used to train a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, and to realize the Kalman filter network for BeiDou satellite positioning.

[0018] The beneficial effects of the method and system of this invention are as follows: This invention initializes parameters using a Kalman filter based on a mathematical model and performs satellite positioning by combining Kalman gain optimization to obtain the current positioning result of the UAV. It introduces a deep reinforcement learning mechanism to achieve adaptive dynamic adjustment of the key intermediate variable in the Kalman filter, the Kalman gain matrix, without relying on manual experience or static parameter design. This improves the robustness and generalization ability of the model in non-ideal environments. Furthermore, it constructs a multi-dimensional Kalman gain satellite positioning environment, sets a Kalman gain correction action space, sets a reward function, and combines the current positioning result of the UAV to construct a deep reinforcement learning Kalman filter network. Finally, it trains the deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, achieving BeiDou satellite positioning via the Kalman filter network. It does not require building a noise prior model but relies on strategy optimization to adaptively adjust the Kalman filter fusion strategy in a high-dimensional observation feature space. In practical applications, this significantly enhances the real-time performance, stability, and generalization ability of the filter. Attached Figure Description

[0019] Figure 1 This is a flowchart of the steps of the BeiDou satellite positioning method based on environmental interaction Kalman filter network of the present invention; Figure 2 This is a structural block diagram of the BeiDou satellite positioning system based on environmental interaction Kalman filter network according to the present invention; Figure 3 This is a schematic diagram of the reinforcement learning environment construction and agent model structure provided in a specific embodiment of the present invention; Figure 4 This is a schematic diagram of the overall process of the positioning model provided in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the BeiDou satellite positioning process provided in a specific embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0021] First, it's important to note that UAVs have become an indispensable part of modern logistics, inspection, surveying, and other industrial fields, making high-precision and highly stable positioning capabilities particularly crucial in this rapidly developing field. Whether it's autonomous flight path planning, precise area inspection, or complex swarm collaborative operations, accurate positioning significantly improves mission success rates and flight safety. However, the complexity of urban environments where UAVs operate, such as dense high-rise buildings, dynamically changing signal blockages, and multipath effects, presents significant challenges to UAV positioning.

[0022] Currently, data-driven GNSS positioning methods can be mainly divided into two categories: one uses deep learning to post-process and correct traditional positioning results, and the other combines deep learning with traditional methods to construct an adaptive Kalman filter (AKF) framework to improve robustness and accuracy in dynamic environments. For example: 1) A GNSS position correction model based on graph convolutional neural network (GCN) is proposed. By constructing a graph structure containing inter-satellite relationships, pseudorange error mapping is learned from the original observation data, which effectively improves the positioning accuracy of smartphones in urban environments. 2) A pseudorange error correction method based on deep reinforcement learning is proposed. The Actor-Critic model is jointly trained using vehicle trajectory and GNSS measurement, and the dynamic change characteristics of positioning error are learned through environmental interaction, thereby realizing the optimization of adaptive positioning strategy for urban traffic environment. 3) For example, reinforcement learning is introduced into the Kalman filtering framework, and the RL-AKF algorithm is proposed. The process noise covariance matrix is ​​dynamically adjusted through reinforcement learning to adapt to changes in process noise, thereby improving the robustness and responsiveness of the filter in ground vehicle navigation tasks. 4) Alternatively, a localization method that deeply couples graph neural networks with Kalman filters was proposed. By directly outputting filter update terms through an end-to-end structure, the ability of the state estimation process to model the correlation of observation data was effectively enhanced, and a dual improvement in stability and accuracy was achieved in smartphone navigation tasks. 5) The proposed EKFNet framework introduces backpropagation into the extended Kalman filter, optimizes the process and measurement noise covariance parameters through time series, avoids manual parameter tuning, realizes automatic learning of noise statistics, and significantly outperforms manually tuned EKF and existing methods on real GPS vehicle tracking data.

[0023] In this regard, data-driven deep learning (DL) methods have received widespread attention in the BeiDou Navigation Satellite System (BDS) field. Existing research mainly focuses on two directions: one is based on deep learning to correct positioning results. Typical examples include the proposed pseudorange error regression model based on graph convolutional networks (GCNs), which learns the error distribution from observation data by constructing an inter-satellite graph structure; another example is using a reinforcement learning Actor-Critic strategy to model the relationship between trajectory and measurement, used to correct initial positioning. While these methods have achieved some accuracy improvements in offline scenarios, they lack unified constraints on vehicle dynamics due to bypassing state-space modeling, resulting in weak system consistency and interpretability. Furthermore, their correction capability is limited by the initial positioning error, making it difficult to achieve stable generalization in dynamic environments.

[0024] Secondly, neural networks or deep reinforcement learning are used to estimate the prior parameters of the Kalman filter. For example, RL-AKF is proposed, which adaptively adjusts the process noise matrix through reinforcement learning; or, for example, a structure that tightly couples graph neural networks with Kalman filters is further constructed to enhance the state estimation capability; or, for example, backpropagation is introduced into extended Kalman filters to achieve adaptive estimation of noise levels by optimizing noise variables through time series. Although such methods improve the response capability to time-varying noise, they still face two problems in practical applications: (1) Correcting the Q / R matrix based on deep learning requires the introduction of parameter constraints or positive definite projection, which not only increases the implementation complexity but also makes it difficult to strictly guarantee the optimality and stability of the filter; (2) The noise correction terms learned by neural networks are mostly based on static estimation of historical data, which makes it difficult to ensure the optimality in the current environment, and the real-time performance and global optimality are insufficient.

[0025] Based on this, this invention proposes a BeiDou navigation method based on optimal filtering strategy learning, which integrates Kalman filtering with deep reinforcement learning. Unlike methods that directly estimate noise statistics, this method introduces a reinforcement learning agent based on positioning error feedback, capable of dynamically sensing environmental changes and adaptively optimizing the Kalman gain, thus achieving online adjustment of the filtering strategy. Compared to traditional schemes relying on covariance constraints, this embodiment does not require forced positive definite projection, and through real-time interaction between the agent and the environment, it can directly learn and obtain the current optimal state fusion strategy in complex urban environments with severe interference and low signal-to-noise ratios, thereby significantly improving the system's accuracy and robustness.

[0026] Reference Figure 1 This invention provides a BeiDou satellite positioning method based on an environment-interactive Kalman filter network, which includes the following steps: S100. The parameters of the Kalman filter based on the mathematical model are initialized, and satellite positioning is performed in combination with Kalman gain optimization to obtain the current positioning result of the UAV. Specifically, a pseudorange measurement equation is constructed based on pseudorange measurements from the BeiDou Navigation Satellite System. This equation is then solved using the weighted least squares method to obtain the coarse positioning result of the UAV, which is used as the observation value for Kalman filtering to construct the measurement equation. The vehicle speed is obtained by constructing a pseudorange rate measurement equation and used as the control variable to construct the state equation. Combining the measurement equation and the state equation, a Kalman filter based on a mathematical model is constructed. The parameters of the Kalman filter based on the mathematical model are initialized, including the process noise covariance matrix, observation noise covariance matrix, prediction error covariance matrix, and innovation error covariance matrix, resulting in an initialized Kalman filter. Based on the initialized Kalman filter and the measurement equation of the Kalman filter, the weight allocation between prediction and measurement is measured, and the Kalman gain is calculated. The Kalman gain is corrected using a reinforcement learning model, and the state is corrected based on the corrected Kalman gain. The optimal state estimate is calculated, the covariance matrix is ​​updated to reflect the uncertainty of the state estimate, and the final positioning solution is output as the UAV's current positioning result.

[0027] Before describing this embodiment, it should be noted that in existing satellite positioning systems, core algorithms (such as Kalman filtering) rely on state-space models to dynamically estimate target motion. The positioning system model strictly follows the discrete-time state-space model (SSM) framework, which can be specifically described in the positioning scenario as follows: 1) State equations (system evolution model):

[0028] State vector This includes the receiver's dynamic parameters (such as three-dimensional position). ,speed ), The current moment; that is .

[0029] Transfer function Describing the kinematics of the target, this method uses a constant velocity (CV) model: , It is a unit array.

[0030] Process noise : Covariance representing disturbances not covered by the model (such as sudden maneuvers, wind resistance). The accuracy of the prediction directly affects the confidence level of the state prediction.

[0031] 2) Observation equation (measurement model):

[0032] Observation vector Composed of the receiver's raw observations (such as pseudorange) carrier phase (or view distance vector), that is ( (Number of visible satellites).

[0033] Observation function : The state vector Mapped to predicted measurements Since the state vector in this method only uses the position of the receiver, the H matrix is ​​defined as the identity matrix.

[0034] Observation noise The typical model uses Gaussian white noise, with covariance... Based on the predefined satellite elevation angle.

[0035] In other words, the filtering problem is the core of real-time tracking: in this embodiment of the invention, on the basis of real-time tracking, it is necessary to filter according to each set of observations. supply Instantaneous estimation. The main focus of this method is that there is some known information about the SSM. Specifically, for the state transition function... and state observation function There is a rough approximation, but the noise statistics Q and R are unknown.

[0036] S200, construct a multi-dimensional Kalman gain satellite positioning environment, set the Kalman gain correction action space, set the reward function, and combine the current positioning results of the UAV to construct a deep reinforcement learning Kalman filter network; Specifically, pseudorange residuals, three-dimensional line-of-sight vectors, carrier-to-noise ratios (CNR), and elevation angles of different satellites are used as satellite observation features. The pseudorange residuals represent the prediction error of the initial position estimation, the three-dimensional line-of-sight vectors represent the coarse directional information from the satellite to the receiver, the CNR represents the current signal quality, and the elevation angle represents the angle between the receiver and the satellite in the perpendicular direction. Based on the historical position constraints of the UAV positioning system, a historical position sequence is defined. Combining the satellite observation features, a robust Kalman gain reinforcement learning observation space is constructed. Combining the actions of adjusting the historical position and adjusting the Kalman gain matrix, a Kalman gain correction action space is constructed. A correction advantage error reward function is defined, and a reinforcement learning reward function for correcting the Kalman gain is constructed. Combining the current UAV positioning result, the robust Kalman gain reinforcement learning observation space, the Kalman gain correction action space, and the reinforcement learning reward function for correcting the Kalman gain, a deep reinforcement learning Kalman filter network is constructed.

[0037] In this embodiment, a pseudorange measurement equation is first constructed based on the pseudorange measurement of BDS, and the coarse positioning result is obtained by solving the equation using the weighted least squares method as the observation value of KF and a measurement equation is constructed. The vehicle speed is obtained by constructing the pseudorange rate measurement equation as the control variable and a state equation is constructed. Then, initialize the prior process noise covariance Q, the measurement noise covariance R, the prediction error covariance matrix P, and the innovation error covariance matrix S; The Kalman gain KG is further calculated, and then the Kalman gain is corrected by the Kalman gain optimization matrix output by reinforcement learning. Finally, the state is corrected using the corrected Kalman gain matrix and the final localization result is output.

[0038] First, combining the SSM mentioned above, we define KF in The state equation and measurement equation for vehicle positioning at time t are as follows:

[0039]

[0040] in, and For KF in Time and The predicted state at any given moment, i.e., the drone's location; For KF in The control quantity at any given time, i.e., the speed of the positioning system; The state transition matrix is ​​defined here as the identity matrix; To represent noise in a random process, we assume it follows a distribution. ,in Represents the process noise covariance matrix; This represents random measurement noise, assumed to follow a distribution. , Represents the measurement noise covariance matrix; In order to be in The measured value at time, i.e., the coarse positioning result calculated by the benchmark method; The transformation matrix from state to measurement is represented here and is defined as the identity matrix.

[0041] Secondly, in the prediction phase, the position at the current time is predicted based on the position at the previous time step using a dynamic model, i.e., the state equation. Then, the prior estimated covariance matrix is ​​calculated, i.e.:

[0042]

[0043] in, express The posterior estimate of the covariance at time t. express The prior estimate of covariance at time t, time It is initialized as a unit diagonal matrix; in time Initialized as the speed control quantity calculated using weighted least squares method The covariance matrix, initialization of the KF state. This indicates the location where KF solution is used.

[0044] Finally, in the update phase, upon receiving the measured values... Then, the KF state is updated to calculate the Kalman gain at the current time step. The Kalman gain matrix is ​​then adjusted based on the output action of the reinforcement learning model, and the posterior estimation error covariance is calculated, as shown below: 1) Calculate the Kalman gain The weights used to measure prediction and measurement: .

[0045] 2) Correct Kalman gain , To enhance learning The constant-time correction action to the Kalman gain matrix.

[0046] 3) Calculate the optimal estimated state of the system Through Kalman gain The optimal state estimate for the current moment is calculated by combining the predicted state and the measurement data with weighted calculations. .

[0047] 4) Covariance matrix Update: .

[0048] 5) The final location solution is as follows: .

[0049] The S300 trains a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, thereby enabling the Kalman filter network to achieve BeiDou satellite positioning.

[0050] Specifically, a deep reinforcement learning agent model is constructed; the deep reinforcement learning agent model interacts with a deep reinforcement learning Kalman filter network to obtain an experience tuple consisting of observation state, executed action, immediate reward, and transition state; based on the experience tuple, by randomly sampling batches of experience data and using temporal difference error as the core training signal, the agent network policy function and evaluator network value function in the deep reinforcement learning agent model are optimized to obtain the optimized evaluator network and optimized agent network; the agent is trained using the PPO algorithm, the optimized evaluator network estimates the value of the correction action, and the optimized agent network optimizes the Kalman gain correction strategy through the PPO-Clip objective function to obtain the optimal filtering strategy in a dynamic and complex urban environment, thus realizing the Kalman filter network for BeiDou satellite positioning.

[0051] In this embodiment, before constructing the environmental features of the intelligent agent, it is necessary to obtain the measurement values ​​from the BDS satellite and perform coarse positioning using the weighted least squares method. The specific process is as follows: First, in At any given time, the receiver receives measurement signals from multiple satellites, obtaining a set of pseudorange observations. At the same time, the position coordinates of each satellite are obtained based on the broadcast ephemeris. ,in First, the total number of currently visible satellites is determined. Second, a pseudorange observation equation is constructed based on pseudorange observations and satellite coordinates. Finally, the weighted least squares method is used to solve the receiver's... Rough position coordinates of time .

[0052] Robust Kalman gain reinforcement learning for environment localization: In reinforcement learning models, the environment is typically modeled using Markov Decision Processes (MDPs). Since environmental information is not fully available during UAV flight, the Kalman gain adaptive correction problem is modeled as a Partially Observable Markov Decision Process (POMDP), which can be represented by a set of symbols. To describe. Among them It is a set of states; It is the set of actions of an intelligent agent; It is a set of conditional transition probabilities between states; It is a reward function; It is the set of all observations; , Indicates the state Next action Observed The probability of, i.e., the conditional observation probability. , It is a discount factor. The goal of reinforcement learning is to learn an optimal policy. , making the trajectory Maximizing the expected value of the cumulative discount rewards obtained, i.e. ,in It expresses expectation.

[0053] To comprehensively characterize the current state of the receiver, we used BeiDou satellite observation features. Historical location sequence To form a robust Kalman gain reinforcement learning observation space, i.e. Since adaptive learning of Kalman gain essentially involves the Kalman filter learning the optimal filtering strategy under a partially known state-space model.

[0054] First, define satellite observation characteristics. To better characterize the current environment surrounding smartphones, embodiments of this invention employ different satellites. pseudo-range residual 3D view distance vector Carrier-to-noise ratio Altitude angle As satellite observation characteristics, pseudorange residuals represent the prediction error of the initial position estimate, the three-dimensional line-of-sight vector represents the coarse directional information from the satellite to the receiver, the carrier-to-noise ratio describes the quality of the current signal, and the elevation angle represents the angle between the receiver and the satellite in the perpendicular direction. Therefore, in Satellite observation characteristics at any given time It can be represented as:

[0055] Among them, satellite exist time Line of sight vector With pseudorange residual The calculation method is as follows:

[0056] Secondly, define the historical location sequence. It represents the historical location constraints of the UAV positioning system, including... A number of historical positions that have been corrected by the model (of which) (for model correction values) and the current coarse bit Placement. Therefore, in Time-based historical location sequence It can be represented as:

[0057] For the Kalman gain correction action space, this embodiment of the invention defines the action as a 12-dimensional continuous vector, i.e. This action can be further divided into two parts: ,in This indicates an action to adjust the historical position. This indicates the action of adjusting the Kalman gain matrix; both are in Execute at all times. Action. Each element in The mean value output by the actor network and variance And generated through Gaussian distribution sampling, i.e. Furthermore, that is:

[0058]

[0059] Further refine the reinforcement learning reward function of the Kalman gain to enable adaptive quantization adjustment of the Kalman gain. To improve positioning accuracy, this method defines a corrected advantage error reward function, which represents the ratio of the improvement in positioning accuracy obtained by the proposed method compared to the baseline method, and is defined as follows:

[0060] in, The position of the Kalman filter output in the baseline method, whose noise covariance matrix Q / R is a fixed, manually preset value, results in a Kalman gain of... It cannot dynamically respond to environmental changes (such as sudden signal changes in urban canyons); The actual location, which can be obtained as a reference, can be obtained using a high-precision inertial navigation and GNSS combined positioning system; To enhance learning, the intelligent agent generates corrective actions through real-time interaction with the environment. Then, the optimal position of the Kalman filter output is achieved by adaptively adjusting the Kalman gain.

[0061] Furthermore, such as Figure 3As shown, the deep reinforcement learning agent model is constructed. The deep reinforcement learning model constructed in this embodiment of the invention consists of an agent. The agent consists of three parts: Kalman gain matrix tuning and localization adjustment. 1) The observation feature extractor integrates the attention module and the LSTM module: the attention module focuses on key environmental perception information by adaptively learning the dynamic weights of different feature dimensions (such as pseudorange residuals and carrier-to-noise ratio); the LSTM module extracts long-term and short-term dependencies from continuous historical observation sequences to capture key dynamic features such as changes in satellite signal quality and user motion state migration.

[0062] 2) Actor Network: Samples and outputs actions from the action space, setting up continuous actions that follow a Gaussian distribution, divided into three directions: X, Y, and Z. , , Correction action The actions taken on each axis are limited by the maximum absolute value M.

[0063] 3) Evaluator network: Predicts the value of the current observed state to guide the training of the actor network.

[0064] Finally, as Figure 4 As shown, the deep reinforcement learning agent model is trained. This embodiment of the invention employs an online reinforcement learning training paradigm based on an experience replay mechanism: First, a robust Kalman gain reinforcement learning positioning environment is constructed using a pre-collected historical satellite positioning dataset; then, an experience replay pool is initialized, and the agent interacts with the environment to update its observation state. Execution of actions Instant rewards and transition status Constructed empirical tuples Continuously write to the pool until saturation; then, use randomly sampled batches of empirical data to calculate the time-series difference error. This serves as the core training signal, while simultaneously optimizing the actor network policy function. With the evaluator network value function .

[0065] During the training phase, the model is trained on an offline platform using collected location data.

[0066] Data preparation: Raw satellite observation data was acquired via receiver, and the baseline positioning results were calculated based on weighted least squares method and traditional Kalman filtering; simultaneously, a high-precision INS / GNSS combined system was used to obtain the true position value. Used for reward calculation; Interactive sampling: Initialize the experience replay pool The agent is in the observation state Below, by the Action Network Generate correction actions The predicted position is output after adaptive Kalman filtering. Calculate instant rewards Store experience tuples to ; Strategy optimization: When Once the capacity threshold is reached, the PPO algorithm is executed for updates. To balance bias and variance and improve learning stability and efficiency, we use generalized dominance estimation. (Generalized advantage estimation, GAE) is used to calculate policy advantage.

[0067] Therefore, this invention provides an interpretable and low-complexity method for Kalman gain optimization using deep reinforcement learning and designs a deep reinforcement learning Kalman Filtering Network (KFNet) with a self-attention mechanism to improve the accuracy of satellite positioning in complex and dynamic urban environments. First, SSM modeling is performed on the UAV positioning system to construct a Kalman filter based on the mathematical model. Second, a deep reinforcement learning Kalman filter network is constructed, specifically including constructing a multi-dimensional Kalman gain satellite positioning environment, setting the Kalman gain correction action space, setting the reward function, and designing an adaptive filtering satellite positioning process. Third, a reinforcement learning agent model is constructed, where the agent interacts in the multi-dimensional Kalman gain environment to optimize the Kalman gain matrix. Finally, the model is trained using the PPO algorithm to obtain the optimal filtering strategy in dynamic and complex urban environments.

[0068] In summary, this invention proposes a Kalman filter network-based BeiDou satellite positioning method that interacts with the environment. It combines Kalman filtering (KF) with deep reinforcement learning (DRL) to achieve real-time adaptive adjustment of the Kalman gain. Unlike traditional methods that rely on precise modeling, this method utilizes data-driven reinforcement learning to adaptively learn and optimize the Kalman gain from the environment through dynamic interaction. This avoids increased positioning errors and instability caused by inaccurate noise modeling, and eliminates the need for additional constraints to ensure positive definiteness. It aims to learn the optimal filtering strategy for the current state in the dynamic and complex urban flight environment of UAVs, achieving accurate and stable BeiDou satellite positioning for low-altitude UAVs.

[0069] Finally, combining Figure 5 Explanation: 1) Establishment of the state space model of the positioning system and definition of challenges.

[0070] Based on the discrete-time state-space model (SSM) framework, the core structure of the UAV positioning system is defined, including the state equation describing the receiver's motion and the observation equation connecting the state with satellite measurements. However, the challenge lies in the fact that while the state transition function and observation function can be approximated, the process noise covariance (Q) and measurement noise covariance (R) are unknown and dynamically changing. Due to the uncertainty of Q and R, traditional Kalman filters cannot achieve accurate positioning estimation in complex dynamic environments. Therefore, this scheme does not rely on precise noise parameter estimation but instead focuses on a more direct objective: correcting the Kalman gain through reinforcement learning.

[0071] 2) Adaptive optimal Kalman gain based on reinforcement learning.

[0072] Given the unknown and dynamic nature of Q and R, the goal of this step is to skip noise parameter estimation and instead learn the optimal filtering strategy dynamically from the environment, i.e., real-time adjustment of the Kalman gain. The agent's input includes satellite observation features, historical location sequences, etc. Through the adaptive mechanism of reinforcement learning, the agent can directly adjust the KG to cope with dynamic noise, thereby optimizing the fusion of prediction and observation information and improving positioning accuracy without knowing the precise noise statistics. This achieves a strategy shift from "estimating noise" to "optimizing gain."

[0073] 3) DRL environment construction and experience data collection.

[0074] The reinforcement learning environment is constructed using measured road data collected by a drone receiver, including raw satellite observations and INS / GNSS ground truth. During this process, the agent interacts with the environment in real time: 1. The feature extractor processes the observation status; 2. The actor network outputs a Kalman gain correction; 3. An optimized position result is calculated using an adaptive Kalman filter; 4. A reward signal is generated based on the actual positioning error. This interactive data is stored as empirical data in an experience replay pool, providing support for subsequent policy training.

[0075] 4) Kalman gain correction training under the PPO framework.

[0076] When sufficient experience pool data is available, the agent is trained using a proximal policy optimization (PPO) framework. The evaluator network is responsible for estimating the value of corrective actions, while the actor network optimizes the corrective policy with Kalman gain through the PPO-Clip objective function. The training process aims to minimize the evaluator's value prediction error (MSRE loss) and maximize the actor's policy advantage through generalized advantage estimation (GAE). This training method ensures that the agent can improve its robustness and accuracy in complex dynamic environments while optimizing the KG.

[0077] 5) Real-time fusion reasoning of reinforcement learning and KF.

[0078] In the practical application (inference) phase, the trained reinforcement learning model participates in the real-time correction of the Kalman filter. The agent dynamically outputs and updates the Kalman gain based on real-time observation data, combined with the system's state transition matrix and observation matrix. This DRL-KF fusion strategy enables the filter to adaptively adjust its weights according to the current environmental state, minimizing prediction errors and enhancing the robustness of the UAV positioning system in non-stationary and uncertain environments.

[0079] Therefore, the embodiments of the present invention differ from the prior art in the following technical aspects: 1) This invention is the first in the field of satellite navigation to utilize DRL (Device Restriction Learning) to dynamically correct Kalman gain, differing from traditional methods that directly fit gain or residual mappings using neural networks. This embodiment transforms the SSM (Sound-Side Filter) failure problem caused by noise non-stationarity in complex urban environments into a partially observable Markov decision process for modeling. A reinforcement learning agent is introduced to continuously interact with the environment during the training phase, dynamically optimizing the state fusion strategy. This method no longer relies on a fixed gain mapping relationship under static learning, but instead obtains the optimal filter gain adjustment strategy under different environments through policy learning, effectively improving the response capability to dynamic disturbances and the overall robustness of the system.

[0080] 2) Unlike existing model parameter adjustment methods based on neural networks to correct the noise covariance matrix (Q / R), this paper skips noise statistics modeling through reinforcement learning and directly learns the optimal filtering strategy for the filter under specific environments. In traditional methods, the output Q / R of the neural network must satisfy symmetric positive definiteness constraints, often requiring the introduction of complex parameterization structures or projection operations to ensure filter stability. Furthermore, the correction results usually rely on historical data fitting and are not optimal estimates for the current environment, making it difficult to cope with frequently changing non-Gaussian and non-stationary noise interference in urban navigation. In contrast, this invention does not require constructing a noise prior model. Instead, it relies on strategy optimization to adaptively adjust the Kalman filter fusion strategy in a high-dimensional observation feature space, significantly enhancing the filter's real-time performance, stability, and generalization ability in practical applications.

[0081] The embodiments of the present invention have the following advantages over the prior art: 1) Existing model-based satellite positioning methods (such as weighted least squares and Kalman filtering) rely on prior modeling of system dynamics and noise characteristics, typically requiring extensive experimentation and repeated adjustments to determine suitable hyperparameters. This highly knowledge-dependent approach is prone to failure in dynamic and complex environments such as urban canyons and overpasses. To address this issue, this invention introduces a deep reinforcement learning mechanism to achieve adaptive dynamic adjustment of the Kalman gain matrix—a key intermediate variable in the Kalman filter—without relying on manual experience or static parameter design, thereby improving the model's robustness and generalization ability in non-ideal environments.

[0082] 2) Most common data-driven localization methods currently employ a cascaded architecture of "model output + neural network correction," whereby a deep learning model is used as a post-processing module to correct the results of traditional algorithms. This approach decouples the optimization process, preventing the formation of a joint optimum, and the localization accuracy depends on the baseline performance of the initial model. Furthermore, these methods are often based on empirical residual learning, failing to model the non-Gaussian and non-stationary characteristics of the error's nature—noise. Consequently, they lack precise modeling and suppression of error sources, limiting the applicability of the algorithms in complex environments.

[0083] 3) Some studies have attempted to introduce reinforcement learning to improve Kalman filter performance, but most methods only adjust the process noise covariance (Q) while ignoring the changes in the observation noise covariance (R) in dynamic environments. Since KF performance essentially depends on the weight balance of Q and R, ignoring the changes in R will directly affect the gain optimality. In contrast, this invention skips the modeling and estimation of Q / R and directly learns the optimal Kalman gain strategy. It adapts to various non-ideal noise scenarios without directly estimating the covariance, significantly improving the response capability and filtering accuracy to random noise interference in complex dynamic urban environments.

[0084] Reference Figure 2 The BeiDou satellite positioning system based on environmental interaction Kalman filter networks includes: The first module 201 is used to initialize the parameters of the Kalman filter based on the mathematical model and combine it with Kalman gain optimization for satellite positioning to obtain the current positioning result of the UAV. The second module 202 is used to construct a multi-dimensional Kalman gain satellite positioning environment, set the Kalman gain correction action space, set the reward function, and combine the current positioning results of the UAV to construct a deep reinforcement learning Kalman filter network. The third module 203 is used to train a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, and to realize the BeiDou satellite positioning of the Kalman filter network.

[0085] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0086] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A BeiDou satellite positioning method based on environmental interaction using a Kalman filter network, characterized in that, Includes the following steps: The parameters of the Kalman filter based on the mathematical model are initialized, and satellite positioning is performed by combining Kalman gain optimization to obtain the current positioning result of the UAV. A multi-dimensional Kalman gain satellite positioning environment is constructed, a Kalman gain correction action space is set, a reward function is set, and a deep reinforcement learning Kalman filter network is constructed by combining the current positioning results of the UAV. By training a deep reinforcement learning Kalman filter network using the PPO algorithm, the optimal filtering strategy in a dynamic and complex urban environment is obtained, enabling the Kalman filter network to achieve BeiDou satellite positioning.

2. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 1, characterized in that, The step of initializing the parameters of the Kalman filter based on the mathematical model and performing satellite positioning in conjunction with Kalman gain optimization to obtain the current positioning result of the UAV specifically includes: Based on the pseudorange measurement of the BeiDou satellite navigation system, a pseudorange measurement equation is constructed, and the pseudorange measurement equation is solved by the weighted least squares method to obtain the coarse positioning result of the UAV, which is then used as the observation value for Kalman filtering to construct the measurement equation. The vehicle speed is obtained by constructing a pseudo-distance rate measurement equation and used as a control variable to construct a state equation. By combining the measurement equation and the state equation, a Kalman filter based on a mathematical model is constructed. The parameters of the Kalman filter based on the mathematical model are initialized. The initialization process includes the noise covariance matrix, observation noise covariance matrix, prediction error covariance matrix, and innovation error covariance matrix, resulting in the initialized Kalman filter. Based on the initialized Kalman filter and the measurement equation of the Kalman filter, the weight allocation of prediction and measurement is measured, and the Kalman gain is calculated. The Kalman gain is corrected by a reinforcement learning model, and the state is corrected based on the corrected Kalman gain. The optimal state estimate is calculated, the covariance matrix is ​​updated to reflect the uncertainty of the state estimate, and the final positioning solution is output as the UAV's current positioning result.

3. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 2, characterized in that, The step of constructing a multi-dimensional Kalman gain satellite positioning environment, setting the Kalman gain correction action space, setting the reward function, and constructing a deep reinforcement learning Kalman filter network by combining the current positioning results of the UAV specifically includes: The pseudorange residuals, three-dimensional line-of-sight vectors, carrier-to-noise ratios, and elevation angles of different satellites are used as satellite observation features. The pseudorange residuals represent the prediction error of the initial position estimation, the three-dimensional line-of-sight vectors represent the coarse directional information from the satellite to the receiver, the carrier-to-noise ratios represent the current signal quality, and the elevation angles represent the angles between the receiver and the satellite in the vertical direction. Based on the historical position constraints of the UAV positioning system, a historical position sequence is defined, and a robust Kalman gain reinforcement learning observation space is constructed by combining satellite observation characteristics. By combining the actions of adjusting historical positions with the actions of adjusting the Kalman gain matrix, a Kalman gain correction action space is constructed. Define a reward function to correct the advantage error and construct a reinforcement learning reward function to correct the Kalman gain; By combining the current location results of the UAV, the robust Kalman gain reinforcement learning observation space, the Kalman gain correction action space, and the reinforcement learning reward function of the corrected Kalman gain, a deep reinforcement learning Kalman filter network is constructed.

4. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 3, characterized in that, The specific expression for the Kalman gain correction action space is as follows: ； ； In the above formula, express The position of the agent after correction at any given moment. express The approximate location of the moment, express The action of constantly adjusting the historical position. express The Kalman gain matrix adjusted by the agent at each time step. express The Kalman gain matrix calculated from the covariance matrix at each time step. express The action of adjusting the Kalman gain matrix at any time.

5. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 4, characterized in that, The expression for the corrected advantage error reward function is as follows: ； In the above formula, This represents the reward function for correcting the advantage error. Indicates the position of the Kalman filter output of the benchmark method. Indicates the actual location being referenced. This indicates the optimal position of the Kalman filter output after the reinforcement learning agent generates corrective actions through real-time interaction with the environment and adaptively adjusts the Kalman gain.

6. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 5, characterized in that, The step of training a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, and realizing the BeiDou satellite positioning of the Kalman filter network, specifically includes: Constructing a deep reinforcement learning agent model; The deep reinforcement learning agent model interacts with the deep reinforcement learning Kalman filter network to obtain an experience tuple consisting of the observation state, the action to be performed, the immediate reward, and the transition state. Based on the empirical tuple, by randomly sampling batches of empirical data and using the temporal difference error as the core training signal, the policy function of the actor network and the value function of the evaluator network in the deep reinforcement learning agent model are optimized to obtain the optimized evaluator network and the optimized actor network. The agent is trained using the PPO algorithm. The optimized evaluator network estimates the value of the correction action, and the optimized actor network optimizes the Kalman gain correction strategy through the PPO-Clip objective function. This yields the optimal filtering strategy in a dynamic and complex urban environment, enabling the Kalman filter network to achieve BeiDou satellite positioning.

7. The BeiDou satellite positioning method based on environmental interaction using a Kalman filter network according to claim 6, characterized in that, The process involves training the agent using the PPO algorithm, optimizing the evaluator network to estimate the value of corrective actions, and then optimizing the Kalman gain correction strategy using the PPO-Clip objective function to obtain the optimal filtering strategy in a dynamic and complex urban environment. This step, which implements the Kalman filter network for BeiDou satellite positioning, specifically includes: The system collects raw satellite observation data through a receiver, and calculates the reference positioning results based on the weighted least squares method and traditional Kalman filtering. Simultaneously, it uses a high-precision INS / GNSS combined system to obtain the true position value for reward calculation. Under observation, the agent generates corrective actions through an optimized actor network, outputs the predicted position through an adaptive Kalman filter, calculates the immediate reward, and initializes the experience replay pool. Once the experience replay pool reaches its capacity threshold, the PPO algorithm is executed for updating. The strategy advantage is estimated through generalized dominance to obtain the optimal filtering strategy in a dynamic and complex urban environment, thus realizing the BeiDou satellite positioning of the Kalman filter network.

8. A BeiDou satellite positioning system based on an environment-interactive Kalman filter network, characterized in that, Includes the following modules: The first module is used to initialize the parameters of the Kalman filter based on the mathematical model and combine it with Kalman gain optimization for satellite positioning to obtain the current positioning result of the UAV. The second module is used to construct a multi-dimensional Kalman gain satellite positioning environment, set the Kalman gain correction action space, set the reward function, and combine the current positioning results of the UAV to construct a deep reinforcement learning Kalman filter network. The third module is used to train a deep reinforcement learning Kalman filter network using the PPO algorithm to obtain the optimal filtering strategy in a dynamic and complex urban environment, and to realize the Kalman filter network for BeiDou satellite positioning.

Images