A GNSS / INS integrated navigation system and method based on reinforcement learning

By constructing a GNSS/INS integrated navigation system based on reinforcement learning, and utilizing a dataset from GNSS-visible scenarios and a reinforcement learning model, the problem of inertial navigation accuracy divergence in GNSS-denied scenarios was solved, achieving high-precision and reliable navigation output from the INS system.

CN118443011BActive Publication Date: 2024-11-26SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410615379.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-17
Publication Date
2024-11-26
Estimated Expiration
2044-05-17

AI Technical Summary

Technical Problem

The accuracy divergence problem of inertial navigation systems in GNSS single positioning and navigation systems under GNSS denial scenarios has not been effectively solved, affecting the robustness and reliability of the navigation system.

Method used

A GNSS/INS integrated navigation system based on reinforcement learning is adopted. An adaptive reinforcement learning network model is constructed using effective datasets in GNSS visible scenarios and reinforcement learning modes to preprocess inertial sensor data, reduce the accuracy divergence rate of pure inertial navigation, and use RL-assisted inertial navigation results as navigation output when GNSS signals are not visible.

Benefits of technology

It improves the long-endurance reliability and navigation accuracy of low-cost INS systems, ensuring high accuracy and reliability of navigation systems in GNSS denied scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118443011B_ABST
    Figure CN118443011B_ABST
Patent Text Reader

Abstract

The present invention discloses a GNSS / INS integrated navigation system based on reinforcement learning, which relates to the fields of machine learning, satellite, and inertial navigation terminal technology, and includes three inertial dead reckoning modules, a GNSS / INS integrated navigation module, and a reinforcement learning module. The system working scenarios are mainly divided into GNSS visible and denied scenarios. In the satellite visible scenario, the position, speed, attitude, and time calculated by the GNSS / INS integrated navigation system are used as the final output results of the system. At the same time, the reinforcement learning module uses the output of the integrated navigation system, the inertial dead reckoning output, and the original inertial device measurement output for training and learning to achieve the best reward. In the GNSS denied scenario, the reinforcement learning module starts to work formally, and the preprocessing of the inertial original observation data is realized through the trained state / action strategy. The present invention also discloses a GNSS / INS integrated navigation method based on reinforcement learning, which realizes the long-flight reliable accuracy of the GNSS / INS integrated navigation system in a comprehensive scenario in an online reinforcement learning mode.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of satellite and inertial navigation terminals, and in particular to a GNSS / INS integrated navigation system and method based on reinforcement learning. Background Art

[0002] With the rapid development of research fields such as smart cities and precision navigation, the industry standards and requirements for location-based services (LBS) are becoming increasingly higher. As an absolute positioning system based on satellite radio navigation, the Global Navigation Satellite System (GNSS) can provide global users with all-weather, high-precision location, speed, time and other important information, and the positioning error will not accumulate over time. Therefore, it is widely used in vehicle navigation, surveying and mapping, geological exploration, target tracking, precision agriculture, military and other fields. Due to the inherent fragility of GNSS, the performance of a single positioning and navigation system based on GNSS will be greatly reduced, and even the positioning service cannot be completed. In order to improve the robustness, accuracy and reliability of the GNSS-based positioning and navigation system, multi-faceted fusion enhancement technology has also been proposed.

[0003] In recent years, in response to challenging environments (such as urban canyons, interference, etc.), positioning and navigation technologies in different fields have also been strengthened accordingly. Multi-source (all-source) positioning and navigation systems have also become a preferred solution, and the integration of different sensor system levels has gradually matured. However, in GNSS denial scenarios, the accuracy divergence of inertial navigation systems (INS) is still a challenging problem. With the rapid development of machine learning (ML) technology, ML has also been tried in the field of GNSS navigation, and has achieved certain results in related sub-directions. In general, ML has great advantages in nonlinear system processing. It does not require physical model support. Through training and learning of data samples, it can obtain the potential mechanism "black box" model of the target system. Neural networks can not only approximate the value of a function, but also approximate the derivatives of different series of the function. The development of ML has certain research in GDOP fitting approximation, attitude estimation, GNSS interference identification, and GNSS navigation system related state estimation, but the problem of INS system accuracy divergence is still in the exploration stage. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a GNSS / INS integrated navigation system and method based on reinforcement learning. The present invention obtains an adaptive reinforcement learning network model through an effective data set in a GNSS visible scene and a reinforcement learning RL model, and implements preprocessing of inertial sensor data IMU through an online state / action strategy, thereby reducing the accuracy divergence speed of pure inertial navigation and greatly improving the long-flight reliability of a low-cost INS system.

[0005] The present invention adopts the following technical solutions to solve the above technical problems:

[0006] A GNSS / INS integrated navigation system based on reinforcement learning proposed in the present invention includes first to third calculation branches, wherein:

[0007] The first computing branch includes a first inertial dead reckoning module and a GNSS / INS combined system module;

[0008] The second computation branch includes a second inertial dead reckoning module;

[0009] The third computing branch includes a reinforcement learning RL module and a third inertial dead reckoning module;

[0010] The six-axis observation data enters the first to third calculation branches for parallel processing, where:

[0011] In the first calculation branch, the six-axis observation data is subjected to inertial dead reckoning and then combined with GNSS / INS to obtain the first calculation branch result PVA1, which includes position, velocity and attitude estimation results;

[0012] In the second calculation branch, the six-axis observation data is processed by the inertial dead reckoning algorithm to obtain the second calculation branch result PVA2, which includes the pure dead reckoning position, velocity and attitude estimation results;

[0013] In the third calculation branch, the six-axis observation data is processed by the RL agent and the inertial dead reckoning algorithm to obtain the third calculation branch result PVA3, which includes the position, velocity and attitude estimation results;

[0014] The first calculation branch result PVA1, the second calculation branch result PVA2 and the third calculation branch result PVA3 are then used to reward the RL agent through the reward calculation unit in the reinforcement learning RL module;

[0015] When GNSS signals are visible, PVA1 is used as navigation output;

[0016] When the GNSS signal is not visible, if the reinforcement learning RL module converges, PVA3 is used as the navigation output. If the reinforcement learning RL module does not converge, PVA2 is used as the navigation output.

[0017] As a further optimization scheme of a GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, a reinforcement learning RL module includes an RL agent, a reward calculation unit, an action strategy unit and a state calculation unit.

[0018] As a further optimization scheme of a GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, the inherent dead reckoning error model information of the inertial navigation system is used for constraints in the reward calculation unit, the action strategy unit and the state calculation unit.

[0019] As a further optimization scheme of a GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, the inertial sensor device outputs six-axis observation data, and the six-axis observation data includes three-axis acceleration and three-axis angular velocity of the carrier coordinate system.

[0020] As a further optimization scheme of the GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, the first inertial dead reckoning module, the second inertial dead reckoning module, and the third inertial dead reckoning module respectively obtain the position, speed, and attitude estimation of the inertial navigation as follows:

[0021] A three-axis gyroscope is used to obtain the three-axis angular velocity of the carrier coordinate system, and a first error compensation is performed on the three-axis angular velocity. The first error compensation includes compensation for non-commutative errors, compensation for errors related to the rotation of the earth, compensation for carrier velocity errors, and compensation provided by a reinforcement learning (RL) module. The three-axis angular velocity after the first error compensation is integrated with the initial attitude information of the carrier to obtain an attitude estimate of the carrier.

[0022] A three-axis accelerometer is used to measure the three-axis acceleration of the carrier coordinate system, and a second error compensation is performed on the three-axis acceleration. The second error compensation includes rowing compensation, rotation compensation and compensation provided by the RL module. The information of the three-axis acceleration after the second error compensation is coordinate projected through the carrier attitude information. Coordinate projection refers to projecting the three-axis acceleration in the carrier coordinate system to the navigation coordinate system. In the navigation coordinate system, after Coriolis compensation, gravity compensation and double integration operations, the estimation of speed and position is obtained.

[0023] As a further optimization scheme of the GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, the combined filtering method in the GNSS / INS combined system module is as follows:

[0024] The position and velocity estimates obtained by the first inertial dead reckoning module are subtracted from the position and velocity provided by the satellite receiver, and the difference is used as measurement information to measure and update the combined Kalman filter, thereby estimating and updating the parameter errors of the inertial navigation system model;

[0025] X=[φ T (δv n ) T (δp n ) T (b g ) T (b a ) T (s g ) T (s a ) T ] T

[0026] Among them, X is the inertial navigation error state, X includes the misalignment angle φ, velocity error δv n 、Position deviation δp n , gyro bias error b g , accelerometer error b a , gyro scale factor error s g and accelerometer scale factor error s a , the state variable superscript n represents the navigation coordinate system, the subscript g represents the gyroscope, the subscript a represents the accelerometer, the superscript T represents the matrix transpose, and δv is an overall variable representing the velocity error.

[0027] As a further optimization scheme of the GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, in the reinforcement learning module, the state input of the RL agent is also calculated by the inertial sensor device measurement calculation, and the RL agent obtains the action according to the state and the trained state / action strategy network. The action is used to compensate the error of the three-axis gyroscope output and the three-axis accelerometer output data, thereby improving the accuracy and endurance of the subsequent inertial dead reckoning PVA3. The following configurations are adopted for reward calculation, state calculation, and RL action design:

[0028] a. Reward calculation

[0029] The reward is set as: Reward = K1·Reward1+K2·Reward2+K3·Reward3, where Reward1 = (|S 12 -S 13 |-K·|S 13 -S 11 |), Reward2=(|S 22 -S 23|-K·|S 23 -S 21 |), Reward3=(|S 32 -S 33 |-K·|S 33 -S 31 |); where S 11 , S 12 , S 13 Respectively represent the position estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch, S 21 , S 22 , S 23 Respectively represent the attitude estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch, S 31 , S 32 , S 33 Represent the velocity estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch respectively. Reward is the total reward of the RL module. Reward1 is the reward in the position domain, Reward2 is the reward in the velocity domain, Reward3 is the reward in the posture domain, K1 is the reward ratio of Reward1, K2 is the reward ratio of Reward2, K3 is the reward ratio of Reward3, and K is the reward ratio of the difference between the first calculation branch and the third calculation branch.

[0030] b. State calculation

[0031] The state configuration is: the difference between the six-axis data output by the inertial sensor at the current moment and the six-axis data output by the inertial sensor at the previous moment, including the difference between the three-axis gyroscope output data before and after, and the difference between the three-axis accelerometer output data before and after;

[0032] c.RL action design

[0033] The RL action configuration is: three-axis gyroscope output compensation and three-axis accelerometer output compensation.

[0034] As a further optimization scheme of the GNSS / INS integrated navigation system based on reinforcement learning described in the present invention, in the reinforcement learning module,

[0035] The first is the initialization of the entire GNSS / INS integrated navigation system, which includes the initial coordinates, attitude, and velocity of the inertial navigation system. In a scenario where the GNSS signal is visible, the GNSS / INS integrated navigation system outputs position, velocity, and attitude as the final navigation output of the system.

[0036] At the same time, the RL model is trained; during the RL model training process, two conditions need to be judged by polling. The first is to judge the convergence of the model training, and the second is to judge the training cycle. When the model training converges, the model available flag T is assigned a value of 1. When the training cycle is greater than the preset cycle threshold Tr, the inertial navigation system dead reckoning is reset and initialized to ensure the effectiveness of the RL model training.

[0037] In the scenario where the GNSS signal is not visible, the convergence sign of the reinforcement learning model is first determined. If T=1, the trained RL strategy network is directly used to compensate the three-axis gyroscope and three-axis accelerometer according to the measurement status of the inertial sensor to suppress the divergence of inertial navigation errors. The RL-assisted inertial navigation position calculation result PVA3 is used as the final navigation output. If the training model does not converge, PVA2 is used as the final navigation output.

[0038] A GNSS / INS integrated navigation method based on reinforcement learning, six-axis observation data enters the first to third calculation branches for parallel processing, where:

[0039] In the first calculation branch, the six-axis observation data is subjected to inertial dead reckoning and then combined with GNSS / INS to obtain the first calculation branch result PVA1, which includes position, velocity and attitude estimation results;

[0040] In the second calculation branch, the six-axis observation data is processed by the inertial dead reckoning algorithm to obtain the second calculation branch result PVA2, which includes the pure dead reckoning position, velocity and attitude estimation results;

[0041] In the third calculation branch, the six-axis observation data is processed by the RL agent and the inertial dead reckoning algorithm to obtain the third calculation branch result PVA3, which includes the position, velocity and attitude estimation results;

[0042] The first calculation branch result PVA1, the second calculation branch result PVA2 and the third calculation branch result PVA3 are then used to reward the RL agent through the reward calculation unit in the reinforcement learning RL module;

[0043] When GNSS signals are visible, PVA1 is used as navigation output;

[0044] When the GNSS signal is not visible, if the reinforcement learning RL module converges, PVA3 is used as the navigation output. If the reinforcement learning RL module does not converge, PVA2 is used as the navigation output.

[0045] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0046] (1) The present invention designs three types of outputs, among which the first type of output is when the GNSS signal is visible, and the GNSS / INS combined navigation result PVA1 is used as the final output. When the GNSS satellite is not visible, if the reinforcement learning model has not been trained, the output PVA2 is used as the final navigation result. If the reinforcement learning model has been trained, the output PVA3 is used as the final navigation result. As a whole, it ensures higher accuracy and reliability than the traditional GNSS / INS combined navigation system.

[0047] (2) An innovative reinforcement learning model is proposed, which meets both online and offline modes. The high-precision navigation results provided by GNSS / INS are used as reference to train the RL model. At the same time, based on the physical knowledge of the INS system, the main elements such as state quantity, reward function (Reward), and action are reasonably designed to obtain the optimal "state-action" strategy, thereby ensuring the accuracy and reliability of RL-assisted INS navigation in GNSS denial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a system structure diagram of the present invention.

[0049] Figure 2 This is a system status flow chart of the present invention.

[0050] Figure 3 This is the flow chart of inertial navigation dead reckoning. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The present invention includes three inertial dead reckoning modules, a GNSS / INS integrated navigation module and a reinforcement learning module; wherein,

[0053] The inertial dead reckoning module mainly uses the output of the inertial measurement unit (IMU), including three-axis acceleration and three-axis angular velocity, to perform related operations to obtain the inertial navigation position, velocity, and attitude estimation. Figure 3As shown, the three-axis gyroscope obtains the acceleration of the three axes of the carrier coordinate system, and first passes through the error compensation module, which mainly includes non-commutative errors, earth rotation-related errors, carrier velocity-related errors, and compensation provided by the RL module. After the error correction, it enters the integration link and directly obtains the attitude estimation result through the initial attitude information. At the same time, the three-axis accelerometer measures the three-axis acceleration of the carrier coordinate system and enters the error compensation, including rowing compensation, rotation compensation, and compensation provided by the RL module. After the measurement and compensation, the acceleration information is coordinate projected through the attitude information, and the carrier coordinate system measurement information is transferred to the navigation coordinate system. In the navigation coordinate system, Coriolis compensation and gravity compensation are further performed, and then a double integration operation is performed to obtain velocity and position estimation information respectively.

[0054]

[0055] in Represents the coordinate transformation matrix from the carrier coordinate system to the navigation coordinate system, Indicates the rotation angular velocity of the carrier coordinate system relative to the navigation coordinate system, is the Coriolis compensation term, g n is the gravity compensation term, is the accelerometer measurement, is the carrier speed, M pv is v in the navigation coordinate system n and the position vector derivative The correlation matrix between .

[0056] The GNSS / INS integrated navigation module mainly performs combined filtering based on the Kalman system on the INS navigation position, velocity estimation and satellite navigation side observations to improve the accuracy and reliability of the entire navigation system. Among them, the GNSS / INS combination method can adopt a conventional loose combination mode. In the loose combination mode, the position / velocity is mainly used as observation in the Kalman observation equation.

[0057] The reinforcement learning RL module includes an RL agent, a reward calculation unit, an action strategy unit, and a state calculation unit. The RL agent includes a deep learning network, which forms an action strategy through data training; a reward calculation unit, which performs reward calculations through PVA1, PVA2, and PVA3 to train the deep learning network; and a state calculation unit, which is used to calculate the current state compensation action strategy. The action strategy unit is obtained through training with PVA1, PVA2, and PVA3 to form a mapping relationship between state and action, namely, an action strategy.

[0058] The reinforcement learning module uses the reinforcement learning (RL) algorithm to suppress the error divergence of the autonomous dead reckoning of the INS system based on the error mechanism of the INS system. First, the target environment model of the RL is constructed according to the principle of the INS system. Figure 1 As shown, the IMU measurement data includes a three-axis gyroscope and a three-axis accelerometer, which pass through three branches respectively. The first branch directly enters the INS dead reckoning and then performs GNSS / INS combination to obtain the position, velocity, and attitude output result PVA1. The second branch enters the INS dead reckoning algorithm, which belongs to the pure dead reckoning position, velocity, and attitude output result PVA2. The third branch enters the dead reckoning module after being processed by the RL agent (RL action) to obtain the position, velocity, and attitude output result PVA3. The last three branches can all obtain PVA results. The three PVA results are used to reward the RL agent through the reward calculation module.

[0059] The other two important elements of the RL model include: state and action. Figure 1 As shown, the state input of the RL agent is also calculated by the IMU measurement calculation. At the same time, the RL obtains the RL action based on the state and the trained action strategy network. This action is used to compensate for the error of the IMU measurement data, thereby improving the accuracy and endurance of the subsequent INS dead reckoning PVA3. Based on the understanding of the INS error system, the following configuration is proposed:

[0060] a. Reward calculation

[0061] Rewards can be set as:

[0062] Reward=K1·Reward1+K2·Reward2+K3·Reward3, where Reward1=(|S 12 -S 13 |-K·|S 13 -S 11 |), Reward2=(|S 22 -S 23 |-K·|S 23 -S 21 |), Reward3=(|S 32 -S 33 |-K·|S 33 -S 31 |); where S 11 , S 12 , S 13 Respectively represent the estimated values ​​of the first branch, the second branch, and the third branch positions, S 21 , S 22 , S 23 Respectively represent the first branch, the second branch, and the third branch posture estimation value, S31 , S 32 , S 33 Represent the velocity estimation values ​​of the first branch, the second branch, and the third branch respectively. Reward is the total reward of the RL model. Reward1 is the reward in the position domain, Reward2 is the reward in the velocity domain, and Reward3 is the reward in the posture domain. K1 is the reward ratio of Reward1, K2 is the reward ratio of Reward2, K3 is the reward ratio of Reward3, and K is the reward ratio of the difference between the estimation calculated by branch one and branch three.

[0063] b. State calculation

[0064] The state configuration is: the difference between the six-axis data output by the inertial sensor at the current moment and the six-axis data output by the inertial sensor at the previous moment, including the difference between the three-axis gyroscope output data before and after, and the difference between the three-axis accelerometer output data before and after;

[0065] c.RL action design

[0066] The RL action configuration is: compensation for the three-axis gyroscope output and the three-axis accelerometer output.

[0067] The state flow of the system of the present invention is as follows Figure 2 As shown, the first step is to initialize the entire GNSS / INS integrated navigation system, including the initial coordinates, attitude, and velocity of the inertial navigation system. In a scenario where the GNSS signal is visible, the GNSS / INS integrated navigation system outputs position, velocity, and attitude as the final navigation output of the system.

[0068] At the same time, the RL model is trained; during the RL model training process, two conditions need to be judged by polling. The first is to judge the convergence of the model training, and the second is to judge the training cycle. When the model training converges, the model available flag T is assigned a value of 1. When the training cycle is greater than the preset cycle threshold Tr, the inertial navigation system dead reckoning is reset and initialized to ensure the effectiveness of the RL model training.

[0069] In the scenario where the GNSS signal is not visible, the convergence sign of the reinforcement learning model is first determined. If T=1, the trained RL strategy network is directly used to compensate the three-axis gyroscope and three-axis accelerometer according to the measurement status of the inertial sensor to suppress the divergence of inertial navigation errors. The RL-assisted inertial navigation position calculation result PVA3 is used as the final navigation output. If the training model does not converge, PVA2 is used as the final navigation output.

[0070] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A GNSS / INS integrated navigation system based on reinforcement learning, characterized in that: It includes first to third calculation branches, wherein, The first computing branch includes a first inertial dead reckoning module and a GNSS / INS combined system module; The second computation branch includes a second inertial dead reckoning module; The third computing branch includes a reinforcement learning RL module and a third inertial dead reckoning module; The six-axis observation data enters the first to third calculation branches for parallel processing, where: In the first calculation branch, the six-axis observation data is subjected to inertial dead reckoning and then combined with GNSS / INS to obtain the first calculation branch result PVA1, which includes position, velocity and attitude estimation results; In the second calculation branch, the six-axis observation data is processed by the inertial dead reckoning algorithm to obtain the second calculation branch result PVA2, which includes the pure dead reckoning position, velocity and attitude estimation results; In the third calculation branch, the six-axis observation data is processed by the RL agent and the inertial dead reckoning algorithm to obtain the third calculation branch result PVA3, which includes the position, velocity and attitude estimation results; The first calculation branch result PVA1, the second calculation branch result PVA2 and the third calculation branch result PVA3 are then used to reward the RL agent through the reward calculation unit in the reinforcement learning RL module; When GNSS signals are visible, PVA1 is used as navigation output; When the GNSS signal is not visible, if the reinforcement learning RL module converges, PVA3 is used as the navigation output. If the reinforcement learning RL module does not converge, PVA2 is used as the navigation output.

2. A GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning RL module includes an RL agent, a reward calculation unit, an action strategy unit, and a state calculation unit.

3. A GNSS / INS integrated navigation system based on reinforcement learning according to claim 2, characterized in that: In the reward calculation unit, action strategy unit and state calculation unit, the dead reckoning error model information inherent in the inertial navigation system is used for constraints.

4. The GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: The inertial sensor device outputs six-axis observation data, which includes three-axis acceleration and three-axis angular velocity of the carrier coordinate system.

5. The GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: Methods for the first inertial dead reckoning module, the second inertial dead reckoning module, and the third inertial dead reckoning module to respectively obtain the position, speed, and attitude estimation of the inertial navigation are as follows: A three-axis gyroscope is used to obtain the three-axis angular velocity of the carrier coordinate system, and a first error compensation is performed on the three-axis angular velocity. The first error compensation includes compensation for non-commutative errors, compensation for errors related to the rotation of the earth, compensation for carrier velocity errors, and compensation provided by a reinforcement learning (RL) module. The three-axis angular velocity after the first error compensation is integrated with the initial attitude information of the carrier to obtain an attitude estimate of the carrier. A three-axis accelerometer is used to measure the three-axis acceleration of the carrier coordinate system, and a second error compensation is performed on the three-axis acceleration. The second error compensation includes rowing compensation, rotation compensation and compensation provided by the RL module. The information of the three-axis acceleration after the second error compensation is coordinate projected through the carrier attitude information. Coordinate projection refers to projecting the three-axis acceleration in the carrier coordinate system to the navigation coordinate system. In the navigation coordinate system, after Coriolis compensation, gravity compensation and double integration operations, the estimation of speed and position is obtained.

6. The GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: The combined filtering method in the GNSS / INS combined system module is as follows: The position and velocity estimates obtained by the first inertial dead reckoning module are subtracted from the position and velocity provided by the satellite receiver, and the difference is used as measurement information to measure and update the combined Kalman filter, thereby estimating and updating the parameter errors of the inertial navigation system model; X=[φ T (δv n ) T (op n ) T (b g ) T (b a ) T (s g ) T (s a ) T )] T Among them, X is the inertial navigation error state, X includes the misalignment angle φ, velocity error δv n 、Position deviation δp n , gyro bias error b g , accelerometer error b a , gyro scale factor error s g and accelerometer scale factor error s a , the state variable superscript n represents the navigation coordinate system, the subscript g represents the gyroscope, the subscript a represents the accelerometer, the superscript T represents the matrix transpose, and δv is an overall variable representing the velocity error.

7. The GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: In the reinforcement learning module, the state input of the RL agent is also calculated by the inertial sensor device measurement calculation. At the same time, the RL agent obtains the action according to the state and the trained state / action strategy network. The action is used to compensate the error of the three-axis gyroscope output and the three-axis accelerometer output data, thereby improving the accuracy and endurance of the subsequent inertial dead reckoning PVA3. The following configurations are used for reward calculation, state calculation, and RL action design: a. Reward calculation The reward is set as: Reward = K1·Reward1+K2·Reward2+K3·Reward3, where Reward1 = (|S 12 -S 13 |-K·|S 13 -S 11 |), Reward2=(|S 22 -S 23 |-K·|S 23 -S 21 |), Reward3=(|S 32 -S 33 |-K·|S 33 -S 31 |); where S 11 , S 12 , S 13 Respectively represent the position estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch, S 21 , S 22 , S 23 Respectively represent the attitude estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch, S 31 , S 32 , S 33 Represent the velocity estimation values ​​of the first calculation branch, the second calculation branch, and the third calculation branch respectively. Reward is the total reward of the RL module. Reward1 is the reward in the position domain, Reward2 is the reward in the velocity domain, Reward3 is the reward in the posture domain, K1 is the reward ratio of Reward1, K2 is the reward ratio of Reward2, K3 is the reward ratio of Reward3, and K is the reward ratio of the difference between the first calculation branch and the third calculation branch. b. State calculation The state configuration is: the difference between the six-axis data output by the inertial sensor at the current moment and the six-axis data output by the inertial sensor at the previous moment, including the difference between the three-axis gyroscope output data before and after, and the difference between the three-axis accelerometer output data before and after; c.RL action design The RL action configuration is: compensation for the three-axis gyroscope output and the three-axis accelerometer output.

8. The GNSS / INS integrated navigation system based on reinforcement learning according to claim 1, characterized in that: In the reinforcement learning module, The first is the initialization of the entire GNSS / INS integrated navigation system, which includes the initial coordinates, attitude, and velocity of the inertial navigation system. In a scenario where the GNSS signal is visible, the GNSS / INS integrated navigation system outputs position, velocity, and attitude as the final navigation output of the system. At the same time, the RL model is trained; During the RL model training process, two conditions need to be judged by polling. The first is to judge the convergence of model training, and the second is to judge the training cycle. When the model training converges, the model available flag T is assigned a value of 1. When the training cycle is greater than the preset cycle threshold Tr, the inertial navigation system dead reckoning is reset and initialized to ensure the effectiveness of RL model training. In the scenario where the GNSS signal is not visible, the convergence sign of the reinforcement learning model is first determined. If T=1, the trained RL strategy network is directly used to compensate the three-axis gyroscope and three-axis accelerometer according to the measurement status of the inertial sensor to suppress the divergence of inertial navigation errors. The RL-assisted inertial navigation position calculation result PVA3 is used as the final navigation output. If the training model does not converge, PVA2 is used as the final navigation output.

9. A GNSS / INS integrated navigation method based on reinforcement learning, characterized in that: The six-axis observation data enters the first to third calculation branches for parallel processing, where: In the first calculation branch, the six-axis observation data is subjected to inertial dead reckoning and then combined with GNSS / INS to obtain the first calculation branch result PVA1, which includes position, velocity and attitude estimation results; In the second calculation branch, the six-axis observation data is processed by the inertial dead reckoning algorithm to obtain the second calculation branch result PVA2, which includes the pure dead reckoning position, velocity and attitude estimation results; In the third calculation branch, the six-axis observation data is processed by the RL agent and the inertial dead reckoning algorithm to obtain the third calculation branch result PVA3, which includes the position, velocity and attitude estimation results; The first calculation branch result PVA1, the second calculation branch result PVA2 and the third calculation branch result PVA3 are then used to reward the RL agent through the reward calculation unit in the reinforcement learning RL module; When GNSS signals are visible, PVA1 is used as navigation output; When the GNSS signal is not visible, if the reinforcement learning RL module converges, PVA3 is used as the navigation output. If the reinforcement learning RL module does not converge, PVA2 is used as the navigation output.