QLEKF-based autonomous navigation method and device for star exploration surrounding section

Abstract

The embodiment of the invention provides a QLEKF-based autonomous navigation method and device for a star detection surrounding section, and the method comprises the steps: obtaining a two-dimensional pixel image of a star satellite plane through a spaceborne camera, determining a relative position vector of each of three satellites of a star and the position of a detector, and further determining an observation variable; in reinforcement learning, according to the observation variable, determining a reinforcement learning Q value corresponding to a reinforcement learning cycle through a first extended Kalman filter, determining state estimation of the running state of the detector through a second extended Kalman filter, and according to the reinforcement learning Q value corresponding to the reinforcement learning cycle, determining the state estimation of the running state of the detector; if it is judged that the reinforcement learning cycle does not reach the convergence condition, continuing to execute the next reinforcement learning cycle of reinforcement learning; if the reinforcement learning cycle is judged to reach the convergence condition, taking the state estimation of the running state of the detector as the target state estimation of the running state of the detector; and calibrating the running state of the detector according to the target state estimation and the preset orbit data.

Description

Technical Field

[0001] The present invention relates to the field of spacecraft navigation, and in particular to a QLEKF-based autonomous navigation method and device for Jupiter exploration orbiting segment. Background Art

[0002] In recent years, Jupiter exploration has become another important area of ​​deep space exploration after the exploration of the moon and Mars. At present, many Jupiter exploration projects have been launched or are being planned around the world. The focus is on the circumnavigation of Jupiter and its satellites, aiming to further explore the Jupiter system and the interplanetary space environment, and to study in detail the distribution of Jupiter's atmosphere, magnetic field, plasma and high-energy particles.

[0003] Due to the long distance between Jupiter and the Earth, there is a communication delay of about 34 minutes, which makes it difficult for the ground to measure and control the probe in real time. Therefore, the probe is required to have strong autonomous management capabilities, including autonomous navigation and control, autonomous mission planning, autonomous fault diagnosis and processing, etc. Autonomous navigation technology can provide real-time information on the status of the probe, which is the key to ensuring the normal implementation of the exploration mission. It requires the probe to not rely on external support, but only use the measurement equipment it carries to determine its position, speed, attitude and other information relative to a certain coordinate system in real time to guide navigation.

[0004] Most of the existing autonomous navigation algorithms are based on the background of Mars and Moon exploration, and are solved by filtering algorithms based on the astrooptical information of navigation stars, gyroscope and accelerometer measurement information, pulsar signals, and measurement information based on intersatellite measurement beacons. Due to limited computing resources and the complex magnetic and gravitational fields around Jupiter, the detector cannot continuously and accurately receive measurement data from distant galaxies. However, the abundant natural satellites around Jupiter are stable and have known ephemeris information. Their line of sight vectors relative to the detector are ideal measurement data sources. However, the complex spatial disturbances and interference sources in the area around Jupiter make it difficult to accurately model process noise and measurement noise, which significantly reduces navigation accuracy. In order to suppress noise uncertainty, researchers have proposed a variety of adaptive filtering methods. For example, the Sage-Husa adaptive filter is improved based on the covariance matching technology to estimate the measurement noise in real time; another example is an AKF algorithm that estimates the process noise covariance matrix based on the measurement sequence and an AKF method based on the model parameter ratio (MPR), which transforms the adaptive estimation problem into a constrained optimization problem. Based on the above adaptive algorithm, in order to further improve the operation speed and estimation accuracy, Xiong et al. proposed an extended Kalman filter algorithm (QLEKF) combined with Q learning, which combines the EKF algorithm with reinforcement learning, obtains feedback through interaction with the environment, and adaptively adjusts the value of the noise covariance. QLEKF can solve the problem of high-precision autonomous navigation in a variety of scenarios and reduce the negative impact of noise uncertainty on EKF. Dai used the QLEKF algorithm to achieve accurate estimation of the MARG sensor attitude based on the real flight data of the drone; Tao used the QLEKF algorithm to solve the high-precision navigation needs of the cruise phase at the boundary of the solar system.

[0005] In the process of implementing the present invention, the applicant discovered that there are at least the following problems in the prior art:

[0006] How to improve navigation accuracy and efficiency in uncertain scenarios. Summary of the invention

[0007] The embodiments of the present invention provide a method and device for autonomous navigation of Jupiter exploration orbit segment based on QLEKF, which solves the problem of how to improve navigation accuracy and navigation efficiency under uncertain scenarios.

[0008] To achieve the above-mentioned purpose, on the one hand, an embodiment of the present invention provides a QLEKF-based autonomous navigation method for Jupiter exploration orbiting segment, comprising:

[0009] Obtaining a two-dimensional pixel image of a Jupiter satellite plane by a satellite-borne camera, and determining a relative position vector between each of three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane;

[0010] Determine the observed variable based on the relative position vector of each of the three pre-set satellites of Jupiter and the position of the probe;

[0011] In each reinforcement learning cycle of reinforcement learning, determining a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to a reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle;

[0012] According to the reinforcement learning state corresponding to the reinforcement learning cycle, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle is obtained from the reinforcement learning state space; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a noise covariance matrix;

[0013] Determine the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determine the state estimation of the operating state of the detector through a second extended Kalman filter;

[0014] According to the reinforcement learning Q value corresponding to the reinforcement learning cycle, judging whether the reinforcement learning cycle has reached a preset convergence condition; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, taking the state estimation of the operating state of the detector as the target state estimation of the operating state of the detector;

[0015] The operating state of the probe is calibrated according to the target state estimation and preset orbit data.

[0016] On the other hand, an embodiment of the present invention provides a QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, comprising:

[0017] A data acquisition unit is used to obtain a two-dimensional pixel image of a Jupiter satellite plane through a satellite-borne camera, and determine a relative position vector between each of three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane;

[0018] an observation variable determination unit, for determining the observation variable according to a relative position vector between the position of each of the three preset satellites of Jupiter and the position of the probe;

[0019] A reinforcement learning state determination unit, configured to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0020] A noise matrix determination unit, configured to obtain, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a noise covariance matrix;

[0021] A filtering processing unit, configured to determine the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and the reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determine the state estimation of the operating state of the detector through a second extended Kalman filter;

[0022] A convergence judgment unit is used to judge whether the reinforcement learning cycle has reached a preset convergence condition according to the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continue to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, use the state estimation of the running state of the detector as the target state estimation of the running state of the detector;

[0023] A state calibration unit is used to calibrate the operating state of the detector according to the target state estimation and preset orbit data.

[0024] The above technical scheme has the following beneficial effects: combining reinforcement learning with the extended Kalman filter to form the QLEKF-single algorithm, which can find suitable noise covariance matrix values ​​within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, and has made significant improvements in the position and velocity estimation after convergence, thereby improving the accuracy of spacecraft navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is a flow chart of an autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF, one of the embodiments of the present invention;

[0027] Figure 2 This is an architecture diagram of an autonomous navigation device for Jupiter exploration and orbiting based on QLEKF, one of the embodiments of the present invention;

[0028] Figure 3 It is a schematic diagram of the reinforcement learning principle of the prior art;

[0029] Figure 4 is a schematic diagram of an action space corresponding to a state set according to one embodiment of the present invention;

[0030] Figure 5 It is a schematic diagram of a QLEKF-single filter design according to one embodiment of the present invention;

[0031] Figure 6 It is a schematic diagram of the trajectory of a probe and a Jupiter satellite during the numerical simulation verification process of one embodiment of the present invention;

[0032] Figure 7 is a simulation flow chart of one embodiment of the present invention;

[0033] Figure 8 This is a performance comparison diagram of four different filtering algorithms;

[0034] Fig. 9 This is a schematic diagram comparing the position errors of the X-axis state estimation under four different filtering algorithms;

[0035] Fig.10 This is a schematic diagram comparing the position errors of the Y-axis state estimation under four different filtering algorithms;

[0036] Fig.11 This is a schematic diagram comparing the position errors of the Z-axis state estimation under four different filtering algorithms;

[0037] Fig.12 This is a schematic diagram comparing the velocity errors of the X-axis state estimation under four different filtering algorithms;

[0038] Fig.13 This is a schematic diagram comparing the velocity errors of the Y-axis state estimation under four different filtering algorithms;

[0039] Fig.14 This is a schematic diagram comparing the velocity errors of the Z-axis state estimation under four different filtering algorithms;

[0040] Fig.15 It is a schematic diagram of an error curve of position estimation in the X-axis direction of the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0041] Fig.16 It is a schematic diagram of an error curve of position estimation in the Y-axis direction of the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0042] Fig.17 It is a schematic diagram of an error curve of the Z-axis direction position estimation of the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0043] Fig.18 It is a schematic diagram of an error curve of the X-axis direction velocity estimation of the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0044] Fig.19 It is a schematic diagram of an error curve of the velocity estimation in the Y-axis direction of the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0045] Fig. 20 It is a schematic diagram of an error curve of the Z-axis direction velocity estimation of the QLEKF single-soft algorithm according to one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] On the one hand, if Figure 1 As shown, an embodiment of the present invention provides a QLEKF-based autonomous navigation method for Jupiter exploration orbiting segment, comprising:

[0048] Step S10, obtaining a two-dimensional pixel image of a Jupiter satellite plane by a satellite-borne camera, and determining a relative position vector between each of three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane;

[0049] Step S11, determining the observation variable according to the relative position vector between each of the three preset satellites of Jupiter and the position of the probe;

[0050] Step S12, in each reinforcement learning cycle of the reinforcement learning, determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0051] Step S13, according to the reinforcement learning state corresponding to the reinforcement learning cycle, obtaining a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle from the reinforcement learning state space; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix;

[0052] Step S14, determining the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining the state estimation of the operating state of the detector through a second extended Kalman filter;

[0053] Step S15, judging whether the reinforcement learning cycle has reached a preset convergence condition according to the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, taking the state estimation of the running state of the detector as the target state estimation of the running state of the detector;

[0054] Step S16: calibrate the operating state of the probe according to the target state estimation and the preset orbit data. Those skilled in the art know how to calibrate the operating state of the probe according to the target state estimation and the preset orbit data.

[0055] The embodiments of the present invention have the following technical effects: reinforcement learning is combined with an extended Kalman filter to form a QLEKF-single algorithm, which can find suitable noise covariance matrix values ​​within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, making significant improvements in position and velocity estimation after convergence, and improving the accuracy of spacecraft navigation.

[0056] Furthermore, the two-dimensional pixel image of the Jupiter satellite plane is obtained by the onboard camera, and the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter is determined in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane, specifically: the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter is determined according to the following formula:

[0057]

[0058] Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, and f is the focal length of the camera. ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors from the three satellites to the probe in the Jupiter inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

[0059] Further, the observation variable is determined according to the relative position vector between each of the three preset satellites of Jupiter and the position of the probe, specifically: the observation variable is determined according to the following formula:

[0060]

[0061] Among them, r ik =[x ik y ik z ik ] T , i = 1, 2, 3, represents the relative position vectors of the three satellites and the probe in the Jupiter inertial system; r k represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

[0062] Further, the determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle includes:

[0063] For each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle is determined according to the following formula:

[0064]

[0065] Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the obtained reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize;

[0066] Selecting the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle;

[0067] The reinforcement learning state corresponding to the reinforcement learning cycle is determined according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

[0068] The embodiments of the present invention have the following technical effects: by introducing the QLEKF-single algorithm and applying the SoftMax algorithm as a stable and effective action selection strategy, it is possible to find a suitable noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, making significant improvements in the position and velocity estimation after convergence, and improving the accuracy of spacecraft navigation.

[0069] Further, the reinforcement learning Q value corresponding to the reinforcement learning cycle is determined by a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and the state estimation of the operating state of the detector is determined by a second extended Kalman filter, including:

[0070] For each time step of a preset number of time steps, according to the following formula, the first state estimate and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the first extended Kalman filter to obtain the first state estimate, the first error covariance matrix and the first new information sequence corresponding to the time step:

[0071]

[0072] in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix;

[0073] According to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step, the reward value corresponding to the current time step is updated according to the following formula:

[0074]

[0075] Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step;

[0076] According to the following formula, the second state estimate corresponding to the previous time step and the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the second extended Kalman filter to obtain the second state estimate corresponding to the time step, the second error covariance matrix, and the second new information sequence:

[0077]

[0078] in, is the second state estimate corresponding to the time step, P kis the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step;

[0079] After all time steps of the preset time step number are executed, the second state estimation value corresponding to the last time step is used as the state estimation of the running state of the detector, and the reward value corresponding to the last time step is used as the reward value corresponding to the reinforcement learning cycle, and according to the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, the reinforcement learning Q value corresponding to the reinforcement learning cycle is determined according to the following formula:

[0080] Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)](7)

[0081] V(s')←max a Q(s',a)(8)

[0082] Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of the Q function updating the reward and inheriting the old value; γ∈(0,1) is the discount factor, which is used to measure the importance of the long-term reward; the maximum Q value that can be achieved based on the state s' is recorded as the value function V(s').

[0083] On the other hand, Figure 2 As shown, an embodiment of the present invention provides an autonomous navigation device for Jupiter exploration and orbiting based on QLEKF, comprising:

[0084] The data acquisition unit 200 is used to obtain a two-dimensional pixel image of a Jupiter satellite plane through a satellite-borne camera, and determine a relative position vector between each of the three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane;

[0085] An observation variable determination unit 201 is used to determine the observation variable according to the relative position vector between the position of each of the three preset satellites of Jupiter and the position of the probe;

[0086] A reinforcement learning state determining unit 202 is used to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0087] A noise matrix determination unit 203 is used to obtain a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle from a reinforcement learning state space according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix;

[0088] A filtering processing unit 204 is used to determine the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determine the state estimation of the operating state of the detector through a second extended Kalman filter;

[0089] A convergence judgment unit 205 is used to judge whether the reinforcement learning cycle has reached a preset convergence condition according to the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continue to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, use the state estimation of the running state of the detector as the target state estimation of the running state of the detector;

[0090] The state calibration unit 206 is used to calibrate the operating state of the probe according to the target state estimation and preset orbit data.

[0091] Furthermore, the data acquisition unit is specifically used to: determine the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter according to formula (1);

[0092] Furthermore, the observed variable determination unit is specifically used to determine the observed variable according to formula (2).

[0093] Furthermore, the reinforcement learning state determination unit includes:

[0094] An execution probability determination module, for determining, for each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle according to formula (3);

[0095] A reinforcement learning action determination module, used to select the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle;

[0096] A reinforcement learning state determination module is used to determine the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

[0097] Furthermore, the filtering processing unit includes:

[0098] A first filtering processing module is used for, for each time step of a preset time step number, inputting the first state estimation value and the first error covariance matrix corresponding to the previous time step, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into a first extended Kalman filter according to formula (4), so as to obtain the first state estimation value, the first error covariance matrix and the first new information sequence corresponding to the time step;

[0099] A reward value updating module, used to update the reward value corresponding to the current time step according to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step according to formula (5);

[0100] A second filtering processing module is used to input the second state estimation value and the second error covariance matrix corresponding to the previous time step, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into a second extended Kalman filter according to formula (6), so as to obtain the second state estimation value, the second error covariance matrix and the second new information sequence corresponding to the time step;

[0101] A reinforcement learning Q value updating module is used to, after all time steps of a preset number of time steps are executed, use the second state estimation value corresponding to the last time step as the state estimation of the running state of the detector, and use the reward value corresponding to the last time step as the reward value corresponding to the reinforcement learning cycle, and determine the reinforcement learning Q value corresponding to the reinforcement learning cycle according to formulas (7) and (8) based on the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and the reinforcement learning state corresponding to the reinforcement learning cycle.

[0102] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0103] Jupiter exploration is of great significance for studying the evolution of the solar system and using planetary gravity for deep space exploration. In view of the influence of noise uncertainty on filtering performance caused by the complex environment of Jupiter exploration, an optical autonomous navigation scheme is established based on the relative line of sight direction information of multiple Jupiter satellites, and the QLEKF (Q learning extended Kalman filter) algorithm simplified by a single filter is used to estimate the position and speed of the detector. QLEKF-single (single filter Q learning extended Kalman filter) designs a reward function based on the new information of a single EKF filter, adaptively selects the value of the noise covariance matrix through the Q learning algorithm, uses the SoftMax strategy for action selection, and finally realizes the iterative estimation of the system state through EKF. The simulation results verify that in the scenario with model uncertainty, the QLEKF-single algorithm effectively improves the navigation accuracy compared with the traditional filtering method; compared with the QLEKF algorithm, the running time is reduced by more than 10% when the accuracy does not change much. The embodiment of the present invention uses the line of sight (LOS) vectors of multiple natural Jupiter satellites obtained during the orbit of the detector to establish an autonomous navigation model based on Jupiter's gravity and J2 perturbation effect. In order to further reduce the model solution time, an embodiment of the present invention proposes a QLEKF-single algorithm with single filter reward feedback. Most of the existing QLEKF filter designs are based on the new information sequence between parallel filters to design the reward value of reinforcement learning, and use a greedy strategy to select the best noise matrix value pair according to the cumulative reward. The QLEKF-single algorithm only uses the logarithmic function of the modulus of the new information sequence of the search EKF as an immediate reward, and uses the SoftMax strategy as the basis for the transfer of the intelligent agent between different noise covariance pairs. Under the premise that the estimation accuracy remains basically unchanged after convergence, the computational efficiency is improved compared to the QLEKF algorithm.

[0104] The following describes the embodiments of the present invention in terms of the autonomous navigation scheme during the Jupiter orbiting mission, the design of the QLEKF-single algorithm, the verification of the effectiveness of the QLEKF-single algorithm through numerical simulation experiments, and the analysis of the experimental results and algorithm performance.

[0105] 1. Jupiter orbital dynamics and measurement model

[0106] The Jupiter flyby mission is a key step in Jupiter exploration. In order to determine the position and velocity of the probe in the selected reference frame, so as to facilitate the establishment and solution of the model, the embodiment of the present invention describes the definition of the coordinate system, probe state variables and measurement variables under discrete time conditions.

[0107] a) Coordinate system:

[0108] 1.) The origin O of Jupiter's inertial coordinate system (i system) is defined as the center of mass of Jupiter, OX is located in the equatorial plane and points to the vernal equinox, OZ points to the North Pole of Jupiter, and OY is located in the equatorial plane, forming a right-handed coordinate system with OX and OZ.

[0109] 2.) The origin O of the body coordinate system (b system) is defined as the center of mass of the detector, and OX, OY, and OZ form a right-handed coordinate system along the three main inertial axes of the detector.

[0110] 3.) The origin O of the camera coordinate system (C system) is defined as the optical center of the camera, OY is parallel to the image plane, and OZ, OX, and OY constitute a right-handed coordinate system.

[0111] b) Detector state variables:

[0112] Define the detector state variables at time k in Jupiter's inertial system:

[0113]

[0114] Among them, r k and v k are the relative position vector and velocity vector from the probe to the center of mass of Jupiter at time k, respectively, k ,y k ,z k is the detector position parameter at time k, is the detector velocity parameter.

[0115] c) Detector observation variables:

[0116] The probe uses the onboard camera to image and identify the three Jovian satellites, capture their two-dimensional pixel data and extract observation values. In the camera coordinate system at time k, the LOS vector of the probe pointing to Jovian satellite i is determined according to formula (1).

[0117] 1.1 Dynamic model

[0118] Assuming that the orbiting probe is mainly affected by the gravity of Jupiter, a dynamic orbiting model of the probe is established in the inertial coordinate system:

[0119] x k =f(x k-1 )+w k (11)

[0120] Among them, the first-order Taylor expansion of the state equation is:

[0121] f(x k-1 )=x k-1 +φ(x k-1 )τ (12)

[0122]

[0123] Among them, r k =||r k ||,w k is process noise, and its statistical characteristics are as follows:

[0124]

[0125] where δ kj is a Dirichlet function that satisfies:

[0126]

[0127] According to the orbital dynamics of Jupiter, the values ​​and meanings of the parameters in the formula are shown in Table 1:

[0128]

[0129]

[0130] Table 1 Dynamic model parameters

[0131] 1.2 Visual Camera Measurement Model

[0132] During the probe's flyby, since natural satellites cannot return ranging data, the LOS vector is used as the observation variable. The size of this observation variable is not affected by the conversion of the coordinate system, and the model can be written as formula (2).

[0133] In the formula, g k To measure noise, its statistical characteristics are:

[0134]

[0135] in,

[0136] And the process noise is independent of the measurement noise, that is, for any k, j:

[0137]

[0138] 1.3 Extended Kalman Filter Algorithm

[0139] Combining the dynamic model (11) and the visual camera measurement model formula (2), the autonomous navigation model for orbiting Jupiter is established:

[0140]

[0141] EKF was first proposed by Stanley Schmidt. It is the basis for the design and implementation of QLEKF filters and is also the most commonly used nonlinear filter. To solve the Jupiter orbiting autonomous navigation model using EKF, first calculate the linearized state matrix:

[0142]

[0143] in

[0144]

[0145] Substitute into matrix A 3×3 The value of each element:

[0146]

[0147] Perform time updates of the state estimate and the estimated error covariance:

[0148]

[0149] According to formula (2), the linearized measurement matrix is ​​obtained, where i = 1, 2, 3:

[0150]

[0151]

[0152] The covariance expressions of state estimation and estimation error are obtained:

[0153]

[0154] Table 2 shows the process of the EKF algorithm:

[0155]

[0156] Table 2 EKF algorithm flow

[0157] 2 Autonomous navigation around Jupiter based on intelligent filtering

[0158] 2.1Q Learning Extended Kalman Filter

[0159] Q-learning is an effective implementation of reinforcement learning (RL). It guides the agent to learn the optimal strategy in a random environment through exploration and exploitation. In this process, the agent performs an action and receives a feedback reward R returned by the environment as an indicator of the benefit of the action taken. The Q value represents the expected cumulative reward that can be obtained by taking a specific action in a given state. The agent maximizes the long-term cumulative reward through a certain action selection strategy. Its basic working principle is as follows Figure 3 .

[0160] Combining the Q-learning method with the EKF, we get the Q-learning extended Kalman filter (QLEKF). The QLEKF can learn interactively with the environment in an uncertain environment and adaptively select the value of the noise covariance matrix based on the feedback of the reward value, thus achieving higher estimation accuracy than the traditional EKF.

[0161] State set and action space design

[0162] The state set S and the action space A that can be taken in each state are both defined as discrete sets. Figure 4 As shown, the constructed state set S is an M×N grid containing M process noise covariance matrix design values and the N measurement noise covariance matrix design values Among them, the M process noise covariance matrix design values and the N measurement noise covariance matrix design values The process noise range and measurement noise range obtained by the pre-simulated extended Kalman filter algorithm can be subdivided into three regions on a plane with M rows and N columns:

[0163] a.) Corner area:

[0164] (S 11 ,S 1N ,S M1 ,S MN )

[0165] b.) Boundary area:

[0166] (S 1j ,j=2,…,N-1;S i1 ,i=2,…,M-1)

[0167] (S Mj ,j=2,…,N-1;S iN ,i=2,…,M-1)

[0168] c.) Central Area:

[0169] (S ij ,i=2,…,M-1,j=2,…,N-1)

[0170] The action space defines the state transitions that the agent can perform, that is, the agent can move to adjacent states without obstacles or stay in the current state. The action space corresponding to the state set is as follows Figure 4 As shown by the arrows in , the state set and action transfer are described, where the action space includes moving up, moving down, moving left, moving right, and maintaining the current state.

[0171] Action selection strategy

[0172] The SoftMax strategy calculates the action selection probability based on the size of the Q-value function, and its mathematical expression is formula (3).

[0173] The difference between SoftMax and the greedy algorithm is that the SoftMax strategy can change the preference for exploitation or exploration based on the relative value of the current state S(Mr, Nr) and the surrounding states. That is, if the Q value of the current state is much larger than other adjacent states, the agent will stay in the current state with a higher probability and visit other states with a lower probability, and the agent tends to exploit; if the Q value of the current state S(Mr, Nr) is similar to one or more adjacent states, the agent will select actions with a relatively equal probability, and the agent tends to explore to avoid falling into the local optimal situation. However, it should be noted that the exponential function in the SoftMax strategy will amplify the Q value, so when designing the reward function and initializing the Q table, the Q value range should be noted.

[0174]

[0175] Table 3: SoftMax strategy

[0176] Reward function design

[0177] After the agent performs an action, the environment will respond to it and reflect the goodness or badness of the action in the short term to the agent in the form of a reward R. The embodiment of the present invention constructs a search EKF filter and an estimation EKF filter to implement the QLEKF-single algorithm.

[0178] To evaluate the performance of a given state-action, we transform the logarithmic function of the innovation sequence into As a reward function. The larger the modulus of the new information sequence, the smaller the reward function. When , the reward function is negative, indicating that the benefit of executing this action is poor. Figure 5 Schematic diagram of QLEKF-single filter design. On the contrary, the environment will give positive feedback to the current state action through the reward function. The closer to 0, the greater the increase in the reward function R; the larger the reward function, the better the choice As the estimation accuracy of the noise covariance matrix is ​​higher, the filtering result of QLEKF is more accurate than that of EKF.

[0179] Q-value function and value function

[0180] The Q value measures the size of the cumulative reward. The value is updated once per cycle. The one-step update rule is as follows:

[0181]

[0182] After executing action a, we reach state s'. The maximum Q value that can be achieved based on state s' is recorded as the value function: that is, formula (8)

[0183] In formula (30), Q(s,a) is the Q value of executing action a in state s, R+γV(s') represents the reward that can be brought by this action a, including short-term reward R and long-term reward V(s'); α∈(0,1) is the learning rate, which measures the proportion of Q function updating rewards and inheriting old values; γ∈(0,1) is the discount factor, which measures the importance of long-term rewards.

[0184] 2.2 Algorithm Design

[0185] According to the state and action space design, the detector is regarded as an intelligent agent, and the QLEKFsingle-soft algorithm is constructed by combining the EKF algorithm and the Q learning principle. The algorithm flow is shown in Table 4. The QLEKFsingle-greedy algorithm is similar to it. It only needs to change the action selection strategy and replace the SoftMax strategy with a greedy strategy. The QLEKF-single algorithm periodically updates the state and action, searches for the appropriate process noise and observation noise matrix, and finally obtains a state estimate with high accuracy. State estimation is an estimation of the position and velocity of the aircraft.

[0186]

[0187]

[0188] Table 4 QLEKFsingle-soft algorithm flow

[0189] The nonlinear time-varying system is simplified into a linear steady-state system. When the system obtains measurement information for more than two consecutive moments, the observability matrix is ​​full rank and the nonlinear system has local observability.

[0190] 3 Simulation Verification

[0191] In order to verify the proposed algorithm, the embodiment of the present invention selects three natural satellites, Io, Callisto and Callisto, as observation sources and verifies them through numerical simulation in the MATLAB environment. The simulation flow chart is shown in FIG. Figure 7 As shown, the orbits of the probe and Jupiter's satellites are as follows Figure 6 The simulation parameter table is shown in Table 5.

[0192]

[0193]

[0194] Table 5 Simulation parameters

[0195] In the simulation, assuming that measurement information is obtained once every 1 minute, the simulation parameters, model errors, and Q learning parameters that need to be considered are shown in Table 5.

[0196] The lower and upper bounds of the state set are logarithmic:

[0197]

[0198] And is divided into M×N grids with equal spacing. If M=3, then Ensured by logarithmic equidistant segmentation The search for values ​​can be performed efficiently while examining as large a range of values ​​as possible.

[0199] Under the above simulation conditions, the state estimation results are shown in Table 6 and Figures 8 to 20 As shown:

[0200]

[0201] Table 6 Program running time

[0202] The embodiment of the present invention uses the root mean square error as the evaluation index of navigation accuracy:

[0203]

[0204] Figure 8 Table 6 shows the performance of four different filtering algorithms, among which EKF has the highest computational efficiency and the lowest accuracy; QLEKF has the highest accuracy but the longest program running time; QLEKF-single reduces the computational time by about 14% compared with QLEKF, and at this time, the accuracy of QLEKF-soft does not change much compared with QLEKF.

[0205] Figures 9 to 11 The estimated deviations of the positions along the three coordinate directions are shown, where XD1, XD2, XD3 and XD4 are the estimated deviation curves of the X-axis position of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; YD1, YD2, YD3 and YD4 are the estimated deviation curves of the Y-axis position of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; ZD1, ZD2, ZD3 and ZD4 are the estimated deviation curves of the Z-axis position of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively. Figure 12 to Figure 14The estimated deviations of the speed along the three coordinate directions are shown, where VX1, VX2, VX3 and VX4 are the velocity estimation deviation curves of the X-axis direction of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; VY1, VY2, VY3 and VY4 are the velocity estimation deviation curves of the Y-axis direction of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; VZ1, VZ2, VZ3 and VZ4 are the velocity estimation deviation curves of the Z-axis direction of the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; the position deviations of the X-axis, Y-axis and Z-axis are 10 4 m, 10 3 m, 10 2 The magnitude of m converges to near 0; the speed errors are all stable around 0, and the oscillation amplitude does not exceed 10m / s. Figure 15 to Figure 17 is the error curve of the position estimation of the QLEKFsingle-soft algorithm, Figure 18 to Figure 20 This is the error curve of the velocity estimation of the QLEKFsingle-soft algorithm. The estimated deviation remains within the 2σ range most of the time. It can be considered that the QLEKFsingle-soft algorithm is suitable for the current Jupiter orbit navigation system.

[0206] During the simulation process, the change in the size of the state set significantly affects the final convergence result. Assuming M = N, the average RMSE values ​​of the last 100 iterations of the three filtering algorithms in 20 experiments are calculated respectively. The results are shown in the following table:

[0207]

[0208] Table 7 RMSE of state estimation position when the state set size changes

[0209] Table 7 shows the impact of the state set size on the performance of the QLEKFsingle-soft and QLEKFsingle-greedy algorithms. In general, the QLEKFsingle algorithm is better than the EKF algorithm. When the state set size is moderate (9≤N≤18), the QLEKF-soft algorithm is better than the QLEKF-greedy algorithm because the Q value table is fully updated. The greedy algorithm selects the best action with probability (1-ε) / A(s), and the suboptimal action is selected with a lower probability. The SoftMax strategy can obtain the global optimal solution more robustly. When the state set is small, the agent can traverse each state and the Q value table is fully updated. At this time, the results obtained by using the SoftMax strategy and the greedy strategy are similar in accuracy. When the state set is too large (N≥21), the Q value table is a sparse matrix. The SoftMax strategy will amplify the selection probability of high-value actions at the current moment

[21] , and continuously accumulate rewards during the Q value cycle update process, making it impossible for the agent to explore other actions and causing large errors. The results obtained by using the greedy strategy have smaller errors.

[0210] 4. Conclusion

[0211] The embodiment of the present invention introduces the QLEKF-single algorithm as an adjustment scheme for the model noise covariance matrix during autonomous navigation around Jupiter, verifies observability and effectiveness, and compares the effects of different action strategies on the estimation results. Through the simulation experiment of the probe flying around Jupiter, it is verified that both the SoftMax algorithm and the greedy algorithm can be used as stable and effective action selection strategies, and the SoftMax algorithm is better than the greedy algorithm under the moderate state set size: it can find the appropriate noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, and has a significant improvement in the position and velocity estimation after convergence.

[0212] Due to the limited computing resources on board and the complexity of the space environment, how to estimate the position, speed and attitude of the probe in real time when there is a delay in obtaining measurement information and the computing speed is slow is the next issue to be considered. In the future, we can further optimize the parallel filter design and state model construction to adapt to the more complex and changeable deep space environment and complete more challenging autonomous navigation tasks.

[0213] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0214] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0215] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0216] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including". In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0217] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0218] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A QLEKF-based autonomous navigation method for Jupiter exploration orbiting, characterized in that: include: Obtaining a two-dimensional pixel image of a Jupiter satellite plane by a satellite-borne camera, and determining a relative position vector between each of three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane; Determine the observed variable based on the relative position vector of each of the three pre-set satellites of Jupiter and the position of the probe; In each reinforcement learning cycle of reinforcement learning, determining a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to a reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle; According to the reinforcement learning state corresponding to the reinforcement learning cycle, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle is obtained from the reinforcement learning state space; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix; Determine the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determine the state estimation of the operating state of the detector through a second extended Kalman filter; According to the reinforcement learning Q value corresponding to the reinforcement learning cycle, judging whether the reinforcement learning cycle has reached a preset convergence condition; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, taking the state estimation of the operating state of the detector as the target state estimation of the operating state of the detector; The operating state of the probe is calibrated according to the target state estimation and preset orbit data.

2. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The two-dimensional pixel image of the Jupiter satellite plane is obtained by the onboard camera, and the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter is determined in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane. Specifically, the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter is determined according to the following formula: Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, and f is the focal length of the camera. ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors from the three satellites to the probe in the Jupiter inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

3. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The observation variable is determined according to the relative position vector of each of the three preset satellites of Jupiter and the position of the probe, specifically: the observation variable is determined according to the following formula: Among them, r ik =[x ik y ik z ik ] T , i = 1, 2, 3, represents the relative position vectors of the three satellites and the probe in the Jupiter inertial system; r k represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

4. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The step of determining the reinforcement learning action and the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle includes: For each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle is determined according to the following formula: Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the obtained reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize; Selecting the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle; The reinforcement learning state corresponding to the reinforcement learning cycle is determined according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

5. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The method of determining the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining the state estimation of the operating state of the detector through a second extended Kalman filter, includes: For each time step of a preset number of time steps, according to the following formula, the first state estimate and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the first extended Kalman filter to obtain the first state estimate, the first error covariance matrix and the first new information sequence corresponding to the time step: in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix; According to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step, the reward value corresponding to the current time step is updated according to the following formula: Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step; According to the following formula, the second state estimate corresponding to the previous time step, the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the second extended Kalman filter to obtain the second state estimate corresponding to the time step, the second error covariance matrix, and the second new information sequence: in, is the second state estimate corresponding to the time step, P k is the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step; After all time steps of the preset time step number are executed, the second state estimation value corresponding to the last time step is used as the state estimation of the running state of the detector, and the reward value corresponding to the last time step is used as the reward value corresponding to the reinforcement learning cycle, and according to the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, the reinforcement learning Q value corresponding to the reinforcement learning cycle is determined according to the following formula: Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)] V(s’)←max a Q(s’,a) Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of the Q function updating the reward and inheriting the old value; γ∈(0,1) is the discount factor, which is used to measure the importance of the long-term reward; the maximum Q value that can be achieved based on the state s' is recorded as the value function 'V(s).

6. A QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, characterized in that: include: A data acquisition unit is used to obtain a two-dimensional pixel image of a Jupiter satellite plane through a satellite-borne camera, and determine a relative position vector between each of three preset Jupiter satellites and the position of the probe in the Jupiter inertial coordinate system according to the two-dimensional pixel image of the Jupiter satellite plane; an observation variable determination unit, for determining the observation variable according to a relative position vector between the position of each of the three preset satellites of Jupiter and the position of the probe; A reinforcement learning state determination unit, configured to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle; A noise matrix determination unit, configured to obtain, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix; A filtering processing unit, configured to determine the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter according to the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and the reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determine the state estimation of the operating state of the detector through a second extended Kalman filter; A convergence judgment unit is used to judge whether the reinforcement learning cycle has reached a preset convergence condition according to the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continue to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, use the state estimation of the running state of the detector as the target state estimation of the running state of the detector; A state calibration unit is used to calibrate the operating state of the detector according to the target state estimation and preset orbit data.

7. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF as claimed in claim 6, characterized in that: The data acquisition unit is specifically used to determine the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter according to the following formula: Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, and f is the focal length of the camera. ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors from the three satellites to the probe in the Jupiter inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

8. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF as claimed in claim 6, characterized in that: The observed variable determination unit is specifically used to determine the observed variable according to the following formula: Among them, r ik =[x ik y ik z ik ] T , i = 1, 2, 3, represents the relative position vectors of the three satellites and the probe in the Jupiter inertial system; r k represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

9. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF as claimed in claim 6, characterized in that: The reinforcement learning state determination unit comprises: The execution probability determination module is used to determine, for each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle according to the following formula: Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the obtained reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize; A reinforcement learning action determination module, used to select the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle; A reinforcement learning state determination module is used to determine the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

10. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF as claimed in claim 6, characterized in that: The filtering processing unit comprises: The first filtering processing module is used for inputting the first state estimation value and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into the first extended Kalman filter for each time step of the preset time step number, so as to obtain the first state estimation value, the first error covariance matrix and the first new information sequence corresponding to the time step according to the following formula: in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix; The reward value updating module is used to update the reward value corresponding to the current time step according to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step according to the following formula: Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step; The second filtering processing module is used to input the second state estimation value and the second error covariance matrix corresponding to the previous time step, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into the second extended Kalman filter according to the following formula to obtain the second state estimation value, the second error covariance matrix and the second new information sequence corresponding to the time step: in, is the second state estimate corresponding to the time step, P k is the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step; The reinforcement learning Q value updating module is used to use the second state estimation value corresponding to the last time step as the state estimation of the running state of the detector after all time steps of the preset time step number are executed, and use the reward value corresponding to the last time step as the reward value corresponding to the reinforcement learning cycle, and determine the reinforcement learning Q value corresponding to the reinforcement learning cycle according to the following formula based on the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle: Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)] V(s’)←max a Q(s’,a) Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of the Q function updating the reward and inheriting the old value; γ∈(0,1) is the discount factor, which is used to measure the importance of the long-term reward; the maximum Q value that can be achieved based on the state s' is recorded as the value function V(s').

Images