Autonomous navigation method and device for Jupiter exploration orbiting phase based on QLEKF

Abstract

An embodiment of the present invention provides a QLEKF-based autonomous navigation method and device for a Jupiter exploration orbit segment, the method comprising: obtaining a two-dimensional pixel image of a Jupiter satellite plane through a satellite-borne camera, determining a relative position vector between each of Jupiter's three satellites and the position of the probe, and then determining an observation variable; in reinforcement learning, determining a reinforcement learning Q value corresponding to a reinforcement learning loop through a first extended Kalman filter according to the observation variable, determining a state estimate of the probe's operating state through a second extended Kalman filter, and continuing to execute the next reinforcement learning loop of the reinforcement learning if it is determined that the reinforcement learning loop has not reached a convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning loop; if it is determined that the reinforcement learning loop has reached a convergence condition, using the state estimate of the probe's operating state as a target state estimate of the probe's operating state; and calibrating the probe's operating state based on the target state estimate and preset orbit data.

Description

Technical Field

[0001] The present invention relates to the field of spacecraft navigation, and in particular to a method and device for autonomous navigation of a Jupiter exploration orbit segment based on QLEKF. Background Art

[0002] In recent years, Jupiter exploration has become another important area of ​​deep space exploration, following lunar and Mars exploration. Currently, numerous Jupiter exploration projects have been launched or are being planned worldwide. These projects primarily focus on orbiting Jupiter and its moons, aiming to further explore the Jovian system and the interplanetary space environment, and to provide detailed studies of Jupiter's atmosphere, magnetic field, and the distribution of ions and high-energy particles.

[0003] Due to the vast distance between Jupiter and Earth, there is a communication delay of approximately 34 minutes, making real-time measurement and control of the probe difficult from the ground. Therefore, the probe is required to possess strong autonomous management capabilities, including autonomous navigation and control, autonomous mission planning, and autonomous fault diagnosis and handling. Autonomous navigation technology, which provides real-time information on the probe's status, is crucial to ensuring the normal implementation of the exploration mission. This requires the probe to rely solely on its own onboard measurement equipment to determine its position, velocity, attitude, and other information relative to a specific coordinate system in real time, independent of external support, to guide navigation.

[0004] Most existing autonomous navigation algorithms are based on Mars and lunar exploration. They utilize filtering algorithms to solve the problem based on astro-optical information from navigational satellites, gyroscope and accelerometer measurements, pulsar signals, and intersatellite beacons. Limited computing resources and the complex magnetic and gravitational fields around Jupiter prevent probes from continuously and accurately receiving measurement data from distant galaxies. However, the abundant natural satellites surrounding Jupiter are stable and have known ephemeris information, making their line-of-sight vectors relative to the probe an ideal source of measurement data. However, the complex spatial perturbations and interference sources in the Jupiter region make it difficult to accurately model both process and measurement noise, significantly reducing navigation accuracy. To mitigate noise uncertainty, researchers have proposed various adaptive filtering methods. For example, an improved Sage-Husa adaptive filter based on covariance matching technology provides real-time measurement noise estimation. Another example is an AKF algorithm that estimates the process noise covariance matrix based on a measurement sequence, and an AKF method based on the model parameter ratio (MPR) that transforms the adaptive estimation problem into a constrained optimization problem. Based on the above adaptive algorithm, to further improve computational speed and estimation accuracy, Xiong et al. proposed an extended Kalman filter (QLEKF) algorithm combined with Q learning. This algorithm combines the EKF algorithm with reinforcement learning, obtains feedback through interaction with the environment, and adaptively adjusts the value of the noise covariance. The QLEKF can solve the problem of high-precision autonomous navigation in a variety of scenarios and reduce the negative impact of noise uncertainty on the EKF. Dai used the QLEKF algorithm to achieve accurate estimation of the MARG sensor attitude based on real UAV flight data; Tao used the QLEKF algorithm to solve the high-precision navigation needs during the cruise phase at the edge of the solar system.

[0005] In the process of implementing the present invention, the applicant discovered that the prior art has at least the following problems:

[0006] How to improve navigation accuracy and efficiency in uncertain scenarios. Summary of the Invention

[0007] The embodiments of the present invention provide a method and device for autonomous navigation of the Jupiter exploration orbit segment based on QLEKF, which solves the problem of how to improve navigation accuracy and efficiency under uncertain scenarios.

[0008] To achieve the above objectives, on the one hand, an embodiment of the present invention provides a QLEKF-based autonomous navigation method for Jupiter exploration orbiting, comprising:

[0009] Obtaining a two-dimensional pixel image of a Jovian satellite plane using an onboard camera, and determining a relative position vector between each of three preset Jovian satellites and the position of the probe in a Jovian inertial coordinate system based on the two-dimensional pixel image of the Jovian satellite plane;

[0010] Determine the observed variable based on the relative position vector of each of the three pre-set satellites of Jupiter and the position of the probe;

[0011] In each reinforcement learning cycle of the reinforcement learning, determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0012] According to the reinforcement learning state corresponding to the reinforcement learning cycle, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle is obtained from the reinforcement learning state space; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a noise covariance matrix;

[0013] Determining a reinforcement learning Q value corresponding to the reinforcement learning cycle using a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining a state estimate of the operating state of the detector using a second extended Kalman filter;

[0014] Determining whether the reinforcement learning cycle has reached a preset convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is determined that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of the reinforcement learning; if it is determined that the reinforcement learning cycle has reached the preset convergence condition, using the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector;

[0015] The operating state of the detector is calibrated based on the target state estimate and preset orbit data.

[0016] On the other hand, an embodiment of the present invention provides a QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, comprising:

[0017] a data acquisition unit configured to obtain a two-dimensional pixel image of a Jovian satellite plane using an onboard camera, and determine, based on the two-dimensional pixel image of the Jovian satellite plane, a relative position vector between each of three preset Jovian satellites and the position of the probe in the Jovian inertial coordinate system;

[0018] an observation variable determination unit, configured to determine an observation variable based on a relative position vector between the position of each of the three preset satellites of Jupiter and the probe;

[0019] a reinforcement learning state determining unit, configured to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle;

[0020] a noise matrix determining unit, configured to obtain, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a noise covariance matrix;

[0021] a filtering processing unit, configured to determine, through a first extended Kalman filter, a reinforcement learning Q value corresponding to the reinforcement learning cycle based on the observed variables, a reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle, a reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and to determine, through a second extended Kalman filter, a state estimate of the operating state of the detector;

[0022] a convergence judgment unit, configured to judge, based on the reinforcement learning Q value corresponding to the reinforcement learning cycle, whether the reinforcement learning cycle has reached a preset convergence condition; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, using the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector;

[0023] A state calibration unit is used to calibrate the operating state of the detector based on the target state estimation and preset orbit data.

[0024] The above technical solution has the following beneficial effects: combining reinforcement learning with the extended Kalman filter to form the QLEKF-single algorithm, which can find the appropriate noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, and making significant improvements in the position and velocity estimation after convergence, thereby improving the accuracy of spacecraft navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is a flowchart of a QLEKF-based autonomous navigation method for Jupiter exploration orbiting segment according to one embodiment of the present invention;

[0027] Figure 2 This is an architecture diagram of a QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, one of the embodiments of the present invention;

[0028] Figure 3 This is a diagram of the reinforcement learning principle of the existing technology;

[0029] Figure 4 is a schematic diagram of the action space corresponding to the state set of one embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of a QLEKF-single filter design according to one embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of the trajectory of the probe and Jupiter's satellite during the numerical simulation verification process of one embodiment of the present invention;

[0032] Figure 7 is a simulation flow chart of one embodiment of the present invention;

[0033] Figure 8 This is a performance comparison diagram of four different filtering algorithms;

[0034] Figure 9 This is a schematic diagram comparing the position error of the X-axis state estimation under four different filtering algorithms;

[0035] Figure 10 This is a schematic diagram comparing the Y-axis state estimation position error under four different filtering algorithms;

[0036] Figure 11 This is a schematic diagram comparing the Z-axis state estimation position error under four different filtering algorithms;

[0037] Figure 12 This is a schematic diagram comparing the velocity errors of the X-axis state estimation under four different filtering algorithms;

[0038] Figure 13 This is a schematic diagram comparing the Y-axis state estimation velocity errors under four different filtering algorithms;

[0039] Figure 14 This is a schematic diagram comparing the velocity errors of the Z-axis state estimation under four different filtering algorithms;

[0040] Figure 15 1 is a schematic diagram of an error curve of the X-axis position estimation using the QLEKF single-soft algorithm according to one embodiment of the present invention;

[0041] Figure 16 Schematic diagram of the error curve of the Y-axis position estimation of the QLEKF single-soft algorithm according to one embodiment of the present invention;

[0042] Figure 17 1 is a schematic diagram of an error curve of the Z-axis position estimation using the QLEKF single-soft algorithm according to one embodiment of the present invention;

[0043] Figure 18 1 is a schematic diagram of an error curve of the X-axis velocity estimation using the QLEKFsingle-soft algorithm according to one embodiment of the present invention;

[0044] Figure 19 1 is a schematic diagram of an error curve of the Y-axis velocity estimation of the QLEKF single-soft algorithm according to one embodiment of the present invention;

[0045] Figure 20 It is a schematic diagram of an error curve of the Z-axis velocity estimation of the QLEKF single-soft algorithm according to one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] On the one hand, if Figure 1 As shown, an embodiment of the present invention provides a QLEKF-based autonomous navigation method for Jupiter exploration orbiting, comprising:

[0048] Step S10, obtaining a two-dimensional pixel image of a Jovian satellite plane using a space-borne camera, and determining a relative position vector between each of three preset Jovian satellites and the position of the probe in a Jovian inertial coordinate system based on the two-dimensional pixel image of the Jovian satellite plane;

[0049] Step S11, determining an observation variable based on a relative position vector between the position of each of the three preset satellites of Jupiter and the position of the probe;

[0050] Step S12, in each reinforcement learning cycle of the reinforcement learning, determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0051] Step S13: obtaining, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix;

[0052] Step S14, determining the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining a state estimate of the operating state of the detector through a second extended Kalman filter;

[0053] Step S15: judging whether the reinforcement learning cycle has reached a preset convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, using the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector;

[0054] Step S16: calibrating the operating state of the probe based on the target state estimation and the preset orbit data. Those skilled in the art know how to calibrate the operating state of the probe based on the target state estimation and the preset orbit data.

[0055] The embodiments of the present invention have the following technical effects: reinforcement learning is combined with the extended Kalman filter to form a QLEKF-single algorithm, which can find a suitable noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, making significant improvements in the position and velocity estimation after convergence, and improving the accuracy of spacecraft navigation.

[0056] Furthermore, the two-dimensional pixel image of the Jovian satellite plane is obtained by the onboard camera, and the relative position vector of the probe's position relative to each of the three preset Jupiter satellites is determined in the Jupiter inertial coordinate system based on the two-dimensional pixel image of the Jovian satellite plane. Specifically, the relative position vector of the probe's position relative to each of the three preset Jupiter satellites is determined according to the following formula:

[0057]

[0058] Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, and f is the focal length of the camera. ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors of the three satellites to the probe in Jupiter’s inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

[0059] Furthermore, the observation variable is determined based on the relative position vector between each of the three preset satellites of Jupiter and the position of the probe, specifically: the observation variable is determined according to the following formula:

[0060]

[0061] Among them, r ik =[x ik y ik z ik ] T , i=1,2,3, represents the relative position vectors of the three satellites and the probe in Jupiter's inertial system; r k Represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

[0062] Furthermore, determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle includes:

[0063] For each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle is determined according to the following formula:

[0064]

[0065] Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize;

[0066] Selecting the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle;

[0067] The reinforcement learning state corresponding to the reinforcement learning cycle is determined according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

[0068] The embodiments of the present invention have the following technical effects: by introducing the QLEKF-single algorithm and applying the SoftMax algorithm as a stable and effective action selection strategy, it is possible to find a suitable noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm when the noise covariance matrix is ​​difficult to accurately determine, thereby achieving a more accurate estimation effect than the EKF algorithm, making significant improvements in the position and velocity estimation after convergence, and improving the accuracy of spacecraft navigation.

[0069] Furthermore, determining the reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining the state estimate of the operating state of the detector through a second extended Kalman filter, includes:

[0070] For each time step of a preset number of time steps, according to the following formula, the first state estimate and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the first extended Kalman filter to obtain the first state estimate, the first error covariance matrix, and the first innovation sequence corresponding to the time step:

[0071]

[0072] in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix;

[0073] Based on the first new information sequence corresponding to the time step and the reward value obtained in the previous time step, the reward value corresponding to the current time step is updated according to the following formula:

[0074]

[0075] Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step;

[0076] According to the following formula, the second state estimate corresponding to the previous time step and the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the second extended Kalman filter to obtain the second state estimate corresponding to the time step, the second error covariance matrix, and the second innovation sequence:

[0077]

[0078] in, is the second state estimate corresponding to the time step, P kis the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step;

[0079] After all time steps of the preset number of time steps are executed, the second state estimation value corresponding to the last time step is used as the state estimation of the operating state of the detector, and the reward value corresponding to the last time step is used as the reward value corresponding to the reinforcement learning cycle, and according to the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, the reinforcement learning Q value corresponding to the reinforcement learning cycle is determined according to the following formula:

[0080] Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)](7)

[0081] V(s')←max a Q(s',a)(8)

[0082] Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of Q function updating rewards and inheriting old values; γ∈(0,1) is the discount factor, which is used to measure the importance of long-term rewards; the maximum Q value that can be achieved based on state s' is recorded as the value function V(s').

[0083] On the other hand, Figure 2 As shown, an embodiment of the present invention provides a QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, comprising:

[0084] The data acquisition unit 200 is configured to obtain a two-dimensional pixel image of a Jovian satellite plane using an onboard camera, and determine a relative position vector between each of three preset Jovian satellites and the position of the probe in the Jovian inertial coordinate system based on the two-dimensional pixel image of the Jovian satellite plane.

[0085] An observation variable determination unit 201 is configured to determine an observation variable based on a relative position vector between the position of each of the three preset satellites of Jupiter and the position of the probe;

[0086] A reinforcement learning state determining unit 202 is configured to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle;

[0087] A noise matrix determining unit 203 is configured to obtain, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix;

[0088] a filtering processing unit 204 for determining a reinforcement learning Q value corresponding to the reinforcement learning cycle using a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining a state estimate of the operating state of the detector using a second extended Kalman filter;

[0089] A convergence judgment unit 205 is configured to judge whether the reinforcement learning cycle has reached a preset convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continue to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, use the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector;

[0090] The state calibration unit 206 is used to calibrate the operating state of the probe according to the target state estimation and preset orbit data.

[0091] Furthermore, the data acquisition unit is specifically configured to: determine the relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter according to formula (1);

[0092] Furthermore, the observed variable determination unit is specifically used to determine the observed variable according to formula (2).

[0093] Furthermore, the reinforcement learning state determination unit includes:

[0094] an execution probability determination module for determining, for each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle according to formula (3);

[0095] A reinforcement learning action determination module, configured to select the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle;

[0096] A reinforcement learning state determination module is configured to determine the reinforcement learning state corresponding to the reinforcement learning cycle based on the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

[0097] Furthermore, the filtering processing unit includes:

[0098] A first filtering processing module is configured to input, for each time step of a preset number of time steps, a first state estimate and a first error covariance matrix corresponding to the previous time step, the observed variables, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into a first extended Kalman filter to obtain the first state estimate, the first error covariance matrix, and the first innovation sequence corresponding to the time step according to formula (4);

[0099] A reward value updating module is used to update the reward value corresponding to the current time step according to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step according to formula (5);

[0100] A second filtering processing module is configured to input the second state estimate corresponding to the previous time step and the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into a second extended Kalman filter according to formula (6), to obtain the second state estimate corresponding to the time step, the second error covariance matrix, and the second innovation sequence;

[0101] The reinforcement learning Q value updating module is used to use the second state estimation value corresponding to the last time step as the state estimation of the running state of the detector after all time steps of the preset time step number are executed, and use the reward value corresponding to the last time step as the reward value corresponding to the reinforcement learning cycle, and determine the reinforcement learning Q value corresponding to the reinforcement learning cycle according to formulas (7) and (8) based on the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle.

[0102] The above technical solutions of the embodiments of the present invention are described in detail below with reference to specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description above.

[0103] Jupiter exploration is of great significance for studying the evolution of the solar system and for deep space exploration using planetary gravity. To address the impact of noise uncertainty on filtering performance caused by the complex environment of Jupiter exploration, an optical autonomous navigation scheme was established based on the relative line-of-sight information of multiple Jovian satellites. A single-filter simplified QLEKF (Q-learning extended Kalman filter) algorithm was used to estimate the probe's position and velocity. The QLEKF-single (Q-learning extended Kalman filter) algorithm designs a reward function based on innovations from a single EKF filter. It adaptively selects the value of the noise covariance matrix through a Q-learning algorithm, selects actions using a softmax strategy, and ultimately iteratively estimates the system state through the EKF. Simulation results demonstrate that the QLEKF-single algorithm effectively improves navigation accuracy compared to traditional filtering methods in the presence of model uncertainty. Compared to the QLEKF algorithm, the runtime is reduced by more than 10% with minimal accuracy change. This embodiment of the present invention utilizes the line-of-sight (LOS) vectors of multiple natural Jovian satellites obtained during the probe's in-orbit orbit to establish an autonomous navigation model based on Jupiter's gravity and J2 perturbation effects. In order to further reduce the model solution time, an embodiment of the present invention proposes a QLEKF-single algorithm with single filter reward feedback. Most existing QLEKF filter designs are based on the design of reinforcement learning reward values ​​based on the new information sequence between parallel filters, and use a greedy strategy to select the best noise matrix value pair based on the cumulative reward. The QLEKF-single algorithm only uses the logarithmic function of the modulus of the new information sequence of the search EKF as an immediate reward, and uses the SoftMax strategy as the basis for the transfer of the intelligent agent between different noise covariance pairs. Under the premise that the estimation accuracy remains basically unchanged after convergence, the computational efficiency is improved compared to the QLEKF algorithm.

[0104] The following describes the embodiments of the present invention, including the autonomous navigation scheme for the Jupiter orbiting mission, the design of the QLEKF-single algorithm, the verification of the effectiveness of the QLEKF-single algorithm through numerical simulation experiments, and the analysis of the experimental results and algorithm performance.

[0105] 1. Jupiter Orbital Dynamics and Measurement Model

[0106] The Jupiter flyby mission is a key step in Jupiter exploration. To determine the probe's position and velocity in a selected reference frame, facilitating model building and solution, this embodiment of the present invention describes the definitions of the coordinate system, probe state variables, and measurement variables under discrete-time conditions.

[0107] a) Coordinate system:

[0108] 1.) The origin O of Jupiter's inertial coordinate system (i-system) is defined as the center of mass of Jupiter. OX is located in the equatorial plane and points to the vernal equinox. OZ points to the North Pole of Jupiter. OY is located in the equatorial plane and forms a right-handed coordinate system with OX and OZ.

[0109] 2.) The origin O of the body coordinate system (b system) is defined as the center of mass of the detector, and OX, OY, and OZ form the right-handed coordinate system along the three main inertial axes of the detector.

[0110] 3.) The origin O of the camera coordinate system (C system) is defined as the optical center of the camera, OY is parallel to the image plane, and OZ, OX, and OY form a right-handed coordinate system.

[0111] b) Detector state variables:

[0112] Define the detector state variables at time k in Jupiter's inertial system:

[0113]

[0114] Among them, r k and v k are the relative position vector and velocity vector from the probe to the center of mass of Jupiter at time k, respectively. k ,y k ,z k is the detector position parameter at time k, is the detector velocity parameter.

[0115] c) Detector observation variables:

[0116] The probe uses its onboard camera to image and identify the three Jovian satellites, capturing their two-dimensional pixel data and extracting observation values. At time k, in the camera coordinate system, the LOS vector of the probe pointing to Jovian satellite i is determined according to formula (1).

[0117] 1.1 Power model

[0118] Assuming that the orbiting probe is mainly affected by the gravity of Jupiter, a dynamic orbiting model of the probe is established in the inertial coordinate system:

[0119] x k =f(x k-1 )+w k (11)

[0120] Among them, the first-order Taylor expansion of the state equation is:

[0121] f(x k-1 )=x k-1 +φ(x k-1 )τ (12)

[0122]

[0123] Among them, r k =||r k ||,w k is the process noise, and its statistical characteristics are as follows:

[0124]

[0125] where δ kj is a Dirichlet function that satisfies:

[0126]

[0127] According to Jupiter's orbital dynamics, the values ​​and meanings of the parameters in the formula are shown in Table 1:

[0128]

[0129]

[0130] Table 1 Dynamic model parameters

[0131] 1.2 Visual Camera Measurement Model

[0132] During the probe's orbit, since natural satellites cannot return ranging data, the LOS vector is used as the observation variable. The magnitude of this observation variable is not affected by the coordinate system transformation, and the model can be written as formula (2).

[0133] In the formula, g k To measure noise, its statistical characteristics are:

[0134]

[0135] in,

[0136] And the process noise is independent of the measurement noise, that is, for any k, j:

[0137]

[0138] 1.3 Extended Kalman Filter Algorithm

[0139] Combining the dynamic model (11) and the visual camera measurement model formula (2), the autonomous navigation model for orbiting Jupiter is established:

[0140]

[0141] The EKF was first proposed by Stanley Schmidt and is the basis for the design and implementation of the QLEKF filter. It is also the most commonly used nonlinear filter. To use the EKF to solve the Jupiter orbiting autonomous navigation model, first calculate the linearized state matrix:

[0142]

[0143] in

[0144]

[0145] Substitute into matrix A 3×3 The value of each element:

[0146]

[0147] Perform time updates of the state estimate and the estimated error covariance:

[0148]

[0149] According to formula (2), the linearized measurement matrix is ​​obtained, where i = 1, 2, 3:

[0150]

[0151]

[0152] The covariance expressions of state estimation and estimation error are obtained:

[0153]

[0154] Table 2 shows the process of the EKF algorithm:

[0155]

[0156] Table 2 EKF algorithm process

[0157] 2 Autonomous navigation around Jupiter based on intelligent filtering

[0158] 2.1Q Learning Extended Kalman Filter

[0159] Q-learning is an effective implementation of reinforcement learning (RL). It guides intelligent agents to learn optimal strategies in a stochastic environment through exploration and exploitation. In this process, the intelligent agent performs an action and receives a reward R as feedback from the environment, which serves as an indicator of the effectiveness of the action taken. The Q value represents the expected cumulative reward that can be obtained by taking a specific action in a given state. The intelligent agent maximizes the long-term cumulative reward through a certain action selection strategy. Its basic working principle is as follows: Figure 3 .

[0160] Combining the Q-learning method with the EKF results in the Q-learning extended Kalman filter (QLEKF). The QLEKF can achieve higher estimation accuracy than the traditional EKF by interactively learning with the environment in uncertain environments and adaptively selecting the value of the noise covariance matrix based on reward feedback.

[0161] State set and action space design

[0162] The state set S and the action space A that can be taken in each state are both defined as discrete sets. Figure 4 As shown, the constructed state set S is an M×N grid containing M process noise covariance matrix design values and N measurement noise covariance matrix design values Among them, the design values ​​of the M process noise covariance matrices are and N measurement noise covariance matrix design values The process noise range and measurement noise range obtained by the pre-simulated extended Kalman filter algorithm can be subdivided. The state set S is divided into three regions on a plane with M rows and N columns:

[0163] a.) Corner area:

[0164] (S 11 ,S 1N ,S M1 ,S MN )

[0165] b.) Boundary area:

[0166] (S 1j ,j=2,…,N-1;S i1 ,i=2,…,M-1)

[0167] (S Mj ,j=2,…,N-1;S iN ,i=2,…,M-1)

[0168] c.) Central Area:

[0169] (S ij ,i=2,…,M-1,j=2,…,N-1)

[0170] The action space defines the state transitions that the agent can perform, i.e., the agent can move to adjacent states without obstacles or stay in the current state. The action space corresponding to the state set is as follows Figure 4 As shown by the arrows in , the state set and action transfer are described, where the action space includes moving up, moving down, moving left, moving right, and maintaining the current state.

[0171] Action selection strategy

[0172] The SoftMax strategy calculates the action selection probability based on the size of the Q-value function, and its mathematical expression is formula (3).

[0173] The difference between SoftMax and Greedy algorithms is that the SoftMax strategy can change its preference for exploitation or exploration based on the relative magnitude of the current state S(Mr, Nr) and surrounding states. Specifically, if the Q-value of the current state is much larger than that of other adjacent states, the agent will likely remain in the current state with a higher probability and visit other states with a lower probability, favoring exploitation. Conversely, if the Q-value of the current state S(Mr, Nr) is similar to that of one or more adjacent states, the agent will select actions with relatively equal probability, favoring exploration and avoiding local optima. However, it should be noted that the exponential function in the SoftMax strategy amplifies the Q-value, so the Q-value range should be carefully considered when designing the reward function and initializing the Q-table.

[0174]

[0175] Table 3: SoftMax strategy

[0176] Reward function design

[0177] After the agent performs an action, the environment responds and reflects the agent's performance in the short term in the form of a reward R. The embodiment of the present invention constructs a search EKF filter and an estimation EKF filter to implement the QLEKF-single algorithm.

[0178] To evaluate the performance of a given state-action, the logarithmic function of the innovation sequence is As a reward function. The larger the modulus of the new information sequence, the smaller the reward function. When , the reward function is negative, indicating that the benefit of executing this action is poor. Figure 5 Schematic diagram for QLEKF-single filter design. Conversely, the environment will give positive feedback to the current state action through the reward function. The closer to 0, the greater the increase in the reward function R; the greater the reward function, the better the choice As the estimation accuracy of the noise covariance matrix is ​​higher, the filtering result of QLEKF is more accurate than that of EKF.

[0179] Q-value function and value function

[0180] The Q value measures the size of the cumulative reward. The value is updated once per cycle. The one-step update rule is as follows:

[0181]

[0182] After executing action a, we reach state s'. The maximum Q value that can be achieved based on state s' is recorded as the value function: that is, formula (8)

[0183] In formula (30), Q(s,a) is the Q value of executing action a in state s, R+γV(s') represents the reward that can be brought by this action a, including short-term reward R and long-term reward V(s'); α∈(0,1) is the learning rate, which measures the proportion of the Q function updating rewards and inheriting old values; γ∈(0,1) is the discount factor, which measures the importance of long-term rewards.

[0184] 2.2 Algorithm Design

[0185] Based on the state and action space design, the detector is treated as an intelligent agent and the QLEKF single-soft algorithm is constructed by combining the EKF algorithm and Q-learning principles. The algorithm flow is shown in Table 4. The QLEKF single-greedy algorithm is similar, except that the action selection strategy is changed from the softmax strategy to a greedy strategy. The QLEKF single algorithm periodically updates the state and action, searching for appropriate process noise and observation noise matrices, ultimately obtaining a highly accurate state estimate. The state estimate is an estimate of the position and velocity of the aircraft.

[0186]

[0187]

[0188] Table 4 QLEKFsingle-soft algorithm flow

[0189] The nonlinear time-varying system is simplified into a linear steady-state system. When the system obtains measurement information for more than two consecutive moments, the observability matrix is ​​full rank and the nonlinear system has local observability.

[0190] 3 Simulation Verification

[0191] In order to verify the proposed algorithm, the embodiment of the present invention selects three natural satellites, Io, Callisto and Callisto, as observation sources and verifies them through numerical simulation in the MATLAB environment. The simulation flow chart is as follows: Figure 7 As shown, the orbits of the probe and Jupiter's satellites are as follows Figure 6 The simulation parameter table is shown in Table 5.

[0192]

[0193]

[0194] Table 5 Simulation parameters

[0195] In the simulation, assuming that measurement information is obtained every 1 minute, the simulation parameters, model errors, and Q-learning parameters that need to be considered are shown in Table 5.

[0196] The lower and upper bounds of the state set are logarithmic:

[0197]

[0198] And it is divided into M×N grids with equal spacing. If M=3, then Ensured by logarithmic equidistant segmentation The value search can be performed efficiently while examining as wide a range of values ​​as possible.

[0199] Under the above simulation conditions, the state estimation results are shown in Table 6 and Figures 8 to 20 As shown:

[0200]

[0201] Table 6 Program running time

[0202] The embodiment of the present invention uses the root mean square error as the evaluation index of navigation accuracy:

[0203]

[0204] Figure 8 Table 6 shows the performance of four different filtering algorithms. Among them, EKF has the highest computational efficiency and the lowest accuracy. QLEKF has the highest accuracy but the longest program running time. Compared with QLEKF, QLEKF-single reduces the computational time by about 14%, and the accuracy of QLEKF-soft does not change much compared with QLEKF.

[0205] Figures 9 to 11 The estimated deviations of the positions along the three coordinate directions are shown, where XD1, XD2, XD3, and XD4 are the X-axis position estimation deviation curves of the EKF, QEKF, QLEKF-single, and QLEKF-soft filtering algorithms, respectively; YD1, YD2, YD3, and YD4 are the Y-axis position estimation deviation curves of the EKF, QEKF, QLEKF-single, and QLEKF-soft filtering algorithms, respectively; and ZD1, ZD2, ZD3, and ZD4 are the Z-axis position estimation deviation curves of the EKF, QEKF, QLEKF-single, and QLEKF-soft filtering algorithms, respectively. Figures 12 to 14The estimated deviations of the velocities along the three coordinate directions are shown, where VX1, VX2, VX3 and VX4 are the velocity estimation deviation curves of the X-axis direction for the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; VY1, VY2, VY3 and VY4 are the velocity estimation deviation curves of the Y-axis direction for the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; VZ1, VZ2, VZ3 and VZ4 are the velocity estimation deviation curves of the Z-axis direction for the EKF, QEKF, QLEKF-single and QLEKF-soft filtering algorithms, respectively; the position deviations of the X-axis, Y-axis and Z-axis are 10 4 m, 10 3 m, 10 2 The magnitude of m converges to near 0; the speed errors are all stable around 0, and the oscillation amplitude does not exceed 10m / s. Figures 15 to 17 is the error curve of the position estimation of the QLEKFsingle-soft algorithm, Figures 18 to 20 This is the error curve of the velocity estimation using the QLEKFsingle-soft algorithm. The estimated deviation remains within the 2σ range most of the time. It can be considered that the QLEKFsingle-soft algorithm is suitable for the current Jupiter orbit navigation system.

[0206] During the simulation process, the change in the state set size significantly affects the final convergence results. Assuming M = N, the average RMSE values ​​of the last 100 iterations of the three filtering algorithms in 20 experiments are calculated. The results are shown in the following table:

[0207]

[0208] Table 7 RMSE of state estimation position when the state set size changes

[0209] Table 7 shows the impact of the state set size on the performance of the QLEKFsingle-soft and QLEKFsingle-greedy algorithms. In general, the QLEKFsingle algorithm outperforms the EKF algorithm. When the state set size is moderate (9≤N≤18), the QLEKF-soft algorithm outperforms the QLEKF-greedy algorithm because the Q value table is fully updated. Under state s, the SoftMax strategy is The probability of selecting the suboptimal action is high; the greedy algorithm selects the optimal action with a probability of (1-ε) / A(s), and the suboptimal action is selected with a smaller probability. The SoftMax strategy can more robustly obtain the global optimal solution. When the state set is small, the agent can traverse each state and the Q value table is fully updated. At this time, the results obtained by using the SoftMax strategy and the greedy strategy are similar in accuracy; when the state set is too large (N≥21), the Q value table is a sparse matrix. The SoftMax strategy will amplify the probability of selecting high-value actions at the current moment

[21] and continuously accumulate rewards during the Q value cycle update process, making it impossible for the agent to explore other actions and causing a large error. The result obtained by using the greedy strategy has a smaller error.

[0210] 4. Conclusion

[0211] This embodiment of the present invention introduces the QLEKF-single algorithm as a solution for adjusting the model noise covariance matrix during autonomous navigation around Jupiter. The algorithm verifies its observability and effectiveness, and compares the impact of different action strategies on the estimation results. Through simulation experiments of a probe flying around Jupiter, it is verified that both the SoftMax algorithm and the Greedy algorithm can be used as stable and effective action selection strategies, and that the SoftMax algorithm outperforms the Greedy algorithm when the state set size is moderate: when the noise covariance matrix is ​​difficult to accurately determine, it can find the appropriate noise covariance matrix value within a certain search range to improve the performance of the QLEKF-single algorithm, thereby achieving a more accurate estimation effect than the EKF algorithm, and significantly improving the position and velocity estimation after convergence.

[0212] Due to limited onboard computing resources and the complex space environment, the next step is to estimate the probe's position, velocity, and attitude in real time, even with delayed measurement information and slow computation speeds. Future optimization of parallel filter design and state model construction will allow for adaptation to the increasingly complex and volatile deep space environment and the completion of even more challenging autonomous navigation missions.

[0213] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0214] In the foregoing detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. On the contrary, as reflected in the appended claims, the invention comprises less than all the features of any individual disclosed embodiment. The appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0215] The above description of the disclosed embodiments is intended to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments presented herein but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0216] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it will be appreciated by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to encompass all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, to the extent the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including". In addition, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

[0217] Those skilled in the art will also appreciate that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0218] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A QLEKF-based autonomous navigation method for Jupiter exploration orbiting, characterized in that: include: Obtaining a two-dimensional pixel image of a Jovian satellite plane using an onboard camera, and determining a relative position vector between each of three preset Jovian satellites and the position of the probe in a Jovian inertial coordinate system based on the two-dimensional pixel image of the Jovian satellite plane; Determine the observed variable based on the relative position vector of each of the three pre-set satellites of Jupiter and the position of the probe; In each reinforcement learning cycle of the reinforcement learning, determining the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle; Obtaining, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix; Determining a reinforcement learning Q value corresponding to the reinforcement learning cycle using a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining a state estimate of the operating state of the detector using a second extended Kalman filter; Determining whether the reinforcement learning cycle has reached a preset convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is determined that the reinforcement learning cycle has not reached the preset convergence condition, continuing to execute the next reinforcement learning cycle of the reinforcement learning; if it is determined that the reinforcement learning cycle has reached the preset convergence condition, using the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector; The operating state of the detector is calibrated based on the target state estimate and preset orbit data.

2. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF according to claim 1, characterized in that: The method comprises obtaining a two-dimensional pixel image of a Jupiter satellite plane by an onboard camera, and determining, in the Jupiter inertial coordinate system, a relative position vector of the probe's position relative to each of the three preset Jupiter satellites based on the two-dimensional pixel image of the Jupiter satellite plane. Specifically, the method comprises determining the relative position vector of the probe's position relative to each of the three preset Jupiter satellites based on the following formula: Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, f is the focal length of the camera; r ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors of the three satellites to the probe in Jupiter’s inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

3. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF according to claim 1, characterized in that: The observation variable is determined based on the relative position vector of each of the three preset satellites of Jupiter and the position of the probe, specifically: the observation variable is determined according to the following formula: Among them, r ik =[x ik y ik z ik ] T , i=1,2,3, represents the relative position vectors of the three satellites and the probe in Jupiter's inertial system; r k Represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

4. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The determining, based on the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and the reinforcement learning state corresponding to the reinforcement learning cycle includes: For each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle is determined according to the following formula: Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize; Selecting the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle; The reinforcement learning state corresponding to the reinforcement learning cycle is determined according to the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

5. The autonomous navigation method for Jupiter exploration orbiting segment based on QLEKF as claimed in claim 1, characterized in that: The method further comprises determining a reinforcement learning Q value corresponding to the reinforcement learning cycle through a first extended Kalman filter based on the observed variables, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and determining a state estimate of the operating state of the detector through a second extended Kalman filter, including: For each time step of a preset number of time steps, according to the following formula, the first state estimate and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the first extended Kalman filter to obtain the first state estimate, the first error covariance matrix, and the first innovation sequence corresponding to the time step: in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix; Based on the first new information sequence corresponding to the time step and the reward value obtained in the previous time step, the reward value corresponding to the current time step is updated according to the following formula: Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step; According to the following formula, the second state estimate corresponding to the previous time step and the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle are input into the second extended Kalman filter to obtain the second state estimate corresponding to the time step, the second error covariance matrix, and the second innovation sequence: in, is the second state estimate corresponding to the time step, P k is the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step; After all time steps of the preset number of time steps are executed, the second state estimation value corresponding to the last time step is used as the state estimation of the operating state of the detector, and the reward value corresponding to the last time step is used as the reward value corresponding to the reinforcement learning cycle, and according to the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, the reinforcement learning Q value corresponding to the reinforcement learning cycle is determined according to the following formula: Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)] V(s’)←max a Q(s’,a) Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of Q function updating rewards and inheriting old values; γ∈(0,1) is the discount factor, which is used to measure the importance of long-term rewards; the maximum Q value that can be achieved based on state s' is recorded as the value function 'V(s).

6. A QLEKF-based autonomous navigation device for Jupiter exploration and orbiting, characterized in that: include: a data acquisition unit configured to obtain a two-dimensional pixel image of a Jovian satellite plane using an onboard camera, and determine, based on the two-dimensional pixel image of the Jovian satellite plane, a relative position vector between each of three preset Jovian satellites and the position of the probe in the Jovian inertial coordinate system; an observation variable determination unit, configured to determine an observation variable based on a relative position vector between the position of each of the three preset satellites of Jupiter and the probe; a reinforcement learning state determining unit, configured to determine, in each reinforcement learning cycle of reinforcement learning, a reinforcement learning action and a reinforcement learning state corresponding to the reinforcement learning cycle according to the reinforcement learning state corresponding to the previous reinforcement learning cycle; a noise matrix determining unit, configured to obtain, from a reinforcement learning state space, a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, according to the reinforcement learning state corresponding to the reinforcement learning cycle; wherein the reinforcement learning state space includes a plurality of reinforcement learning states arranged in a matrix form; each reinforcement learning state corresponds to a different noise covariance matrix; a filtering processing unit, configured to determine, through a first extended Kalman filter, a reinforcement learning Q value corresponding to the reinforcement learning cycle based on the observed variables, a reinforcement learning state corresponding to a previous reinforcement learning cycle of the reinforcement learning cycle, a reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle, and a noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, and to determine, through a second extended Kalman filter, a state estimate of the operating state of the detector; a convergence judgment unit, configured to judge whether the reinforcement learning cycle has reached a preset convergence condition based on the reinforcement learning Q value corresponding to the reinforcement learning cycle; if it is judged that the reinforcement learning cycle has not reached the preset convergence condition, continue to execute the next reinforcement learning cycle of the reinforcement learning; if it is judged that the reinforcement learning cycle has reached the preset convergence condition, use the state estimate of the operating state of the detector as the target state estimate of the operating state of the detector; A state calibration unit is used to calibrate the operating state of the detector based on the target state estimation and preset orbit data.

7. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF according to claim 6, characterized in that: The data acquisition unit is specifically configured to determine a relative position vector of the position of the probe relative to each of the three preset satellites of Jupiter according to the following formula: Among them, x ik,c ,y ik,c is the imaging plane coordinate of the centroid feature point of satellite i extracted from the image taken by the camera, f is the focal length of the camera; r ik =[x ik y ik z ik ] T ,i=1,2,3,represents the relative position vectors of the three satellites to the probe in Jupiter’s inertial system; is the rotation matrix from Jupiter's inertial coordinate system to the probe's body coordinate system; is the rotation matrix from the detector body coordinate system to the camera coordinate system; g k is the measurement noise.

8. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF according to claim 6, characterized in that: The observed variable determination unit is specifically used to determine the observed variable according to the following formula: Among them, r ik =[x ik y ik z ik ] T , i=1,2,3, represents the relative position vectors of the three satellites and the probe in Jupiter's inertial system; r k Represents the relative position vector from the probe to the center of mass of Jupiter, g k To measure noise.

9. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF according to claim 6, characterized in that: The reinforcement learning state determination unit includes: An execution probability determination module is configured to determine, for each reinforcement learning action in the reinforcement learning action space, the execution probability corresponding to the reinforcement learning action under the reinforcement learning state corresponding to the previous reinforcement learning cycle according to the following formula: Among them, Q(s,a i ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle i , the reinforcement learning Q value, Q(s,a j ) is to execute the reinforcement learning action a in the reinforcement learning action space under the reinforcement learning state s corresponding to the previous reinforcement learning cycle j , the obtained Q value, the temperature parameter temp measures the agent's preference for exploration and utilization. When temp tends to ∞, the SoftMax algorithm tends to only explore, and when temp tends to 0, it tends to only utilize; a reinforcement learning action determination module, configured to select the reinforcement learning action corresponding to the maximum execution probability as the reinforcement learning action corresponding to the reinforcement learning cycle; A reinforcement learning state determination module is configured to determine the reinforcement learning state corresponding to the reinforcement learning cycle based on the reinforcement learning action corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle, and the reinforcement learning state space, wherein the reinforcement learning action space includes multiple reinforcement learning actions; the reinforcement learning state space includes multiple reinforcement learning states; and the reinforcement learning action is used to trigger the transfer of the reinforcement learning state.

10. The autonomous navigation device for Jupiter exploration and orbiting based on QLEKF according to claim 6, characterized in that: The filtering processing unit includes: The first filtering processing module is configured to input the first state estimate and the first error covariance matrix corresponding to the previous time step, the observed variables, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into the first extended Kalman filter for each time step of a preset number of time steps, to obtain the first state estimate, the first error covariance matrix, and the first innovation sequence corresponding to the time step according to the following formula: in, is the first state estimate corresponding to the time step, is the first error covariance matrix corresponding to the time step, is the first new information sequence corresponding to the time step, The first state estimate corresponding to the previous time step, is the first error covariance matrix corresponding to the previous time step, y k is the observed variable, is the process noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle, A measurement noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop, wherein the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning loop includes a process noise covariance matrix and a measurement noise covariance matrix; The reward value update module is used to update the reward value corresponding to the current time step according to the first new information sequence corresponding to the time step and the reward value obtained in the previous time step according to the following formula: Among them, R t is the reward value obtained at this time step, R t-1 is the reward value obtained in the previous time step, is the first new information sequence corresponding to the time step; The second filtering processing module is used to input the second state estimate corresponding to the previous time step and the second error covariance matrix, the observed variable, and the noise covariance matrix corresponding to the reinforcement learning state corresponding to the reinforcement learning cycle into the second extended Kalman filter according to the following formula to obtain the second state estimate corresponding to the time step, the second error covariance matrix and the second innovation sequence: in, is the second state estimate corresponding to the time step, P k is the second error covariance matrix corresponding to the time step, is the second new information sequence corresponding to the time step, The second state estimate corresponding to the previous time step, P k-1 is the second error covariance matrix corresponding to the previous time step; The reinforcement learning Q-value update module is configured to, after all time steps of a preset number of time steps have been executed, use the second state estimation value corresponding to the last time step as the state estimation of the detector's operating state, and use the reward value corresponding to the last time step as the reward value corresponding to the reinforcement learning cycle, and determine the reinforcement learning Q-value corresponding to the reinforcement learning cycle according to the following formula based on the reward value corresponding to the reinforcement learning cycle, the reinforcement learning state corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, the reinforcement learning action and reinforcement learning state corresponding to the reinforcement learning cycle: Q(s,a)←Q(s,a)+α[R+γV(s')-Q(s,a)] V(s’)←max a Q(s’,a) Among them, Q(s,a) is the Q value of executing the reinforcement learning action a corresponding to the reinforcement learning cycle under the reinforcement learning state s corresponding to the previous reinforcement learning cycle of the reinforcement learning cycle, R is the reward value corresponding to the reinforcement learning cycle, that is, the short-term reward, R+γV(s') represents the reward that can be brought by the reinforcement learning action a, including the short-term reward R and the long-term reward V(s'); α∈(0,1) is the learning rate, which is used to measure the proportion of Q function updating rewards and inheriting old values; γ∈(0,1) is the discount factor, which is used to measure the importance of long-term rewards; the maximum Q value that can be achieved based on state s' is recorded as the value function V(s').

Images