Inter-satellite navigation method, system storage medium and computer equipment

Abstract

The embodiment of the invention provides an inter-satellite navigation method, a system storage medium and computer equipment, and the method comprises the following steps: 1, establishing a detector state variable under a lunar center inertial coordinate system; the method comprises the following steps: constructing observation variables of a spaceborne camera through pictures of two satellites which are shot by the spaceborne camera on a detector and are subjected to the gravity of the moon; wherein the detector is arranged on the spacecraft; 2, establishing a kinetic model and a measurement equation of the satellite; step 3, based on a kinetic model and a measurement equation of the satellite, constructing a QLEKF state equation improved based on a reward function; and step 4, solving according to a QLEKF state equation improved based on a reward function to obtain the state of the detector. According to the method, the detector state estimation result can be quickly converged under the condition that noise in a space environment is uncertain, the problem of state estimation divergence in an autonomous navigation algorithm is effectively relieved, and the operation efficiency and the positioning precision are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of navigation, and in particular to an inter-satellite navigation method, system storage medium and computer device. Background Art

[0002] Deep space exploration has become a key field to promote scientific progress and expand the boundaries of scientific research, and has received extensive attention and continuous investment from countries around the world.

[0003] In deep space exploration missions, the high-precision autonomous navigation of detectors is the key to ensuring the success of space missions. Due to the complex space environment and limited on-board resources (a satellite refers to a spacecraft carrying a detector, and resources mainly refer to the computing resources used by the detector for navigation), detectors usually cannot obtain sufficient measurement information through ground stations or external signals. Therefore, an inter-satellite navigation method that uses the relative position and velocity information between the detector and satellites (one detector and three lunar satellites providing observation information) for positioning has become an effective option. Taking an optical sensor as an example, the detector obtains the line-of-sight direction vector (LOS) of the observation target (the observation target is a lunar satellite) through an on-board camera, and combines it with the orbital dynamics model of the spacecraft (carrying the detector), and estimates the position and velocity vectors of the detector through a filtering algorithm.

[0004] Due to the fact that the space environment is easily affected by factors such as space rays, cosmic radiation, space dust and extreme temperature differences, there are uncertainties in the error characteristics during the state estimation of the detector and the measurement of the sensor, which significantly affects the accuracy of the filtering algorithm. In response to these error uncertainties, a variety of adaptive filtering methods are currently applied to estimate the noise matrix, such as the adaptive extended Kalman filter (AEKF), the multiple model adaptive estimation (MMAE) algorithm, the adaptive filtering algorithm based on variational Bayesian (VB) theory, etc.

[0005] The above algorithms have been widely applied in noise-uncertain scenarios. However, there are challenges such as limited on-board computing resources and high real-time requirements in space exploration missions.

[0006] With the development of intelligent algorithms, many adaptive navigation systems have begun to be optimized by combining machine learning (such as deep learning, reinforcement learning, etc.). Bekhtaoui combined the Kalman filter with the temporal difference method and proposed the QLKF algorithm to solve the tracking problem of a single maneuvering target. This algorithm can adjust the process noise of the Kalman filter in a timely manner when the target mode changes. Further, based on the principle of reinforcement learning, the EKF algorithm is combined with the Q-learning algorithm to propose the Q-learning extended Kalman filter algorithm (QLEKF). The QLEKF algorithm obtains feedback by interacting with the environment, adaptively adjusts the estimated value of the noise covariance, and at the same time ensures high operating efficiency and small computational complexity. Its estimation accuracy has been verified in various scenarios. At the same time, research on high-precision navigation using the QLEKF algorithm has been carried out in the context of cruising around the solar system boundary. The attitude estimation effect of a rigid body based on MARG sensors has been improved by the Q-learning method. The rigid body speed estimation has been improved using the QLEKF algorithm. In addition, the Q-learning unscented Kalman filter (QLUKF) is used to accurately estimate the relative position and speed of satellites in a formation.

[0007] However, due to the impact of noise uncertainty brought by complex environments, it is impossible to achieve high-precision inter-satellite navigation positioning to realize the high-precision autonomous navigation of detectors. Summary of the Invention

[0008] Embodiments of the present invention provide an inter-satellite navigation method, system storage medium, and computer device, which can solve the problem in the prior art of "due to the impact of noise uncertainty brought by complex environments, it is impossible to achieve high-precision inter-satellite navigation positioning to realize the high-precision autonomous navigation of detectors".

[0009] To achieve the above object, in the first aspect, an inter-satellite navigation method provided by an embodiment of the present invention includes:

[0010] Step 1: Establish a detector state variable in the lunar-centered inertial coordinate system; construct an observation variable of the on-board camera through pictures of three satellites affected by the lunar gravity taken by the on-board camera of the detector; wherein, the detector is arranged on a spacecraft.

[0011] Step 2: Establish a dynamic model and a measurement equation of the detector.

[0012] Step 3: Based on the dynamic model and the measurement equation of the detector, construct a QLEKF state equation improved based on a reward function.

[0013] Step 4: Solve according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

[0014] In the second aspect, an inter-satellite navigation system provided by an embodiment of the present invention includes:

[0015] A variable setting unit is configured to establish the state variables of the detector in the selenocentric inertial coordinate system; and construct the observation variables of the on-board camera by using the pictures of three satellites affected by the lunar gravity taken by the on-board camera of the detector; wherein, the detector is disposed on a spacecraft.

[0016] A modeling unit is configured to establish the dynamic model and measurement equation of the detector.

[0017] A reward function construction unit is configured to construct a QLEKF state equation improved based on the reward function based on the dynamic model and measurement equation of the detector.

[0018] A solving unit is configured to solve according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

[0019] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium storing one or more programs, and when the one or more programs are executed by a computer device, the computer device is caused to execute the aforementioned inter-satellite navigation method.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer device, including:

[0021] a processor; and a memory arranged to store computer-executable instructions, and when the executable instructions are executed, the processor is caused to execute the aforementioned inter-satellite navigation method.

[0022] The above technical solution has the following beneficial effects: it can make the detector state estimation result converge quickly in the case of noise uncertainty in the space environment, effectively alleviate the problem of state estimation divergence in the autonomous navigation algorithm, and significantly improve the operation efficiency and positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0024] Figure 1 is a flowchart of an inter-satellite navigation method according to an embodiment of the present invention;

[0025] Figure 2 is the detector camera measurement principle according to an embodiment of the present invention;

[0026] Figure 4 is the state space and action transfer according to an embodiment of the present invention;

[0027] Figure 6 It is the lateral movement transfer of the embodiment of the present invention;

[0028] Figure 5 It is the longitudinal movement transfer of the embodiment of the present invention;

[0029] Figure 7 It is the algorithm flow chart of the embodiment of the present invention;

[0030] Figure 3 It is the lunar satellite trajectory diagram of the embodiment of the present invention;

[0031] Figure 8 It is the filtered estimated position result of the embodiment of the present invention;

[0032] Figure 9 It is the box plot of the filtered estimated result under the measurement error variation of the embodiment of the present invention;

[0033] Figure 10 It is the line chart of the filtered estimated result under the measurement error variation of the embodiment of the present invention;

[0034] Figure 11 It is the position deviation of the QLEKF-step algorithm of the embodiment of the present invention;

[0035] Figure 12 It is the filtered estimated position error value under the variation of the γ parameter of the embodiment of the present invention;

[0036] Figure 13 It is the filtered estimated position error value under the variation of the ε parameter of the embodiment of the present invention;

[0037] Figure 14 It is the box plot of the filtered estimated position error under the variation of the γ parameter of the embodiment of the present invention;

[0038] Figure 15 It is the box plot of the filtered estimated position error under the variation of the ε parameter of the embodiment of the present invention. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] As Figure 1 shown, in combination with the embodiments of the present invention, a method for inter-satellite navigation is provided, including:

[0041] Step 1: Establish the state variables of the detector in the lunar-centered inertial coordinate system; construct the observation variables of the on-board camera through the pictures of three satellites affected by the lunar gravity taken by the on-board camera of the detector; where the detector is installed on the spacecraft.

[0042] Step 2: Establish the dynamic model and measurement equation of the detector.

[0043] Step 3: Based on the dynamic model and measurement equation of the detector, construct a QLEKF state equation improved based on the reward function.

[0044] Step 4: Solve according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

[0045] The above technical solutions of the embodiments of the present invention will be described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, reference can be made to the relevant descriptions above.

[0046] Inter-satellite autonomous navigation is of great significance for deep space exploration and resource development. In order to handle the influence of noise uncertainty brought by complex environments and achieve high-precision inter-satellite navigation and positioning,

[0047] Taking the lunar satellite constellation scenario as an example, the state of the detector is estimated based on inter-satellite relative measurement. An inter-satellite navigation method based on the improved QLEKF algorithm, a Q-learning extended Kalman filter (QLEKF-step) algorithm improved based on the reward function. The Q-learning extended Kalman filter (QLEKF-step) algorithm adaptively adjusts the process noise matrix and measurement noise matrix respectively based on the innovation sequence and one-step estimation error by designing a synchronously updated reward function, so as to optimize the estimation of the noise covariance.

[0048] In order to further reduce the estimation error boundary, two parallel updated reward functions are adopted, and the noise matrix values with smaller estimation errors are continuously selected respectively based on the innovation sequence and one-step update error. Applying this algorithm to the inter-satellite navigation scenario of the lunar constellation verifies its effectiveness.

[0049] The experimental results show that compared with the traditional method of estimating the noise matrix only based on the innovation sequence, the QLEKF-step algorithm speeds up the Q-value table update process, and at the same time, effectively reduces the divergence phenomenon and improves the convergence accuracy by more than 10%.

[0050] I. Dynamic and Measurement Models

[0051] Taking the lunar satellite constellation scenario as an example, the state of the detector is estimated based on inter-satellite relative measurement. The definitions of the coordinate system, detector state variables, and measurement variables under discrete-time conditions are introduced below. As Figure 2 shown.

[0052] (1) Coordinate System

[0053] 1. Lunar-centered Inertial Coordinate System (i-system): The origin O is defined as the center of the moon. OX lies in the lunar equatorial plane and points towards the vernal equinox. OZ points towards the positive normal direction of the lunar equatorial plane, and OY, together with OX and OZ, forms a right-handed coordinate system.

[0054] 2. Probe Body Coordinate System (b-system): The origin is defined as the center of mass O of the probe. OX, OY, and OZ are respectively along the three inertial principal axes of the probe and form a right-handed coordinate system.

[0055] 3. Camera Coordinate System (c-system): The origin is defined as the optical center O of the spaceborne camera. OY is parallel to the image plane, and OZ is perpendicular to the image plane, forming a right-handed coordinate system with OX and OY.

[0056] (2) Probe State Variables

[0057] In the lunar-centered inertial coordinate system, the state variables of the probe at time k are defined as follows:

[0058]

[0059] where r k and v k are respectively the relative position vector and velocity vector of the probe to the center of the moon at time k. x k , y k , z k are the position parameters in three directions, and are the velocity parameters in three directions.

[0060] (3) Observation Variables of the Spaceborne Camera

[0061] The probe uses the spaceborne camera to image and identify three satellites, capture the two-dimensional pixel data of their plane (the camera imaging plane), and extract the observed values. Denote the LOS vector pointing to satellite i in the camera coordinate system at time k as:

[0062]

[0063] where x ik,c , y ik,c are the coordinates of the centroid feature point of satellite i in the imaging plane extracted from the image taken by the spaceborne camera, and f represents the camera focal length. r k is the relative position vector of the probe to the center of the moon at time k, r ik = [x ik y ik z ik T , i = 1, 2, 3 represents the relative position vectors of the three satellites to the center of the moon in the lunar-centered inertial coordinate system; ​

[0064] is the rotation matrix from the lunar-centered inertial coordinate system (i-system) to the detector body coordinate system (b-system); is the rotation matrix from the detector body coordinate system (b-system) to the camera coordinate system (c-system), and η k represents the measurement noise at time k.

[0065] 1.1 Dynamic Model

[0066] Assume that the three (observation-providing) satellites are mainly affected by the lunar gravity during the fly-around phase. Only considering the J2 perturbation term, the process noise w k is used to describe the error caused by neglecting other perturbation terms. The fly-around model of the detector's vehicle dynamics is established in the lunar-centered inertial coordinate system:

[0067] x k = f(x k-1 ) + w k (4)

[0068] where w k is the process noise, and f(x k-1 ) represents the non-linear state transition function of the detector;

[0069] Among them, the first-order Taylor expansion of the non-linear state transition function is:

[0070] f(x k-1 ) = x k-1 + φ(x k-1 )τ (5)

[0071]

[0072] Among them, r k in formula (6) represents the distance from the detector to the lunar center, τ represents the time step, μ m is the gravitational constant of the moon, J m2 is the lunar spherical harmonic coefficient, w k is the process noise (vector), and the statistical characteristics of the process noise are as follows:

[0073]

[0074] where w j represents the process noise at time j, Q k represents the system process noise covariance matrix at time k, and δ kt is the Kronecker function, which satisfies:

[0075]

[0076] where t represents a certain moment during the operation of the detector.

[0077] According to the lunar orbit dynamics, the values and meanings of the parameters are shown in Table 1.

[0078] Table 1 Dynamic model parameters

[0079]

[0080] 1.2 Measurement model

[0081] During the on-orbit circumvolution of the detector, the on-board camera can use the line-of-sight vector (LOS) of adjacent constellation satellites around the moon as the measurement information source. Since the attitude information of the on-board camera is known, the influence of coordinate system conversion on the magnitude of the observed variable is not considered, that is, it is considered that is the identity matrix, and the detector measurement model can be written as:

[0082]

[0083] Equation (9) is part of Equation (3). The first one in Equation (3) is used to obtain data, and the second one is used to establish a model.

[0084] Among them, r 1k represents the relative position vector from the first satellite to the lunar center in the lunar-centered inertial coordinate system, r 2k represents the relative position vector from the second satellite to the lunar center in the lunar-centered inertial coordinate system, r 3k represents the relative position vector from the third satellite to the lunar center in the lunar-centered inertial coordinate system, r k is the relative position vector of the detector to the lunar center at time k, η k is the detector measurement noise at time k, and the statistical characteristics of the measurement noise are:

[0085]

[0086] Among them, η k represents the detector measurement noise at time k, R k represents the system measurement noise covariance matrix at time k, δ kt is the Kronecker function, satisfying:

[0087]

[0088] And the process noise w k is independent of the measurement noise η k , that is, for any k, t there is:

[0089]

[0090] Among them, t represents a certain moment during the operation of the detector.

[0091] II. QLEKF Algorithm Improved Based on Reward Function

[0092] 2.1 State Space and Action Transition

[0093] As Figure 4 shown, both the state space S and the action space A that can be taken by the current state are defined as discrete sets. To reduce the volume of the Q-table and search for the noise estimation value directionally, two one-dimensional state sets are defined respectively, representing the covariance matrix value sets of the process noise w k and the measurement noise η k :

[0094]

[0095] The Cartesian product of the two one-dimensional state sets is taken to obtain the two-dimensional state space S = I × J, and a mapping is defined in the state space:

[0096]

[0097] The action space is defined as A = L × W, where L = {-1, 0, 1} and W = {-1, 0, 1}. The action selected at time k based on the current state s k is denoted as a k =(l k , w k ) ∈ A, and l k ∈ L, w k ∈ W. By taking the action a k the agent transfers to the next-time state s k+1 =(i k + l k , j k + w k ). When i k = 1, l k ≠ -1; when i k = M, l k ≠ 1, M represents the number of design values of the process noise covariance matrix, and i k represents the serial number of the covariance matrix design value selected at time k, corresponding to j k Similarly, similar boundary constraints can be made: when j k = 1, w k ≠ -1; when i k = N, w k ≠ 1, N represents the number of design values of the measurement noise covariance matrix, and j k represents the serial number of the covariance matrix design value selected at time k, corresponding to As Figure 6 and Figure 5 shown.

[0098] 2.2 Reward Function and Filter Design

[0099] Construct parallel filters: Use the benchmark EKF (trEKF) with nominal noise covariance values (Q0, R0), and use Q-learning to update the noise covariance values of the search EKF (seEKF), and record the innovation sequences separately and denote the innovation sequence of trEKF, denote the innovation sequence of seEKF, and take the difference between the two as the reward function. The reward function at time k is:

[0100]

[0101] By synchronously adjusting the value of, continuously accumulate the rewards and record the cumulative reward values in the form of a table for each state, which is called the Q-value table (store the Q-values in the corresponding state); further, select actions according to the Q-value size through the greedy strategy, and execute the actions to obtain new values. The synchronous adjustment strategy requires maintaining a Q-value table of size M×N×|A|, where |A| is the number of actions that can be selected in the current state , |A| ∈ [3, 5], M represents the number of design values of the process noise covariance matrix, and N represents the number of design values of the measurement noise covariance matrix.

[0102] To further optimize the update process, combined with the plasticity of the reward function based on the potential function, the embodiment of the present invention adopts a new reward function:

[0103] R sa (k) = R1(k) + R2(k) (13)

[0104]

[0105] where, denotes the estimated error variance matrix of the state vector at time k-1, R sa (k) is the total reward at time k, and R1(k) and R2(k) respectively reflect the measurement error and the state error, denotes the one-step estimation error of seEKF, denotes the one-step estimation error of the estimated EKF (esEKF). Construct parallel filters: Use the benchmark EKF (trEKF) with nominal noise covariance values (Q0, R0), and use Q-learning to gradually update of the search EKF (seEKF) and update of the estimated EKF (esEKF), denotes that the estimated process noise covariance matrix at time k is the i-th design value, It is indicated that the measurement noise covariance matrix at time k is the j-th design value.

[0106] Its working principle is as Figure 7 shown.

[0107] Record the innovation sequences of the reference EKF and the search EKF, as well as the one-step update errors of the search EKF and the estimation EKF, to calculate the reward function value:

[0108]

[0109] Gradually adjust the value by alternating updates. In this way, it only needs to store and update a table of size (M + N)×3 at most. In addition, alternating updates can alleviate the influence of one parameter with inaccurate estimation on the other parameter in a pair of parameters, that is, the coupling is not so strong and they can be adjusted separately.

[0110] 2.3 Value function and Q function update

[0111] Similar to the Q value table in QLEKF, establish tables Q1 and Q2 to record the cumulative reward functions R1 and R2 under the current state s(i,j) and action a(l,w) respectively, and update them once per period (T time instants). The one-step update rule is as follows:

[0112]

[0113] Among them, After executing action a(l,w), the state s′(i′,j′) is reached. i′ represents the value state of the process noise matrix after action transfer, taking j′ represents the value state of the measurement noise matrix after action transfer, taking l and w are related to the current state i and j, and are the actions obtained by the greedy algorithm under the current state. i and j restrict the values of l and w to make the action transfer meaningful.

[0114] Denote the maximum Q value that can be achieved based on state s′ as the value function:

[0115]

[0116] In equations (18) and (19), Q1(i,l) and Q2(j,w) are the Q values of executing action a(l,w) in state s(i,j), represents the Q value before update, represents the Q value after update, represents the Q value before update, represents the Q value after update;

[0117] R1 + γV1(i′) and R2 + γV2(j′) represent the rewards that the current action a(l, w) can bring, including the short-term rewards R1, R2 and the long-term rewards V1(i′), V2(j′); α ∈ (0, 1) is the learning rate, which measures the proportion of updating the reward of the Q function and inheriting the Q value before updating; γ ∈ (0, 1) is the discount factor, which measures the importance of the long-term reward.

[0118] Update the process noise table and the measurement noise table respectively. Take the initialized Q table when M = N = 6. M represents the number of design values of the process noise covariance matrix, and N represents the number of design values of the measurement noise covariance matrix, as shown in Tables 2 and 3.

[0119] Table 2 Process Noise Table Q1

[0120]

[0121] Table 3 Measurement Noise Table Q2

[0122]

[0123] 2.4 Algorithm Design

[0124] Regard the detector as an agent. According to the state and action space design, combine the EKF algorithm and the Q learning principle to construct the QLEKF-step algorithm.

[0125] Table 4 Algorithm Flow

[0126]

[0127] In Step 4, in the first time period, at each time step k, randomly select the state s(mr, nr), and calculate Q1(mr, l) = R1 and Q2(nr, w) = R2 respectively, and update all actions in this state. In this way, initialize the Q value table. It should be noted that if there are too many time steps in the time period, the time steps consumed by initialization should be reduced to prevent excessive error accumulation caused by randomly selecting states during the initialization process.

[0128] In Step 5, use the greedy algorithm to select the horizontal action l and the vertical action w according to the tables Q1(mr, l) = R1 and Q2(nr, w) = R2 respectively. Taking the horizontal action as an example, the principle of the greedy policy action selection is shown in Table 5:

[0129] Table 5 Greedy Policy

[0130]

[0131] In the greedy action selection strategy, explore randomly with a probability of ε and exploit with a probability of 1 - ε: If there are multiple alternative actions with the same Q1 or Q2, randomly select one of them.

[0132] III. Theoretical Proof

[0133] 3.1 Consistency

[0134] Definition Briefly analyze the Q function update process. From formulas (18) and (19), the one-step update formulas for the Q1(i, l) and Q2(j, w) functions are:

[0135]

[0136] Q2(j, w) ← (1 - α)Q2(j, w) + α[R2 + γV2(j′)] (22)

[0137] Corollary 1:

[0138] Assume 0 < α ≤ 1, 0 < γ ≤ 1. Under this condition, according to the recurrence formulas (21) and (22), the function There exists an optimal policy π s such that in the state space s(i, j), it converges to the optimal Q function Q target (s, a) through the step-by-step action transition a(l, w), that is

[0139] Proof:

[0140] Taking the Q1 function as an example, analyze the error change using the recurrence method. Let's denote Q 1,target as the optimal Q1 function, and Q 1,new (s, a), Q 1,old (s, a) are respectively denoted as Let's assume the maximum error at the nth step is:

[0141]

[0142] The Q 1,target function under the optimal policy is:

[0143]

[0144] To simplify the analysis, assume that the reward function values obtained for the same state-action pair (i, l) are the same. Then we have:

[0145]

[0146] For the term corresponding to the weight coefficient 1 - α, we have

[0147]

[0148] For the term corresponding to the weight coefficient α, there is

[0149]

[0150] Therefore, there is

[0151]

[0152] Let the error between the initialized Q1(x, l) function and the Q 1,target function obtained by the optimal policy be Δ0. Then when 0 < γ < 1, The maximum error between and Q 1,target does not exceed [(1 - α) + αγ] n Δ0, and when n → ∞, Δ 1,n → 0. Therefore, for any state-action pair (i, l), there exists an optimal policy π i such that converges to Q 1,target (i, l); Similarly, it can be proved that there exists an optimal policy π j such that converges to the optimal Q function Q 2,target (j, w).

[0153] On the other hand, define Examine whether there exists an optimal policy π s such that through the step-by-step action transfer a(l, w) in the state space s(i, j), it converges to the optimal Q function Q target (s, a). Since the reward function is linearly additive, then:

[0154] R sa (s, a) = R1(i, l) + R2(j, w) (27)

[0155] And the action sets L, W are sets rather than vectors. Therefore:

[0156]

[0157] Then there is

[0158]

[0159] From The convergence of finally leads to:

[0160]

[0161] That is, there exists an optimal policy π s such that the optimal Q function Q target (s, a) is reachable.

[0162] 3.2 Boundedness

[0163] Prove the effectiveness of the algorithm by the boundedness of the estimation error. Denote:

[0164]

[0165] Let be Taylor-expanded at :

[0166]

[0167] Denote Let be decomposed and Taylor-expanded at to obtain:

[0168] where

[0169]

[0170] To analyze the boundedness of the estimation error, we establish sufficient conditions and use Theorem 1 to verify that the filtering error is exponentially bounded in the mean square sense.

[0171] Theorem 1:

[0172] (1) If there exist real constants such that the following is satisfied in the estimation filter (esEKF):

[0173]

[0174] and there exist positive real numbers ε φ , ε χ such that the nonlinear functions φ and χ satisfy the following boundedness conditions:

[0175]

[0176] (2) Let the parameter such that Then for any k ≥ 0, under the conditions that (1) and (2) hold, the expectation of the estimation error has an upper bound and converges exponentially:

[0177]

[0178] Denote l as the dimension of matrix Q k and m as the dimension of matrix R k There is in Equation (35):

[0179]

[0180] Lemma 1:

[0181] For any vectors a, b and matrix M > 0, as well as any constant β > 0, the following inequality holds:

[0182] (a + b) T M(a + b) ≤ (1 + β -1 )a T Ma + (1 + β)b T Mb(37)

[0183] Based on Definition 1 and Lemma 1, we can obtain Corollary 2, which verifies that the cumulative reward function value in the reinforcement learning process can reduce the upper bound of the error expectation, making the convergence result of the filter more stable.

[0184] Corollary 2:

[0185] Suppose there exists a real constant such that in the search filter (seEKF), it satisfies Then there exists an upper bound for the error expectation of the estimation filter (esEKF), and it decreases as the reward function value accumulates:

[0186]

[0187] Proof:

[0188] From equations (16) and (17), define the reward function:

[0189]

[0190]

[0191] Denote Then:

[0192]

[0193] There is

[0194]

[0195] Combined with the definition of the reward function, the inequality holds:

[0196]

[0197] Therefore, the second term in equation (40) can be rewritten as:

[0198]

[0199] Because And from the equivalent form of the filtering gain:

[0200]

[0201] The upper bound of the filtering gain of the search filter can be determined:

[0202]

[0203] Therefore

[0204]

[0205] The second term of equation (40) is equivalent to

[0206]

[0207] Also, because Assume that the estimation error at the previous moment is uncorrelated with the process noise w at the current moment k Consider the first term in (40):

[0208]

[0209] According to Lemma 1, there exists a constant β1 > 0 such that

[0210]

[0211] According to the Jensen inequality, we have It is easy to know from the triangle inequality that:

[0212]

[0213] According to Theorem 1 There is an upper bound, so let where

[0214]

[0215] So

[0216]

[0217] As the reward function values R1(k) and R2(k) accumulate, the upper bound of the error becomes smaller.

[0218] IV. Simulation verification

[0219] To verify the effectiveness of the QLEKF algorithm, assume that the initial orbital parameters of three lunar constellation satellites are shown in Table 6, and the lunar satellite trajectory diagram is as Figure 3 shown.

[0220] Table 6 Initial orbital parameters of constellation satellites

[0221]

[0222] Table 7 Simulation parameter table

[0223]

[0224] During the simulation process, the standard deviation of the initial state estimation position error is set to σ x = σ y = σ z = 400 m, and the standard deviation of the velocity error σ vx = σ vy = σ vz = 3 m / s. Under the simulation conditions in Table 7, experiments are conducted on a computer with RAM 16.0 GB, Intel(R) Core(TM) i7-13620H CPU@2.40 GHz, and the state estimation results are obtained as shown in Figure 8 and Table 8.

[0225] In terms of estimation accuracy, compared with the traditional EKF algorithm and the QLEKF algorithm based on a single reward function source, the convergence accuracy of the QLEKFstep algorithm is improved by more than 10%, and the convergence speed of the QLEKF algorithm is significantly faster than that of the EKF algorithm.

[0226] Table 8 Program running duration

[0227] EKF / s QLEKF / s QLEKFstep / s 0.3283 0.5565 0.5779

[0228] The average running duration is recorded through 50 experiments. In terms of computational efficiency, the QLEKF-step algorithm is slightly lower than QLEKF. Compared with the QLEKF algorithm, its running time increases by 3.8%. As shown in Figure 9 is the box plot of the filtering estimation results under the variation of measurement errors.

[0229] From Figure 10 , as the measurement error changes, the QLEKF algorithm and the QLEKF-step algorithm are always superior to the traditional EKF algorithm. Figure 11 The position estimation error diagrams of each axis are plotted. The estimation error is within the range of 3σ, indicating that the estimation results have high stability. That is, the QLEKF and QLEKF-step algorithms can better suppress the influence of errors, thereby improving the positioning accuracy and reliability of the navigation system.

[0230] Take ε ∈ (0.05, 0.15) in the greedy algorithm and γ ∈ (0.85, 0.95) in the reinforcement learning parameters, and conduct 50 simulation experiments respectively, and take the average value of the last 1000 moments, as shown in Figure 12 , Figure 13 , Figure 14 and Figure 15 shown:

[0231] From Figure 12 , Figure 13 , Figure 14and Figure 15 It can be seen that with the change of the reinforcement learning parameters, the change of the filtering result is not significant, that is, the estimation error is insensitive to the parameter change to a certain extent, which indicates that the Q-learning filtering algorithm has good robustness in the parameter adjustment process. In addition, compared with the QLEKF algorithm, the QLEKF-step algorithm has a smaller average error and higher position estimation accuracy.

[0232] Combined with the embodiments of the present invention, an inter-satellite navigation system is provided, including:

[0233] A variable setting unit, configured to establish a detector state variable in the selenocentric inertial coordinate system; construct an observation variable of the on-board camera through pictures of three satellites affected by the lunar gravity taken by the on-board camera of the detector; wherein, the detector is arranged on the spacecraft;

[0234] A modeling unit, configured to establish a dynamic model and a measurement equation of the detector;

[0235] A reward function construction unit, configured to construct a QLEKF state equation improved based on the reward function based on the dynamic model and the measurement equation of the detector;

[0236] A solving unit, configured to solve according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

[0237] Preferably, the variable setting unit is specifically configured to:

[0238] Construct a selenocentric inertial coordinate system (i-system): the origin O is defined as the center of the moon, OX is located in the selenocentric equatorial plane and points to the vernal equinox direction; OZ points to the positive normal direction of the lunar equatorial plane, and OY forms a right-handed coordinate system with OX and OZ;

[0239] Construct a detector body coordinate system (b-system): the origin is defined as the center of mass O of the detector, OX, OY, and OZ are respectively along the three inertial principal axis directions of the detector, and form a right-handed coordinate system;

[0240] Construct a camera coordinate system (c-system): the origin is defined as the optical center O of the on-board camera, OY is parallel to the image plane, OZ is perpendicular to the image plane, and forms a right-handed coordinate system with OX and OY;

[0241] In the selenocentric inertial coordinate system, define the detector state variable at time k as:

[0242]

[0243] where r k and v k are respectively the relative position vector and velocity vector of the detector to the center of the moon at time k, x k , y k,z k are the position parameters in three directions, and are the velocity parameters in three directions;

[0244] Three satellites affected by the lunar gravity are imaged and identified through the on - satellite camera on the detector, and the two - dimensional pixel data of the camera imaging plane are captured and the observed values are extracted; Denote the LOS vector pointing to satellite i in the camera coordinate system at time k as:

[0245]

[0246] where, x ik,c ,y ik,c are the coordinates of the centroid feature point of satellite i in the imaging plane extracted from the image taken by the on - satellite camera, f is the focal length of the on - satellite camera, r k is the relative position vector from the detector to the center of the moon at time k, r ik =[x ik y ik z ik T ,i = 1,2,3 represent the relative position vectors of the three satellites to the center of the moon in the lunar - centered inertial coordinate system; is the rotation matrix from the lunar - centered inertial coordinate system (i - system) to the detector body coordinate system (b - system); is the rotation matrix from the detector body coordinate system (b - system) to the camera coordinate system (c - system), and η k represents the measurement noise at time k.

[0247] Preferably, the modeling unit is specifically used for:

[0248] Assume that the three satellites are affected by the lunar gravity during the fly - around phase, consider the J2 perturbation term, and use the process noise w k to describe the error caused by ignoring other perturbation terms; Establish the dynamic fly - around model of the detector in the lunar - centered inertial coordinate system:

[0249] x k =f(x k-1 )+w k (4)

[0250] In the formula, w k is the process noise, and f(x k-1 ) represents the non - linear state transition function of the detector:

[0251] f(x k-1 )=x k-1 +φ(x k-1 )τ (5)

[0252]

[0253] where, r​k = ||r k ||, representing the distance from the detector to the center of the moon at time k; τ represents the time step, and μ m is the gravitational constant of the moon, and J m2 is the lunar spherical harmonic coefficient, and w t is the process noise at time t. The statistical characteristics of the process noise are as follows:

[0254]

[0255] where w t represents the process noise vector at time t, and Q k represents the system noise variance matrix at time k, and δ kt is the Kronecker function and satisfies:

[0256]

[0257] Preferably, the modeling unit is specifically used for:

[0258] The spaceborne camera uses the LOS vector pointing to the satellite in the line-of-sight direction of the lunar satellite as the measurement information source; since the attitude information of the spaceborne camera is known and the influence of coordinate system conversion on the magnitude of the observation variable is not considered, it is considered that is the identity matrix, and the measurement model of the detector can be written as:

[0259]

[0260] where r 1k represents the relative position vector from the first satellite to the center of the moon in the lunar-centered inertial coordinate system, r 2k represents the relative position vector from the second satellite to the center of the moon in the lunar-centered inertial coordinate system, r 3k represents the relative position vector from the third satellite to the center of the moon in the lunar-centered inertial coordinate system, r k is the relative position vector from the detector to the center of the moon at time k, and η k is the detector measurement noise at time k. The statistical characteristics of the measurement noise are:

[0261]

[0262] where η t represents the detector measurement noise vector at time t, and R k represents the measurement noise variance matrix, and δ kt is the Kronecker function and satisfies:

[0263]

[0264] The process noise w k and the measurement noise η kIndependent of each other, for any k and t, the following relationship exists:

[0265]

[0266] Preferably, the reward function construction unit is specifically used for:

[0267] Construct the state space and action transition, specifically including:

[0268] Define both the state space S and the action space A that can be taken by the current state as discrete sets; estimate the directional search noise value, and define two one-dimensional state sets, respectively representing the process noise w k and the covariance matrix design value set of the measurement noise η k :

[0269]

[0270] Take the Cartesian product of the two one-dimensional state sets to obtain the two-dimensional state space S = I × J, and define a mapping in the state space:

[0271]

[0272] The action space is defined as A = L × W, L = {-1, 0, 1}, W = {-1, 0, 1}, and at time k, based on the current state s k The selected action is denoted as a k =(l k , w k ) ∈ A, and l k ∈ L, w k ∈ W. By taking the action a k The agent transfers to the next time state s k+1 =(i k + l k , j k + w k ). When i k = 1, l k ≠ -1; i k = M, l k ≠ 1, M represents the number of process noise covariance matrix design values, and i k represents the serial number of the covariance matrix design value selected at time k, corresponding to

[0273] When j k = 1, w k ≠ -1; i k = N, w k ≠ 1, N represents the number of measurement noise covariance matrix design values, and j k represents the serial number of the covariance matrix design value selected at time k, corresponding to

[0274] Preferably, the reward function construction unit is specifically configured to:

[0275] Construct an initial reward function, specifically including:

[0276] Use the reference EKF with nominal noise covariance values (Q0, R0), denoted as trEKF, and use Q-learning to update the noise covariance value The search EKF of and Take the difference between the two as the reward function. The reward function at time k is:

[0277]

[0278] By synchronously adjusting the value of constantly accumulate rewards and update the Q-value table. The size of the Q-value is used to select actions through the greedy policy, and execute the action to obtain a new value; the greedy policy of synchronous adjustment requires maintaining a Q-value table of size M×N×|A|, where |A| is the number of actions that can be selected in the current state

[0279] Combined with the initial reward function, based on the plasticity of the reward function of the potential function, construct a new reward function, specifically including:

[0280] R sa (k) = R1(k) + R2(k) (13)

[0281]

[0282] Where represents the estimated error variance matrix of the state vector at time k-1, and R sa (k) is the total reward at time k, and R1(k) and R2(k) represent the measurement error and state error respectively, represents the one-step estimation error of seEKF, represents the one-step estimation error of the estimated esEKF; construct a parallel filter: based on the reference EKF with nominal noise covariance values (Q0, R0), use Q-learning to gradually update the search EKF of and update the estimated EKF of represents the estimated process noise covariance matrix at time k as the i-th design value,

[0283] Record the innovation sequences corresponding to trEKF and seEKF, as well as the one-step update errors of seEKF and esEKF, to calculate the reward function value:

[0284]

[0285] Gradually adjust by alternately updating the value of to alleviate the

[0286] Preferably, the reward function construction unit is specifically configured to:

[0287] Construct the value function and update the Q function, specifically including:

[0288] The Q1 and Q2 values are updated once per cycle and are used to measure the magnitudes of the cumulative rewards R1 and R2 within the cycle respectively;

[0289] Establish tables Q1 and Q2 to record the cumulative reward functions R1 and R2 in the current state s(i,j) and action a(l,w), and update them once per cycle (T time instants). The one-step update rule is as follows:

[0290]

[0291] Where After executing the action a(l,w), the state s′(i′,j′) is reached. i′ represents the estimated process noise value after action transfer and takes j′ represents the estimated measurement noise value after action transfer and takes Constrain the values of l and w through i and j;

[0292] Based on the maximum Q value achievable from state s′, it is denoted as the value function:

[0293] V1(i′) = Max l Q1(i′,l),

[0294] V2(j′) = Max w Q2(j′,w) (20)

[0295] In equations (18) and (19), Q1(i,l) and Q2(j,w) are the Q values for executing action a(l,w) in state s(i,j), represents the Q value before update, represents the Q value after update, represents the Q value before update, represents the Q value after update;

[0296] R1 + γV1(i′) and R2 + γV2(j′) represent the rewards brought by the current action a(l, w), where the rewards include short-term rewards R1, R2 and long-term rewards V1(i′), V2(j′); α ∈ (0, 1) is the learning rate, which measures the proportion of updating the reward of the Q function and inheriting the Q value before the update; γ ∈ (0, 1) is the discount factor to measure the importance of long-term rewards.

[0297] In combination with the embodiments of the present invention, a computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a computer device, the computer device executes any one of the inter-satellite navigation methods.

[0298] In combination with the embodiments of the present invention, a computer device is provided, including:

[0299] a processor; and a memory arranged to store computer-executable instructions, and when the executable instructions are executed, the processor executes any one of the inter-satellite navigation methods.

[0300] The beneficial technical effects obtained by the embodiments of the present invention are as follows:

[0301] An improved QLEKF-step algorithm significantly improves the estimation accuracy of the filtering algorithm by constructing a parallel reward function, respectively accumulating the innovation sequence and the one-step update error value, and adaptively updating the covariance matrices of the process noise and the measurement noise. This method can enable the detector state estimation result to converge quickly in the case of noise uncertainty in the space environment, effectively alleviate the problem of state estimation divergence in the autonomous navigation algorithm, and significantly improve the operation efficiency and positioning accuracy.

[0302] The performance of Q-learning in dealing with noise uncertainty has been verified under Gaussian noise conditions. It is expected that reinforcement learning algorithms such as Q-learning will play an important role in these complex scenarios.

[0303] It should be understood that the specific order or hierarchy of the steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of the steps in the process can be rearranged without departing from the protection scope of the present disclosure. The appended method claims list the elements of the various steps in an exemplary order and are not limited to the specific order or hierarchy described.

[0304] In the foregoing detailed description, various features are combined in a single embodiment to simplify the present disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly recited in each claim. On the contrary, as reflected in the appended claims, the invention lies in less than the full scope of features of the single disclosed embodiment. Accordingly, the appended claims are hereby expressly incorporated into the detailed description, where each claim stands on its own as a separate preferred embodiment of the invention.

[0305] The above-described disclosed embodiments are described to enable any person skilled in the art to make or use the present invention. For those skilled in the art, various modifications to these embodiments are obvious, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0306] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but those of ordinary skill in the art should recognize that each embodiment can be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, this term is encompassed in a manner similar to the term "including", as interpreted when "including" is used as a transitional word in a claim. In addition, any use of the term "or" in the claims or specification is intended to mean "non-exclusive or".

[0307] The specific embodiments described above further elaborate on the object, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An inter-satellite navigation method, characterized in that, Including: Step 1: In the lunar-centered inertial coordinate system, establish the state variables of the detector; construct the observation variables of the on-board camera through the pictures of three satellites affected by the lunar gravity taken by the on-board camera of the detector; wherein, the detector is arranged on the spacecraft. Step 2: Establish the dynamic model and measurement equation of the detector. Step 3: Based on the dynamic model and measurement equation of the detector, construct the QLEKF state equation improved based on the reward function. Step 4: Solve according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

2. The inter-satellite navigation method according to claim 1, characterized in that Step 1 includes: Construct the lunar-centered inertial coordinate system (i-system): The origin O is defined as the center of the moon, OX is located in the lunar equatorial plane and points to the vernal equinox direction; OZ points to the positive normal direction of the lunar equatorial plane, and OY forms a right-handed coordinate system with OX and OZ. Construct the detector body coordinate system (b-system): The origin is defined as the center of mass O of the detector, and OX, OY, and OZ are respectively along the three inertial principal axis directions of the detector and form a right-handed coordinate system. Construct the camera coordinate system (c-system): The origin is defined as the optical center O of the on-board camera, OY is parallel to the image plane, OZ is perpendicular to the image plane, and forms a right-handed coordinate system with OX and OY. In the lunar-centered inertial coordinate system, define the detector state variable at time k as: where r k and v k are the relative position vector and velocity vector of the detector to the center of the moon at time k, respectively, and x k , y k , z k are the position parameters in three directions, and are the velocity parameters in three directions; Perform imaging recognition on three satellites affected by the lunar gravity through the on-board camera of the detector, capture the two-dimensional pixel data of the camera imaging plane and extract the observed values; denote the LOS vector pointing to satellite i in the camera coordinate system at time k as: where x ik,c , y ik,c are the coordinates of the centroid feature points of satellite i in the imaging plane extracted from the images taken by the spaceborne camera, f is the focal length of the spaceborne camera, r k is the relative position vector of the detector to the center of the moon at time k, r ik = [x ik y ik z ik T , and i = 1, 2, 3 represent the relative position vectors of the three satellites to the center of the moon in the lunar inertial coordinate system; is the rotation matrix from the lunar inertial coordinate system (i-system) to the detector body coordinate system (b-system); is the rotation matrix from the detector body coordinate system (b-system) to the camera coordinate system (c-system), and η k represents the measurement noise at time k.​ 3. The inter-satellite navigation method according to claim 2, wherein Step 2 includes: Suppose that three satellites are affected by the lunar gravity during the fly-around phase. Considering the J2 perturbation term, the process noise w is used to k describe the error caused by neglecting other perturbation terms; A dynamic fly-around model of the detector is established in the lunar-centered inertial coordinate system: x k = f(x k-1 ) + w k (4) where w k is the process noise, and f(x k-1 ) represents the nonlinear state transition function of the detector; f(x k-1 ) = x k-1 + φ(x k-1 )τ (5) where r k = ||r k ||, representing the distance from the detector to the center of the moon at time k; τ represents the time step, μ m is the gravitational constant of the moon, J m2 is the lunar spherical harmonic coefficient, w t is the process noise at time t, and the statistical characteristics of the process noise are as follows: Among them, w t represents the process noise vector at time t, and Q k represents the system noise variance matrix at time k, and δ kt is the Kronecker function and satisfies:

4. The inter-satellite navigation method according to claim 3, wherein Step 2 also includes: The spaceborne camera uses the LOS vector pointing to the satellite in the line-of-sight direction of the lunar satellite as the measurement information source; since the attitude information of the spaceborne camera is known and the influence of coordinate system conversion on the magnitude of the observed variable is not considered, it is considered that is the identity matrix, and the measurement model of the detector can be written as: where r 1k represents the relative position vector from the first satellite to the lunar center in the lunar-centered inertial coordinate system, r 2k represents the relative position vector from the second satellite to the lunar center in the lunar-centered inertial coordinate system, r 3k represents the relative position vector from the third satellite to the lunar center in the lunar-centered inertial coordinate system, r k is the relative position vector of the detector to the lunar center at time k, η k is the measurement noise of the detector at time k, and the statistical characteristics of the measurement noise are as follows: where η t represents the detector measurement noise vector at time t, R k represents the measurement noise variance matrix, and δ kt is the Kronecker function and satisfies: Process noise w k is independent of the measurement noise η k and for any k and t, the following relationship holds:

5. The inter-satellite navigation method according to claim 4, characterized in that, Step 3 includes: Construct the state space and action transfer, specifically including: Define both the state space S and the action space A that can be taken by the current state as discrete sets; for the directional search noise estimation value, define two one-dimensional state sets respectively, which represent the process noise w k and the measurement noise η k of the covariance matrix design value set: Take the Cartesian product of two one-dimensional state sets to obtain the two-dimensional state space S = I × J, and define a mapping in the state space: The action space is defined as A = L × W, where L = {-1, 0, 1} and W = {-1, 0, 1}. At time step k, based on the current state s k The selected action, denoted as a k =(l k , w k ) ∈ A, and l k ∈ L, w k ∈ W. By taking action a k the agent transitions to the next state s k+1 =(i k + l k , j k + w k ). When i k = 1, l k ≠ -1; when i k = M, l k ≠ 1, where M represents the number of design values of the process noise covariance matrix, and i k represents the sequence number of the covariance matrix design value selected at time step k, corresponding to When j k = 1, w k ≠ -1; when i k = N, w k ≠ 1, where N represents the number of design values of the measurement noise covariance matrix, and j k represents the serial number of the covariance matrix design value selected at time k, corresponding to 6. The inter-satellite navigation method according to claim 5, characterized in that Step 3 also includes: Construct the initial reward function, specifically including: The baseline EKF using the nominal noise covariance values (Q0, R0), denoted as trEKF, updates the noise covariance values using Q-learning. The search EKF of is denoted as seEKF, and the innovation sequences are recorded separately. The difference between the two is used as the reward function, and the reward function at time k is: By synchronous adjustment of the value, continuously accumulate rewards and update the Q-value table. The size of the Q-value is used to select actions through the greedy strategy, and execute the actions to obtain new values; The greedy strategy of synchronous adjustment requires maintaining a Q-value table of size M×N×|A|, where |A| is the number of actions that can be selected in the current state and |A| ∈ [3, 5]. M represents the number of design values of the process noise covariance matrix, and N represents the number of design values of the measurement noise covariance matrix; Combined with the initial reward function, based on the plasticity of the reward function of the potential function, construct a new reward function, specifically including: R sa R(k) = R1(k) + R2(k) (13) Among them, represents the estimated error variance matrix of the state vector at time k-1, and R sa (k) is the total reward at time k. R1(k) and R2(k) respectively represent the measurement error and state error at time k, represents the one-step estimation error of seEKF, represents the one-step estimation error of estimating esEKF; construct a parallel filter: a reference EKF based on the nominal noise covariance values (Q0, R0), and use Q-learning to gradually update the search EKF of and update the estimation EKF of represents that the estimated process noise covariance matrix at time k is the i-th design value, represents that the measurement noise covariance matrix at time k is the j-th design value; Record the innovation sequences corresponding to trEKF and seEKF, as well as the one-step update errors of seEKF and esEKF to calculate the reward function value: Gradually adjust by alternating updates to mitigate the error propagation caused by inaccurate estimation of any one of the parameter pairs.

7. The inter-satellite navigation method according to claim 6, wherein Step 3 also includes: Construct the value function and update the Q function, specifically including: The values of Q1 and Q2 are updated once per period, and are used to measure the magnitudes of the cumulative rewards R1 and R2 within the period respectively. Establish the tables Q1 and Q2, which respectively record the cumulative reward functions R1 and R2 in the current state s(i,j) and action a(l,w), and are updated once per period (T moments), and the one-step update rule is as follows: Among them, After executing the action a(l, w), it reaches the state s′(i′, j′), where i′ represents the estimated process noise value after the action transfer, taking j′ represents the estimated measurement noise value after the action transfer, taking The values of l and w are restricted by i and j; Based on the maximum Q value that can be achieved by the state s′, denoted as the value function: V1(i′) = Max l Q1(i′, l), V2(j′) = Max w Q2(j′, w) (20) In Formula (18) and Formula (19), Q1(i, l) and Q2(j, w) are Q-values for executing action a(l, w) in state s(i, j). Indicates the Q-value before update, Indicates the Q-value after update, Indicates the Q-value before update, Indicates the Q-value after update; R1 + γV1(i′), R2 + γV2(j′) represent the rewards brought by this action a(l,w), and the rewards include the short-term rewards R1 and R2 and the long-term rewards V1(i′) and V2(j′); α ∈ (0,1) is the learning rate, which measures the proportion of the Q function updating the reward and inheriting the Q value before the update; γ ∈ (0,1) is the discount factor to measure the importance of the long-term reward.

8. An inter-satellite navigation system, characterized in that, Including: A variable setting unit for establishing detector state variables in the lunar-centered inertial coordinate system; constructing observation variables of the on-board camera through pictures of three satellites affected by lunar gravity taken by the on-board camera of the detector; wherein the detector is provided on a spacecraft; A modeling unit for establishing a dynamic model and a measurement equation of the detector; A reward function construction unit for constructing a QLEKF state equation improved based on a reward function based on the dynamic model and the measurement equation of the detector; A solving unit for solving according to the QLEKF state equation improved based on the reward function to obtain the state of the detector.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a computer device, the computer device executes the inter-satellite navigation method according to any one of claims 1-7.

10. A computer device, characterized in that, Comprising: A processor; And a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the inter-satellite navigation method according to any one of claims 1-7.

Images