Method and system for predicting behaviors of space craft

Abstract

The invention relates to the technical field of satellite orbit prediction, and provides a space craft behavior prediction method and system, and the method comprises the steps: S1, obtaining the continuous space states of a space craft at a plurality of moments; s2, selecting a space state as a current space state; s3, calculating a reward value for executing a flight action in the current space state; s4, acquiring a next space state after the flight action is executed in the current space state; s5, calculating income values of executing all the next flight actions in the next space state; s6, updating the income value; s7, taking the next space state as the current space state; s8, repeating the step S4 to the step S7 until the income value is greater than a specified income value; and S9, selecting the flight action with the maximum profit value as the predicted action of the space craft behavior according to the current space state and the profit value of the space craft. According to the scheme, real-time monitoring and high-precision prediction of the space aircraft can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of satellite orbit prediction, and in particular to a method and system for predicting the behavior of a spacecraft. Background Art

[0002] With the rapid development of global space activities, especially the intensification of commercial space and space exploration between countries, the number of satellites in orbit has shown explosive growth. According to statistics from the International Space Research Institute, there are currently more than 10,000 satellites in orbit, and this number is expected to double in the next decade. The diverse applications of satellites, including communications, remote sensing, meteorological monitoring, scientific experiments, and navigation and positioning, have greatly promoted the progress of human science and technology. However, the problems that followed, such as non-cooperative satellites and space debris, have gradually attracted widespread attention.

[0003] Non-cooperative satellites are generally those that do not actively provide status information, are not properly coordinated, and may pose a potential threat to other satellites or space missions. The behavior of such satellites is unpredictable and may occur due to failure, loss of control, abandonment, or intentional malicious behavior. Predicting their behavior is not only critical to the safety of space missions, but also poses a huge challenge to ground station operations, protection of space assets, and national security.

[0004] Reinforcement learning, as an important branch of machine learning, is particularly suitable for solving decision-making problems in dynamic, complex and uncertain environments. It optimizes strategies through interactive learning with the environment, enabling the agent to gradually improve the success rate of tasks in unknown environments. In the prediction of non-cooperative satellite behavior, reinforcement learning can guide the agent to explore and utilize through reward mechanisms, and adapt to the complex dynamic behavior of non-cooperative satellites. Therefore, the non-cooperative satellite behavior prediction algorithm based on reinforcement learning has gradually become a research hotspot.

[0005] In recent years, many international space agencies and research organizations have begun to explore non-cooperative satellite behavior prediction technology based on reinforcement learning. For example, NASA, ESA and other institutions have carried out a series of related research to improve the monitoring and prediction capabilities of non-cooperative satellites. In addition, domestic and foreign academic circles have also conducted a lot of exploration in this field and proposed a variety of innovative algorithms based on deep reinforcement learning. These studies not only provide a new theoretical basis for non-cooperative satellite behavior prediction, but also provide a feasible technical route for practical application.

[0006] Although existing research has made some progress, it still faces many challenges. First, how to effectively construct the state space and action space so that the reinforcement learning algorithm can accurately reflect the complex behavior of non-cooperative satellites still needs to be further explored. Secondly, the design of the reward function is crucial. How to quantify the risks and impacts of non-cooperative satellite behavior and how to guide learning through a reasonable reward mechanism are urgent issues to be solved. In addition, in response to the real-time prediction needs of non-cooperative satellite behavior, how to achieve fast and efficient online learning and decision-making is also an important direction for future research.

[0007] In summary, with the increasing number of non-cooperative satellites and the intensification of their potential threats, traditional behavior prediction methods can no longer effectively cope with the complex and changing aerospace environment. The non-cooperative satellite behavior prediction algorithm based on reinforcement learning has shown broad application prospects due to its adaptability and intelligent decision-making capabilities. In the future, with the continuous optimization of the algorithm, the construction of the data set and the maturity of the technology, the non-cooperative satellite behavior prediction technology based on reinforcement learning will provide strong support for aerospace safety and help the further development of the aerospace industry.

[0008] Traditional non-cooperative satellite behavior prediction methods mostly rely on physical models and empirical rules. These methods usually assume that the movement of satellites follows the laws of classical mechanics and rely on accurate orbital data and environmental models. However, this method shows obvious limitations in a changing space environment. Since the orbital data of non-cooperative satellites may lack real-time updates, and their behavior is often affected by a variety of complex factors, such as interactive collisions, gravitational perturbations, solar radiation pressure, etc., the effectiveness of the prediction model is reduced. In addition, traditional methods are difficult to adapt to the complexity and randomness of non-cooperative satellite behavior, resulting in insufficient timeliness and accuracy of predictions, and cannot meet the needs of modern space missions in terms of refinement and efficiency.

[0009] With the rapid development of machine learning, especially deep learning technology, data-driven methods have provided new ideas for the prediction of non-cooperative satellite behavior. Deep learning can automatically extract features through large-scale data sets, capture the potential laws in the behavior of non-cooperative satellites, and significantly improve the adaptability and flexibility of prediction models. However, deep learning algorithms usually require a large amount of labeled data. For the relatively scarce data environment of non-cooperative satellites, building a training set is a daunting task. At the same time, the black box characteristics of deep learning algorithms make their decision-making process difficult to explain, which increases the difficulty of application in space missions and cannot meet the timeliness and accuracy of predictions at the same time.

[0010] Therefore, there is a need to provide a method and system for predicting space vehicle behavior that can achieve real-time monitoring and high-precision prediction of space vehicles such as non-cooperative satellites and space debris, and overcome the problems of insufficient timeliness and accuracy of predictions.

[0011] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0012] The main purpose of the present invention is to solve the problem that existing prediction methods cannot simultaneously meet the timeliness and accuracy of prediction, and to provide a method and system for predicting space vehicle behavior, which can realize real-time monitoring and high-precision prediction of space vehicles such as non-cooperative satellites and space debris.

[0013] To achieve the above-mentioned purpose, the present invention provides a method for predicting the behavior of a spacecraft in a first aspect, comprising the following steps:

[0014] S1: Obtain the space state of the spacecraft at multiple consecutive moments;

[0015] S2: Initialize the benefit value and select a space state as the current space state;

[0016] S3: Calculate the reward value of executing a flying action in the current space state;

[0017] S4: Obtain the next space state after the current space state completes the flight action;

[0018] S5: Calculate the benefit value of executing all next flight actions in the next space state;

[0019] S6: Update the benefit value according to the benefit value, the reward value and the benefit value of executing all the next flight actions in the next space state;

[0020] S7: taking the next space state as the current space state;

[0021] S8: Repeat steps S4 to S7 until the benefit value is greater than the specified benefit value;

[0022] S9: According to the current space state and benefit value of the space vehicle, the flight action with the maximum benefit value is selected as the predicted action of the space vehicle behavior.

[0023] As an exemplary implementation of the present invention, in step S1, the spatial state includes spatial position, velocity, acceleration and environmental parameters.

[0024] As an example implementation of the present invention, in step S3, the reward value is calculated using Formula 1:

[0025] R(s,a)=R accuracy +R stability -R penalty +R cooperationFormula 1;

[0026] Among them, R(s, a) represents the reward value, s represents the current space state, a represents a flight action performed in the current space state, and R accuracy represents the prediction error reward, R stability represents the trajectory stability reward, R penalty represents the stability reward, R cooperation Represents the distance bonus between the spacecraft and other satellites.

[0027] As an example implementation of the present invention, the prediction error reward is calculated using Formula 2:

[0028] R accuracy =-||P actual -P predicted || Formula 2;

[0029] Among them, R accuracy represents the prediction error reward, P actual represents the actual position of the spacecraft, P predicted Represents the predicted position of a space vehicle.

[0030] As an example implementation of the present invention, the trajectory stability reward is calculated using Formula 3:

[0031] R stability =-||△a|| Formula 3;

[0032] Among them, R stability represents the trajectory stability reward, and △a represents the difference between the current acceleration and the previous acceleration.

[0033] As an exemplary embodiment of the present invention, the difference between the current acceleration and the previous acceleration is equal to the current acceleration minus the previous acceleration.

[0034] As an example implementation of the present invention, the stability reward is calculated using Formula 4:

[0035] R penalty =C penalty ×|△a| Formula 4;

[0036] Among them, R penalty represents the stability reward, C penalty is a constant, C penalty represents the penalty intensity, and △a represents the difference between the current acceleration and the previous acceleration.

[0037] As an example implementation of the present invention, the distance reward between the spacecraft and other satellites is calculated using Formula 5:

[0038] Rcooperation =C cooperation ×(D threshold -D current ) Formula 5;

[0039] Among them, R cooperation represents the distance reward between the spacecraft and other satellites, C cooperation is a constant that determines the strength of the distance reward; D threshold represents the ideal distance threshold; D current Indicates the actual distance between the current space vehicle and other satellites.

[0040] As an example implementation of the present invention, in step S6, the revenue value is updated according to the revenue value, the reward value and the next space state using Formula 6:

[0041]

[0042] Formula 6;

[0043] Wherein, Q(s, a) represents the benefit value of executing a flight action in the current spatial state, s represents the current spatial state, a represents a flight action executed in the current spatial state, R(s, a) represents the reward value, s represents the current spatial state, a represents a flight action executed in the current spatial state, Q(s', a') represents the benefit value of executing the next flight action in the next spatial state, s' represents the next spatial state, a' represents the next flight action executed in the next spatial state, It represents the maximum benefit value that can be obtained by executing all the next flight actions in the next space state, α represents the learning rate, α is a constant, γ is the discount factor, and γ is a constant.

[0044] According to a second aspect of the present invention, the present invention provides a prediction system for space vehicle behavior, comprising: a space state acquisition module, a benefit value storage module, a benefit value calculation module, and a benefit value output module;

[0045] The space state acquisition module is used to acquire the space state of the spacecraft at multiple consecutive moments;

[0046] The benefit value calculation module is connected with the space state acquisition module and the benefit value storage module, and is used to initialize the benefit value, select a space state as the current space state; calculate the reward value of executing a flight action in the current space state; obtain the next space state after the current space state completes the flight action; calculate the benefit value of executing all next flight actions in the next space state; update the benefit value according to the benefit value, the reward value and the benefit value after executing all next flight actions in the next space state; use the next space state as the current space state; and repeatedly update the benefit value until the benefit value is greater than the specified benefit value.

[0047] The benefit value output module is connected to the benefit value storage module and is used to select the flight action with the maximum benefit value as the predicted action of the space vehicle behavior according to the current space state and benefit value of the space vehicle.

[0048] This scheme first constructs a multi-dimensional spatial state, including the trajectory, speed, acceleration, etc. of the space vehicle. By comparing with historical data, the algorithm uses the strategy evaluation mechanism of reinforcement learning to dynamically update the prediction model, analyze the behavior patterns of the space vehicle in different time periods, and combines it with real-time data streams for training. It has adaptability and intelligent decision-making capabilities, achieves high-precision prediction of satellite behavior, overcomes the problems of insufficient timeliness and accuracy of prediction, and shows broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the present application will become more apparent by describing in detail the exemplary embodiments thereof with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative effort.

[0050] Figure 1 A diagram schematically shows the steps of a method for predicting the behavior of a space vehicle. DETAILED DESCRIPTION

[0051] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0052] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0053] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0054] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0055] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the concepts of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0056] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0057] According to a first specific embodiment of the present invention, the present invention provides a space vehicle behavior prediction system, including: a space state acquisition module, a benefit value storage module, a benefit value calculation module, and a benefit value output module.

[0058] The space state acquisition module is used to acquire the space state of the spacecraft at multiple consecutive moments.

[0059] The benefit value calculation module is connected with the space state acquisition module and the benefit value storage module, and is used to initialize the benefit value, select a space state as the current space state; calculate the reward value of executing a flight action in the current space state; obtain the next space state after the current space state completes the flight action; calculate the benefit value of executing all next flight actions in the next space state; update the benefit value according to the benefit value, the reward value and the benefit value after executing all next flight actions in the next space state; take the next space state as the current space state; and repeatedly update the benefit value until the benefit value is greater than the specified benefit value.

[0060] The benefit value output module is connected to the benefit value storage module and is used to select the flight action with the maximum benefit value as the predicted action of the space vehicle behavior according to the current space state and benefit value of the space vehicle.

[0061] According to a second specific embodiment of the present invention, the present invention provides a method for predicting the behavior of a spacecraft, using the prediction system for the behavior of a spacecraft of the first specific embodiment, such as Figure 1 As shown, the following steps are included:

[0062] S1: Obtain the spatial state of the spacecraft at multiple consecutive moments.

[0063] The spatial state includes spatial position, velocity, acceleration and environmental parameters.

[0064] The space status is obtained as follows:

[0065] 1. Predicted orbit data

[0066] Space vehicle prediction is an important process to ensure the normal operation of space vehicles in orbit. It mainly includes orbit calculation, environmental factor analysis, attitude prediction and mission execution, so as to better ensure the smooth operation of satellites.

[0067] Input parameters:

[0068] TLE (two row elements) data:

[0069] Line 1: 141456U 16025A 24066.14192917.00005348 00000+0 57368-309997;

[0070] The first line has the following meanings in order:

[0071] 1: row number (always 1), the data of the first row;

[0072] 41456: The satellite’s international designer (NORAD Catalog Number), used to uniquely identify the satellite;

[0073] U: Satellite status identifier (U means undetermined status).

[0074] 16025A: The international designator of the launch, indicating that the satellite is the Ath satellite launched in the 25th time in 2016;

[0075] 24066.14192917: day number and decimal part, indicating the number of days since January 1, 1950 (24066 means the 24066th day, and the number after that is the partial day of that day);

[0076] .00005348: orbital decay rate, in cycles / day 2 ;

[0077] 00000+0: indicates the correction value of orbital attenuation, usually used to supplement the information of orbital attenuation;

[0078] 57368-3: Orbital increments and units, usually expressed to one significant figure;

[0079] 0: the epoch number of the orbit (usually zero);

[0080] 9997: Checksum digits, usually used for data integrity verification;

[0081] Line 2 241456 98.1738 83.43360022473 0.2493359.873014.88888585419294;

[0082] The meanings of the second line in order are:

[0083] 2: row number (always 2);

[0084] 41456: International designator of the satellite, same as the first row;

[0085] 98.1738: orbital inclination, in degrees;

[0086] 83.4336: the longitude of the satellite's ascending node, in degrees;

[0087] 0022473: The satellite's orbital eccentricity, given as a value expressed as a valid figure;

[0088] 0.2493: perigee height, which indicates the height relative to the average radius of the Earth;

[0089] 359.8730: orbital phase, in degrees, indicating the position of the satellite in its orbit;

[0090] 14.88888585419294: The orbital period of the satellite, in minutes.

[0091] Start Time: The start time of orbit prediction, in UTC format.

[0092] End Time: The end time of orbit prediction, in UTC format.

[0093] According to the input parameters, the orbit status information is output, including the changes in position (longitude, latitude, altitude) and velocity (velocity component) within the specified time period. The orbit status information is calculated as follows:

[0094] (1) Initialize the environment: Build the required ground data directory so that the program can access the necessary geophysical data and parameters required for orbit calculations.

[0095] (2) Analyzing TLE data:

[0096] Extract satellite orbital elements based on TLE data, including:

[0097] semi-major axis a;

[0098] Eccentricity e;

[0099] Orbital inclination i;

[0100] Longitude of ascending node Ω;

[0101] Argument of perigee ω;

[0102] Mean anomaly angle M.

[0103] (3) Create the initial track:

[0104] Based on the parsed TLE data, the initial orbit is generated according to the extracted orbital elements, and the orbital object will be used for subsequent orbit propagation calculations. Using the Kepler orbital equation:

[0105]

[0106] Where E is the eccentric anomaly angle, p = a(1-e 2 ) is the semi-diameter, μ is the standard gravitational parameter of the Earth.

[0107] (4) Define the time range:

[0108] Determine the start and end time of the orbit forecast and convert them into the required format for time calculation.

[0109] (5) Setting up the track propagator:

[0110] Create an orbit propagator and select the numerical integration method to perform orbit propagation. Set the time step so that the orbital state is evaluated over time.

[0111]

[0112] Among them, r is the position vector and v is the velocity vector.

[0113] (6) Execution orbit prediction:

[0114] The calculation is gradually advanced within the time range. At each time point tn, the orbit propagator is used to calculate the state of the satellite. For each time point, the orbit propagator is used to calculate the state of the current satellite and obtain its position information and velocity information, as follows:

[0115] r n+1 =r n +V n ·dt

[0116]

[0117] (7) Record and output results:

[0118] The orbital state (position and velocity) at each time point is recorded.

[0119] 2. Environment modeling and state definition

[0120] Environmental modeling is crucial in predicting the behavior of spacecraft. The state of a spacecraft moving in space can be described by a series of parameters that reflect the position and motion state of the spacecraft. We define a state space S, where each state s can be represented by a multidimensional vector, containing the satellite's spatial position, velocity, and other possible characteristics.

[0121] Input parameters:

[0122] Location coordinates: (x, y, z) (meters)

[0123] Velocity component: (V x , V y , V z )(meters per second)

[0124] Acceleration: (a x , a y , a z )(meters per square second)

[0125] x, y, and z represent the position coordinates of the satellite in three-dimensional space, respectively. These coordinates are expressed in the geocentric inertial coordinate system (ECI) with the center of the earth as the origin.

[0126] V x , V y , V z It is the component of the satellite's velocity in all directions, measured in meters per second (m / s).

[0127] The specific calculation is as follows:

[0128] (1) Initialize the state vector: Initialize the state vector s (i.e., spatial state) based on the input position information, velocity information, and acceleration information.

[0129] (2) Environmental parameter modeling: Based on specific environmental conditions, a mathematical model of the effects of gravity, air resistance, and solar radiation pressure is established.

[0130] (3) Status update: Use the mathematical model of environmental parameters to update and predict the status of the satellite, that is, use the state transfer equation to update the current space status:

[0131]

[0132] Here, Δt is the time interval.

[0133] (4) Integrate the output state vector: Integrate the updated position, velocity, and acceleration information into a new state vector s.

[0134] (5) Output state vector: Output the adjusted spatial state vector s.

[0135] Output: State vector s: A multidimensional vector representing the current spatial state of the spacecraft.

[0136] s=[x,y,z,V x , V y , V z , a x , a y , a z , environmental parameters].

[0137] Environmental parameters include gravity, air resistance, solar radiation pressure and other environmental parameters.

[0138] S2: Initialize the benefit value and select a space state as the current space state.

[0139] The benefit value is the Q value. Initialize the benefit value: initialize the Q value corresponding to each space state and each flight action, usually set to zero.

[0140] S3: Calculate the reward value for executing a flying action in the current space state.

[0141] The reward function plays a vital role in reinforcement learning, especially in the learning framework with Q-learning as the core. It not only guides the learning process of the agent, but also directly affects the final learning effect. In the non-cooperative satellite behavior prediction, we design a reward function to evaluate the quality of satellite behavior.

[0142] The goal of the reward function is to take into account multiple factors in order to give the agent (space vehicle) feedback on the quality of its behavior at each time step, thereby optimizing its decision-making process. In particular, the following aspects should be paid attention to:

[0143] Prediction accuracy: The error between the predicted position of the satellite in its current state and its actual position.

[0144] Trajectory stability: The smoothness of satellite trajectory changes to avoid sudden acceleration or deceleration.

[0145] The reward value is calculated using Formula 1:

[0146] R(s,a)=R accuracy +R stability -R penalty +R cooperation Formula 1;

[0147] Among them, R(s, a) represents the reward value, s represents the current space state, a represents a flight action performed in the current space state, and R accuracy represents the prediction error reward, R stability represents the trajectory stability reward, R penalty represents the smoothness reward (i.e., imposes a penalty on sharply changing acceleration), R cooperation Represents the distance bonus between the spacecraft and other satellites.

[0148] The prediction error reward is calculated using Formula 2:

[0149] R accuracy =-||P actual -P predicted || Formula 2;

[0150] Among them, R accuracy represents the prediction error reward, P actual represents the actual position of the spacecraft, P predicted Represents the predicted position of a spacecraft. The actual position of a spacecraft can be obtained historically or from a database of the actual position of the spacecraft, and can be put together to form a more comprehensive spatial state, that is, the spatial position included in the spatial state includes the actual position and the predicted position.

[0151] Purpose of prediction error reward: To evaluate the prediction accuracy of the satellite. The smaller the error, the higher the reward value.

[0152] The trajectory stability reward is calculated using Formula 3:

[0153] R stability =-||△a|| Formula 3;

[0154] Among them, R stability represents the trajectory stability reward, △a represents the difference between the current acceleration and the previous acceleration, and ||△a|| represents the norm of the △a vector.

[0155] The difference between the current acceleration and the previous acceleration is equal to the current acceleration minus the previous acceleration. The purpose is to evaluate the change in acceleration and understand the smoothness of the space vehicle's motion.

[0156] Trajectory stability bonus: Smaller acceleration changes mean a smoother trajectory, leading to a higher reward.

[0157] The stability reward is calculated using Formula 4:

[0158] R penalty =C penalty ×|△a| Formula 4;

[0159] Among them, R penalty represents the stability reward, C penalty is a constant, C penalty represents the penalty intensity, and △a represents the difference between the current acceleration and the previous acceleration.

[0160] The difference between the current acceleration and the previous acceleration is equal to the current acceleration minus the previous acceleration.

[0161] The purpose of the smoothness bonus is to penalize sharply changing accelerations to avoid jerky motion.

[0162] The distance bonus between a spacecraft and other satellites is calculated using Formula 5:

[0163] R cooperation =C cooperation ×(D threshold -D current ) Formula 5;

[0164] Among them, R cooperation represents the distance reward between the spacecraft and other satellites, C cooperation is a constant that determines the strength of the distance reward; D threshold represents the ideal distance threshold; D current Indicates the actual distance between the current space vehicle and other satellites.

[0165] R accuracy , R stability , R cooperation and R penalty Synthesize the final reward: Substitute each reward value into the comprehensive calculation formula 1 to obtain the final reward R. Update the Q-value strategy: Based on the reward R, use the Q-learning algorithm to update the spacecraft's strategy to optimize its behavior.

[0166] S4: Obtain the next space state after the current space state completes the flight action.

[0167] Q-learning is a value-based algorithm in reinforcement learning. Q is Q(s, a), which is the expected benefit of taking flight action a (a∈A) in a spatial state s at a certain moment (s∈S), and then selects the action that can get the maximum benefit based on the Q value. Among them, S is the set of spatial states and A is the set of flight actions.

[0168] S5: Calculate the benefit value of executing all next flight actions in the next space state.

[0169] S6: Update the benefit value according to the benefit value, the reward value and the benefit value of executing all next flight actions in the next space state.

[0170] Formula 6 is used to update the benefit value based on the benefit value, reward value and next space state:

[0171]

[0172] Formula 6;

[0173] Wherein, Q(s, a) represents the benefit value of executing a flight action in the current spatial state, s represents the current spatial state, a represents a flight action executed in the current spatial state, R(s, a) represents the reward value, s represents the current spatial state, a represents a flight action executed in the current spatial state, Q(s', a') represents the benefit value of executing the next flight action in the next spatial state, s' represents the next spatial state, a' represents the next flight action executed in the next spatial state, It represents the maximum benefit value that can be obtained by executing all the next flight actions in the next space state, α represents the learning rate, α is a constant, γ is the discount factor, and γ is a constant.

[0174] The learning rate is used to update the step size of the Q value and determines the weight of the new and old Q values; the discount factor determines the importance of future rewards.

[0175] Updated Q value Q(s,a): represents the expected long-term return (profit value) of taking flight action a in space state s.

[0176] S7: Taking the next spatial state as the current spatial state.

[0177] S8: Repeat steps S4 to S7 until the profit value is greater than the specified profit value.

[0178] S9: According to the current space state and benefit value of the space vehicle, the flight action with the maximum benefit value is selected as the predicted action of the space vehicle behavior.

[0179] Update the Q value and continuously perform offline learning to obtain the optimal strategy and select the best orbit from the calculated predicted orbits.

[0180] This scheme first constructs a multi-dimensional space state, including the trajectory, speed, acceleration, etc. of the spacecraft. By comparing with historical data, the algorithm uses the policy evaluation mechanism of reinforcement learning to dynamically update the prediction model, analyze the behavior patterns of the spacecraft in different time periods, and trains in combination with real-time data streams. This scheme optimizes the strategy through interactive learning with the environment, so that the intelligent agent (an entity that learns strategies through interaction with the environment to maximize its cumulative rewards) can gradually improve the success rate of tasks in unknown environments, guide the intelligent agent to explore and utilize through the reward mechanism, adapt to the complex dynamic behavior of spacecraft, have adaptability and intelligent decision-making capabilities, achieve high-precision prediction of satellite behavior, overcome the problems of insufficient timeliness and accuracy of prediction, and show broad application prospects.

[0181] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A method for predicting the behavior of a spacecraft, characterized in that: The following steps are involved: S1: Obtain the space state of the spacecraft at multiple consecutive moments; S2: Initialize the benefit value and select a space state as the current space state; S3: Calculate the reward value of executing a flying action in the current space state; S4: Obtain the next space state after the current space state completes the flight action; S5: Calculate the benefit value of executing all next flight actions in the next space state; S6: Update the benefit value according to the benefit value, the reward value and the benefit value of executing all the next flight actions in the next space state; S7: taking the next space state as the current space state; S8: Repeat steps S4 to S7 until the benefit value is greater than the specified benefit value; S9: According to the current space state and benefit value of the space vehicle, the flight action with the maximum benefit value is selected as the predicted action of the space vehicle behavior.

2. The method for predicting the behavior of a spacecraft according to claim 1, characterized in that: In step S1, the spatial state includes spatial position, velocity, acceleration and environmental parameters.

3. The method for predicting the behavior of a spacecraft according to claim 1, characterized in that: In step S3, the reward value is calculated using formula 1: R(s, a) = R accuracy + R stability - R penalty + R cooperation Equation 1; Among them, R(s, a) represents the reward value, s represents the current space state, a represents a flight action performed in the current space state, and R accuracy represents the prediction error reward, R stability represents the trajectory stability reward, R penalty represents the stability reward, R cooperation Represents the distance bonus between the spacecraft and other satellites.

4. The method for predicting the behavior of a spacecraft according to claim 3, characterized in that: The prediction error reward is calculated using Formula 2: R accuracy =-||P actual -P predicted || Formula 2; Among them, R accuracy represents the prediction error reward, P actual represents the actual position of the spacecraft, P predicted Represents the predicted position of a space vehicle.

5. The method for predicting the behavior of a spacecraft according to claim 3, characterized in that: The trajectory stability reward is calculated using Formula 3: R stability =-||△a|| Formula 3; Among them, R stability represents the trajectory stability reward, and △a represents the difference between the current acceleration and the previous acceleration.

6. The method for predicting the behavior of a spacecraft according to claim 5, characterized in that: The difference between the current acceleration and the previous acceleration is equal to the current acceleration minus the previous acceleration.

7. The method for predicting the behavior of a spacecraft according to claim 3, characterized in that: The stability reward is calculated using Formula 4: R penalty =C penalty ×|△a| Formula 4; Among them, R penalty represents the stability reward, C penalty is a constant, C penalty represents the penalty intensity, and △a represents the difference between the current acceleration and the previous acceleration.

8. The method for predicting the behavior of a spacecraft according to claim 3, characterized in that: The distance bonus between the spacecraft and other satellites is calculated using Formula 5: R cooperation =C cooperation ×(D threshold -D current ) Formula 5; Among them, R cooperation represents the distance reward between the spacecraft and other satellites, C cooperation is a constant that determines the strength of the distance reward; D threshold represents the ideal distance threshold; D current Indicates the actual distance between the current space vehicle and other satellites.

9. The method for predicting the behavior of a spacecraft according to claim 1, characterized in that: In step S6, the revenue value is updated according to the revenue value, the reward value and the next space state using formula 6: Formula 6; Wherein, Q(s, a) represents the benefit value of executing a flight action in the current spatial state, s represents the current spatial state, a represents a flight action executed in the current spatial state, R(s, a) represents the reward value, s represents the current spatial state, a represents a flight action executed in the current spatial state, Q(s', a') represents the benefit value of executing the next flight action in the next spatial state, s' represents the next spatial state, a' represents the next flight action executed in the next spatial state, It represents the maximum benefit value that can be obtained by executing all the next flight actions in the next space state, α represents the learning rate, α is a constant, γ is the discount factor, and γ is a constant.

10. A prediction system for spacecraft behavior, characterized in that: include: Space state acquisition module, benefit value storage module, benefit value calculation module, benefit value output module; The space state acquisition module is used to acquire the space state of the spacecraft at multiple consecutive moments; The revenue value storage module is used to store the revenue value; The benefit value calculation module is connected to the space state acquisition module and the benefit value storage module, and is used to initialize the benefit value and select a space state as the current space state; Calculate the reward value of executing a flight action in the current space state; obtain the next space state after the current space state completes the flight action; calculate the benefit value of executing all the next flight actions in the next space state; update the benefit value according to the benefit value, the reward value and the benefit value of executing all the next flight actions in the next space state; take the next space state as the current space state; repeatedly update the benefit value until the benefit value is greater than the specified benefit value; The benefit value output module is connected to the benefit value storage module and is used to select the flight action with the maximum benefit value as the predicted action of the space vehicle behavior according to the current space state and benefit value of the space vehicle.

Images