Spacecraft attitude planning method under complex constraints based on deep reinforcement learning

Abstract

The present application is directed to the problem of spacecraft attitude maneuver planning under forbidden pointing constraints, and proposes a spacecraft attitude planning method under complex constraints based on deep reinforcement learning. The method of the present application comprehensively considers the shortest rotation path, energy optimization, forbidden pointing constraints, dynamics constraints and the like, and through deep reinforcement learning training, a simple multiple constraint model is obtained, which can determine the approximate optimal path of the attitude maneuver and is suitable for the control of micro-satellites. The trained model can directly perform end-to-end reasoning on the starting and target attitudes, and compared with the low solving efficiency and high complex and accurate model degree of the traditional method, the present application has a relatively efficient and simple solving process and strong generalization performance. The simulation results show that the planning results meet all the constraints, including four forbidden pointing constraints, dynamics constraints and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of satellite attitude maneuver planning, and relates to a spacecraft attitude planning method under complex constraints based on deep reinforcement learning. BACKGROUND

[0002] With the rapid development of space technology and the increasing frequency of human space exploration, the world's space launch activities are increasing year by year, and space missions are developing in the direction of diversification and unmanned autonomy. Emerging space missions represented by space exploration and deep space exploration have been attracting the attention and investment of space powers in recent years. These missions require spacecraft to perform a large number of attitude maneuvers, and during the attitude maneuver, the spacecraft must meet a series of attitude pointing constraints.

[0003] Attitude pointing constraints include forbidden pointing constraints: some light-sensitive and heat-sensitive elements (star sensors, infrared cameras, cryogenic cooled telescopes, etc.) cannot be exposed to strong light sources or heat sources, that is, the sensitive elements cannot be aligned with the direction of the strong light (heat) source; secondly, forced pointing constraints: when observing a target, the optical lens must be pointed at the observation target; the solar sail needs to be always pointed at the sun to maintain power supply, and the satellite antenna needs to be pointed at the ground station to ensure data transmission. These attitude pointing constraints exist on the Hubble Space Telescope, Deep Space One probe, and Cassini probe, which greatly limits the feasible region of the spacecraft during attitude maneuver.

[0004] Attitude pointing constraints can be regarded as constraints of the external environment of the spacecraft, and the attitude dynamics constraints of the spacecraft itself also need to be considered. On the one hand, the output amplitude of the spacecraft actuator is limited. On the other hand, in the spacecraft attitude control system, the attitude sensor and the actuator require that the angular velocity of the attitude maneuver cannot be too large. During the execution of the spacecraft mission, different attitude pointing constraints will be generated according to different external environments. When the constraints caused by the complex external environment and the spacecraft's own dynamics constraints are coupled together, it is difficult to solve the attitude maneuver path of the spacecraft, so the spacecraft attitude maneuver planning must be improved.

[0005] At the same time, due to the diversity of space missions, the attitude maneuver capability of the spacecraft must be improved. For example, when performing disaster warning, scientific exploration, and military reconnaissance missions, the spacecraft is often required to maneuver from the current attitude to a given desired attitude in a short time with limited energy consumption. The specific requirements mainly include the following three aspects: 1) maneuver path length; 2) maneuver energy consumption; 3) maneuver time. This requires in-depth research on the attitude maneuver problem under complex constraints from a theoretical point of view, and under the comprehensive consideration of the above complex constraints and mission indexes, a safe and efficient attitude maneuver path is planned.

[0006] The current common attitude maneuver planning includes potential function, geometric method, discrete method, stochastic programming algorithm and numerical optimization planning method. These methods have corresponding defects, such as the potential function method is easy to fall into local minimum value; the geometric method has limited application of more constraints; the discrete method depends on the structure of off-line solution; the stochastic programming method generally only generates a feasible path, and the optimality is not considered; and the numerical optimization method needs accurate mathematical modeling and strong computing power, which is not conducive to practical application. SUMMARY

[0007] The application provides a spacecraft attitude planning method under complex constraints based on deep reinforcement learning, aiming at solving the problems of low solving efficiency and strong model dependency of traditional methods. Deep reinforcement learning has the ability to build complex intelligent agents, thereby realizing end-to-end processing from perception to planning. The application comprehensively considers the shortest rotation path, optimal energy, prohibited pointing constraint, dynamic constraint and the like, and a simple multiple constraint model is obtained through deep reinforcement learning training, which can determine the approximate optimal path of attitude maneuver and is suitable for the control of micro spacecraft. The simulation results show that the planning results meet all the constraints, and the advantages of the method in terms of average energy consumption of maneuver and planning speed are verified.

[0008] A spacecraft attitude planning method under complex constraints based on deep reinforcement learning, comprising the following steps:

[0009] (1) constructing a basic spacecraft attitude description method, establishing a prohibited pointing constraint, a control moment constraint, an angular velocity constraint, an attitude kinematics and dynamics constraint, and converting the attitude maneuver problem in a three-dimensional attitude celestial sphere into a two-dimensional plane path planning problem by setting an initial attitude, a target attitude and a prohibited pointing area;

[0010] (2) setting an initial training environment, an action, a state and a reward based on the two-dimensional plane path planning problem, then establishing a deep reinforcement learning model by using a deep Q network (DQN) and training the model to obtain a trained model;

[0011] (3) inputting an initial attitude, a target attitude and a random environment into the trained model to obtain a satellite planning path.

[0012] In the above technical solution, further, the prohibited pointing constraint in step (1) is mainly established because a light-sensitive and heat-sensitive element (a star sensor, an infrared camera, a cryogenic cooled telescope and the like) cannot be exposed to a strong light source or a heat source such as the sun, that is, the included angle between the direction vector of the strong light celestial body and the line-of-sight axis direction of the sensitive optical element cannot be lower than the field of view angle, and the prohibited pointing constraint is expressed as:

[0013]

[0014] wherein r b represents the direction vector of the sensitive optical element in the body coordinate system of the spacecraft, r I represents the direction vector of the strong light celestial body in the inertial coordinate system, C bI represents the attitude transfer matrix from the inertial coordinate system to the body coordinate system, and φ represents the field of view angle of the sensitive optical element.

[0015] In an actual attitude maneuvering task, the attitude control torque output by the attitude execution mechanism of the spacecraft is limited, forming a control torque constraint, the control torque constraint being:

[0016] |T i |<γ1, i = 1, 2, 3

[0017] wherein T i represents the control torque of the spacecraft, i represents the component on the three axes, and γ1 represents the upper limit amplitude of the control torque of the spacecraft.

[0018] The angular velocity constraint is established because the range of the angular velocity sensor is limited, and the angular velocity of the spacecraft must be kept within a limited range, the angular velocity constraint being:

[0019] |ω i |<γ2, i = 1, 2, 3

[0020] wherein ω i is the angular velocity of the spacecraft, and γ2 represents the upper limit amplitude of the angular velocity of the spacecraft.

[0021] The spacecraft is regarded as a rigid body in three-axis motion, and the attitude kinematics and dynamics constraints are described in the form of quaternions:

[0022]

[0023]

[0024] wherein ω = [ω1, ω2, ω3] is the angular velocity of the spacecraft, ω × is the skew-symmetric matrix of ω, is the derivative of ω, J is the moment of inertia of the spacecraft, and T is the control torque of the spacecraft.

[0025] The spacecraft attitude description mode is specifically that the attitude of the spacecraft is described by using Euler axis / angle and unit quaternion, and the conversion relationship is as follows:

[0026]

[0027] In the formula, q0, q v are the scalar part and vector part of the quaternion respectively, and q1, q2, q3 represent q vThree components in the complex plane; e x y z Respectively represent the three-axis components corresponding to the Euler rotation axis; represents the rotation angle of the spacecraft around the Euler rotation axis e. The present application adopts the minimum angle maneuvering mode, and the spacecraft is maneuvered from the initial attitude to the target attitude. According to the Euler rotation theorem, the attitude maneuvering path is to rotate around the e-axis by an angle of θ.

[0028] Further, the state s t Is set to:

[0029]

[0030] Wherein Represents the included angle between the current attitude vector and the obstacle in each direction Represents the relative direction of the current attitude vector and the target attitude target vector And the included angle

[0031] The action is set to rotate the spacecraft from the current motion direction to the target direction by an angle In different rotation directions, so as to obtain different actions; wherein the angle The calculation formula is:

[0032]

[0033] Wherein, n represents the number of designed actions. Since there is a certain angle between the target direction and the current motion direction, different rotation directions need to be designed, that is, rotation around different rotation axes, so as to reach the target direction, thereby realizing the conversion of the line-of-sight axis rotation planning problem into the plane particle motion path planning problem.

[0034] The reward is designed into three parts, first the basic reward and the greedy reward, which are respectively used to realize the basic goal of real-time path planning and accelerate the convergence speed of the algorithm. In addition, the smoothness reward aims to improve the smoothness of the planned path. The basic reward is used to tell the agent where the target is and where the obstacle is, and is designed as: when reaching the target attitude, a positive reward r T = 20 is given, and when entering the forbidden pointing area, a penalty (i.e. negative reward) r F = -20 is given. The greedy reward is based on the greedy strategy of using only target information and ignoring obstacle information to guide the spacecraft to reach the target attitude, which can make the agent reduce some aimless wandering in the training process and accelerate the convergence speed of the reinforcement learning algorithm. Therefore, the greedy reward is designed as:

[0035] ​​

[0036] is the reward design based on the included angle distance: the included angle at the current moment is greater than the current moment, then-0.1, otherwise-0.3, the design of negative number is because it is needed to tell the agent that the more steps, the lower the reward, so if the included angle is getting larger, it will get a larger penalty;

[0037]

[0038] is the reward design based on the target direction: the included angle of the target direction at the current moment is greater than the current moment, then+0.1, otherwise-0.1.

[0039] The smoothness reward is used to reduce unnecessary direction switching between two consecutive actions and increase the smoothness of the planned path. To this end, the smoothness reward is designed as:

[0040]

[0041] The deep reinforcement learning model constructed by the application combines deep learning and reinforcement learning. Deep learning is suitable for inductive and summary experience knowledge, and reinforcement learning uses sample information obtained by interaction between an agent and an environment to provide experience data for deep learning, which makes the deep reinforcement learning model have the potential to construct a complex agent, so that the satellite can realize end-to-end self-learning from perception to decision control. The deep reinforcement learning model constructed by the application has strong generalization performance in different environments; avoids kinematics and dynamics modeling; and reduces real-time calculation amount.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] (1) Firstly, reasonable plane mapping is carried out on the motion space of a single line-of-sight axis. By designing appropriate actions, the line-of-sight axis rotation planning problem is converted into a planar particle motion path planning problem.

[0044] (2) The deep Q network (DQN) can comprehensively consider various constraint conditions. The path planning result of the method of the application can meet the requirements of the shortest rotation path, the optimal energy, the prohibition constraint, the dynamic constraint and the arrival terminal attitude at the same time.

[0045] (3) The method of the application has generalization ability and strong universality in various complex environments, especially for uncertain environments and suddenly appearing prohibition constraints, and it can generate planning results in real time and quickly. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 a 3D schematic diagram is designed for the action of the application;

[0047] Figure 2 a 2D schematic diagram is designed for the action of the application;

[0048] Figure 3 2D schematic diagram for the state of the invention;

[0049] Figure 4 Network structure diagram of the invention;

[0050] Figure 5 2D schematic diagram of the training data environment of the invention;

[0051] Figure 6 Schematic diagram of the simulation example attitude quaternion changing over time of the invention;

[0052] Figure 7 Schematic diagram of the simulation example angular velocity changing over time of the invention;

[0053] Figure 8 Schematic diagram of the simulation example control torque changing over time of the invention.

[0054] Figure 9 2D schematic diagram of the simulation example planning attitude maneuver path of the invention.

[0055] Figure 10 3D schematic diagram of the simulation example planning attitude maneuver path of the invention. DETAILED DESCRIPTION

[0056] The design scheme of the invention will be described in detail below with reference to the accompanying drawings.

[0057] A spacecraft attitude planning method under complex constraints based on deep reinforcement learning, comprising the following steps:

[0058] 1. Spacecraft attitude description and complex attitude constraint description

[0059] When performing tasks, spacecrafts usually need large-angle attitude maneuvers to complete various space exploration or rendezvous and docking space missions. Due to the many constraints during the maneuver process, the attitude system needs to establish a corresponding attitude maneuver description method.

[0060] 1.1 Spacecraft attitude description method

[0061] Firstly, the invention uses Euler axis / angle and unit quaternion to describe the spacecraft attitude, and the conversion relationship is as follows.

[0062]

[0063] In the formula: q0, q v are the scalar and vector parts of the quaternion, respectively, q1, q2, q3 represent three components of q v in the complex plane; e x , e ye z respectively, and θ represents the rotation angle of the spacecraft around the Euler rotation axis e. The present application adopts the minimum angle maneuvering mode, and the spacecraft maneuvers from the initial attitude to the target attitude. According to the Euler rotation theorem, the attitude maneuvering path is rotating around the e axis by θ angle.

[0064] 1.2 Attitude kinematics and dynamics constraints

[0065] The establishment and analysis of the spacecraft attitude kinematics and dynamics constraints are the prerequisite for designing the spacecraft attitude planning method, so as to ensure the safe attitude maneuvering of the spacecraft from the initial attitude to the target attitude. The present application regards the spacecraft as a rigid body in three-axis motion, and adopts the quaternion form to describe the attitude kinematics and dynamics constraints:

[0066]

[0067]

[0068] wherein ω = [ω1, ω2, ω3] is the angular velocity of the spacecraft, ω × is the skew-symmetric matrix of ω, is the derivative of ω, and J is the rotational inertia of the spacecraft.

[0069] 1.3 Prohibition pointing constraint

[0070] The spacecraft needs to avoid the sensitive optical element pointing to the strong light celestial body, and the direction vector of the strong light celestial body and the line-of-sight axis direction of the sensitive optical element cannot be less than the field of view angle during the large-angle attitude maneuvering. The prohibition pointing constraint is expressed as

[0071]

[0072] wherein r b represents the direction vector of the sensitive optical element in the body coordinate system of the spacecraft, r I represents the direction vector of the strong light celestial body in the inertial coordinate system, C bI represents the attitude transfer matrix from the inertial coordinate system to the body coordinate system, and φ represents the field of view angle of the sensitive optical element.

[0073] 1.4 Control torque constraint

[0074] In the actual attitude maneuvering task, the output attitude control torque of the attitude actuator of the spacecraft is limited, forming the control torque constraint:

[0075] |T i |<γ1, i = 1, 2, 3

[0076] In the formula: T irepresents the control moment of the spacecraft, i represents the component on the three axes; γ1 represents the upper limit amplitude of the control moment of the spacecraft.

[0077] 1.5 Angular velocity constraint

[0078] At the same time, due to the limited range of the angular velocity sensor of the spacecraft, the angular velocity of the spacecraft must be kept within a limited range, forming the angular velocity constraint of the spacecraft:

[0079] |ω i |<γ2, i = 1, 2, 3

[0080] In the formula: ω i is the angular velocity of the spacecraft; γ2 represents the upper limit amplitude of the angular velocity of the spacecraft.

[0081] 2 Deep reinforcement learning setting

[0082] 2.1 Action setting

[0083] Due to the prohibition of pointing constraints, the angular velocity and control moment of the attitude maneuver are limited. Therefore, it is necessary to set a discrete action so that the planned trajectory satisfies the constraints.

[0084] As Figure 1 shown. The attitude direction vector at the last time is r t-1 , the attitude vector at the current time is r t , and the action is performed. It can be inferred that the pointing vector at the next time is The invention uses the minimum angle rotation path method to realize the maneuver from the initial attitude to the target attitude. This maneuver is achieved by rotating an angle θ1 around the rotation axis e t . e t is calculated by the following formula.

[0085]

[0086] The specific design of the action is viewed in a two-dimensional plane space. As Figure 2 shown, the current action direction, i.e. the angular velocity direction around the rotation axis e t , is determined by the direction of r t-1 and r t . r f represents the target direction vector. Since the angle between the target direction and the current motion direction is , it is necessary to design different rotation directions. Different actions can be obtained by rotating around different axes, for example, vector r1 t+1 can be obtained by rotating vector around vector r i tRotation angle The calculation formula of the angle is obtained as follows:

[0087]

[0088] In the present application, n is designed as 32, so that the agent can go in 32 different directions around the last time point.

[0089] 2.2 State setting

[0090] Due to the uncertainty of the real-time complex environment, the satellite must learn to avoid the forbidden constraints and find the target. Therefore, in order to make real-time decision, at least two types of information need to be obtained: the observation state of the forbidden constraint around the current attitude And the position relationship between the target attitude and the current attitude Therefore, the state setting is as follows:

[0091]

[0092] As shown in Figure 3 , wherein represents the angle between the current attitude vector and the obstacle in each direction That is, the vector around the current direction vector The angle between the center direction vector of the forbidden pointing area, represents the relative direction of the current attitude vector and the target attitude target vector And the angle Indicates the maximum detection angle.

[0093] 2.3 Reward setting

[0094] First, the basic reward and the greedy reward, respectively, realize the basic goal of real-time path planning and accelerate the convergence speed of the algorithm. In addition, the smoothness reward aims to improve the smoothness of the planned path. The basic reward R bs is used to tell the agent where the target is and where the obstacle is, and is designed as: when reaching the target attitude, a positive reward r T = 20 is given, and when entering the forbidden pointing area, a penalty (i.e. negative reward) r F = -20 is given. The greedy reward is based on the greedy strategy of using only target information and ignoring obstacle information to guide the spacecraft to reach the target attitude, which can reduce some aimless wandering of the agent in the training process and accelerate the convergence speed of the reinforcement learning algorithm. For this purpose, the greedy reward is designed as:

[0095]

[0096] is a distance-based reward design: the current moment distance (angle) is greater than the current moment angle, then -0.1, otherwise -0.3, the design of negative number is due to the need to tell the agent that the more steps, the lower the reward, if the distance (angle) is getting farther and farther, then a greater penalty is obtained.

[0097]

[0098] is a target direction-based reward design: the current moment target direction angle is greater than the current moment, then +0.1, otherwise -0.1.

[0099] The smoothness reward is used to reduce unnecessary direction switching between two consecutive actions, and increase the smoothness of the planned path. For this purpose, the smoothness reward is designed as:

[0100]

[0101] 2.4 Network architecture

[0102] The structure of the deep reinforcement learning network is shown in Figure 4 . The observation is fed into a convolutional neural network module (Convolutional Neural Network Module), which includes: a one-dimensional convolutional layer (Conv1D, which includes a filter Filter with a size of 5*2), a maximum pooling layer (MaxPool) with a 2*2 convolution kernel, and a flattening layer (Flatten). Then its output is spliced with the target pose information , and fed into a fully connected network module (Fully Connected Network Module), which includes three 32-dimensional fully connected layers (Dense). Finally, the action value Q(s t ,·) is obtained, so as to enter the next state.

[0103] 3 Simulation experiment

[0104] The spacecraft attitude maneuver under multiple constraint conditions is simulated to verify the effectiveness of the attitude maneuver planning method proposed in the present application.

[0105] 3.1 Training dataset setting

[0106] The present application uses a DQN model to solve the satellite attitude planning problem. The training set constructs a general path planning environment, which includes four bright celestial bodies that the spacecraft needs to avoid during attitude maneuver. As Figure 5The environment is a two-dimensional plane representation of the celestial sphere extended to longitude and latitude. The directions of these bright celestial bodies in the inertial frame are r1 = [0.37, 0.87, -0.31] T , r2 = [-0.53, -0.28, 0.8] T , r3 = [0.16, -0.96, 0.22] T , and r4 = [0.76, 0, 0.64] T , the field of view angles are θ1 = 35°, θ2 = 25°, θ3 = 30°, and θ4 = 20°, the target attitude is q f = [0.733, 0.362, -0.544, 0.181] T . The start points are generated in 8 directions around each forbidden pointing region, and the end points are located at the center of the environment. Setting the start points near the forbidden pointing region can make the neural network learn better how to avoid the forbidden pointing constraint region.

[0107] The test set is composed of 500 randomly generated 4-angles of 20-30° forbidden constraints and 30 pairs of randomly generated start and target attitudes.

[0108] 3.2 Simulation results

[0109] In the spacecraft attitude maneuver simulation verification, the trained DQN model generates end-to-end results using the samples in Table 1. The spacecraft is equipped with an optical camera mounted in the z-axis direction, and the direction vector of the optical camera in the spacecraft body coordinate system is represented by r b . The spacecraft inertia J, the directions r I and the field of view angle data θ i of the four bright celestial bodies that the spacecraft needs to avoid during the attitude maneuver, as well as the initial attitude q0 and target attitude q f of the spacecraft, and the initial angular velocity ω s and target angular velocity ω f are listed in Table 1.

[0110] Table 1

[0111]

[0112]

[0113] Figure 6 to Figure 8 The attitude quaternion, angular velocity, and control torque curves of the spacecraft during the attitude maneuver are shown, showing the simulation results. Figure 6 It is shown that the spacecraft can achieve attitude maneuver from the initial attitude to the target attitude, verifying the effectiveness of the attitude maneuver planning. Figure 7 and Figure 8It can be seen that the path obtained by the method satisfies the boundary constraint, and the control torque and angular velocity in the entire path do not exceed the upper limit value. Meanwhile, the attitude quaternion and angular velocity are also relatively stable.

[0114] Figure 9 and Figure 10 are two-dimensional and three-dimensional schematic diagrams of the planning result of the application respectively, and it can be seen that the planning path satisfies the forbidden pointing constraint and is smooth.

[0115] Then, the test set in 3.1 is used for testing, 8000 test results are obtained, and the success rate is 98.47%. At the same time, the 8000 test results are calculated, and the average energy consumption is 0.1333N 2 ·m 2 ·s / scene, and the average planning time is 0.384s per environment.

[0116] From the above simulation results, we can see that the DQN model is trained with a large amount of data, so as to learn how to reasonably allocate control torque and angular velocity between nodes, generate smoother paths and reduce energy consumption.

[0117] Table 2

[0118]

[0119]

[0120] In addition, generalization and handling of uncertainty are advantages of deep reinforcement learning. In the face of many complex, uncertain and unknown environments, sudden emergence of forbidden pointing areas can also be handled based on the observed DQN surrounding environment. As shown by the data in Table 2, increasing the number of forbidden constraints has little effect on the success rate and planning duration of the algorithm compared to other algorithms. Although the algorithm is trained in a simple environment, it has the ability to generalize for use in more complex environments.

Claims

1. A spacecraft attitude planning method under complex constraints based on deep reinforcement learning, characterized in that, a basic spacecraft attitude description method is constructed, and prohibition pointing constraints, control moment constraints, angular velocity constraints, and attitude kinematics and dynamics constraints are established, by setting an initial attitude, a target attitude, and a prohibited pointing area, the attitude maneuver problem in a three-dimensional attitude celestial sphere is converted into a two-dimensional plane path planning problem; based on the two-dimensional plane path planning problem, an initial training environment, an action, a state, and a reward are set, then a deep reinforcement learning model is established and the model is trained to obtain a trained model; an initial attitude, a target attitude, and a random environment are input into the trained model to obtain a satellite planning path; wherein the state is set to: ， wherein, represents the current time instant spacecraft attitude state, represents the current attitude vector and the angle with the obstacles in each direction , represents the current attitude vector and the relative direction with the target attitude vector and the angle ; The action is configured to rotate the spacecraft from a current moving direction to a different rotating direction by an angle to a target direction, thereby obtaining a different action; wherein the angle is calculated according to a formula ， wherein, representing the number of actions of the design; the reward setting is divided into three parts: a basic reward and a greedy reward, which are used to realize the basic goal of real-time path planning and accelerate the convergence speed of the algorithm, respectively; and a smoothness reward is used to improve the smoothness of the planning path; the greedy reward is based on a greedy strategy that only uses target information and ignores obstacle information to guide the spacecraft to reach the target attitude, and the greedy reward is designed in two parts: ， is the reward design based on the included angle distance: the included angle at the current moment is greater than the included angle at the current moment, then -0.1, otherwise -0.3, the negative design is because it is necessary to tell the agent that the more steps, the lower the reward, so if the included angle becomes larger and larger, a larger penalty is obtained; , is the target direction reward design: the current moment target direction angle is greater than the current moment, then +0.1, otherwise -0.1; the smoothness reward is designed as: 。 2. The spacecraft attitude planning method under complex constraints based on deep reinforcement learning according to claim 1 A method, characterized in that the basic spacecraft attitude description method is constructed, specifically, the spacecraft attitude is described by Euler axis / angle and unit quaternion, and the specific formula is as follows: ， wherein: , are the scalar and vector parts of the quaternion, respectively, denote the three components in the complex plane; denote the three-axis components corresponding to the Euler rotation axis; denote the rotation angle of the spacecraft around the Euler rotation axis .

3. The spacecraft attitude planning method under complex constraints based on deep reinforcement learning according to claim 2 A method, characterized in that the prohibition pointing constraints are: ， wherein, denotes the direction vector of the sensitive optical element in the spacecraft body coordinate system, denotes the direction vector of the bright celestial body in the inertial coordinate system, denotes the attitude transformation matrix from the inertial coordinate system to the spacecraft body coordinate system, denotes the field of view angle of the sensitive optical element.

4. The spacecraft attitude planning method under complex constraints based on deep reinforcement learning according to claim 1 A method, characterized in that the control moment constraints are: ， ， wherein, denotes the control moment of the spacecraft, denotes the component on the three axes; denotes the upper limit amplitude of the control moment of the spacecraft.

5. The spacecraft attitude planning method under complex constraints based on deep reinforcement learning according to claim 1 A method, characterized in that the angular velocity constraints are: ， ， wherein, is the angular velocity of the spacecraft, represents the components on the three axes; denotes the upper amplitude value of the angular velocity of the spacecraft.

6. The spacecraft attitude planning method under complex constraints based on deep reinforcement learning according to claim 1 A method, characterized in that the spacecraft is regarded as a rigid body with three-axis motion, and the attitude kinematics and dynamics constraints are described as follows: ， ， wherein is an angular velocity of the spacecraft, is a skew-symmetric matrix of is a derivative of is is a derivative of is a moment of inertia of the spacecraft, is a control moment of the spacecraft.

Images