Satellite attitude adaptive control algorithm and system based on DSAC-T

Abstract

The invention relates to the related field of satellite communication, discloses a DSAC-T-based satellite attitude adaptive control algorithm and system, and solves the problem that the angular velocity of a satellite continuously swings between a positive value and a negative value in the existing algorithm, so that the stable operation of the satellite can be better controlled, and meanwhile, compared with the prior art, the satellite attitude adaptive control algorithm and system have the advantage that the satellite attitude adaptive control efficiency is improved. The DSAC-T algorithm has a faster control speed and faster algorithm convergence, improves the real-time performance of satellite attitude control, improves the anti-interference degree, has good control stability in different interference experiment environments, has good robustness for interference which is not seen in the training process, and can be applied to the field of satellite attitude control. The trained agency also has good adaptability to satellites with different rotational inertia, so that the trained agency can be conveniently migrated to a new satellite, and has better universality.

Description

Technical Field

[0001] The present invention relates to the field related to satellite communications, and in particular to a satellite attitude adaptive control algorithm and system based on DSAC-T. Background Art

[0002] Attitude control is very important for the safe and normal operation of space satellites. When performing different tasks, satellites need to be transformed into corresponding attitude directions through control. In the existing technology, satellite attitude control schemes include nanosatellite attitude control algorithms based on DDPG, satellite attitude control algorithms based on PPO algorithms, and satellite attitude adaptive continuous control algorithms based on TD3 (Twin Delayed Deep Deterministic Policy Gradient).

[0003] The existing methods converge slowly, and there are certain unstable factors in the control accuracy and stability. In addition, they are prone to jittering back and forth during the control process, which is not conducive to the stable control of the satellite's attitude. The moment of inertia affects the difficulty of an object changing its rotation. The smaller the moment of inertia, the easier it is for an object to change its rotation state. The existing scheme based on the TD3 algorithm is not stable enough. It is easy to give a larger control torque in the direction where the satellite's moment of inertia is smaller, causing the satellite to jitter back and forth. Summary of the invention

[0004] The object of the present invention is to provide a satellite attitude adaptive control algorithm and system based on DSAC-T to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A satellite attitude adaptive control algorithm based on DSAC-T, including:

[0007] Initialize the parameters and environment, and calculate and obtain the desired control amount based on the current posture, wherein the desired control amount includes the rotation axis and the corresponding desired rotation angle;

[0008] Responding to and executing the desired control amount, and calculating based on the current state to obtain a reward value, wherein the reward value is used to represent the degree of approach of the satellite to the desired target;

[0009] The intermediate data is packaged and cached in the revisit buffer area, and the storage of the revisit buffer area is monitored. If the data volume reaches a preset value, a training mark is generated;

[0010] In response to the training identifier, batch data is obtained by revisiting the cache area for training to update its own parameters, and the results are evaluated, and the training evaluation process is cycled with a predetermined number of training steps to confirm the optimal model.

[0011] As a further solution of the present invention: the step of calculating and obtaining the desired control amount based on the current posture specifically includes:

[0012] The satellite state is modeled in a simulation environment, and the modeling model of the state is characterized as follows:

[0013]

[0014] The error quaternion q e Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis;

[0015] Based on the error quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by:

[0016] Wherein, φ is the desired rotation angle.

[0017] As a further solution of the present invention: in the step of performing fitting calculation on the desired control amount to obtain the rotation axis and the corresponding desired rotation angle, the specific fitting calculation method is:

[0018] Error quaternion q e ,q e The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

[0019] As a further solution of the present invention: the step of responding to and executing the desired control amount and calculating based on the current state to obtain a reward value specifically includes:

[0020] receiving a control command including the desired control amount, executing the control command, and generating corresponding control torques at preset time intervals to control the rotation of the satellite;

[0021] When the satellite returns to a new state, a reward value is calculated and given, and the reward value is represented by:

[0022]

[0023] in, q s,t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

[0024] As a further solution of the present invention: the step of confirming the optimal model specifically includes:

[0025] The indicator is evaluated based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is set as the optimal result;

[0026] If the optimal model and the optimal result are currently stored, the current result will be compared with the optimal result. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

[0027] The embodiment of the present invention aims to provide a satellite attitude adaptive control system based on DSAC-T, comprising:

[0028] An initial calculation module is used to initialize parameters and environment, and calculate and obtain the desired control amount based on the current posture, wherein the desired control amount includes a rotation axis and a corresponding desired rotation angle;

[0029] A reward granting module, used to respond to and execute the desired control amount, and to calculate a reward value based on the current state, wherein the reward value is used to represent the degree of approach of the satellite to the desired target;

[0030] A data monitoring module is used to pack and cache the intermediate data in the revisit buffer area, and to perform storage monitoring on the revisit buffer area. If the data volume reaches a preset value, a training mark is generated;

[0031] The loop evaluation module is used to respond to the training identifier, obtain batch data for training by revisiting the cache area to update its own parameters, and evaluate the results, and loop the training evaluation process with a predetermined number of training steps to confirm the optimal model.

[0032] As a further solution of the present invention: the initial calculation module specifically includes:

[0033] The state modeling unit is used to model the satellite state in a simulation environment, and the modeling model of the state is characterized as follows:

[0034]

[0035] The error quaternion qe Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis;

[0036] Parameter fitting unit for error-based quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by:

[0037] Wherein, φ is the desired rotation angle.

[0038] As a further solution of the present invention: the parameter fitting unit specifically includes:

[0039] Expected assignment subunit, used to assign error quaternion q e ,q e The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

[0040] As a further solution of the present invention: the reward granting module specifically includes:

[0041] A command response unit, configured to receive a control command including the desired control amount, execute the control command, and generate a corresponding control torque at a preset time interval to control the satellite rotation;

[0042] The reward feedback unit is used to calculate and give a reward value when the satellite returns to a new state. The reward value is represented by:

[0043]

[0044] in, q s,t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

[0045] As a further solution of the present invention: the cycle evaluation module includes:

[0046] The optimal setting unit is used to evaluate the index based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is used as the optimal result;

[0047] The optimal judgment unit is used to compare the current result with the optimal result if the optimal model and the optimal result are currently stored. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

[0048] Compared with the prior art, the beneficial effects of the present invention are: it solves the problem that the prior algorithm causes the satellite angular velocity to constantly swing between positive and negative values, thereby enabling better control of the smooth operation of the satellite. At the same time, compared with the prior art, the DSAC-T algorithm has a faster control speed and faster algorithm convergence, which improves the real-time performance of satellite attitude control and the degree of anti-interference. It has good control stability in different interference experimental environments and has good robustness to interference that has not been seen during the training process. The trained agent also has good adaptability to satellites with different moments of inertia, which allows it to be easily migrated to new satellites and has better versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of a satellite attitude adaptive control algorithm based on DSAC-T.

[0050] Figure 2 This is a flow chart of a satellite attitude adaptive control algorithm based on DSAC-T.

[0051] Figure 3 This is a block diagram of the satellite attitude adaptive control system based on DSAC-T. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0054] like Figure 1 and Figure 2 The satellite attitude adaptive control algorithm based on DSAC-T provided by an embodiment of the present invention comprises the following steps:

[0055] S10, initializing parameters and environment, and calculating and obtaining a desired control amount based on the current posture, wherein the desired control amount includes a rotation axis and a corresponding desired rotation angle;

[0056] S20, responding to and executing the desired control amount, and calculating based on the current state to obtain a reward value, wherein the reward value is used to represent the degree of approach of the satellite to the desired target;

[0057] S30, packaging and caching the intermediate data into a revisit buffer area, and performing storage monitoring on the revisit buffer area, and generating a training mark if the data volume reaches a preset value;

[0058] S40, responding to the training identifier, obtaining batch data for training by revisiting the cache area to update its own parameters, and evaluating the results, and looping the training evaluation process with a predetermined number of training steps to confirm the optimal model.

[0059] In this embodiment, a satellite attitude adaptive control algorithm based on DSAC-T is provided, which solves the problem that the existing algorithm causes the satellite angular velocity to constantly swing between positive and negative values, so that the smooth operation of the satellite can be better controlled. At the same time, compared with the existing technology, the DSAC-T algorithm has a faster control speed and a faster algorithm convergence, which improves the real-time performance of satellite attitude control and the degree of anti-interference. It has good control stability in different interference experimental environments and has good robustness to interference that has not been seen during the training process. The trained agent also has good adaptability to satellites with different moments of inertia, which makes it easy to migrate to new satellites and has better versatility. Specifically, the satellite attitude adaptive continuous control algorithm based on DSAC-T (Distributional Soft Actor-Critic with Three refinements) achieves faster and more stable control, and can resist external disturbances to a certain extent. DSAC-T is the deep reinforcement learning algorithm with the best training effect. It is based on artificial intelligence and neural network methods for learning and training. It can achieve good control ability by learning state-action return distribution, and also introduces expected value replacement. Three improvements, namely substituting, twin value distribution learning, and variance-based critic gradient adjusting, have improved learning stability, made it adaptive to a certain extent, reduced the complexity and uncertainty caused by manual design, achieved better results, and made it generalizable and resistant to situations not seen during training.

[0060] As another preferred embodiment of the present invention, the step of calculating and obtaining the desired control amount based on the current posture specifically includes:

[0061] The satellite state is modeled in a simulation environment, and the modeling model of the state is characterized as follows:

[0062]

[0063] The error quaternion q e Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis;

[0064] Based on the error quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by:

[0065] Wherein, φ is the desired rotation angle.

[0066] Furthermore, in the step of performing fitting calculation on the desired control amount to obtain the rotation axis and the corresponding desired rotation angle, the specific fitting calculation method is:

[0067] Error quaternion q e ,q e The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

[0068] In this embodiment, at the beginning, the parameters required by the algorithm and the simulation environment are initialized first, and then the training cycle iteration process begins; in the simulation environment, the state is modeled as The error quaternion q e It is used to indicate the current attitude of the satellite, e indicates the rotation axis, and φ indicates the desired rotation angle. Indicates q e The derivative with respect to time t, ω represents the angular velocity of the satellite; therefore, it is assumed here that when q e =[0,0,0,1], When ω=0, it is considered that the optimal state has been reached. In each round of iteration, the environment is first reset, the satellite's rotation speed is reset to 0, and then a desired rotation angle φ∈[30°, 150°] and a rotation axis e are randomly generated as the target for the agent to control the satellite's rotation in this round. The environment sets two termination conditions. When the maximum allowed number of steps is reached (here the training period is set to 500 steps) or the maximum allowed angular velocity (0.25rad / s) is reached, the simulation will be terminated and the next round of simulation will begin. At each step, the agent uses the state of the satellite at this time as the input of the neural network, calculates an output through the network, and applies the output as the control torque to the satellite to control its rotation.

[0069] As another preferred embodiment of the present invention, the step of responding to and executing the desired control amount and calculating based on the current state to obtain a reward value specifically includes:

[0070] receiving a control command including the desired control amount, executing the control command, and generating corresponding control torques at preset time intervals to control the rotation of the satellite;

[0071] When the satellite returns to a new state, a reward value is calculated and given, and the reward value is represented by:

[0072]

[0073] in, q s，t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

[0074] In this embodiment, after receiving the control command from the agent, the satellite starts to rotate. Since the control torque is generated at intervals, the rotation will continue for a period of time before the next control torque is generated. Then the satellite returns to its new state and gives a reward value. The reward is calculated according to the following formula, where: q s，t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, if the current q s A smaller value indicates that the spacecraft is closer to the desired goal than the previous step, so it is given a positive reward, otherwise a negative reward is given, which will encourage the agent to continuously control the spacecraft to rotate in the desired direction and curb the agent's regression; m represents the maximum number of steps, n represents the current number of steps, and at each step, if q s,t≥0.99999762 (i.e., the error angle φ is ≤0.25° at this time), an additional reward of 9 will be given. When the simulation ends, if it can still maintain this state, a reward of 50 will be given directly. If the angular velocity of the spacecraft exceeds the maximum allowable angular velocity (0.5rad / s) at the end, a penalty of -25 will be imposed. By setting the reward, the agent can be guided to learn how to continuously keep the attitude of the spacecraft within a desired range and avoid the agent falling into a bad environment.

[0075] As another preferred embodiment of the present invention, the step of confirming the optimal model specifically includes:

[0076] The indicator is evaluated based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is set as the optimal result;

[0077] If the optimal model and the optimal result are currently stored, the current result will be compared with the optimal result. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

[0078] In this embodiment, in the aforementioned embodiment, the data finally generated will be packaged into a tuple and stored in the replay buffer. If there is enough data stored in the replay buffer at this time, the agent will extract a batch of data from the buffer for training and update its own parameters, and the algorithm will save the current model every certain number of steps.

[0079] After several training iterations, the algorithm will evaluate the current model several times. During the evaluation, a new environment will be created and the latest saved model parameters will be used to initialize a new model instead of the environment and model during training. The reward obtained by the agent (i.e., the cumulative sum of the rewards obtained) will be the evaluation indicator of the evaluation result. If the optimal model and the optimal result have not been saved yet, the current model will be the optimal model and the current result will be the optimal result. If the optimal model and the optimal result already exist, the current result will be compared with the optimal result. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result. By adding the evaluation link to the training process, it is helpful to better judge the training effect and save the model parameters in time. This ensures that even if unexpected performance degradation occurs in the subsequent training process, the model parameters with the optimal performance can still be guaranteed. The training-evaluation cycle will continue until the specified number of training steps is reached and stops. Since the optimal model is saved in the evaluation stage, the model can be easily applied to the deployment of downstream tasks and subsequent further applications.

[0080] like Figure 3 As shown, the present invention also provides a satellite attitude adaptive control system based on DSAC-T, which comprises:

[0081] The initial calculation module 100 is used to initialize the parameters and environment, and calculate and obtain the desired control amount based on the current posture, wherein the desired control amount includes the rotation axis and the corresponding desired rotation angle;

[0082] A reward granting module 200, for responding to and executing the desired control amount, and calculating based on the current state to obtain a reward value, wherein the reward value is used to represent the degree of approach of the satellite to the desired target;

[0083] The data monitoring module 300 is used to pack and cache the intermediate data in the revisit buffer area, and perform storage monitoring on the revisit buffer area. If the data volume reaches a preset value, a training mark is generated;

[0084] The loop evaluation module 400 is used to respond to the training identifier, obtain batch data for training by revisiting the cache area to update its own parameters, and evaluate the results, and loop the training evaluation process with a predetermined number of training steps to confirm the optimal model.

[0085] As another preferred embodiment of the present invention, the initial calculation module 100 specifically includes:

[0086] The state modeling unit is used to model the satellite state in a simulation environment, and the modeling model of the state is characterized as follows:

[0087]

[0088] The error quaternion q e Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis;

[0089] Parameter fitting unit for error-based quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by:

[0090] Wherein, φ is the desired rotation angle.

[0091] As another preferred embodiment of the present invention, the parameter fitting unit specifically further includes:

[0092] Expected assignment subunit, used to assign error quaternion q e ,qe The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

[0093] As another preferred embodiment of the present invention, the reward granting module specifically includes:

[0094] A command response unit, configured to receive a control command including the desired control amount, execute the control command, and generate a corresponding control torque at a preset time interval to control the satellite rotation;

[0095] The reward feedback unit is used to calculate and give a reward value when the satellite returns to a new state. The reward value is represented by:

[0096] in, q s,t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

[0097] As another preferred embodiment of the present invention, the cyclic evaluation module includes:

[0098] The optimal setting unit is used to evaluate the index based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is used as the optimal result;

[0099] The optimal judgment unit is used to compare the current result with the optimal result if the optimal model and the optimal result are currently stored. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0101] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0102] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A satellite attitude adaptive control algorithm based on DSAC-T, characterized in that: Include: Initialize the parameters and environment, and calculate and obtain the desired control amount based on the current posture, wherein the desired control amount includes the rotation axis and the corresponding desired rotation angle; Responding to and executing the desired control amount, and calculating based on the current state to obtain a reward value, wherein the reward value is used to represent the degree of approach of the satellite to the desired target; The intermediate data is packaged and cached in the revisit buffer area, and the storage of the revisit buffer area is monitored. If the data volume reaches a preset value, a training mark is generated; In response to the training identifier, batch data is obtained by revisiting the cache area for training to update its own parameters, and the results are evaluated, and the training evaluation process is cycled with a predetermined number of training steps to confirm the optimal model.

2. The satellite attitude adaptive control algorithm based on DSAC-T according to claim 1, characterized in that: The step of calculating and obtaining the desired control amount based on the current posture specifically includes: The satellite state is modeled in a simulation environment, and the modeling model of the state is characterized as follows: The error quaternion q e Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis; Based on the error quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by: Wherein, φ is the desired rotation angle.

3. The satellite attitude adaptive control algorithm based on DSAC-T according to claim 2, characterized in that: In the step of performing fitting calculation on the desired control amount to obtain the rotation axis and the corresponding desired rotation angle, the specific fitting calculation method is: Error quaternion q e ,q e The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

4. The satellite attitude adaptive control algorithm based on DSAC-T according to claim 3 is characterized in that: The step of responding to and executing the desired control amount and calculating based on the current state to obtain a reward value specifically includes: receiving a control command including the desired control amount, executing the control command, and generating corresponding control torques at preset time intervals to control the rotation of the satellite; When the satellite returns to a new state, a reward value is calculated and given, and the reward value is represented by: in, q s,t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

5. The satellite attitude adaptive control algorithm based on DSAC-T according to claim 1, characterized in that: The step of confirming the optimal model specifically includes: The indicator is evaluated based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is set as the optimal result; If the optimal model and the optimal result are currently stored, the current result will be compared with the optimal result. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

6. A satellite attitude adaptive control system based on DSAC-T, characterized in that: Include: An initial calculation module is used to initialize parameters and environment, and calculate and obtain the desired control amount based on the current posture, wherein the desired control amount includes a rotation axis and a corresponding desired rotation angle; A reward granting module, used to respond to and execute the desired control amount, and to calculate a reward value based on the current state, wherein the reward value is used to represent the degree of approach of the satellite to the desired target; A data monitoring module is used to pack and cache the intermediate data in the revisit buffer area, and to perform storage monitoring on the revisit buffer area. If the data volume reaches a preset value, a training mark is generated; The loop evaluation module is used to respond to the training identifier, obtain batch data for training by revisiting the cache area to update its own parameters, and evaluate the results, and loop the training evaluation process with a predetermined number of training steps to confirm the optimal model.

7. The satellite attitude adaptive control system based on DSAC-T according to claim 6, characterized in that: The initial calculation module specifically includes: The state modeling unit is used to model the satellite state in a simulation environment, and the modeling model of the state is characterized as follows: The error quaternion q e Used to characterize the current attitude of the satellite, To characterize q e The derivative with respect to time, ω, is used to characterize the angular velocity of the satellite, and the e is used to characterize the rotation axis; Parameter fitting unit for error-based quaternion q e The desired control amount is fitted and calculated to obtain the rotation axis and the corresponding desired rotation angle. The error quaternion q e Characterized by: Wherein, φ is the desired rotation angle.

8. The satellite attitude adaptive control system based on DSAC-T according to claim 7, characterized in that: The parameter fitting unit specifically includes: Expected assignment subunit, used to assign error quaternion q e ,q e The derivative with respect to time t And the angular velocity ω of the satellite is assigned so that the q e =[0,0,0,1], ω=0, that is, the fitting satellite reaches the optimal attitude, and the expected rotation angle at this time is calculated.

9. The satellite attitude adaptive control system based on DSAC-T according to claim 8, characterized in that: The reward granting module specifically includes: A command response unit, configured to receive a control command including the desired control amount, execute the control command, and generate a corresponding control torque at a preset time interval to control the satellite rotation; The reward feedback unit is used to calculate and give a reward value when the satellite returns to a new state. The reward value is represented by: in, q s,t Indicates the current q s value, and q s,t-1 Indicates the q of the previous step s Value, m represents the maximum number of steps, and n represents the current number of steps.

10. The satellite attitude adaptive control system based on DSAC-T according to claim 6, characterized in that: The cycle assessment module includes: The optimal setting unit is used to evaluate the index based on the cumulative sum of the rewards obtained. If the optimal model and the optimal result are not currently stored, the current model is set as the optimal model and the current result is used as the optimal result; The optimal judgment unit is used to compare the current result with the optimal result if the optimal model and the optimal result are currently stored. If the current evaluation result is better than the existing optimal evaluation result, the currently used model will be saved as the optimal model and the current result will be saved as the optimal result.

Images