Spacecraft attitude preset time cooperative control method based on reinforcement learning
By adopting a pre-set time cooperative control method based on reinforcement learning, the problem of balancing energy consumption and control performance in spacecraft formation systems is solved, and efficient attitude tracking and optimal control are achieved within a pre-set time.
Patent Information
- Application Number
- CN202310936647.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-28
- Publication Date
- 2026-02-13
- Estimated Expiration
- 2043-07-28
AI Technical Summary
Existing spacecraft formation cooperative control strategies improve system control performance but neglect energy consumption, leading to increased energy consumption and difficulty in precisely controlling convergence time and steady-state threshold.
A pre-set time cooperative control method based on reinforcement learning is adopted. By constructing a pre-set time distributed observer and obstacle function transformation technology, a distributed optimal attitude cooperative control law is designed. The controller is optimized by combining reinforcement learning methods to ensure that attitude tracking is completed within the pre-set time and reduce energy consumption.
It achieves spacecraft attitude tracking within a preset time, meeting transient and steady-state performance indicators while reducing energy consumption and improving the system's energy utilization efficiency.
Smart Images
Figure CN117208230B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of spacecraft control, and particularly relates to a spacecraft attitude preset time cooperative control method based on reinforcement learning. BACKGROUND
[0002] The spacecraft formation system can break through the constraints of single spacecraft in physical structure, and improve the information acquisition and resolution capability. Effective spacecraft attitude cooperative control is the key to guarantee the success of spacecraft formation flight missions such as on-orbit service, earth monitoring and space rescue, and thus has attracted widespread attention.
[0003] The cooperative ability of fast maneuvering and high-precision stability is the premise and guarantee for the spacecraft formation system to complete complex tasks such as high-precision observation and measurement. The main cooperative control methods include finite time cooperative control, fixed time cooperative control and preset performance control. Although the finite time cooperative control has the advantages of fast convergence speed, high control precision and strong robustness, the upper bound of the convergence time is related to the initial state of the system, which limits its application in engineering. The fixed time cooperative control makes the upper bound of the convergence time independent of the initial value. However, like the finite time cooperative control, the convergence time and steady-state threshold of the system can only be obtained by post-estimation. The preset performance cooperative control can quantitatively design the transient and steady-state performance of the system.
[0004] The common spacecraft formation cooperative control strategy only considers how to improve the control performance of the system (such as convergence speed, transient performance, steady-state performance, etc.), ignoring the energy consumption problem in the cooperative control process. The energy carried by the actual spacecraft is limited and valuable, and the above cooperative algorithm will inevitably increase the energy consumption while improving the performance of the formation system. SUMMARY
[0005] The application provides a spacecraft attitude preset time cooperative control method based on reinforcement learning, which can guarantee that the spacecraft formation system meets the preset convergence time, transient and steady-state performance while considering the energy consumption problem.
[0006] Technical scheme: The application provides a spacecraft attitude preset time cooperative control method based on reinforcement learning, comprising the following steps:
[0007] (1) Mathematical description of the multi-spacecraft attitude cooperative control problem: according to the dynamic characteristics of the spacecraft, a single spacecraft attitude dynamics model is established; the communication topological relationship between the member spacecraft and the leader and the member spacecraft and its neighbor spacecraft is described by graph theory;
[0008] (2) Prescheduled time distributed observer design: A prescheduled time distributed observer is constructed to guarantee that the follower can obtain the observation information of the leader's state within a prescheduled time;
[0009] (3) Prescheduled time performance function design and system equivalence transformation: The attitude cooperative tracking error estimation value is defined according to the attitude estimation value of each member spacecraft to the leader; the prescheduled time performance function is determined to quantitatively characterize the convergence time, transient performance and steady-state performance constraints of the cooperative tracking error; the error transformation based on the barrier function converts the cooperative tracking error system subject to the prescheduled performance constraints into an unconstrained system:
[0010] (4) Distributed optimal attitude cooperative control law design: For the state equation without constraints, the performance index function and its corresponding Hamilton-Jacobi-Bellman equation are determined, and the optimal control input is obtained by solving the partial derivative of the Hamilton-Jacobi-Bellman with respect to the optimal function.
[0011] (5) Reinforcement learning method is used to design an approximate optimal controller in the evaluation network framework.
[0012] Further, the single-spacecraft attitude dynamics model in step (1) is expressed as:
[0013]
[0014]
[0015] where σ i represents the modified Rodrigues parameter of the attitude of spacecraft i relative to the inertial system, ω i represents the angular velocity of spacecraft i, is its skew-symmetric matrix, J i represents the moment of inertia of the i-th spacecraft, τ i represents the control torque of the i-th spacecraft, and the expression of the matrix G(σ i ) is:
[0016]
[0017] Further, the process of step (1) described above for describing the communication topological relationship between the member spacecraft and the leader and between the member spacecraft and its neighbor spacecraft using graph theory is as follows:
[0018] The communication topological structure between spacecraft formation members is an undirected graph, denoted as where N = {n1,..., n n} represents the set of member spacecraft, is the set of edges, (n i , n j) represents that spacecraft j can directly interact with spacecraft i; A = [a ij ] is the weighted adjacency matrix of the undirected graph , if (n i , n j ) ∈ E, then the adjacency matrix element a ij > 0, otherwise a ij = 0; the master-slave architecture of the multi-spacecraft system assumes that there is a virtual leader, which is numbered 0, and its state is set to the given desired trajectory, if spacecraft i has a direct communication connection with the leader, then a i0 > 0, otherwise a i0 = 0.
[0019] Further, the step (2) is implemented as follows:
[0020] For spacecraft i, according to its attitude and velocity estimation value of the leader with its neighbor spacecraft, the preset time distributed observer is designed as:
[0021]
[0022]
[0023] wherein α0, α1, α2, α3> 0 are design parameters of the distributed observer, p i and represent the attitude σ0 and velocity of the leader estimated by the i-th spacecraft, k u is a constant, θ(t, t f1 , ε1) = 1 / (ε1 + θ0(t, t f1 )), t f1 > 0 is the transition time of the distributed observer specified by the designer, ε0, ε1> 0 are design parameters, which are used to represent the estimation error of the observer, and:
[0024]
[0025] Further, the attitude cooperative tracking error estimation value of step (3) is:
[0026]
[0027] wherein is the skew-symmetric matrix of p i .
[0028] Further, the preset time performance function of step (3) is:
[0029]
[0030] where t f and η ijs denote the specified convergence time upper bound and steady-state value, respectively, and a ijk k = 2, 3, 4 are determined by
[0031]
[0032] The performance constraint required to be satisfied by the cooperative tracking error is expressed as
[0033]
[0034] where are the design parameters used to characterize the upper and lower bounds of the pre-specified tracking error performance function, are the components of .
[0035] Further, the error transformation based on the barrier function in step (3) is expressed as
[0036]
[0037] where ε ij is the transformed cooperative tracking error, is the design parameter used to avoid excessive control input;
[0038] The definition of the transformed unconstrained system is expressed as
[0039]
[0040] where
[0041] Further, the performance index function in step (4) is expressed as
[0042]
[0043] where is the designed utility function, Q i is the designed positive definite matrix used to represent the proportion of the cooperative tracking error in the utility function, and Ψ i (τ i ) is the positive definite integrand used to constrain the control input:
[0044]
[0045] where λ i > 0 is the upper bound of the system control input, satisfying ||τ i || < λ i, is a positive definite matrix designed to represent the proportion of control input in the utility function.
[0046] Further, the Hamilton-Jacobi-Bellman equation in step (4) is:
[0047]
[0048] where, is the optimal control input, is the optimal cost function, is V i * the partial derivative of s i with respect to s
[0049] Taking the partial derivative of the Hamilton-Jacobi-Bellman equation with respect to s , the expression of the optimal control input is:
[0050]
[0051] Further, the implementation process of step (5) is as follows:
[0052] Based on the approximation ability of neural networks for nonlinear functions, an evaluation network is constructed to estimate the optimal performance index function online, and the actual optimal attitude cooperative control strategy is obtained based on the online approximation of the optimal performance index function; the expressions of the optimal performance index function and the optimal attitude cooperative control strategy are as follows:
[0053]
[0054]
[0055] where, W i represents the ideal evaluation network weight matrix; represents the basis function vector, represents the approximation error; define as the estimate of the ideal weight matrix W i , then the optimal performance index function and the attitude cooperative control strategy are approximated as:
[0056]
[0057]
[0058] where, the update law of s is designed as follows:
[0059]
[0060] where, βi For the learning law of design,
[0061] Beneficial effects: compared with the prior art, the preset distributed observer designed in the application can guarantee that all member spacecrafts obtain the state estimation information of the virtual leader within the preset time; the application introduces an error conversion technology based on an obstacle function to equivalently convert a performance-limited cooperative optimal control problem into a traditional unconstrained optimal stabilization problem, and an approximate optimal controller is obtained under an evaluation network framework, which not only guarantees that the cooperative tracking error satisfies the preset performance constraint, but also makes the control performance optimal. BRIEF DESCRIPTION OF DRAWINGS
[0062] Figure 1 For the flowchart of the application;
[0063] Figure 2 For the communication topology between the spacecrafts as shown in the figure;
[0064] Figure 3 For the cooperative tracking error convergence curve of the multi-spacecraft attitude system shown in the embodiment of the application; wherein (a) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed in the application; (b) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed in the application; (c) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed in the application;
[0065] Figure 4 For the control input component of the follower spacecraft under the cooperative strategy proposed in the application; wherein (a) is the change curve of the control input component τ i,1 ; (b) is the change curve of the control input component τ i,2 ; (c) is the change curve of the control input component τ i,3 ;
[0066] Figure 5 For the cooperative tracking error convergence curve under the preset attitude distributed cooperative control strategy without considering the optimal performance index; wherein (a) is the convergence curve of the cooperative tracking error component ; (b) is the convergence curve of the cooperative tracking error component ; (c) is the convergence curve of the cooperative tracking error component ;
[0067] Figure 6 For the control input curve under the preset attitude distributed cooperative control strategy without considering the optimal performance index; wherein (a) is the change curve of the control input component τ i,1the variation curve of the control input component τ i,2 the variation curve of the control input component τ i,3 . DETAILED DESCRIPTION
[0068] The application will be further described below in conjunction with the accompanying drawings.
[0069] The application proposes a spacecraft attitude preset time cooperative control method based on reinforcement learning, aiming to ensure that the attitude of the follower spacecraft cooperatively tracks the given reference attitude trajectory within the preset time, while making the control performance optimal. Figure 1 As shown in the figure, the method specifically comprises the following steps:
[0070] Step 1: Mathematical description of the multi-spacecraft attitude cooperative control problem: according to the dynamic characteristics of the spacecraft, the dynamic equation of a single spacecraft is established; the communication topological relationship between the member spacecraft and the leader and between the member spacecraft and its neighbor spacecraft is described by graph theory.
[0071] The attitude dynamics of a single spacecraft is expressed as:
[0072]
[0073]
[0074] wherein σ i represents the modified Rodrigues parameter of the attitude of the spacecraft i relative to the inertial system, ω i represents the angular velocity of the spacecraft i, is its skew-symmetric matrix, J i represents the moment of inertia of the i-th spacecraft, τ i represents the control torque of the i-th spacecraft, and the expression of the matrix G(σ i ) is:
[0075]
[0076] The attitude variation trajectory of the virtual leader is generated by the following kinematic equation:
[0077]
[0078] wherein ω d is the reference angular velocity, σ d is the modified Rodrigues parameter of the reference attitude relative to the inertial system attitude, and the expression of the matrix G(σ d ) is:
[0079]
[0080] The communication topology between spacecraft formation members is an undirected graph, denoted as G = (V, E), where V = {n1, n2,..., nN} represents the set of member spacecrafts, and E = { (n , n n ) | n i , n j ∈ V} is the set of edges, where (n ij , n i ) represents that spacecraft j and spacecraft i can directly exchange information. A = [a j ] is the weighted adjacency matrix of the undirected graph G, where a ij > 0 if (n ij , n i0 ) ∈ E, otherwise a i0 = 0. The master-slave architecture of the multi-spacecraft system assumes that there is a virtual leader, which is set as n0, and its state is set as the given desired trajectory. a i > 0 if spacecraft i has a direct communication connection with the leader, otherwise a u = 0. The communication topology of the entire formation system is shown in FIG. 1, and at least one follower can directly communicate with the leader.
[0081] Step 2: Prescheduled time distributed observer design: A prescheduled time distributed observer is constructed to ensure that the follower obtains the observation information of the state of the leader within the prescheduled time.
[0082] The member spacecraft i designs a prescheduled time distributed observer according to the attitude and velocity estimation values of the leader of the neighbor spacecrafts of the member spacecraft i, as follows:
[0083]
[0084]
[0085] wherein a0, a1, a2, a3> 0 are design parameters of the distributed observer, p u and represent the attitude and velocity estimation values of the leader of the ith spacecraft, k f1 is used to mark a constant k i = 0.2785, t f > 0 is a transition time of the distributed observer specified by the designer, and e0, e1> 0 are design parameters used to represent the estimation error of the observer, and:
[0086]
[0087] Step 3: Prescribed time performance function design and system equivalent transformation: Define the attitude cooperative tracking error estimation based on the leader's attitude estimation of each member spacecraft; Determine the prescribed performance function to quantitatively characterize the convergence time, transient performance and steady-state performance constraints of the cooperative tracking error; Transform the cooperative tracking error system with prescribed performance constraints into an unconstrained system based on the error transformation of barrier function.
[0088] The estimation of the attitude cooperative tracking error is:
[0089]
[0090] where, is the skew-symmetric matrix of p i .
[0091] The prescribed time performance function design is:
[0092]
[0093] where, t f and η ijs represent the specified upper bound of convergence time and steady-state value, and the design parameter a ijk , k = 2, 3, 4 are determined by:
[0094]
[0095] The performance constraints required to be satisfied by the cooperative tracking error are expressed as:
[0096]
[0097] where, is the design parameter used to characterize the upper and lower bounds of the prescribed tracking error performance function, is the component of .
[0098] The error transformation based on the barrier function is expressed as:
[0099]
[0100] where, ε ij is the transformed cooperative tracking error, is the design parameter used to avoid excessive control input.
[0101] Define , then the transformed unconstrained system is expressed as:
[0102]
[0103] where,
[0104] Step 4: Distributed optimal attitude cooperative control law design: For the unconstrained state equation, the performance index function and its corresponding Hamilton-Jacobi-Bellman equation are determined, and the optimal control input is obtained by solving the partial derivative of Hamilton-Jacobi-Bellman with respect to the optimal function.
[0105] For the unconstrained state equation, the performance index function determined is:
[0106]
[0107] where, is the designed utility function, Q i is the designed positive definite matrix, which is used to represent the proportion of cooperative tracking error in the utility function, Ψ i (τ i ) is a positive definite integral function used to constrain the control input:
[0108]
[0109] where, λ i > 0 is the upper limit of the system control input, which satisfies ||τ i || < λ i , is the designed positive definite matrix, which is used to represent the proportion of control input in the utility function.
[0110] The Hamilton-Jacobi-Bellman equation is:
[0111]
[0112] where, is the optimal control input, is the optimal cost function, is V i * the partial derivative with respect to s i .
[0113] The partial derivative of the above equation with respect to is taken to obtain the expression of the optimal control input:
[0114]
[0115] Step 5: Design an approximate optimal controller under the evaluation network framework using reinforcement learning method.
[0116] Based on the approximation ability of neural network to nonlinear function, the evaluation network is constructed to online estimate the optimal performance index function, and the actual optimal attitude cooperative control strategy is obtained based on the online approximation of the optimal performance index function. The optimal performance index function and the optimal attitude cooperative control strategy are expressed as follows:
[0117]
[0118]
[0119] wherein, W i represents the ideal evaluation network weight matrix; represents the base function vector, represents the approximation error. Since W i is an unknown matrix in practical application, the parameter matrix is introduced to estimate it, and the approximation of the optimal performance index function and the optimal attitude cooperative control strategy is obtained as follows:
[0120]
[0121]
[0122] wherein, the update law expression of the parameter matrix is as follows:
[0123]
[0124] wherein, β i is the designed learning law,
[0125] To verify that the preset time optimal cooperative control input based on reinforcement learning proposed in the embodiment can ensure that the multi-spacecraft attitude system can complete the cooperative tracking task within the preset time, corresponding simulation verification is performed. The attitude cooperative tracking control problem of five spacecrafts is considered, and the communication topology between the spacecrafts is shown in Figure 2 . The communication topology is a undirected connected graph. The inertia matrix, initial attitude and angular velocity of the follower spacecraft are shown in Table 1.
[0126] Table 1 Inertia matrix, initial attitude and angular velocity of the follower spacecraft
[0127] Numbering Inertia matrix Initial attitude Initial angular velocity 1 <![CDATA[J1 = diag([20, 17, 16])]]> [σ1 = [0.12, -0, 1, 0.09]] [0.0125, -0,004, -0.0038]] 2 <![CDATA[J2 = diag([20, 16, 15])]]> [σ2= [0.15, -0.13, 0.13]] [0.012, -0,0048, -0.0035]] 3 [j3 = diag([17, 14, 16])] [σ3 = [0.14, -0.10, 0.12]] [0.0128, -0,005, -0.0037] <!-- 8 -->]]> 4 [σ4 = [0.15, -0.15, 0.13]] [0.0122, -0,0046, -0.004]] 5 <![CDATA[J5 = diag([16, 14, 18])]]> [σ5 = [0.13, -0.13, 0.13]] [0.0119, -0,004, -0.0036]]
[0128] The initial value of the desired reference attitude is σ d (0) = [0.1 0.1 0.01] T , and the desired angular velocity trajectory is ω d(t) = 0.01 [cos(t / 40) - sin(t / 50) cos(t / 30)] T The parameters of the preset time performance function are selected as: a ij0 = [0.16 0.2 0.16] T η ijs = 0.005 [1 1 1] T t f = 50. Figure 3 are the cooperative tracking error convergence curves of the multi-spacecraft formation system shown in the embodiments of the present application; wherein (a) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed by the present application; (b) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed by the present application; (c) is the convergence curve of the cooperative tracking error component under the cooperative strategy proposed by the present application; it is not difficult to see that the cooperative tracking error converges to the origin within the preset time t f = 50, and meets the preset transient and steady-state performance indicators. Figure 4 are the control input components of the follower spacecraft under the cooperative strategy proposed by the present application; wherein (a) is the change curve of the control input component τ i,1 ; (b) is the change curve of the control input component τ i,2 ; (c) is the change curve of the control input component τ i,3 ; Figure 5 and Figure 6 are the cooperative tracking error convergence curves and control input curves under the preset attitude distributed cooperative control strategy without considering the optimal performance indicators; wherein Figure 5 in which (a) is the convergence curve of the cooperative tracking error component ; (b) is the convergence curve of the cooperative tracking error component ; (c) is the convergence curve of the cooperative tracking error component ; Figure 6 in which (a) is the change curve of the control input component τ i,1 ; (b) is the change curve of the control input component τ i,2 ; (c) is the change curve of the control input component τ i,3 . It can be seen through comparison that both cooperative control strategies can ensure that the formation system completes the specified cooperative task within the preset time, and meets the preset transient and steady-state performance indicators, but the optimal cooperative tracking strategy proposed by the present application requires smaller control torque, which can effectively reduce the energy consumption of formation flight
[0129] The above merely illustrates the embodiments of the present application, and is not intended to limit the patent protection scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which are made by using the content of the present application specification and drawings, are also included in the protection scope of the present application.
Claims
1. A spacecraft attitude pre-set time cooperative control method based on reinforcement learning, characterized in that, Includes the following steps: (1) Mathematical description of the attitude cooperative control problem of multiple spacecraft: Based on the dynamic characteristics of spacecraft, a single spacecraft attitude dynamic model is established; graph theory is used to describe the communication topology between member spacecraft and the navigator, as well as between member spacecraft and their neighboring spacecraft; (2) Design of pre-set time distributed observer: Construct a pre-set time distributed observer to ensure that the follower obtains the observation information of the leader's state within a pre-set time. (3) Design of pre-set time performance function and system equivalent transformation: Define attitude cooperative tracking error estimate based on the attitude estimate of each member spacecraft to the navigator; A predefined time performance function is used to quantitatively characterize the convergence time, transient performance, and steady-state performance constraints of the cooperative tracking error; an error transformation based on a barrier function converts the cooperative tracking error system subject to predefined performance constraints into an unconstrained system. (4) Design of distributed optimal attitude cooperative control law: For the unconstrained state equation, the performance index function and its corresponding Hamilton-Jacobi-Bellman equation are determined. The expression of the optimal control input with respect to the optimal function is obtained by solving the partial derivative of Hamilton-Jacobi-Bellman with respect to the optimal control. (5) Design an approximate optimal controller within the evaluation network framework using reinforcement learning methods; The implementation process of step (5) is as follows: Based on the ability of neural networks to approximate nonlinear functions, an evaluation network is constructed to estimate the optimal performance index function online and obtain the actual optimal attitude cooperative control strategy. The expressions for the optimal performance index function and the optimal attitude cooperative control strategy are as follows: Among them, W i This represents the ideal evaluation network weight matrix; Represents a basis function vector. Represents the approximation error; Definition For the ideal weight matrix W i Based on the estimation, the optimal performance index function and attitude cooperative control strategy are approximated as follows: in, The update law is designed in the following form: Where, β i The learning law designed for 2. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The attitude dynamics model of the single spacecraft described in step (1) is expressed as follows: Where, σ i ω represents the corrected Rodrigues parameter representing the attitude of spacecraft i relative to the inertial frame. i This represents the angular velocity of spacecraft i. For its antisymmetric matrix, J i Let τ represent the moment of inertia of the i-th spacecraft. i G(σ) represents the control torque of the i-th spacecraft. i The expression for ) is:
3. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The process of describing the communication topology between member spacecraft and the navigator, as well as between member spacecraft and their neighboring spacecraft, using graph theory in step (1) is as follows: The communication topology among spacecraft formation members is an undirected graph, denoted as . Where N = {n1,...,n} n } represents a set of member spacecraft. It is the set of edges, (n i ,n j This indicates that spacecraft j and spacecraft i can directly exchange information; A = [a ij [A graph is an undirected graph] The weighted adjacency matrix, if (n i ,n j If )∈E, then the adjacency matrix element a ij >0, otherwise a ij =0; In a master-slave multi-spacecraft system, a virtual navigator is assumed, numbered 0, and its state is set to a given desired trajectory. If spacecraft i has a direct communication connection with the navigator, then a i0 >0, otherwise a i0 =0.
4. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The implementation process of step (2) is as follows: For spacecraft i, based on its attitude and velocity estimates of the navigator from its neighboring spacecraft, a pre-defined time-distributed observer is designed as follows: Where α0, α1, α2, α3 > 0 are the design parameters of the distributed observer, p i and Let σi represent the attitude σ0 and velocity of the i-th spacecraft relative to the navigator. The estimate, k u It is a constant value. θ(t,t f1 ,ε1)=1 / (ε1+θ0(t,t f1 )), t f1 >0 represents the transition time of the distributed observer specified by the designer, ε0,ε1>0 are design parameters used to represent the estimation error of the observer, and:
5. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The estimated value of the attitude cooperative tracking error in step (3) is: in, For p i An antisymmetric matrix.
6. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The preset time performance function mentioned in step (3) is: Among them, t f and η ijs This represents the upper bound of the specified convergence time and the steady-state value, design parameter a. ijk k = 2, 3, 4 is determined by the following formula: The performance constraint that the cooperative tracking error needs to satisfy is expressed as follows: in, These are design parameters used to characterize the upper and lower boundaries of the preset tracking error performance function. yes The amount.
7. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The error transformation based on the barrier function described in step (3) is expressed as follows: Where, ε ij For the converted cooperative tracking error, θ ij >0 is a design parameter used to avoid excessively large control inputs; definition The transformed unconstrained system is represented as: in, 8. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The performance index function mentioned in step (4) is: in, For the utility function of the design, Q i Ψ is a designed positive definite matrix used to represent the proportion of cooperative tracking error in the utility function. i (τ i ) is a positive definite integrand used to constrain and control the input: Where, λ i >0 represents the upper limit of the system's control input, satisfying ||τ i ||<λ i , This is a positive definite matrix designed to represent the proportion of the control input in the utility function.
9. The spacecraft attitude pre-set time cooperative control method based on reinforcement learning according to claim 1, characterized in that, The Hamilton-Jacobi-Bellman equation mentioned in step (4) is: in, For optimal control input, The optimal cost function is... For V i * Compared to s i The partial derivatives; Find the equations for Hamilton-Jacobi-Bellman with respect to... The partial derivatives yield the expression for the optimal control input as follows:
Citation Information
Patent Citations
Data-driven heterogeneous missile formation switching communication topology cooperative control method
CN115877718A