An optimal impedance learning method for space robots based on sampling data

Abstract

The present invention relates to a space robot optimal impedance learning method based on sampled data. First, a dynamic model of a multi-degree-of-freedom space robot system is established, and a mass-spring-damper approximate interaction model is adopted; secondly, an interaction performance function is designed to convert the optimal impedance control problem into a special linear quadratic tracking (LQT) problem; then, assuming that the robot terminal position and speed are measurable, a different strategy model-free integral reinforcement learning method based on full state feedback is designed for impedance parameter solution; finally, considering the problem of missing robot terminal speed, a different strategy model-free integral reinforcement learning algorithm based on historical data feedback is designed to solve the optimal impedance parameters with the help of state reconstruction technology. The present invention uses limited sampled data to learn the optimal impedance parameters offline, which can be used for systems with missing terminal speed, unknown interaction models and instruction generator dynamics, to achieve safe and compliant interactive control of space robots and unknown target surfaces.

Description

Technical Field

[0001] The invention belongs to the field of intelligent compliance control of space robots, and in particular relates to an optimal impedance learning method for space robots based on sampling data. Background Art

[0002] Space robots can assist or replace astronauts in completing more dangerous on-orbit operation tasks, such as the assembly and maintenance of space stations, the release and recovery of satellites, the capture and towing of space debris, etc., which greatly expands the human activity and exploration scope in the universe and has broad application prospects. The world's major space powers have carried out a large number of on-orbit verification studies, such as Canada's Shuttle Remote Manipulator System (SRMS), the International Space Station Mobile Servicing System (MSS), the European Robotic Arm (ERA), Japan's ETS-VII Robotic Arm System and Experimental Module Remote Operation System (JEMRMS), the U.S. Orbital Express program, and the Chinese Space Station Robotic Arm System.

[0003] During on-orbit operation, the space robot will inevitably come into contact with the target. Once the contact force is too large or out of control, it will seriously threaten the safety of the entire spacecraft system and cause immeasurable economic losses. Therefore, studying the safe interaction technology between the space robot and the target is the core of the on-orbit operation mission and is of great significance to the development of space technology in various countries. In order to achieve the goal of safe interaction, the impedance control method is usually used to respond to the external force to adapt to the contact environment. However, the traditional method of fixed impedance parameters cannot adapt to complex and changeable operation tasks, which requires the impedance parameters to be able to self-adjust according to different interaction environments. Common methods for achieving variable impedance control include adaptive control (CN112372637A), iterative learning control (CN113043266A), etc., but the traditional adaptive control method cannot guarantee the optimal interaction performance, and iterative learning control requires the robot to repeat operation training.

[0004] As an intelligent algorithm, reinforcement learning can handle problems such as partially known system models and completely unknown models from the perspective of optimal control, and has been developed and applied in different fields. When dealing with impedance control problems, reinforcement learning algorithms can learn the optimal impedance parameters through historical sampling data when the interaction model is unknown, avoiding the problem of traditional fixed impedance parameters and iterative learning methods that require multiple selections or experiments, and can ensure the optimality of the interaction process. Existing optimal adaptive impedance control technologies based on reinforcement learning (X.Liu, SSGe, F.Zhao, and X.Mei, "Optimized Interaction Control for Robot Manipulator Interacting WithFlexible Environment," IEEE / ASME Transactions on Mechatronics, vol.26, no.6, pp.2888-2898, 2021; X.Liu, SSGe, F.Zhao, and X.Mei, "Optimized ImpedanceAdaptation of Robot Manipulator Interacting With Unknown Environment," IEEE Transactions on Control Systems Technology, vol.29, no.1, pp.411-419, 2021) are all designed based on discrete interaction models, but the interaction models in real systems are all continuous systems, and the discretization of the model will inevitably affect the learning accuracy. In addition, the reinforcement learning algorithm designed based on continuous systems involves integral and derivative problems, which has greater theoretical challenges than discrete learning algorithms. Therefore, the present invention aims at the continuous interaction system, considers the two working conditions of measurable and unmeasurable terminal velocity, designs a model-free reinforcement learning impedance control method based on sampling data, realizes safe, intelligent and optimal interaction between the space robot and the target, and provides a theoretical basis and technical support for enhancing the intelligent autonomy level of the space robot's on-orbit operation. Summary of the invention

[0005] In order to solve the above technical problems, considering the complex and diverse on-orbit operation tasks of space robots, the present invention provides a space robot optimal impedance learning method based on sampled data in view of the problems that the existing control methods need to repeat experiments, rely on full-state measurement, and cannot meet the optimal interaction performance. The method is developed based on impedance control, state reconstruction technology and an integral reinforcement learning algorithm based on different strategies. It can use sampled data to autonomously learn impedance control parameters and ensure optimal interaction performance when the terminal speed is unmeasurable and the interaction model is completely unknown. It can be used for optimal impedance control during the on-orbit operation of space robots.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] An optimal impedance learning method for space robots based on sampled data is designed to solve the optimal impedance control problem of space robot systems with completely unknown interaction models and unmeasurable terminal velocity. First, a multi-degree-of-freedom space robot dynamics model including terminal interaction forces is established based on the Kane equation, and the interaction model is approximately described using a mass-spring-damper. Second, an interaction performance function is designed according to the control objective, and the optimal impedance control problem is converted into a special linear quadratic tracking (LQT) problem. Then, assuming that the terminal position and velocity are measurable, a different-strategy model-free integral reinforcement learning method based on full-state feedback is designed to solve the optimal impedance parameters. Finally, in the case of missing terminal velocity information, a different-strategy model-free integral reinforcement learning algorithm based on historical data feedback is designed using state reconstruction technology to achieve optimal impedance control. The specific implementation steps are as follows:

[0008] (1) Based on the Kane equation, a multi-degree-of-freedom spatial robot dynamics model including end-to-end interaction forces is established, and the interaction model is approximately described using a mass-spring-damper.

[0009] The Kane equation is used to establish the dynamic model of a multi-degree-of-freedom space robot including the end interaction force as follows:

[0010]

[0011] Where q represents the generalized coordinates of the space robot system, They represent the first and second order derivatives of q respectively; D(q) represents the global mass matrix of the system, is the global nonlinear term of the system, τ represents the system control force / torque, J e represents the Jacobian matrix related to the motion of the space robot, F e represents the end interaction force, which is usually measured by a force sensor; the superscript "T" represents the transpose of the matrix;

[0012] End interaction force F eBy approximating the interaction model as a mass-spring-damper system, we obtain:

[0013]

[0014] Where M e , D e and K e They are respectively represented as the mass coefficient, damping coefficient and stiffness coefficient of the system; x represents the component of the robot end position in the inertial system, represent the first and second derivatives of x respectively; x e The component representing the position of the interaction surface in the inertial system;

[0015] The expression of the impedance controller is:

[0016]

[0017] Where M d , D d0 Indicates the preset expected inertia and damping coefficient, D d , K d They represent the desired damping and stiffness coefficients to be further designed, respectively, and x d represents the desired trajectory of the robot end;

[0018] By adding the interaction model to the impedance controller, the impedance interaction system expression is as follows:

[0019]

[0020] Among them, x e represents the constant vector of the environmental position, M e1 Defined as M e1 =M d +M e , D e1 Defined as D e1 =D d0 +D e , represents the non-inertial interaction force;

[0021] Get state variables The control input is u = f, and the linear state equation of the impedance interaction system is obtained:

[0022]

[0023] in, represents the derivative of ξ,

[0024]

[0025] Where y represents the output vector of the system. When both the velocity and the end position can be measured, the output matrix C is expressed as When only the end positions are measurable, the output matrix is ​​represented by C=[0, I]; 0 and I represent a zero matrix and an identity matrix of appropriate dimensions, respectively.

[0026] (2) Designing the interactive performance function according to the control objective, the optimal impedance control problem is transformed into a special linear quadratic tracking (LQT) problem;

[0027] Assume that the desired trajectory x at the end d Produced by the following linear instruction generator:

[0028]

[0029] In the formula, Represents x d The first-order derivative of , Ω represents the dynamics matrix of the instruction generator;

[0030] The performance function is set to the following form with discount factors:

[0031] V(y(t),x d (t))=∫ t ∞ e -γ(τ-t) [(yx d ) T Q0(yx d )+u T Ru]dτ,

[0032] Among them, Q0≤0 and R>0 are the corresponding symmetric weight matrices, γ is the discount factor, and t represents the current time;

[0033] Then the state of the augmented system is constructed as Ξ=[ξ T , x d ] T , the augmented system is:

[0034]

[0035] The performance function is thus rewritten as:

[0036] V(Ξ(t))=∫ t ∞ e -γ(τ-t) (Ξ T QΞ+u T Ru)dτ=Ξ T (t)PΞ(t),

[0037] Among them, Q=[0, I, -I] T Q0[0, I, -I];

[0038] The optimal controller is denoted as u * =-K * Ξ form, where K * is the optimal control gain, which is calculated as:

[0039]

[0040] When the augmented system does not exist Gx e When Gx e Due to the existence of the term, the traditional solution method of the linear quadratic tracking problem is no longer applicable. It is necessary to regard this problem as a special type of linear quadratic tracking problem and solve P through the following improved discounted ARE equation:

[0041]

[0042] (3) Assuming that the terminal position and speed can be measured, a different-strategy model-free integral reinforcement learning method based on full-state feedback is designed to solve the optimal impedance parameters;

[0043] If the end position and speed of the impedance interaction system can be measured, in order to learn the optimal impedance parameters, the different-strategy model-free integral reinforcement learning algorithm based on full state feedback is as follows:

[0044] a) Initialization: Give an initial stable control strategy u0;

[0045] b) Solve the kernel matrix P i and the optimal control gain K i+1 :

[0046]

[0047] Among them, Δt represents the system sampling period; at the i-th moment, Q i Defined as Q i =K iT R i +Q,K i represents the control gain at the i-th moment; P i represents the kernel matrix at the i-th moment;

[0048] c) Convergence condition: Determine the iteration stop condition || P i+1 -P i ||≤ε, if satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

[0049] (4) In the case of missing terminal velocity information, a different-strategy model-free integral reinforcement learning algorithm based on historical data feedback is designed using state reconstruction technology;

[0050] If the end position of the impedance interaction system can be measured but the speed cannot be measured, the full state information Ξ of its augmented system can be reconstructed through historical input and output data, and the expression is:

[0051]

[0052] in, is the measurable historical data, the input data set from time t-Δt to time t-NΔt Output dataset and a constant position vector x e , expected trajectory x d Composition; N represents the number of sampling points; Γ is the state reconstruction matrix, expressed as:

[0053]

[0054] Among them, U N 、V N , Φ N They all represent parameter matrices, Represents Φ N The pseudo-reversal,

[0055]

[0056]

[0057] Applying the reconstructed state expression to the different-strategy model-free integral reinforcement learning algorithm based on full-state feedback designed in the third step, the different-strategy model-free integral reinforcement learning algorithm based on historical data feedback is obtained as follows:

[0058] a) Initialization: Give an initial stable control strategy u0;

[0059] b) Solve the kernel matrix With the optimal control gain

[0060]

[0061] Among them, the kernel matrix at the i-th moment is defined as Optimal control gain They are defined as At the i-th moment Defined as

[0062] c) Convergence conditions: Determine the conditions for stopping iteration If it is satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

[0063] The advantages of the present invention compared with the prior art are:

[0064] 1) The algorithm designed in this invention is aimed at interactive systems with measurable end positions and speeds. It autonomously learns the optimal impedance parameters through single data sampling and can achieve optimal interactive control when the model is completely unknown.

[0065] 2) Aiming at the interactive system with unmeasurable terminal velocity, the present invention improves the existing integral reinforcement learning method by using the state reconstruction technology that does not depend on the controller expression, thus overcoming the influence of exploration noise on the reconstruction accuracy; the improved algorithm is applied to the optimal impedance learning control with completely unknown model;

[0066] 3) The impedance learning algorithms involved in the present invention are all designed for continuous-time systems and take into account the influence of environmental position, which is closer to the real system; they can be used for intelligent and safe interactive control of the on-orbit operation process of space robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A flowchart of a space robot optimal impedance learning method based on sampling data of the present invention;

[0068] Figure 2 This is a schematic diagram of the space robot and the interaction model;

[0069] Figure 3 The end position response curves of approximate optimal gain and theoretical optimal gain under two working conditions;

[0070] Figure 4 These are the interaction force response curves of the approximate optimal gain and the theoretical optimal gain under two working conditions. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0072] like Figure 1 As shown, the specific implementation steps of the optimal impedance learning method for a space robot based on sampling data of the present invention are as follows:

[0073] The first step is to Figure 2 The space robot system shown in the figure establishes a multi-degree-of-freedom space robot dynamic model including the end interaction force based on the Kane equation, and uses the mass-spring-damper to approximate the interaction model;

[0074] The Kane equation is used to establish the dynamic model of a multi-degree-of-freedom space robot including the end interaction force as follows:

[0075]

[0076] Where q represents the generalized coordinates of the space robot system, They represent the first and second order derivatives of q respectively; D(q) represents the global mass matrix of the system, is the global nonlinear term of the system, τ represents the system control force / torque, J e represents the Jacobian matrix related to the motion of the space robot, F e represents the end interaction force, which is usually measured by a force sensor; the superscript "T" represents the transpose of the matrix;

[0077] During the implementation process, a spatial six-degree-of-freedom robotic arm system was used for simulation experiments. The system parameters are as follows:

[0078]

[0079] Among them, the length of the matrix at “#” should actually be the position vector of the connection point between the arm 1 and the matrix in the matrix, which is [-10.88; -2.45; 0.93] m.

[0080] End interaction force F e By approximating the interaction model as a mass-spring-damper system, we obtain:

[0081]

[0082] Where M e , D e and K e They are respectively represented as the mass coefficient, damping coefficient and stiffness coefficient of the system; x represents the component of the robot end position in the inertial system, represent the first and second derivatives of x respectively; x e It represents the component of the position of the interaction surface in the inertial system. In the actual simulation, it is assumed that the tool at the end of the robot arm moves along the axial direction of the interaction surface, so the tangential force of the interaction surface can be ignored and only the axial interaction force is considered. Take the stiffness coefficient K e =400N / m, damping coefficient D e =10Ns / m, mass coefficient M e =1kg, the axial component at the interaction surface is x e =2.33m;

[0083] The expression of the impedance controller is:

[0084]

[0085] Where M d , D d0 Indicates the preset expected inertia and damping coefficient, D d , K d They represent the desired damping and stiffness coefficients to be further designed, respectively, d Indicates the desired trajectory of the robot end; in the implementation process, take M d =1kg, D d0 =50Ns / m;

[0086] Adding the interaction model to the impedance controller, the impedance interaction system expression is as follows:

[0087]

[0088] Among them, x e represents the constant vector of the environmental position, M e1 Defined as M e1 =M d +M e , D e1 Defined as D e1 =D d0 +D e , represents the non-inertial interaction force;

[0089] Get state variables The control input is u = f, and the linear state equation of the impedance interaction system is obtained:

[0090]

[0091] in, represents the derivative of ξ,

[0092]

[0093] Where y represents the output vector of the system. When both the velocity and the end position can be measured, the output matrix C is expressed as When only the end positions are measurable, the output matrix is ​​represented by C=[0, I]; 0 and I represent a zero matrix and an identity matrix of appropriate dimensions, respectively.

[0094] The second step is to design the interactive performance function according to the control objective and transform the optimal impedance control problem into a special linear quadratic tracking (LQT) problem.

[0095] Assume that the desired trajectory x at the end d Produced by the following linear instruction generator:

[0096]

[0097] In the formula, Represents x d The first-order derivative of Ω represents the dynamic matrix of the instruction generator; in the simulation, it is assumed that the desired trajectory is a constant vector x d =2.40m, that is, Ω=0.

[0098] The performance function is set to the following form with discount factors:

[0099] V(y(t),x d (t))=∫ t ∞ e -γ(τ-t) [(yx d ) T Q0(yx d )+u T Ru]dτ,

[0100] Where t represents the current time; the symmetric weight matrix is ​​Q0=50000 and R=0.0002, and the discount factor is γ=500;

[0101] Then the state of the augmented system is constructed as Ξ=[ξ T , x d ] T , the augmented system is:

[0102]

[0103] The performance function is thus rewritten as:

[0104] V(Ξ(t))=∫ t ∞ e -γ(τ-t) (Ξ T QΞ+u T Ru)dτ=Ξ T (t)PΞ(t),

[0105] Among them, Q=[0, I, -I] T Q0[0, I, -I];

[0106] The optimal controller is denoted as u * =-K * Ξ form, where K * is the optimal control gain, which is calculated as:

[0107]

[0108] When the augmented system does not exist Gx e When Gx eDue to the existence of the term, the traditional solution method of the linear quadratic tracking problem is no longer applicable. It is necessary to regard this problem as a special type of linear quadratic tracking problem and solve P through the following improved discounted ARE equation:

[0109]

[0110] In the third step, assuming that the terminal position and speed can be measured, a different-strategy model-free integral reinforcement learning method based on full-state feedback is designed to solve the optimal impedance parameters;

[0111] If the end position and speed of the impedance interaction system can be measured, in order to learn the optimal impedance parameters, the different-strategy model-free integral reinforcement learning algorithm based on full state feedback is as follows:

[0112] a) Initialization: Give an initial stable control strategy u0;

[0113] b) Solve the kernel matrix P i and the optimal control gain K i+1 :

[0114]

[0115] Among them, Δt represents the system sampling period; at the i-th moment, Q i Defined as Q i =K iT R i +Q,K i represents the control gain at the i-th moment; P i represents the kernel matrix at the i-th moment;

[0116] c) Convergence condition: Determine the iteration stop condition || P i+1 -P i ||≤ε, if satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

[0117] The fourth step is to design a different-strategy model-free integral reinforcement learning algorithm based on historical data feedback using state reconstruction technology in the case of missing terminal velocity information.

[0118] If the end position of the impedance interaction system can be measured but the speed cannot be measured, the full state information Ξ of its augmented system can be reconstructed through historical input and output data, and the expression is:

[0119]

[0120] in, is the measurable historical data, the input data set from time t-Δt to time t-NΔt Output dataset and a constant position vector x e, expected trajectory x d Composition; N represents the number of sampling points; T is the state reconstruction matrix, expressed as:

[0121]

[0122] Among them, U N 、V N , Φ N They all represent parameter matrices, Represents Φ N The pseudo-reversal,

[0123]

[0124]

[0125] Applying the reconstructed state expression to the different-strategy model-free integral reinforcement learning algorithm based on full-state feedback designed in the third step, the different-strategy model-free integral reinforcement learning algorithm based on historical data feedback is obtained as follows:

[0126] a) Initialization: Give an initial stable control strategy u0;

[0127] b) Solving the kernel matrix With the optimal control gain

[0128]

[0129] Among them, the kernel matrix at the i-th moment is defined as Optimal control gain They are defined as At the i-th moment Defined as

[0130] c) Convergence conditions: Determine the conditions for stopping iteration If it is satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

[0131] The simulation step size / sampling period is set to Δt = 5ms, and the number of sampling data points used to recover the state information is N = 2. In order to meet the full rank condition, the initial control strategy used to collect data needs to set the exploration noise u noise The present invention takes Where randn represents a random number between -1 and 1, and the end position of the environment used for training is x e = 0.3m, the expected position of the end of the robot arm is x d =0.4m, and the initial position of the end is x0=0.31m.

[0132] When both the terminal velocity and position can be measured, the learning algorithm in the third step can be applied to impedance control to obtain the terminal position and interaction force response curves as follows: Figure 3 and Figure 4 As shown. Among them, A1 Apr It means that when the model is completely unknown, the algorithm designed in the third step is used to learn the approximate optimal solution and apply it to the response curve of the impedance control of the space robot; A1 Opt It represents the response curve corresponding to the theoretical optimal solution when the model information is completely known.

[0133] When the terminal velocity is unmeasurable but the position is measurable, the learning algorithm in the fourth step can be applied to impedance control to obtain the terminal position and interaction force response curves as shown in Figure 3 and Figure 4 As shown. A2 Apr It means that when the model is completely unknown, the algorithm designed in the fourth step is used to learn the approximate optimal solution and apply it to the response curve of the impedance control of the space robot; A2 Opt It represents the response curve corresponding to the theoretical optimal solution when the model information is completely known.

[0134] according to Figure 3 and Figure 4 It can be seen that when the terminal speed is measurable / unmeasurable, the impedance control response curves obtained by the approximate optimal solution of the two algorithms and the theoretical optimal solution are almost completely overlapped, and the error is within an acceptable range; the response curves obtained by the algorithm designed in the fourth step and the algorithm designed in the third step are only slightly different in the transition stage. When reaching stability, the force / position curves are almost completely overlapped, and the error is within an acceptable range. This verifies the effectiveness of the design method of the present invention.

[0135] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. It is easy for the technical personnel in the field to understand that the above description is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A spatial robot optimal impedance learning method based on sampling data, characterized in that: The following steps are involved: In the first step, a multi-DOF spatial robot dynamics model including the end-point interaction force is established based on the Kane equation, and the interaction model is approximately described using the mass-spring-damper. The first step is specifically implemented as follows: The dynamic model of a multi-degree-of-freedom space robot including the end interaction force based on the Kane equation is as follows: Where q represents the generalized coordinates of the space robot system, They represent the first and second order derivatives of q respectively; D(q) represents the global mass matrix of the system, is the global nonlinear term of the system, τ represents the system control force / torque, J e represents the Jacobian matrix related to the motion of the space robot, F e represents the end interaction force, which is usually measured by a force sensor; the superscript "T" represents the transpose of the matrix; End interaction force F e By approximating the interaction model as a mass-spring-damper system, we obtain: Where M e , D e and K e They are respectively represented as the mass coefficient, damping coefficient and stiffness coefficient of the system; x represents the component of the robot end position in the inertial system, represent the first and second derivatives of x respectively; x e The component representing the position of the interaction surface in the inertial system; The expression of the impedance controller is: Where M d , D d0 Indicates the preset expected inertia and damping coefficient, D d , K d They represent the desired damping and stiffness coefficients to be further designed, respectively, d represents the desired trajectory of the robot end; Adding the interaction model to the impedance controller, the impedance interaction system expression is as follows: Among them, x e represents the constant vector of the environmental position, M e1 Defined as M e1 =M d +M e , D e1 Defined as D e1 =D d0 +D e , represents the non-inertial interaction force; Get state variables The control input is u = f, and the linear state equation of the impedance interaction system is obtained: in, represents the derivative of ξ, Where y represents the output vector of the system. When both the velocity and the end position can be measured, the output matrix C is expressed as When only the end position is measurable, the output matrix is ​​represented by C = [0, Ι]; 0 and Ι represent the zero matrix and the identity matrix of appropriate dimensions, respectively; The second step is to design the interactive performance function according to the control objective and transform the optimal impedance control problem into a special linear quadratic tracking problem; In the third step, assuming that the terminal position and speed can be measured, a different-strategy model-free integral reinforcement learning method based on full-state feedback is designed to solve the optimal impedance parameters; The fourth step is to use state reconstruction technology to design a different-strategy model-free integral reinforcement learning algorithm based on historical data feedback to achieve optimal impedance control in the case of missing terminal velocity information.

2. The optimal impedance learning method for a space robot based on sampling data according to claim 1, characterized in that: The second step is specifically implemented as follows: Assume that the desired trajectory x at the end d Produced by the following linear instruction generator: In the formula, Represents x d The first-order derivative of , Ω represents the dynamics matrix of the instruction generator; The performance function is set to the following form with discount factors: Among them, Q0≥0 and R>0 are the corresponding symmetric weight matrices, γ is the discount factor, and t represents the current time; Then the state of the augmented system is constructed as Ξ=[ξ T ,x d ] T , the augmented system is: The performance function is thus rewritten as: where Q = [0, Ι, -I] T Q0 [0, Ι, -I]; The optimal controller is denoted as u * =-K * Ξ form, where K * is the optimal control gain, which is calculated as: The special linear quadratic tracking problem is solved by the following modified discounted ARE equation P:

3. The optimal impedance learning method for a space robot based on sampling data according to claim 2, characterized in that: The third step is specifically implemented as follows: For the impedance interaction system with measurable end position and velocity, in order to learn the optimal impedance parameters, the heterogeneous strategy model-free integral reinforcement learning algorithm based on full state feedback is as follows: a) Initialization: Give an initial stable control strategy u0; b) Solve the kernel matrix P i and the optimal control gain K i+1 : Among them, Δt represents the system sampling period; at the i-th moment, Q i Defined as Q i =K iT R i +Q,K i represents the control gain at the i-th moment; P i represents the kernel matrix at the i-th moment; c) Convergence condition: Determine the iteration stop condition || P i+1 -P i ||≤ε, if satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

4. The space robot optimal impedance learning method based on sampling data according to claim 3 is characterized in that: The fourth step is specifically implemented as follows: For the impedance interaction system with measurable end position and unmeasurable velocity, the full state information Ξ of the augmented system is reconstructed through historical input and output data, and the expression is: in, is the measurable historical data, the input data set from time t-Δt to time t-NΔt Output dataset and a constant position vector x e , expected trajectory x d Composition; N represents the number of sampling points; Γ is the state reconstruction matrix, expressed as: Among them, U N 、V N , Φ N They all represent parameter matrices, Represents Φ N The pseudo-reversal, Applying the reconstructed state expression to the different-strategy model-free integral reinforcement learning algorithm based on full-state feedback designed in the third step, the different-strategy model-free integral reinforcement learning algorithm based on historical data feedback is obtained as follows: a) Initialization: Give an initial stable control strategy u0; b) Solve the kernel matrix With the optimal control gain Among them, the kernel matrix at the i-th moment is defined as Optimal control gain They are defined as At the i-th moment Defined as c) Convergence conditions: Determine the conditions for stopping iteration If it is satisfied, stop the iteration, otherwise set i=i+1 and go to step b).

Images