Robot adaptive neural optimal tracking control method and device considering dynamic constraint

Abstract

The invention discloses a robot adaptive neural optimal tracking control method and device considering dynamic constraints, and the method comprises the steps: building a kinetic model of a robot in a joint space, and considering the input torque, inertia matrix, centripetal force and Coriolis force matrix and gravity torque of a robot system; the dynamics of the robot system is mapped to a Cartesian space, a position vector and a speed vector are defined, and a state space model of the robot system is established; and under the condition that dynamic constraints exist, a self-adaptive optimal nerve tracking controller is designed based on an optimal backstepping method and an actor-commentator Sheng network method so as to process the dynamic constraints and realize optimal tracking performance. Under the dynamic constraint environment, the robot can track the expected trajectory more accurately, and meanwhile movement safety is ensured.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a robot adaptive neural optimal tracking control method and device considering dynamic constraints. Background Art

[0002] With the development of robotics technology, robots are increasingly used in industries such as industry, medicine, and services, especially in complex and dynamic environments. The demand for high-precision tasks is becoming more and more prominent. However, existing robot control methods still face many challenges in achieving stable and safe trajectory tracking while facing dynamic constraints and optimal control performance. Robot systems usually exhibit high nonlinearity and multivariable coupling characteristics. Although traditional control methods such as proportional-integral-differential control (PID), fuzzy control, and sliding mode control can meet the stability of the system to a certain extent, it is difficult to achieve precise control of complex constraints in dynamically changing environments. In addition, these methods are difficult to respond when the dynamic characteristics of the robot change rapidly, and cannot meet the real-time optimal performance requirements of the system.

[0003] In recent years, robot adaptive control methods based on neural networks have received extensive attention in academia and industry due to their strong learning and nonlinear approximation capabilities. Neural networks can model the complex dynamics of a system with less prior knowledge and adapt to changes in the system state. In particular, the "Actor-Critic" neural network architecture has become an effective means to solve the optimal control problem of complex nonlinear systems because of its ability to approximate the unknown internal dynamics of the system. However, when faced with dynamic state constraints, existing neural network control schemes usually rely on preset excitation conditions, making it difficult to ensure the optimal trajectory tracking effect of the system under dynamic constraints. In addition, most neural network-based adaptive control methods may not be able to maintain the robustness of the control system when the constraints change greatly, and cannot meet the requirements of safety control.

[0004] In terms of constraint control, barrier Lyapunov functions are widely used to ensure that the system does not violate constraints in terms of output, state, etc. By adding barrier functions to the Lyapunov control design, the state can be prevented from exceeding the preset limits. However, traditional barrier Lyapunov functions are mainly used in static constraint scenarios and are difficult to apply directly to robotic systems with time-varying and dynamic asymmetric constraints. Although symmetric, asymmetric, steady-state, and time-varying barrier Lyapunov functions have been evolved based on the barrier Lyapunov function, the existing models still find it difficult to solve the stability and control accuracy problems of robotic systems under highly dynamic asymmetric constraints.

[0005] Existing optimal control schemes have made some progress in meeting specific performance indicators, but they usually require solving the complex Hamilton-Jacobi-Bellman equations, which is computationally expensive for complex nonlinear systems. Although the adaptive optimization method based on reinforcement learning provides a way to solve the Hamilton-Jacobi-Bellman equations without accurately solving the Hamilton-Jacobi-Bellman equations, it has high requirements for the system control incentive conditions and lacks effective design to deal with dynamic constraints, which makes it easy for constraint violations to occur during the control process. Therefore, the existing control schemes cannot meet the requirements of system safety and optimal performance under dynamic state constraints.

[0006] In summary, the existing technology has significant deficiencies in the dynamic constraint control and optimal performance balance of robots, and it is difficult to cope with the complex dynamic changes of highly nonlinear systems, and there is a lack of efficient dynamic constraint processing methods in practical applications. Therefore, there is an urgent need for a robot adaptive control method that can achieve optimal control while satisfying dynamic constraints, so as to meet the safety, stability and optimal trajectory tracking requirements of robots in complex environments. Summary of the invention

[0007] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a robot adaptive neural optimal tracking control method and device taking dynamic constraints into account.

[0008] Aiming at the existing problems of robot motion safety and its optimal tracking performance, this paper designs a robot adaptive neural optimal control method with dynamic constraints, which enables the system to track the desired trajectory, all closed-loop signals are semi-globally consistent and ultimately bounded, and never violate dynamic constraints.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] The robot adaptive neural optimal tracking control method considering dynamic constraints includes the following steps:

[0011] S1. Establish the dynamic model of the robot in the joint space, considering the input torque, inertia matrix, centripetal force and Coriolis force matrix and gravity torque of the robot system;

[0012] S2. Map the dynamics of the robot system into Cartesian space, define the position vector and velocity vector, and establish a state space model of the robot system;

[0013] S3. In the presence of dynamic constraints, an adaptive optimal neural tracking controller is designed based on the optimal backstepping method and the actor-critic network method to handle the dynamic constraints and achieve optimal tracking performance;

[0014] As a preferred technical solution of the present invention, S1 comprises the following steps:

[0015] In joint space The dynamic equation of the linkage robot is:

[0016] (1)

[0017] In the formula, Represent the joint position vector, velocity vector and acceleration vector respectively, represents the input torque of the robot system, is a positive definite symmetric inertia matrix, represents the centripetal force and Coriolis force matrices, represents the gravity matrix;

[0018] As a preferred technical solution of the present invention, S2 comprises the following steps:

[0019] Joint Angle Can be mapped to the end position of the robot's positive kinematics , taking the derivative of the forward kinematics with respect to time, we get , is the Jacobian matrix; from the inverse kinematics of the robot, we can get , , for of pseudo-rebellion; , is the dimension of the end effector coordinate vector; They are the robot end position, end velocity and acceleration vector respectively;

[0020] The robot dynamics model in Cartesian space is:

[0021] (2)

[0022] In the formula, , , , ;

[0023] definition , Represents the position vector and velocity vector of the robot in Cartesian space; the general nonlinear equation describing the robot dynamics model is:

[0024] (3)

[0025] In the formula, , , Represents the upper and lower bounds of dynamic constraints;

[0026] As a preferred technical solution of the present invention, S31 in S3 includes the following steps:

[0027] Decomposing complex systems and defining tracking errors: The core idea of ​​using the optimal backstepping method is to decompose complex nonlinear systems into subsystems that do not exceed the order of the system. For each subsystem , define the control tracking error

[0028] (4)

[0029] In the formula, , is the expected trajectory, , is the virtual controller to be designed, and The relationship with other variables is ;

[0030] As a preferred technical solution of the present invention, S32 in S3 includes the following steps:

[0031] Based on the error defined in step S31 and , the asymmetric time-varying integral barrier Lyapunov function is designed as:

[0032] (5)

[0033] In the formula, is positive definite and continuously differentiable within the constraints, represents the integration variable.

[0034] As a preferred technical solution of the present invention, S33 in S3 comprises the following steps:

[0035] Subsystem , define the optimal performance indicator function :

[0036] (6)

[0037] To reflect the performance of the system in achieving the desired trajectory;

[0038] right Subsystem Design Hamilton-Jacobi-Bellman Equations , to achieve optimal control:

[0039] (7)

[0040] In the formula, , , These are design parameters. , , is the process variable; this function can ensure that the designed controller meets the corresponding optimal performance requirements;

[0041] As a preferred technical solution of the present invention, S34 in S3 comprises the following steps:

[0042] Solution , and get the ideal virtual controller :

[0043] (8)

[0044] In the formula, can be approximated as

[0045] (9)

[0046] In the formula, is the ideal weight vector, is the kernel function vector, is the approximate error;

[0047] As a preferred technical solution of the present invention, S35 in S3 includes the following steps:

[0048] Based on the optimal virtual controller obtained in S34 in step S3 , introducing an actor-critic neural network to approximate the ideal adaptive weights ; Use the function fitting ability of neural networks to fit unknown uncertain nonlinear functions in robot systems Approximate the optimal solution to the equation :

[0049] Defining Hamiltonian approximation error :

[0050] (10)

[0051] Used to measure the accuracy of the approximate solution.

[0052] To ensure the convergence of neural network weights, a critic neural network is designed , Actor Neural Network The adaptive weight update law is expressed as:

[0053] (11)

[0054] (12)

[0055] Then we can get the approximate optimal virtual control law for:

[0056] (13)

[0057] As a preferred technical solution of the present invention, S36 in S3 comprises the following steps:

[0058] Subsystem based on step S1 , repeat S32 to S35 in step S3, In the subsystem, based on Defining optimal performance metrics , Hamilton-Jacobi-Bellman equation :

[0059] (14)

[0060] (15)

[0061] Solution , and obtain the ideal optimal controller :

[0062] (16)

[0063] In the formula, is the design parameter, can be approximated by a neural network as

[0064] (17)

[0065] In the formula, represents the ideal weight vector, is the kernel function vector, represents the approximation error;

[0066] Introducing the "actor-critic" neural network to approximate the solution , we get the approximate optimal solution of the equation ;

[0067] Defining Hamiltonian approximation error :

[0068] (18)

[0069] Get the approximately optimal controller :

[0070] (19)

[0071] Minimize the approximation error , we can deduce and The adaptive weight update law is expressed as:

[0072] (20)

[0073] (twenty one)

[0074] The second aspect of the present invention relates to a robot adaptive neural optimal control device under dynamic constraints, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the robot adaptive neural optimal control method under dynamic constraints of the present invention.

[0075] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the robot adaptive neural optimal control method under dynamic constraints of the present invention.

[0076] The innovative features of the present invention are:

[0077] The present invention can achieve more accurate trajectory tracking in a dynamic environment and ensure the safety of system operation by introducing an asymmetric time-varying integral barrier Lyapunov function.

[0078] The present invention uses the optimal backstepping method to design performance index functions of different subsystems to achieve hierarchical dynamic optimization. This strategy balances dynamic constraints and optimal performance, ensuring both control accuracy and safety.

[0079] The present invention utilizes the nonlinear approximation and self-learning capabilities of neural networks, approximates the dynamic function of the unknown system through an "actor-critic" structure, and updates the optimal control law in real time.

[0080] The device of the present invention is combined with a computer-readable storage medium to provide a complete hardware and software solution, so that the method can be deployed in an actual system. The present invention has the following beneficial effects:

[0081] 1. Aiming at the problem that it is difficult for robots to achieve optimal trajectory tracking and ensure motion safety under dynamic constraints, this paper adopts asymmetric time-varying integral barrier Lyapunov function to deal with dynamic constraints, and combines the optimal backstepping method and the "actor-critic" neural network structure to design an adaptive neural optimal controller. Under dynamic constraint environment, the robot can track the desired trajectory more accurately while ensuring motion safety, providing new technology and methods for the precise control and safe operation of robots in the field of safety constraints.

[0082] 2. Aiming at the highly nonlinear characteristics existing in robot control systems and the difficulty of accurate control by traditional control methods, the present invention utilizes the self-learning and nonlinear approximation capabilities of neural networks, approximates unknown uncertain functions in the robot system through the "actor-critic" neural network, and updates the optimal control law online. The present invention not only improves the control capability of complex nonlinear systems, but also ensures certain real-time performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a structural schematic diagram of a robot adaptive neural optimal control method with dynamic constraints of the present invention.

[0084] Figure 2 The present invention provides a position state trajectory diagram under dynamic constraints of a robot adaptive neural optimal control method with dynamic constraints.

[0085] Figure 3 The present invention is a velocity state trajectory diagram under dynamic constraints of a robot adaptive neural optimal control method with dynamic constraints.

[0086] Figure 4 The invention discloses an adaptively updated weight trajectory diagram of a robot adaptive neural optimal control method with dynamic constraints.

[0087] Figure 5 This is a control input trajectory diagram of a robot adaptive neural optimal control method with dynamic constraints of the present invention.

[0088] Figure 6 It is a Hamilton function approximation trajectory diagram of a robot adaptive neural optimal control method with dynamic constraints of the present invention.

[0089] Figure 7 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0090] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0091] Example 1

[0092] See also Figure 1 The invention provides a technical solution: a robot adaptive neural optimal tracking control method considering dynamic constraints, comprising the following steps:

[0093] S1. Establish the dynamic model of the robot in the joint space, considering the input torque, inertia matrix, centripetal force and Coriolis force matrix and gravity torque of the robot system;

[0094] S2. Map the dynamics of the robot system into Cartesian space, define the position vector and velocity vector, and establish a state space model of the robot system;

[0095] S3. In the presence of dynamic constraints, an adaptive optimal neural tracking controller is designed based on the optimal backstepping method and the actor-critic network method to handle the dynamic constraints and achieve optimal tracking performance;

[0096] S1 includes the following steps:

[0097] In the joint space, the robot dynamics model is:

[0098] (1)

[0099] In the formula, They are the joint positions, velocities, and acceleration vectors of the interactive robot. It is the control input of the robot joints.

[0100] The dynamic parameters of the two-link manipulator in the joint space are

[0101] (2)

[0102] (3)

[0103] (4)

[0104] In the formula, the dynamic parameters of the corresponding robot model are: the mass of connecting rod 1 is 2kg, the mass of connecting rod 2 is 0.85kg, the length of connecting rod 1 is 0.35m, the length of connecting rod 2 is 0.31m, the inertia of connecting rod 1 for , the inertia of connecting rod 2 for ;

[0105] S2 includes the following steps:

[0106] Joint Angle Can be mapped to the end position of the robot's positive kinematics , taking the derivative of the forward kinematics with respect to time, we get , is the Jacobian matrix; from the inverse kinematics of the robot, we can get , , for of pseudo-rebellion; , is the dimension of the end effector coordinate vector; They are the robot end position, end velocity and acceleration vector respectively;

[0107] The robot dynamics model in Cartesian space is:

[0108] (5)

[0109] In the formula, , , , ;

[0110] In Cartesian space, the standard nonlinear model of robot kinematics is established as:

[0111] (6)

[0112] In the formula, and Represent the robot end effector position and velocity vector respectively, Represents the dimension of the end effector coordinate vector. , , ;

[0113] S3 includes the following steps:

[0114] S31 includes the following steps:

[0115] Decomposing complex systems and defining tracking errors: The core idea of ​​using the optimal backstepping method is to decompose complex nonlinear systems into subsystems that do not exceed the order of the system. For each subsystem , define the control tracking error

[0116] (7)

[0117] In the formula, , is the expected trajectory, , is the virtual controller to be designed, and and The relationship is ;

[0118] S32 includes the following steps:

[0119] Based on the error defined in step S31 and , the asymmetric time-varying integral barrier Lyapunov function is designed as:

[0120] (8)

[0121] In the formula, is positive definite and continuously differentiable within the constraints, represents the integral variable. The upper and lower bounds of the dynamic constraint are and ,Right now and .

[0122] S33 includes the following steps:

[0123] Defining Subsystems The optimal performance indicator function is as follows:

[0124] (9)

[0125] In the formula, represents the cost function, It is the ideal optimal virtual controller. is a design parameter.

[0126] right Subsystem Design Hamilton-Jacobi-Bellman Equations , to achieve optimal control:

[0127] (10)

[0128] In the formula, , , These are design parameters. , , is the process variable; this function can ensure that the designed controller meets the corresponding optimal performance requirements;

[0129] S34 includes the following steps:

[0130] Solution , and get the ideal virtual controller :

[0131] (11)

[0132] In the formula, can be approximated as:

[0133] (12)

[0134] In the formula, is the ideal weight vector, is the kernel function vector, is the approximate error;

[0135] S35 includes the following steps:

[0136] Based on the optimal virtual controller obtained in step S34 in step S3 , introducing an actor-critic neural network to approximate the ideal adaptive weights ; Use the function fitting ability of neural networks to fit unknown uncertain nonlinear functions in robot systems Approximate the optimal solution to the equation :

[0137] Defining Hamiltonian approximation error :

[0138] (13)

[0139] To ensure the convergence of neural network weights, a critic neural network is designed , Actor Neural Network The adaptive weight update law is expressed as:

[0140] (14)

[0141] (15)

[0142] Then we can get the approximate optimal virtual control law for:

[0143] (16)

[0144] S36 includes the following steps:

[0145] Subsystem based on step S1 , repeat S32 to S35 in step S3, In the subsystem, based on Defining optimal performance metrics , Hamilton-Jacobi-Bellman equation :

[0146] (17)

[0147] (18)

[0148] Solution , and obtain the ideal optimal controller :

[0149] (19)

[0150] In the formula, is the design parameter, can be approximated by a neural network as:

[0151] (20)

[0152] In the formula, represents the ideal weight vector, is the kernel function vector, represents the approximation error;

[0153] Introducing the "actor-critic" neural network to approximate the solution , we get the approximate optimal solution of the equation ;

[0154] Defining Hamiltonian approximation error :

[0155] (twenty one)

[0156] Get the approximately optimal controller :

[0157] (twenty two)

[0158] Minimize the approximation error , we can deduce and The adaptive weight update law is expressed as:

[0159] (twenty three)

[0160] (twenty four)

[0161] In order to verify the effectiveness of the proposed method, this method is simulated for the robotic arm system. The system initialization parameters are set as follows:

[0162] The expected trajectory is , given the asymmetric time-varying state constraint condition is , The initial conditions are , , , The control parameters are given as , , , , , The initial values ​​of the actor-critic neural network are preset to , and the width of the Gaussian kernel function is , simulation time .

[0163] The simulation results are as follows Figure 2-6 Shown: Figure 2 and Figure 3 The robot state trajectory under the dynamic state constraints is plotted, and it can be seen that good optimal tracking performance of the desired state is achieved and the dynamic constraint boundary is never violated. Figure 4 Depicting the adaptive updating of weights in an actor-critic neural network architecture. Figure 5 Represents the state error of the robot system. Figure 6 The approximation of the Hamilton-Jacobi-Bellman equation is plotted. Figure 2-6 As shown, all signals of the closed-loop system are bounded and convergent.

[0164] In summary, the present invention can achieve the goal of adaptive neural optimal trajectory tracking control while meeting the robot motion safety.

[0165] Example 2

[0166] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the method for adaptive neural optimal control of a robot under dynamic constraints of the present invention is implemented. The computer-readable storage medium can provide control codes and store data generated during the experiment. Based on the experimental data stored in the computer-readable storage medium, a Figure 2-6 . Figure 2 and Figure 3 The robot state trajectory under dynamic state constraints is plotted. Figure 4 Depicting the adaptive updating of weights in an actor-critic neural network architecture. Figure 5 Represents the state error of the robot system. Figure 6 An approximation to the Hamilton-Jacobi-Bellman equation is plotted.

[0167] Example 3

[0168] like Figure 7 This embodiment relates to a robot adaptive neural optimal control device under dynamic constraints, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the robot adaptive neural optimal control method under dynamic constraints of Example 1.

[0169] The above describes the excellent optimization effect shown by an embodiment of the present invention. Obviously, the present invention is not limited to the above embodiment, and various modifications can be made to it without departing from the basic spirit of the present invention and without exceeding the scope of the essential content of the present invention.

Claims

1. A robot adaptive neural optimal tracking control method considering dynamic constraints, characterized in that: The following steps are involved: S1. Establish the dynamic model of the robot in the joint space, considering the input torque, inertia matrix, centripetal force and Coriolis force matrix and gravity torque of the robot system; S2. Map the dynamics of the robot system into Cartesian space, define the position vector and velocity vector, and establish a state space model of the robot system; S3. In the presence of dynamic constraints, an adaptive optimal neural tracking controller is designed based on the optimal backstepping method and the "actor-critic" network method to handle the dynamic constraints and achieve optimal tracking performance.

2. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: In step S1, In the joint space, the robot dynamics model is: （1） In the formula, They are the joint positions, velocities, and acceleration vectors of the interactive robot; is the symmetric inertia matrix, are the vectors of the Coriolis force and the centripetal force, is the gravity vector; It is the control input of the robot joints.

3. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S2 specifically includes: Joint Angle Can be mapped to the end position of the robot's positive kinematics , taking the derivative of the forward kinematics with respect to time, we get , is the Jacobian matrix; from the inverse kinematics of the robot, we can get , , for of pseudo-rebellion; , is the dimension of the end effector coordinate vector; They are the robot end position, end velocity and acceleration vector respectively; The robot dynamics model in Cartesian space is: （2） in , , , ; definition , Represents the position vector and velocity vector of the robot in Cartesian space; the general nonlinear equation describing the robot dynamics model is: （3） In the formula, .

4. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S3 specifically includes: S31 system order reduction, decomposing the original complex nonlinear system into optimal subsystems that do not exceed the system order; S32, based on the subsystem established in step S31, designing a corresponding asymmetric time-varying integral barrier Lyapunov function for the subsystem to process the corresponding asymmetric time-varying dynamic constraint condition; S33, based on the subsystem established in step S31, defining a corresponding optimal performance index function for the subsystem, and obtaining the Hamilton-Jacobi-Bellman equation of each optimal subsystem to meet the optimality requirement; S34 derives a theoretical optimal virtual controller corresponding to the subsystem based on the function established in step S32 and step S33; S35 designs an "actor-critic" neural network architecture based on the optimal virtual controller of step S34 to approximate the unknown uncertain function of the optimal virtual controller and update the optimal control law online to obtain an approximate optimal virtual controller; S36 returns from the lowest-order subsystem to the original system, repeats steps S32 to S35 for all subsystems, integrates all approximate optimal virtual controllers, and obtains the final adaptive optimal neural tracking controller.

5. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S31 specifically includes: Defining Tracking Error and , define the virtual control law ; and Defined as: （4） In the formula, is the expected trajectory; and The differential relationship between subsystems, that is, The derivative with respect to time is: （5）。 6. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S32 specifically includes: Based on the subsystem defined in step S31 , the asymmetric time-varying integral barrier Lyapunov function is designed as: （6） In the formula, is positive definite and continuously differentiable within the constraints, represents the integral variable, Represents the upper and lower bounds of dynamic constraints; this function can effectively handle dynamic constraints and ensure the safety of the robot during movement.

7. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S33 specifically includes: Based on the subsystem defined in step S31 , design the optimal performance index function as for: （7） In the formula, , , These are design parameters. , , is the process variable; this function can ensure that the designed controller meets the corresponding optimal performance requirements.

8. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1 is characterized in that: Step S34 specifically includes: Based on the optimal index function defined in step S33 , by solving , we can find the optimal virtual controller for: （8） In the formula, can be approximated as , is the ideal weight vector, is the kernel function vector, is an approximate error.

9. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1, characterized in that: Step S35 specifically includes: Based on the optimal virtual controller obtained in step S34 , designing an actor-critic neural network architecture to approximate the ideal adaptive weights ; By minimizing the approximation error , and the adaptive weight update law of the critic neural network and the actor neural network is derived , , and then get the approximate optimal virtual control law for: （9） Implementing adaptive weights and to ensure the convergence of right An effective approximation of .

10. The robot adaptive neural optimal tracking control method considering dynamic constraints according to claim 1, characterized in that: Step S36 specifically includes: Based on the subsystem defined in step S31 , repeat steps S32 to S35 to derive the theoretical optimal controller for: （10） In the formula, is the design parameter; can be approximated by NNs as , represents the ideal weight vector, is the kernel function vector, represents the approximation error; Design a corresponding actor-critic neural network architecture to approximate the ideal adaptive weights , we get the adaptive weight update law of the critic neural network and the actor neural network as , ; Substituting the above results, we can finally obtain the approximate adaptive optimal neural control law for: （11）。

Images