Online learning control method of nonlinear discrete time system

Abstract

The invention discloses an online learning control method of a nonlinear discrete time system. The method comprises a behavior strategy selection step, an optimal Q-function definition step, an evaluation network and execution network introduction step, an estimation error calculation step and a final optimal weight calculation step, wherein after the weights of an evaluation network and an execution network are converged, the output of the execution network is the approximate value of an optimal controller. According to the invention, repeated iteration between strategy evaluation and strategy improvement is not needed, and real-time online learning of the optimal controller can be realized; a deorbital strategy learning mechanism is adopted, so that the problem that a direct heuristic dynamic planning method is insufficient in state-strategy space exploration is effectively solved, and an execution network and an evaluation network can use activation functions in any form; and online learning of an optimal controller can be achieved, a system model is not needed, and only state data generated by the behavior strategy is needed.

Description

technical field [0001] The invention relates to the field of industrial production control, in particular to an online learning control method for nonlinear discrete time systems. Background technique [0002] In the process of industrial production, engineers and technicians often need to optimize the design of the controllers of the control objects such as robots, drones, and unmanned vehicles to meet certain control indicators. Because the above-mentioned control objects often exhibit strong nonlinearity, the optimization of the controller faces great difficulties. From the perspective of optimal control, obtaining the optimal control controller needs to solve the complex Hamilton-Jacobi-Bellman equation (HJB equation), but the HJB equation is a nonlinear partial differential equation, which is very difficult to solve. Traditional dynamic programming, variational methods, spectral methods, etc. often face great limitations in practical applications due to their extremely...

AI Technical Summary

Problems solved by technology

Although the direct heuristic dynamic programming algorithm can achieve online adaptive optimal control, the algorithm still has the following shortcomings: 1) The algorithm uses an on-policy learning mechanism, which has the problem of insufficient exploration of the state-policy space , easy to fall into the local optimal solution; 2) The hyperbolic tangent function is used in both the activation function of the execution network and the evaluation network, and all the current convergence and stability analysis results are based on the hyperbolic tangent function. For other types of The activation function is no longer applicable

Images