A method for controlling mobile robots by automatically creating abstract actions

Abstract

The invention discloses a method for controlling a mobile robot by automatically creating abstract actions, including: 1) initialization; 2) updating all state-action evaluation values ​​according to the update formula of the Q learning module, and saving the experience track, removing the 3) Calculate the diversity density value of the remaining states; 4) If there is a state whose diversity density value reaches the preset threshold, go to step 5); otherwise, repeat steps 2) to 4) ;5) Select the state whose diversity density value is greater than the threshold as the sub-goal point, create an abstract action module, set the starting state, strategy and termination conditions of the abstract action, add the abstract action module to the optional action set of the robot, and use the updated An optional set of actions for controlling the mobile robot. The invention can effectively create abstract actions, and the robot can use the created abstract actions to perform layered reinforcement learning, thereby accelerating the learning of the robot in complex environments.

Description

technical field [0001] The invention relates to a robot control method, which belongs to the field of machine learning, in particular to a method for controlling a mobile robot by using abstract actions, especially a method for the mobile robot to automatically create abstract actions to realize movement control. Background technique [0002] With the continuous expansion of the application field of robots, the tasks faced by robots are becoming more and more complex. Although many algorithms can pre-design and program the possible behaviors of robots, when faced with complex tasks, designers often cannot make effective predictions for the rapidly changing external environment, so they cannot make reasonable predictions about the behavior of robots. Therefore, autonomous robots that can perceive the environment and complete tasks by taking relevant actions have become the focus of research. [0003] Reinforcement Learning (Reinforcement Learning), represented by path planni...

AI Technical Summary

Problems solved by technology

[0004] However, when solving large-scale problems with big data, the performance of traditional reinforcement learning algorithms will drop sharply, because reinforcement learning needs to continuously interact with the environment through robots, so when the problem scale expands, the number of features in the environment will increase exponentially, Therefore, the complexity of the algorithm will rise sharply, and the robot needs a lot of time and experience to interact with the environment to obtain a good strategy

Images