Method for controlling mobile robot through automatic establishment of abstract action

Abstract

The invention discloses a method for controlling a mobile robot through automatic establishment of abstract action. The method comprises the steps of 1, initialization; 2, updating all state-action assessed values according to a Q-learning module updating formula, saving an experiment track, and removing the states in repeated state intervals in the experiment track; 3, calculating the diversity density values of residual states; 4, executing step 5 if the diversity density value of a certain state reaches a preset threshold value, and repeating step 2, 3 and 4 otherwise; 5, taking states with diversity density values larger than the threshold value as sub-goal points, establishing an abstract action module, setting the start state, strategy and stop condition of the abstract action, adding the abstract action module to a selectable action set of the robot, and controlling the mobile robot with the updated selectable action set. By the adoption of the method, the abstract action can be effectively established, the robot can conduct hierarchical reinforcement learning by means of the established abstract action, and then the learning speed of the robot in a complicated environment is increased.

Description

technical field [0001] The invention relates to a robot control method, which belongs to the field of machine learning, in particular to a method for controlling a mobile robot by using abstract actions, especially a method for the mobile robot to automatically create abstract actions to realize movement control. Background technique [0002] With the continuous expansion of the application field of robots, the tasks faced by robots are becoming more and more complex. Although many algorithms can pre-design and program the possible behaviors of robots, when faced with complex tasks, designers often cannot make effective predictions for the rapidly changing external environment, so they cannot make reasonable predictions about the behavior of robots. Therefore, autonomous robots that can perceive the environment and complete tasks by taking relevant actions have become the focus of research. [0003] Reinforcement Learning (Reinforcement Learning), represented by path planni...

AI Technical Summary

Problems solved by technology

[0004] However, when solving large-scale problems with big data, the performance of traditional reinforcement learning algorithms will drop sharply, because reinforcement learning needs to continuously interact with the environment through robots, so when the problem scale expands, the number of features in the environment will increase exponentially, Therefore, the complexity of the algorithm will rise sharply, and the robot needs a lot of time and experience to interact with the environment to obtain a good strategy

Images