Hierarchical reinforcement learning task graph evolution method based on cause-and-effect diagram

Abstract

The invention provides a hierarchical reinforcement learning task graph evolution method based on a cause-and-effect diagram. The method comprises the following steps of: (1) carrying out parameter setting; (2) exploring a cause-and-effect diagram in a target environment; (3) carrying out initialization on a species group N; (4) calculating a fitness value; (5) carrying out genetic operation, wherein the genetic operation comprises selection, intersection and variation; and maintaining the cause-and-effect relationship between nodes in the operation; (6) judging whether to stop; (7) saving K task graphs G1, G2,..., Gk with the highest fitness corresponding to the cause-and-effect diagram; and (8) outputting the task graph G1 with the highest fitness. Compared with the prior art, the automatization and high efficiency of the task graphs constructed by the method of the invention can be suitable for large-scale complex systems and can be applied to the dynamic change situations of the system environment. The method provided by the invention only depends on the changes of the cause-and-effect diagram of the target environment, predicts the task level changes of the target environment when the cause-and-effect diagram of the target environment is changed regularly, and rapidly and efficiently generates an MAXQ task graph.

Description

technical field [0001] The invention relates to a method for optimizing a task graph of hierarchical reinforcement learning by using a computer. technical background [0002] Hierarchical reinforcement learning is an important method to solve the curse of dimensionality problem in reinforcement learning. At present, three typical methods in hierarchical reinforcement learning are Option proposed by Sutton, HAM proposed by Parr, and MAXQ proposed by Dietterich. An important issue in hierarchical reinforcement learning is that the hierarchical task graph needs to be given in advance by the designer based on expert knowledge. Since the manual construction of the hierarchical structure of hierarchical reinforcement learning requires relevant expert knowledge and cannot meet the needs of dynamic unknown environments, how to automatically discover and construct the hierarchical structure of tasks has become a major problem in hierarchical reinforcement learning. At present, many...

AI Technical Summary

Problems solved by technology

[0003] At present, there is mainly the HI-MAT method for the MAXQ automatic layering method, but the task graph obtained by HI-MAT depends on an observed successful path, so as to obtain a task graph consistent with this trajectory in the task graph structure space, so easy stuck in local optimum

Images