An automatic discovery method for maxq task graph structure in complex systems

Abstract

The invention provides an automatic discovery method of a complex system oriented MAXQ task graph structure. The method comprises the following steps of: firstly searching state variables influencing actions by adopting a Q-learning exploration environment; then scheduling a clustering algorithm based on the action execution effect, wherein the policy uses a data object as one atomic class, and the atomic classes are aggregated; gradually aggregating the atomic classes to a crescent class until the end condition is met; the process of an aggregation algorithm is as follows: forming a single cluster by each member in the beginning, and combining the adjacent clusters into one cluster in the later iterative process until all the members form one cluster, wherein the time and space complexities are O (n2); combining two clusters through a clustering method, which cannot be separated to the prior state; and (3) obtaining a hierarchical task graph. The method establishes a clustering model by using all learned and perceived information, and automatically constructs the MAXQ task graph through clustering and finally realizes automatic hierarchy of the MAXQ.

Description

technical field [0001] The invention relates to an automatic layering method of layered reinforcement learning under large-scale task solving in a complex system by using a computer. Background technique [0002] At present, no computer has been found to solve the MAXQ automatic stratification problem under large-scale tasks by combining clustering methods. Although there are some methods that can solve the hierarchical problem of hierarchical reinforcement learning, such as: bottleneck and landmark state method, shared subspace method, multidimensional state method and Markov space method, etc., these methods have certain connections with the present invention, namely Both are problems in the automatic layering domain of hierarchical reinforcement learning. But the specific solution is a completely different problem. Most of the previous methods are based on Option, or Q-learning and other methods, and my invention is based on the hierarchical reinforcement learning of the...

AI Technical Summary

Problems solved by technology

The MAXQ method can clearly express the hierarchical structure of the task with the task graph, and the method has strong online learning ability, but the MAXQ automatic layering ability is weak, and there are often subtasks whose state space is still large and cannot be further divided

Images