Device and method for training control strategy by means of reinforcement learning

Abstract

A method for training a control strategy by means of reinforcement learning is described, having performing a plurality of reinforcement learning training traversal, in each traversal, selecting an action to be performed starting from an initial state of the control traversal for each state of a sequence of states of an agent, selecting an action by specifying a planned range for at least some states, and training the selected action by specifying the planned range for at least some states. The planned range specifies the number of states; determining a plurality of sequences of reachable states starting from respective states having a specified number of states by applying an answer set programming solver to an answer set programming program modeling a relationship between actions and subsequent states reached by the actions; selecting a sequence that provides the maximum reward among the sequences, wherein the reward provided by the determined sequence is the sum of the rewards obtained when the state of the sequence is reached; and selecting, as an action for the respective state, an action usable to start from the respective state to a first state of the selected sequence.

Description

technical field [0001] In general, various embodiments relate to an apparatus and method for training a control policy with reinforcement learning. Background technique [0002] Control devices for machines like robots can be trained by so-called reinforcement learning, English Reinforcement Learning (RL), to perform specific tasks, eg in a production process. The implementation of a task usually involves choosing an action for each state of a sequence of states, that is to say it can be viewed as a sequential decision problem. According to the state reached by the selected action, especially the final state, these actions are rewarded (English return), for example, according to whether the action is allowed to reach it (for example, for achieving the goal of the task). to the final state of the . [0003] Reinforcement learning enables an agent (such as a robot) to learn from experience by adapting its behavior to maximize the rewards it earns over time. Simple trial-and...

Images