Crowd evacuation simulation method in dynamic environment based on deep reinforcement learning

Abstract

The invention provides a crowd evacuation simulation method in a dynamic environment based on deep reinforcement learning, and the method comprises the steps: using a crowd as a multi-agent system, employing a deep neural network to approach a mapping function from a state to an action for a single pedestrian agent to serve as a behavior controller of the agent; and by the pedestrian agent, makinga behavior decision by observing the dynamic environment state and utilizing the mapping function, and taking corresponding actions from the action space. In the evacuation simulation process of a discrete time sequence, one strategy refers to probability distribution of agent actions on the time sequence in a given state. The information feedback of the environment to the intelligent agent is reflected as a return value. The target of the intelligent agent is to maximize the expectation of the cumulative return value, that is, an optimal action value function is searched, and an optimal strategy is obtained. According to the method, crowd evacuation simulation which is difficult to realize in classical crowd simulation in a high-dynamic environment can be realized, the simulation effectis closer to the real situation, and the randomness of crowd behaviors is enhanced.

Description

technical field [0001] The invention belongs to the technical field of crowd simulation and computer simulation, and in particular relates to a crowd evacuation simulation method in a dynamic environment based on deep reinforcement learning. Background technique [0002] From the perspective of crowd evacuation simulation, the commonly used crowd simulation models can be divided into macroscopic models and microscopic models. The macro model mainly models the group behavior, regards the group as a continuum following the laws of fluid mechanics, and is suitable for large-scale crowd simulation in a large area. The micro model uses a single individual as the basic unit of modeling, and through the movement of a large number of individuals and the interaction between individuals, group behavior emerges. In contrast, microscopic models are more suitable for modeling and simulating individual behaviors, and have received more research. [0003] Each model has its own applicabl...

AI Technical Summary

Problems solved by technology

On the one hand, the state of the environment and the movement of the agent are continuous processes, that is, continuous state space and action space. However, algorithms such as Q-Learning and SARSA only support discrete state space and action space, and the huge continuous space may lead to Curse of Dimensionality Problem

On the other hand, this type of algorithm usually uses Q table as a state-action mapping storage body. In the case of continuous state space and action space, huge storage space and indexing time are required, resulting in an exponential increase in computational complexity.

[0007] Due to the low accuracy and weak randomness of the crowd simulated by the social force model in the existing technology; the traditional reinforcement learning method for crowd simulation only supports discrete state space and action space, and the huge continuous space may lead to the curse of dimensionality problem, and in the case of continuous state space and action space, huge storage space and indexing time are required, resulting in technical problems such as an exponential increase in computational complexity. Therefore, the present invention researches and designs a dynamic environment based on deep reinforcement learning Crowd Evacuation Simulation Method

Images