A multi-agent reinforcement learning-based personnel evacuation simulation method and system

Abstract

This invention discloses a method and system for evacuation simulation based on multi-agent reinforcement learning, relating to the fields of computer simulation and public safety and emergency management. The method parametrically models the evacuation environment, constructs a global dynamic scene image and local observation states, establishes a discrete action space for each evacuation agent, and performs safety displacement updates in conjunction with a collision avoidance algorithm. Furthermore, it constructs an immediate reward based on local density, motion smoothness, evacuation time, and target distance, implements multi-agent collaborative training using a convolutional neural network, and employs a fuzzy constraint-based action sampling mechanism to generate evacuation decisions that balance behavioral rationality and randomness. Finally, it outputs individual trajectories, evacuation time, evacuation flow rate, and exit neighborhood density distribution results. This method can be used for evacuation simulation, safety assessment, model validation, and emergency decision support in complex scenarios, exhibiting strong engineering applicability and simulation analysis value.

Description

Technical Field

[0001] This invention belongs to the field of computer simulation and public safety and emergency management technology, and relates to the characterization of crowd evacuation behavior in complex building environments, multi-agent collaborative decision-making training, evacuation path evolution simulation and result evaluation, and in particular to a personnel evacuation simulation method and system based on multi-agent reinforcement learning. Background Technology

[0002] In emergency events such as building fires and public safety accidents, the evacuation process is often accompanied by a high degree of uncertainty and complexity. Individuals are not only affected by objective environmental factors such as spatial layout, exit location, and evacuation instructions during evacuation, but also by their individual risk preferences, information comprehension abilities, and the continuous influence of the behavior of the surrounding crowd, thus exhibiting significant individual differences and behavioral randomness.

[0003] To describe and analyze the behavioral characteristics of evacuated crowds, numerous studies have proposed various evacuation crowd simulation methods based on experimental observations and empirical summaries. Among these, the most typical include rule-based methods, mechanics-based methods, and cellular automata models (CAMs). Rule-based methods typically describe individual path selection, avoidance, and following behaviors through pre-defined behavioral rules; mechanics-based methods, represented by the Social Force Model (SFM), simulate collective movement patterns formed during evacuation by constructing forces between individuals and between individuals and the environment; cellular automata models describe crowd movement and congestion phenomena through discrete space and local transfer rules. These models have been widely applied in engineering practice and experimental reproduction, and can reproduce typical evacuation phenomena such as detours, following, and local congestion to a certain extent.

[0004] However, the aforementioned traditional evacuation models generally rely on pre-designed rule structures or fixed parameters, and the behavioral patterns they can represent are largely limited by the assumptions introduced during the model design phase. Although these rules and parameters are usually derived from experimental observations or empirical inductions, they are essentially still abstract expressions of known evacuation phenomena. In actual evacuation processes, crowd behavior does not stably converge to a certain pattern, but may evolve, shift, or even mutate among multiple potential behavioral patterns as environmental conditions, information acquisition methods, or individual cognitive differences change. Existing models still have significant shortcomings in systematically exploring the diversity, randomness, and dynamic evolution of such collective behaviors in evacuation, and their applicability and generalization ability are limited in complex or insufficiently observed evacuation scenarios. For example, Chinese patent CN119579376B discloses a method for simulating the evacuation of people in a building, which determines the target exit by calculating the predicted queuing and evacuation time of each exit in real time and determines the simulated speed by combining the forward distance of pedestrian characteristics. However, this method is essentially a deterministic optimization model based on the shortest time path, ignoring the limited cognition of pedestrians in emergency situations and the random disturbances in the decision-making process. CN118551561A discloses a low-posture evacuation simulation method for fires. It introduces physical and psychological forces from the fire source into a social force model and combines low-posture force analysis to solve the trajectory of personnel. However, this method is still a local modification of the mechanical interaction mechanism, focusing on the rigid mapping between physiological posture and environmental constraints, and lacks the characterization of individual strategy learning and behavioral evolution in the process of interaction with the environment.

[0005] In recent years, with the development of computing power and intelligent algorithms, reinforcement learning technology has been gradually introduced into crowd evacuation simulation research. Unlike traditional models based on fixed rules or parameters, reinforcement learning constructs an interaction process between the environment and individuals, enabling evacuated individuals to gradually form behavioral strategies based on feedback information during continuous interaction with the environment. This type of method does not rely on pre-set behavioral patterns, but guides the evolution of individual behavior through reward or constraint mechanisms. It can generate multiple potential individual behavioral strategies and collective evacuation patterns in the same evacuation environment, thereby alleviating the dependence of traditional models on existing empirical rules to a certain extent. However, existing learning-based evacuation models still have significant limitations in behavioral selection mechanisms: First, the description of individual cognitive constraints, risk judgment, and decision uncertainty in the behavior generation process is still insufficient, leading to deviations between the obtained strategies and the real crowd decision-making process at the behavioral mechanism level; Second, the action selection stage often adopts a relatively direct random exploration mechanism, which is prone to problems such as insufficient expression of randomness or unreasonable random distribution.

[0006] In summary, existing personnel evacuation simulation technologies still have significant shortcomings in uniformly characterizing the rationality of individual decision-making, the randomness of behavior, and the evolutionary process of group patterns in complex emergency scenarios. Therefore, how to reasonably represent the cognitive constraints, behavioral diversity, and decision-making uncertainty of evacuation individuals within a learning-based evacuation modeling framework is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] (a) Purpose of the invention The purpose of this invention is to address the problem that existing evacuation crowd simulation methods struggle to simultaneously characterize the rationality of individual decision-making and the randomness of behavior. Addressing the shortcomings of traditional evacuation crowd simulation models, such as rule and parameter constraints on simulated behavior patterns, tendency to converge to a single evacuation pattern, and insufficient randomness in the behavior selection phase of existing learning-based methods, this invention proposes a personnel evacuation simulation method and system based on multi-agent reinforcement learning. Within the framework of evacuation behavior modeling and reinforcement learning, a fuzzy stochastic decision-making mechanism consistent with human cognitive characteristics is introduced. Fuzzy mathematics is used to improve the reward function and action sampling method of the evacuation agents, enabling individual behavior to exhibit appropriate fuzziness and randomness while satisfying the rationality of evacuation behavior. This method can generate more diverse individual behaviors and collective evacuation patterns under the same evacuation environment and conditions, simulating the diversity and uncertainty of crowd behavior. It more realistically reflects the diversity, uncertainty, and evolutionary characteristics of crowd behavior in actual emergency situations, providing technical support for simulation analysis and safety assessment in complex evacuation scenarios.

[0008] (II) Technical Solution To achieve the objective of this invention and solve its technical problems, the present invention adopts the following technical solution: The first objective of this invention is to provide a method for simulating evacuation based on multi-agent reinforcement learning, used to model, train, and simulate the behavior of multiple evacuation individuals in a preset evacuation scenario, and output evacuation routes, evacuation times, and congestion distribution results, comprising at least the following steps: SS1. Evacuation Environment and Scene Modeling: Discrete parameterized modeling of the simulated evacuation environment is performed, defining the evacuation space boundary, obstacles, passable areas and evacuation exits, generating the initial positions and target exits of multiple evacuation agents, and establishing the mapping relationship between physical space coordinates and image pixel coordinates, providing a foundation for subsequent visualization and deep reinforcement learning models based on convolutional neural networks. SS2. Agent Perception and State Representation: The static obstacle layout of the evacuation environment and the dynamic position of each evacuation agent are mapped into a global dynamic scene image. Each evacuation agent is distinguished and a local observation area is cropped with its own position as the center. After resampling and normalization, the state input is formed. SS3. Action Space Construction: Construct a discrete action set for each evacuation agent, which includes a stationary action and multiple motion directions. Map the selected discrete action to the preferred velocity vector. Combine the maximum velocity constraint and collision avoidance algorithm to determine the safe velocity in the feasible velocity domain and perform displacement update. SS4. Evacuation Reward and Constraint Feedback: After fuzzing, rule activation, aggregation and defuzzing the three types of state variables, local density, motion smoothness and evacuation time, a continuous reward is generated. The distance improvement reward based on the optimal goal is calculated simultaneously. The two types of rewards are integrated to form an instant reward. SS5. Multi-agent collaborative training and learning: Construct a deep reinforcement learning framework based on convolutional neural networks. Each sparse agent learns collaboratively by sharing network parameters and centralized experience replay, and updates the policy network and target network. SS6. Fuzzy Constraint-Based Action Sampling: Based on the discrete action values ​​output by the policy network, a fuzzy membership degree of evacuated acceptable actions is constructed, and it is coupled with and normalized with random diffusion weights based on action values ​​to obtain the selection probability of each discrete action and complete the action sampling accordingly. SS7. Evacuation Process Evolution and Result Output: Drive each evacuation agent to iteratively execute the sampled actions until the preset termination condition is reached, and output the individual trajectory, individual evacuation time, total evacuation time, evacuation flow rate and exit neighborhood density distribution.

[0009] The second objective of this invention is to provide a personnel evacuation simulation system, comprising multiple modules, for executing the personnel evacuation simulation method based on multi-agent reinforcement learning described above.

[0010] (III) Technical Effects Compared with the prior art, the personnel evacuation simulation method and system based on multi-agent reinforcement learning of the present invention has the following beneficial and significant technical effects: (1) This invention can improve the realism of evacuation behavior simulation by introducing controllable randomness while ensuring the rationality of evacuation behavior. This invention introduces fuzzy mathematics methods in the reward generation stage and proposes action sampling rules based on fuzzy constraints in the action selection stage, thus modeling the randomness of evacuation behavior as a continuous fuzzy set. This mechanism allows the agent to randomly diffuse only within the set of actions that satisfy the basic rationality constraints of evacuation, thereby avoiding unreasonable behavior caused by completely random exploration. Therefore, the result of this invention is that, under the same evacuation scenario and initial conditions, it can generate more diverse individual trajectories and collective evacuation patterns that all conform to the evacuation goals, making the simulation results more statistically close to the behavioral distribution characteristics of real population evacuation. This feature enables this invention to maintain good generalization ability when facing complex building environments or evacuation scenarios that are not fully observed, and it is suitable for application scenarios such as evacuation risk assessment, extreme situation analysis, and emergency management decision support.

[0011] (2) This invention can effectively alleviate the action oscillation problem caused by sparse rewards in reinforcement learning evacuation modeling. In multi-exit evacuation environments, traditional distance-based reward functions tend to cause agents to frequently switch between multiple targets, resulting in path oscillations or policy instability, which affects training convergence and result interpretability. This invention introduces nonlinear distance-based reward improvement based on the optimal target in the reward design and weights the target distance by importance, making the agent more inclined to continuously advance towards a certain target exit over a period of time. This design maintains policy flexibility while suppressing frequent decision reversals caused by short-term value differences. In multi-exit evacuation scenarios, this reward design can significantly reduce the frequency of agents repeatedly switching between adjacent exits, reduce the fluctuation amplitude of Q-values ​​during the learning process, and make the training process more convergent, thereby improving simulation efficiency and stability. Attached Figure Description

[0012] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood in conjunction with the following description of the embodiments, in which: Figure 1 This is a flowchart of the personnel evacuation simulation method based on multi-agent reinforcement learning of the present invention; Figure 2 This is a schematic diagram of the multi-agent reinforcement learning network structure of the present invention; Figure 3 This is a simulated room background image of Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of evacuation behavior simulated by the present invention, wherein: (a) represents pedestrian queuing behavior, which is more rational; and (b) represents bottleneck behavior, which is more random. Detailed Implementation

[0013] This invention aims to provide a method and system for simulating personnel evacuation based on multi-agent reinforcement learning. It is used to model, train, and simulate the behavior of multiple evacuation individuals in a preset evacuation scenario, and output evacuation routes, evacuation times, and congestion distribution results. To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments will be described in more detail below with reference to the accompanying drawings. The described embodiments are some, but not all, embodiments of this invention, and are exemplary, intended to explain the invention, and should not be construed as limiting the invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0014] Example 1: Personnel Evacuation Simulation Method like Figure 1 As shown in the embodiments of the present invention, the personnel evacuation simulation method based on multi-agent reinforcement learning models the evacuation process as a continuous decision-making process of multiple agents in a confined building environment. By constructing a closed-loop mechanism of perception-decision-execution-feedback, evacuation individuals can randomly choose among a variety of acceptable behaviors under the premise of meeting basic evacuation rationality constraints, thereby generating a collective evacuation behavior pattern with diversity and uncertainty. The method mainly includes the following steps in implementation: evacuation environment and scenario modeling, agent perception and state representation, behavior action space construction, evacuation reward and constraint feedback, multi-agent collaborative training and learning, action sampling based on fuzzy constraints, evacuation process evolution and result output. Each step works together to achieve a systematic characterization of the diversity and uncertainty of crowd behavior in complex evacuation situations.

[0015] SS1. Evacuation Environment and Scene Modeling: Discrete parameterized modeling is performed on the simulated evacuation environment, defining evacuation space boundaries, obstacles, passable areas, and evacuation exits. Initial positions and target exits for multiple evacuation agents are generated, and a mapping relationship between physical space coordinates and image pixel coordinates is established, providing a foundation for subsequent visualization and deep reinforcement learning models based on convolutional neural networks. Step SS1 is used to construct a simulation environment for agents to evacuate. Its workflow involves the following steps to construct the physical scene: S110. Global Coordinate System Construction: Based on the topological information of the building or evacuation space to be simulated, a unified global coordinate system is established in two-dimensional or three-dimensional space as the reference coordinate system for geometric modeling of the evacuation space. The evacuation environment is then discretized and modeled within this coordinate system. The evacuation environment includes, but is not limited to, geometric elements such as room boundaries, passable areas, exit locations and dimensions, and internal fixed obstacles.

[0016] S111. Define the main region and walls: Define a region centered at the origin and with a width of [missing information] in the global coordinate system. W Height is H The internal accessible main area is constructed based on the topological information of the building or evacuation space to be simulated, and a preset width is built. W ,high H and wall thickness D Boundary walls are used to limit the activity range and environmental boundary conditions of evacuation agents. Boundary walls can be described as enclosing rectangles, polygonal lines, or closed boundaries formed by connecting multiple line segments end-to-end. For architectural spaces with locally impassable recessed areas, additional restrictive boundaries can be defined within the main area. Obstacles can be represented as fixed walls, facilities, fences, or other non-passable objects that cause crowds to detour or experience localized congestion. Exits can be further assigned width, location, direction, and target importance parameters to facilitate the subsequent construction of target distance improvement rewards.

[0017] S112. Define exits and obstacles: Based on the topological information of the building or evacuation space to be simulated, open one or more independent evacuation exits with preset location and width parameters, and configure obstacles in the main passable area to form an evacuation scenario to be simulated that includes passage constraints.

[0018] After completing the environmental geometry modeling, an initial evacuation scenario is generated within the coordinate system according to the evacuation scenario requirements. Specifically, based on the preset crowd size and distribution rules, the initial position coordinates of individuals within the passable area are generated randomly or distributed according to given locations, and each evacuation individual is assigned an ID. To avoid situations where agents overlap with walls or obstacles or where agents overlap excessively during the initialization phase, minimum spacing constraints and boundary feasibility checks can be further applied during initial position generation.

[0019] Specifically, the module executes the following definition process to initialize and deploy the agent: S120. Determine the initial generation area: Within the main passable area, based on the purpose of the building to be simulated and the distribution characteristics of the crowd, delineate the initialization area for placing the intelligent agent.

[0020] S121. Crowd Size and Identity Configuration: Generate multiple evacuation agents based on the preset crowd size, and assign a unique identity ID to each evacuation agent for indexing agent status, actions, rewards, trajectories and statistical results during training.

[0021] S122. Generate Initial Positions: According to preset distribution rules, a two-dimensional coordinate point (x, y) is randomly generated within the initialization area for each agent to be simulated, serving as its initial position. This ensures that all evacuation agents enter a unified scene and form an initial crowd distribution state at the start of the simulation. During deployment, initial positions can be generated through random sampling or sampling based on a given density field. If a candidate position does not meet boundary constraints, obstacle occupancy constraints, or minimum neighbor constraints, resampling can be performed until a valid position is obtained.

[0022] S123. Associating Target Exits: Assign one or more initial target exit coordinates to each agent, enabling it to have target pointing information corresponding to the evacuation exit at the initial moment, and allowing the agent to adjust the target exit selection according to environmental changes and evacuation rules during subsequent scenario evolution. The agent will also change its target selection accordingly based on scenario changes and evacuation rules.

[0023] S124. Coordinate Mapping: Establish a coordinate mapping system to linearly map the physical space coordinates (unit: meters) defined above to the screen pixel coordinates used for graphics rendering, thus providing a foundation for subsequent visualization and deep reinforcement learning models based on convolutional neural networks. The physical space is mainly used for position updates, distance calculations, velocity clipping, and evacuation statistics, while the pixel space is mainly used for global dynamic scene image generation, local observation region clipping, and convolutional neural network state input. The two maintain a strict correspondence through the mapping relationship.

[0024] SS2. Agent Perception and State Representation: The static obstacle layout of the evacuation environment is mapped to the dynamic positions of each evacuation agent to form a global dynamic scene image. Each evacuation agent is distinguished, and its local observation region is cropped centered on its own position. After resampling and normalization, this becomes the state input. This step transforms the global physical state of the simulated environment into a set of egocentric local observations usable by each agent's decision-making model; this is the state space input to the subsequent deep reinforcement learning network. The core function is to generate a two-dimensional raster image. I i ∈R H×W As an intelligent agent i Status input, H and W These represent the height and width of the state image, respectively. By using an egocentric local state representation method, the geometric relationships of the neighborhood environment and the distribution relationships of adjacent individuals can be preserved while reducing the problems of excessive dimensionality and redundant information caused by direct input of the global state.

[0025] Specifically, mapping the static obstacle layout of the evacuation environment to the dynamic positions of each evacuation agent into a global dynamic scene image includes at least the following sub-steps: S210. Loading the reference image: First, load the reference image, which represents the static obstacle layout of the evacuation environment and was pre-generated in step SS1. I base Its image size is W world × Height H world Pixels. In I base In the middle, the obstacle area is filled with grayscale values. C wall The passable area is represented by grayscale values. C free .

[0026] S211. Pixel coordinate mapping: For each agent currently existing in the simulation environment. j (Total number is) N agent The module sets its two-dimensional position coordinates in the world coordinate system. p j =( x j , y j Mapping to the image pixel coordinate system, this mapping is achieved through a linear transformation function. f map-to-creen ( · )accomplish: ( u j , v j )= f map-to-creen ( p j ; extent , W world , H world ) in, extent =( x min , y min , x max , y max This defines the boundary correspondence between world coordinates and image pixel coordinates. x min , x max These are the minimum and maximum values ​​of the x-coordinate in the world coordinate system. y max , y minThese are the minimum and maximum values ​​of the ordinate in the world coordinate system; x j , y j Represents evacuation agents j The horizontal and vertical coordinates in the world coordinate system u j , v j Represents evacuation agents j The horizontal and vertical coordinates in the image pixel coordinate system.

[0027] S212. Agent position rendering: on the reference image I base The copy image is represented by each evacuation agent. j The pixel coordinates obtained by mapping u j , v j With ) as the center, and r j = R j / s scale To draw the radius (where R j The physical radius of the intelligent agent. s scale (Scaling from world units to pixel units), using grayscale values. C agent Draw a solid circle. This step generates a global dynamic binary image of the position information of all agents. I global .

[0028] S213. Global Dynamic Scene Generation: Define the image after rendering the locations of all evacuated agents as the global dynamic scene. I global This binary image simultaneously contains static obstacle layout information and the dynamic position information of all evacuation agents at the current time step. During training, I global It is continuously refreshed over time, and its update frequency is consistent with the environmental state update frequency.

[0029] When differentiating and cropping the local observation area for each evacuation agent, at least the following should be included: S220. Feature Agent Highlighting and Differentiation: For the feature agents in the state space to be acquired... i Create a global dynamic scene image I global copy In this copy, the intelligent agent will be... iThe corresponding solid circle from itself C agent Change to a preset highlight color C self This operation aims to distinguish the agent itself from other neighboring agents and obstacles in the state space image.

[0030] S221. Boundary padding: processing global dynamic scene image copies Boundary filling is performed so that the target evacuation agent can still obtain a complete local observation area when it is close to the edge of the image.

[0031] S222. Self-centered pruning: targeting the intelligent agent i Its own pixel coordinates ( u i , v i Centered on the image after boundary padding, Cut out a piece with a side length of L crop square area This step defines the agent's observation space. It is used to characterize the distribution of obstacles, the distribution of adjacent evacuation agents, and the positional relationship of itself in a local space centered on itself. In reality, it generally corresponds to the range of human vision or the range of information received. The specific parameters are determined by the building environment and purpose to be simulated.

[0032] When resampling, normalizing, and forming state input for a local observation region, at least the following should be included: S230. Resampling: Resample the cropped local observation window image. The target size is resampled using a bilinear interpolation algorithm. H × W Obtain the state image after uniform size This ensures that the tensor size input to subsequent models is uniform. Using bilinear interpolation helps maintain the continuity of spatial structure when compressing or enlarging local observation regions, and avoids the introduction of obvious false boundaries due to coarse sampling.

[0033] S231. Batch normalization: The set of observed images of all agents obtained after the previous resampling step. Perform batch normalization processing Each image is normalized to ensure that the input data distribution has zero mean and unit variance. m This is the average of a batch of images. s For the standard deviation of the batch images, e To prevent zero from being a very small positive number, which is beneficial to the stability and convergence speed of model training.

[0034] S232. State Tensor Assembly: Assemble all state images after normalization. Assembled according to the evacuation agent dimension, outputting a shape of ( N agent , H , W The three-dimensional tensor of the agent encodes the binary spatial layout information of static obstacles, other agents, and the agent's own position within a certain perception range centered on the agent, thus constituting the agent's position. i State representation It is used as input for subsequent deep reinforcement learning networks to estimate action value and update policies.

[0035] SS3. Construction of the Behavior and Action Space: For each evacuation agent, a discrete action set is constructed, comprising one stationary action and multiple motion directions. The selected discrete action is mapped to a preferred velocity vector. Combining maximum velocity constraints and collision avoidance algorithms, a safe velocity is determined within the feasible velocity domain, and displacement updates are performed. This step defines the set of discrete actions that the deep reinforcement learning model can output and provides a deterministic function that maps the discrete action to a continuous two-dimensional preferred velocity vector. This preferred velocity vector serves as a key input to the underlying collision avoidance physics simulator. Considering the local movement characteristics of individuals during evacuation and computational efficiency requirements, this invention restricts the individual's selectable motion directions to their Moore's neighborhood. By decoupling high-level discrete decision-making from low-level continuous velocity execution, it facilitates stable training of the reinforcement learning network within a limited action space and also allows for the introduction of a classic collision avoidance model to ensure the safety and physical rationality of displacement updates.

[0036] When constructing a discrete set of actions for each evacuation agent and completing the action indexing representation, it should include at least: S310. Definition of Discrete Action Direction Set: The set of actions taken by the i-th evacuation agent at the current time step... t Current location Centered on a point, a discrete set of motion directions is constructed, consisting of 8 directions and 1 stationary motion, defined as follows: A i ={ a 0, a 1,…, a 8}, of which A i Indicates the first i A set of discrete actions of a evacuated agent. a 0 represents a stationary action. a 1~ a 8 represents actions in the front, back, left, right, and four diagonal directions.

[0037] S311. Action direction vector association: for a discrete set of actions. A i Every action in a k Pre-associate a direction vector e k and satisfy And static movement a 0 corresponds to the zero vector e 0 = (0,0), where k =1,2,…,8, Represents direction vector e k The length of the module.

[0038] S312. Action Index Space Establishment: In the deep reinforcement learning framework, the above effective action set is... A i Mapped to discrete action index space k ∈{0,1,…,8}. This allows the evacuation agent to make decisions based on the current observed state at each decision-making time. Select an action from the action set.

[0039] When mapping discrete motions to preferred velocity vectors and applying kinematic constraints, at least the following should be included: S320. Preferred velocity vector generation: for evacuated agents i At time step t Selected discrete actions Generate the preferred velocity vector according to the deterministic mapping function. g(•) represents the mapping function from discrete motion to continuous velocity. v i,max Represents evacuation agents i Maximum speed scalar, e k This represents the direction vector corresponding to the selected action. When the selected action is a stationary action... a 0 o'clock .

[0040] S321. Upper limit constraint for single-step displacement: Let the simulation step size be Δ t Then evacuate the intelligent agents i Maximum allowable displacement within a single time step d i,max satisfy d i,max = v i,max Δ t .

[0041] S322. Preferred motion expression can be executed: with preferred velocity vector As the target input of the underlying collision avoidance solver, and with v i,max With Δ t Commonly defined evacuation agents i The velocity amplitude and displacement range at the current time step are kept consistent with the actual local movement capability of the evacuated individuals.

[0042] The Reciprocal Velocity Obstacles (RVO) algorithm, a classic method in evacuation simulation, is introduced to prune velocities for agents. i A set of velocity obstacles is constructed from the surrounding impassable areas (environmental obstacles, other agents, etc., specifically determined by the building to be simulated), and the safest velocity closest to the desired direction is selected within the defined feasible velocity domain. The action selected in the previous algorithm is then executed to generate the agent's displacement update for the next time step. Specifically, when pruning the preferred velocity vector and performing displacement updates based on the RVO algorithm, at least the following steps are included: S330. Speed ​​Barrier Construction: Constructing Evacuation Agents i At time step t Based on the set of inaccessible objects around it Speed ​​obstacle set Impassable objects include environmental obstacles and nearby evacuation agents. VO ( i , o ) represents the evacuation agent i relative to objects o Speed ​​obstacle zone.

[0043] S331. Feasible velocity domain determination: Construct the feasible velocity domain while satisfying the maximum velocity constraint. ,in Represents evacuation agents i At time step t The feasible speed range, v Represents the candidate velocity vector. This represents the magnitude of the candidate velocity vector.

[0044] S332. Safe speed selection and displacement update: in the feasible speed domain Internal selection and preferred velocity vector closest safe speed And complete the position update according to the safe speed. ,in Represents evacuation agents i At time step t Safe speed, Represents evacuation agentsi The position vector at the next time step.

[0045] SS4. Evacuation Rewards and Constraints Feedback: After sequentially fuzzifying, activating rules, aggregating, and defuzzifying three types of state variables—local density, motion smoothness, and evacuation time—continuous rewards are generated. Simultaneously, distance improvement rewards based on the optimal goal are calculated, and the results are integrated to form an immediate reward. This step is one of the core innovations of this invention. Based on fuzzy mathematics theory, a multi-dimensional evaluation function is constructed to generate immediate reward signals for the agent during evacuation. This step simulates the reasoning and decision-making process of humans in emergency situations, transforming environmental state characteristics into continuous reward values, enabling the agent to exhibit more realistic evacuation behavior. Compared to methods that only use distance reduction or whether an exit has been reached as rewards, this step simultaneously introduces three types of variables: congestion level, directional smoothness, and evacuation urgency. This ensures that reward feedback not only guides the agent to continuously move towards the exit but also suppresses obviously unreasonable frequent turns, blind squeezing into locally congested areas, and target oscillation behavior in multi-exit scenarios.

[0046] Specifically, the calculation of the nonlinear distance improvement reward based on the optimal goal, used to evaluate the agent's continuous improvement in moving towards the goal, includes at least: S410. Construction of Multi-Target Distance Sets: For each agent i Given the set of distances from the current time step to each target point. The set of distances from the previous time step to each target point , K The target export volume. k =1,2,…, K First, the distance is weighted according to its importance; this step is to reflect the agent's preference for different evacuation exits in the building. S411. Target Importance Weighting: Let the importance weight vector of each target exit be... r important =[ r 1, r 2,…, r K ],in r k Indicates the first k The importance weight of each target export k =1,2,…, K We obtain the following by weighting the set of distances between the current and previous time steps element by element: , in, This indicates an element-wise division operation. These represent the weighted distance set of the current time step and the distance set of the previous time step, respectively. By introducing target importance weights, the priority, accessibility, or desired guidance direction of different exits in the evacuation organization can be differentiated and encoded.

[0047] S412. Calculate the distance improvement reward value: Calculate the evacuation agent based on the weighted distance set. i Nonlinear distance improvement reward ,in Represents evacuation agents i At time step t Distance improvement reward w 1 indicates the distance improvement reward weighting coefficient. t d and l This indicates the parameter for adjusting sensitivity to distance improvement.

[0048] This reward function is designed based on the nonlinear distance to the optimal target and observations of real evacuation behavior. The agent receives a positive reward for consistently and effectively approaching a target, aiming to encourage a continued tendency to move towards the target exit. This is because, in real evacuation, pedestrians often continuously move towards a particular exit. Using traditional distance-based reward reduction methods could cause the agent to oscillate between two exits that are close to each other, negatively impacting training effectiveness. t d and l The sensitivity to distance improvement is adjusted, and the importance weights allow for differentiated evaluation of different objectives.

[0049] When fuzzing local density variables, motion smoothness variables, and time variables, at least the following should be included: S420. State Variable Acquisition: Acquiring the Evacuation Agent i At time step t Local density variables Motion smoothness variables and the evacuation process time variable T t , Represents evacuation agents i The number of evacuated individuals per unit area. This indicates the angle between the preferred direction of movement in the next step and the actual direction of movement in the previous step. T t Indicates the time step from the start of evacuation to the current time. t The cumulative elapsed time; S421. Fuzzy set partitioning: for local density variables The number of pedestrians per unit area around the agent is divided into three fuzzy sets: low, medium, and high; for the motion smoothness variable... The angle between the agent's next movement direction and its previous movement direction is divided into three fuzzy sets in the universe of discourse: smaller, general, and larger; for the time variable... T t The time elapsed since the start of evacuation is used to assess the level of urgency, and the domain of discourse is divided into four fuzzy sets: safe, general, and dangerous.

[0050] S422. Quantization of motion smoothness: ... Represented as ,in Represents evacuation agents i At time step t The preferred velocity vector, This represents its actual execution speed vector at the previous time step. To prevent extremely small positive numbers with a denominator of zero.

[0051] S423. Construction of Symmetric Triangular Membership Functions: For each of the above input variables , T t A corresponding fuzzy set is predefined, and the input variables are fuzzified using a symmetric triangular membership function, where the symmetric triangular membership function is... , a , b , c These represent the left, right, and left endpoints of the trigonometric membership function, respectively. (The time variable is used as an example.) T t For example, its fuzzy set includes three categories: "safe," "general," and "dangerous." For any time variable value... T t Each time point is assigned a membership degree to each fuzzy set, which is used to describe the degree to which the time point belongs to multiple semantic states simultaneously.

[0052] S424. Membership Mapping: Mapping each input variable to multiple membership values, resulting in... Among them, superscript L , M , H They represent low, medium, and high, respectively, with superscript. S , M , L These represent smaller, general, and larger, respectively, and are indicated by superscript. S , M , D These represent safe, normal, and dangerous, respectively.

[0053] In this way, the original precise numerical input is converted into multiple membership values, thereby realizing the expression of the uncertainty and continuity of the environmental state.

[0054] When performing rule activation, aggregation, and defuzzification based on a fuzzy rule base, at least the following should be included: S430. Fuzzy Rule Base Construction: A set of fuzzy rule bases is pre-constructed, generated based on the safety and efficiency principles of evacuation scenarios. Each rule in the rule base adopts the following general form: IF (density state) AND (smoothness state) AND (time state) THEN ( R (Reward status) in, R This represents a fuzzy set of rewards.

[0055] S431. Rule activation strength calculation: at the current time step t For each rule in the rule base, activation calculation is performed to obtain the membership degree of each fuzzy set in the preconditions of the rule; the activation strength of the rule is calculated using the minimum operator. The activation strength is used to indicate the applicability of the rule under the current environmental conditions. Indicates the first m Rule at time step t activation intensity, m =1,2,…, M , M This represents the total number of fuzzy rules. These represent the membership degrees of the fuzzy sets corresponding to the local density, motion smoothness, and time variables in the preconditions of the rule, respectively.

[0056] Therefore, this invention does not select a single rule for judgment, but allows multiple rules to participate in the reward decision at the same time step to simulate the uncertainty of real pedestrian evacuation.

[0057] S432. Reward Fuzzy Set Aggregation: For all activated fuzzy rules, the corresponding reward fuzzy sets in the rule conclusions are weighted according to their activation strength, and then aggregated using the maximum operator to obtain a comprehensive output fuzzy reward set. ,in y This represents a discrete reward point in the reward universe. Indicates the first m The fuzzy set of rewards corresponding to the rule conclusions at the reward points y Membership degree of location This represents the aggregated output fuzzy reward set. This output fuzzy set will form a continuous distribution over the reward universe, used to represent the overall evaluation result of the reward bias in the current state.

[0058] S433. Centroid Defuzzification: After obtaining the fuzzy reward set, a centroid defuzzification strategy is used to convert the fuzzy set into a single precise reward value. ,in Represents evacuation agents i At time step t Fuzzy reward value, Y Let represent the set of the discretized reward universe. e R This represents a very small positive number to prevent the denominator from being zero. Specifically, by discretizing the reward universe, a weighted average of the membership degrees corresponding to each reward value is calculated to obtain the immediate reward value at the current time step. This reward value is a continuous real number and can be directly used as the immediate reward signal in reinforcement learning algorithms.

[0059] After the aforementioned fuzzy reward is generated, the module further integrates this reward with the sparse event reward during the evacuation process, which in this invention is the distance improvement reward, to obtain the final reward for each action. Specifically, when integrating the fuzzy reward and the distance improvement reward to form instant reward feedback, it includes at least: S440. Reward Integration: Integrate time steps t Fuzzy reward value Distance Improvement Rewards Integration is performed to obtain evacuation intelligent agents. i At time step t The final instant reward .

[0060] S441. Motion Feedback Association: [This will be used in conjunction with other methods] As an evacuation intelligent agent i In state Next action The feedback signal after that, and the state at the next time step. Together they constitute the experience of state transition.

[0061] S442. Reward Output: Output the immediate reward to the multi-agent collaborative training and learning module for subsequent action value estimation, experience playback storage, and network parameter updates.

[0062] SS5. Multi-agent collaborative training and learning: A deep reinforcement learning framework based on convolutional neural networks is constructed, in which evacuation agents collaboratively learn by sharing network parameters and centralized experience replay, and update the policy network and target network. This step implements a multi-agent deep reinforcement learning training framework based on convolutional neural networks. Through sharing neural network parameters, centralized experience replay, and collaborative learning mechanisms, multiple agents learn collaborative policies in evacuation scenarios. The key to this step is that although the multiple evacuation agents are in different local states, they share the same decision network parameters, thereby learning generalized evacuation behavior patterns in a unified policy space and reducing parameter size and training resource consumption.

[0063] Specifically, this step performs the following defined process for multi-agent collaborative training, such as... Figure 2 As shown: S510. Module S510 uses a deep convolutional neural network (CNN) to process the agent's visual state input. This CNN network consists of the following key components: a. Feature Extraction Backbone Network: Given an input image size of 84×84, the network comprises three convolutional layers specifically designed for processing 84×84 grayscale images. The first convolutional layer uses 32 8×8 convolutional kernels with a stride of 4 to extract low-level spatial features from the input channels. The second convolutional layer uses 64 4×4 convolutional kernels with a stride of 2 to capture medium-scale spatial patterns. The third convolutional layer uses 64 3×3 convolutional kernels with a stride of 1 to extract high-level abstract features. Batch normalization and ReLU activation functions are applied after each convolutional layer to extract multi-level state representations from low-level spatial features to high-level abstract features.

[0064] b. Feature Flattening and Fully Connected Layers: The 3D feature map output from the convolutional layers is flattened into a 1D feature vector, which is then processed through two fully connected layers. The network employs a Dueling architecture, feeding the feature vectors into the value stream and the dominance stream, respectively. The value stream outputs a single-state value estimate, while the dominance stream outputs the dominance value for each action.

[0065] c. Noise injection mechanism: The fully connected layer uses a noisy linear layer instead of a traditional linear layer. The weights and biases of the noisy linear layer are decomposed into deterministic mean parameters and random noise parameters.

[0066] S520. The module creates two Deep Q Networks (Dueling Deep Q Networks) with the same structure as described above, i.e., the Dueling-DQN algorithm: the current policy network. Q θ and target network ,in i For trainable parameters, i － The target parameters are used. All agents share a set of network parameters. i The current policy network is used to output the value estimate of each discrete action in the current state, and the target network is used to calculate the relatively stable target Q value to reduce target drift during training.

[0067] S530. Construct a unified experience replay pool to store the transition experiences of all agents. The replay pool adopts a priority sampling mechanism.

[0068] During training, all agents work in parallel using their current local observations. As input, through the shared current policy network Q θ Calculate the value of each action and select actions using the fuzzy constraint diffusion rule from Module 6. Subsequently, all agents execute actions simultaneously, the environment returns the next state and reward, and the experience of each agent is stored in its corresponding queue. When conditions are met, multi-step rewards are calculated and stored in the shared replay pool. Next, when the number of experiences in the replay pool reaches a threshold, batch experiences are periodically sampled from the replay pool, the temporal difference loss is calculated, and the current network parameters are updated through backpropagation. i Finally, periodically update the current network parameters. i Copy to target network parameters i － This achieves stable updates of the target Q value.

[0069] SS6. Action sampling based on fuzzy constraints: Based on the discrete action values ​​output by the policy network, fuzzy membership degrees of acceptable evacuation actions are constructed. These degrees are then coupled with and normalized to obtain the selection probability of each discrete action, and action sampling is performed accordingly. This step is one of the core innovations of this invention, proposing an action sampling method called the Fuzzy-Constrained Diffusion Rule. This method integrates fuzzy mathematics theory, deterministic drift, and random diffusion mechanisms to generate a decision-making strategy in the discrete action space that considers both value orientation and exploratory robustness. Importantly, this method's action calculation module can introduce randomness that conforms to the actual evacuation behavior of the agent, adaptable to the simulated building, while ensuring that the agent's action selection is realistic, in order to discover new stable evacuation patterns.

[0070] Specifically, when constructing the fuzzy membership of evacuation-acceptable actions based on the discrete action values ​​output by the policy network, it should include at least: S610. Action Value Acquisition: For evacuated agents i At time stept Current state Obtain its discrete action set Value estimation for each discrete action ,in Represents evacuation agents i At time step t For the first k Discrete actions a k Action value estimation, This represents the total number of discrete actions. Q θ Indicates parameters i The current policy network.

[0071] S611. Determining the maximum action value: Determine the maximum action value in the current state based on the action value estimate. ,in Represents evacuation agents i At time step t The maximum action value is used to characterize the optimal action reference point in the value orientation sense under the current state. These value estimates constitute the deterministic drift term of action selection.

[0072] S612. Construction of Acceptable Action Fuzzy Set: Definition Q The fuzzy membership function is innovatively introduced in this step, which introduces a fuzzy concept of evacuation acceptable action and defines a fuzzy set. A acceptable and the first k The degree to which a discrete action belongs to the fuzzy set is denoted as Its membership function Represents evacuation agents i At time step t Action a k The acceptable level of action; S613. Calculation of exponential fuzzy membership degree: Membership degree Calculated using the following exponential decay function: in, d >0 is a key hyperparameter called the diffusion gate, which controls the degree of boundary fuzziness in fuzzy sets. When d When the value is large, even actions whose value differs significantly from the maximum value have a high degree of membership and are more exploratory; when d When the value is small, only actions with values ​​close to the maximum are considered acceptable, and the strategy is closer to greed. Membership degree calculation defines a set of membership degrees. Qmax Using as a reference point, d It is an exponential membership function for the decay parameter.

[0073] When constructing random diffusion components based on action values ​​and obtaining unnormalized selection weights, at least the following should be included: S620. Construction of Random Diffusion Logarithmic Weights: Introducing a softmax-based stochastic process for evacuated agents. i At time step t For each action, generate unnormalized logits values: Indicates the first k Logarithmic weights of discrete actions t >0 is a temperature parameter that controls the intensity of randomness; S621. Numerical stabilization: To improve numerical stability, the logarithmic weights of logits are adjusted. Standardization process is required to obtain , This represents the standardized logarithmic weights after stabilization.

[0074] S622. Calculation of random diffusion components: Subsequently, the exponential values ​​of the random diffusion components for each discrete action are calculated based on the standardized logarithmic weights. , Indicates action a k At time step t The random diffusion component.

[0075] S623. Unnormalized selection weight calculation: Finally, calculate the fuzzy membership degrees. With random diffusion components Multiply to obtain the unnormalized selection weight for each action. , Indicates action a k The unnormalized selection weights, with the fuzzy membership degree serving as a fuzzy constraint gate, filter and weight the random diffusion process, so that actions with lower acceptability are significantly suppressed in subsequent sampling.

[0076] m k As a fuzzy constraint gate, it filters or weights random diffusion processes. Only those belonging to the acceptable action fuzzy set (i.e., those with higher...) are considered... m k Actions with a value of ( ) will have their random diffusion components significantly preserved; while actions that are clearly unacceptable ( ) will have their random diffusion components significantly preserved. mk (Approaching 0) is almost completely suppressed.

[0077] When normalizing unnormalized weights and generating action selection probabilities, at least the following should be included: S630. Normalized denominator construction: for sparse agents i At time step t All discrete actions are summed using unnormalized weights to construct a normalized denominator. , e >0 is a very small positive number, preventing division by zero errors.

[0078] S631. Action selection probability calculation: Calculate the final selection probability of each action based on the normalized denominator. , Represents evacuation agents i At time step t Select Action a k The probability of.

[0079] S632. Probability constraint satisfied: The final selection probability satisfies... and This makes all discrete actions form a complete probability distribution; S633. Probability Distribution Output: Outputs the action selection probability distribution. The output is sent to the action sampling execution unit as the basis for action sampling at the current time step.

[0080] When sampling an action based on its selection probability and outputting the sampling result, at least the following should be included: S640. Polynomial sampling execution: Based on the generated evacuation agent i At time step t Action selection probability distribution Multinomial sampling is used to select the action for each agent at the current time step. ,satisfy .

[0081] S641. Discrete Action Determination: Based on Determine the discrete action at the current time step .

[0082] S642. Action Output Association: Associate discrete actions The output is sent to the preferred velocity mapping and safe velocity pruning process in step SS3 to generate the corresponding preferred velocity vector and safe execution velocity; thereby enabling a closed-loop connection between high-level policy sampling and low-level physical execution.

[0083] S643. Sampling Mechanism Constraint Effect: The action value orientation term, fuzzy acceptable constraint term, and random diffusion term are applied together to the action sampling process, so that the action selection retains adjustable randomness only within the acceptable range of actions that meet the basic requirements of evacuation rationality.

[0084] SS7. Evacuation Process Evolution and Result Output: The module drives each evacuation agent to iteratively execute sampled actions until a preset termination condition is reached, and outputs individual trajectories, individual evacuation times, total evacuation times, evacuation flow rates, and density distribution in the exit neighborhood. When the system evolves to the termination condition (all individuals complete evacuation or the maximum simulation time is reached), this module summarizes and outputs the evacuation results. Specifically, the module executes the following definition process to construct its action space: outputs the motion trajectory and evacuation time of each individual; outputs the total evacuation time; outputs the evacuation flow rate curve; outputs the local density distribution in the exit neighborhood; and outputs the overall evacuation visualization process.

[0085] Preferably, in step SS7, the preset termination condition is that all evacuation agents complete the evacuation and / or the simulation time reaches the maximum preset duration; when the preset termination condition is met, the motion trajectory of each evacuation agent, individual evacuation time, total evacuation time, evacuation flow rate curve, local density distribution in the exit neighborhood, and the overall evacuation visualization process are uniformly summarized and output to form a comprehensive evaluation result of the spatiotemporal evolution characteristics of the evacuation process.

[0086] Example 2: Application Case Based on the above embodiment 1, this embodiment 2 further provides an application example based on a single-exit room scenario to verify the applicability of the personnel evacuation simulation method based on multi-agent reinforcement learning of the present invention in typical confined spaces, as well as its ability to represent different evacuation behavior patterns.

[0087] like Figure 3 As shown, this embodiment simulates a single-exit room, 50m long and wide, with the initial positions of the crowd randomly distributed. A total of 30 agents are evenly distributed throughout the room, tasked with rapidly evacuating the room. No additional fixed obstacles are placed within this scenario to highlight behavioral evolution phenomena such as crowd convergence, queuing, competition for passage, and localized congestion within the exit neighborhood under the same geometric boundary conditions. Specific parameters are shown in Table 1.

[0088] After parameter settings were completed, the multi-agent reinforcement learning training framework began operation. Each evacuation agent input its local observation state into a convolutional neural network to obtain an action value estimate for each discrete action. This estimate, combined with local density, motion smoothness, evacuation time, and improvement in target distance, generated an immediate reward. Subsequently, a fuzzy constraint-based action sampling mechanism was employed to generate discrete actions for each agent at the current time step, while ensuring the basic rationality of evacuation behavior. A mutual velocity barrier method was used to safely prune the preferred velocity, ultimately obtaining the actual execution speed and updating the individual's position. This process was iterated until all evacuation agents completed the evacuation or the maximum simulation duration was reached.

[0089] Table 1 Summary of parameters for reinforcement learning evacuation motion model The final simulated evacuation effect diagram is as follows Figure 4 As shown, by adjusting the randomness parameters of the crowd in the model (such as δ, τ), the evacuation patterns of the crowd exhibit rational queuing and rapid orderly evacuation (as shown in Figure 4(a), where most evacuation agents can move relatively stably towards the exit and gradually form a queue with consistent direction and clear order before the exit. At this time, the individual path deflection is relatively small, the exit utilization is relatively continuous, and the overall evacuation process shows good orderliness and high passage efficiency) and congestion before the bottleneck (as shown in Figure 4(a)). Figure 4 (b) The lateral disturbances of evacuation agents in the exit neighborhood are more obvious, the local competition is stronger, and they are more likely to form high-density aggregation and blockage in front of the bottleneck. Some individuals have greater turning and cross movements, and the overall evacuation process shows stronger uncertainty and volatility.

[0090] As the results above demonstrate, the method of this invention can not only reproduce queuing behavior in single-exit scenarios within a unified modeling framework, but also further characterize congestion, competition, and local disorder in bottleneck areas by adjusting fuzzy constraint action sampling parameters. This indicates that the method of this invention can effectively simulate the random disturbances, cognitive differences, and group evolution characteristics widely present in real evacuation behavior while maintaining goal orientation.

[0091] The objectives of this invention have been fully and effectively achieved through the above embodiments. Those skilled in the art will understand that this invention includes, but is not limited to, the contents described in the accompanying drawings and the specific embodiments described above. Although the invention has been described with reference to what is currently considered the most practical and preferred embodiments, it should be understood that the invention is not limited to the disclosed embodiments, and any modifications that do not depart from the functional and structural principles of the invention will be included within the scope of the claims.

Claims

1. A method for simulating personnel evacuation based on multi-agent reinforcement learning, characterized in that, It should include at least the following steps: SS1. Perform discrete parameterized modeling of the simulated evacuation environment, define the evacuation space boundary, obstacles, passable areas and evacuation exits, generate the initial positions and target exits of multiple evacuation agents and establish the mapping relationship between physical space coordinates and image pixel coordinates; SS2. Map the static obstacle layout of the evacuation environment and the dynamic position of each evacuation agent to a global dynamic scene image. Distinguish each evacuation agent and crop the local observation area with its own position as the center. After resampling and normalization, form the state input. SS3. For each evacuation agent, construct a set of discrete actions containing one stationary action and multiple motion directions. Map the selected discrete actions to the preferred velocity vector. Combine the maximum velocity constraint and collision avoidance algorithm to determine the safe velocity within the feasible velocity domain and perform displacement update. SS4. After sequentially performing fuzzing, rule activation, aggregation, and defuzzing on three types of state variables—local density, motion smoothness, and evacuation time—a continuous reward is generated. Simultaneously, the distance improvement reward based on the optimal goal is calculated. The two types of rewards are integrated to form an instant reward. SS5. Construct a deep reinforcement learning framework based on convolutional neural networks, in which each evacuated agent learns collaboratively by sharing network parameters and centralized experience replay, and updates the policy network and target network. SS6. Construct fuzzy membership degrees of evacuation acceptable actions based on the discrete action values ​​output by the policy network, and couple and normalize them with random diffusion weights based on action values ​​to obtain the selection probability of each discrete action and complete action sampling accordingly. SS7. Drive each evacuation agent to iteratively execute the sampled actions until the preset termination condition is reached, and output the individual trajectory, individual evacuation time, total evacuation time, evacuation flow rate and exit neighborhood density distribution.

2. The method according to claim 1, characterized in that, In step SS1, when performing discrete parameterized modeling of the evacuation environment, at least the following sub-steps are included: S110. Global coordinate system construction: Based on the topological information of the building or evacuation space to be simulated, establish a unified global coordinate system in two-dimensional or three-dimensional space; S111. Definition of Main Area and Walls: Define the main area that can be accessed inside the building or evacuation space to be simulated, and construct the outer boundary walls of the main area based on the preset width, height and wall thickness parameters; S112. Exit and obstacle configuration: Based on the topological information of the building or evacuation space to be simulated, set one or more evacuation exits with preset location and size parameters on the boundary wall, and configure obstacles in the main passable area to form an evacuation scenario to be simulated that includes passage constraints. The initial deployment of evacuation agents should include at least the following sub-steps: S120. Initialization Area Delineation: Within the main passable area, based on the purpose of the building or evacuation space to be simulated and the characteristics of crowd dispersal, delineate the initialization area for deploying evacuation agents. S121. Crowd Size and Identity Configuration: Generate multiple evacuation agents based on the preset crowd size, and assign a unique identity to each evacuation agent; S122. Initial position generation: According to the preset distribution rules, generate two-dimensional initial position coordinates for each evacuation agent in the initialization area, so that each evacuation agent enters the unified scene at the start of the simulation and forms the initial crowd distribution state. S123. Target Exit Association: Assign one or more initial target exit coordinates to each evacuation agent, so that it has target pointing information corresponding to the evacuation exit at the initial moment.

3. The method according to claim 1 or 2, characterized in that, In step SS2, when mapping the static obstacle layout of the evacuation environment and the dynamic positions of each evacuation agent to a global dynamic scene image, at least the following sub-steps are included: S210. Reference Image Loading: Load a reference image used to characterize the static obstacle layout of the evacuation environment. I base Its image size is width W world × Height H world The obstacle area is filled with grayscale values. C wall The passable area is filled with grayscale values. C free ; S211. Pixel coordinate mapping: Mapping each evacuation agent j Two-dimensional position coordinates in the world coordinate system p j =( x j , y j ) through linear transformation function ( u j , v j )= f map-to-creen ( p j ; extent , W world , H world Mapped to the image pixel coordinate system, where j =1,2,… N agent , N agent To evacuate the total number of intelligent agents, x j , y j Represents evacuation agents j The horizontal and vertical coordinates in the world coordinate system u j , v j Represents evacuation agents j The horizontal and vertical coordinates in the image pixel coordinate system extent =( x min , y min , x max , y max This represents the range between the world coordinate boundary and the image pixel boundary. x min , x max These are the minimum and maximum values ​​of the x-coordinate in the world coordinate system. y max , y min These are the minimum and maximum values ​​of the ordinate in the world coordinate system; S212. Agent position rendering: on the reference image I base On the copy image, each evacuation agent j Corresponding pixel coordinates ( u j , v j With ) as the center, and r j = R j / s scale Plot the grayscale value for the radius. C agent A solid circle, R j To evacuate intelligent agents j The radius in physical space S scale Indicates the scaling ratio; S213. Global Dynamic Scene Generation: Define the image after rendering the locations of all evacuated agents as the global dynamic scene. I global .

4. The method according to claim 3, characterized in that, In step SS2, when distinguishing each evacuation agent and cropping the local observation area centered on its own position, at least the following is included: S220. Feature-based agent highlighting and differentiation: Target evacuation agent for the state input to be acquired. i Create a global dynamic scene image I global copy and its corresponding solid circle grayscale value is changed from C agent Replace with preset highlight grayscale value C self ; S221. Boundary padding: processing global dynamic scene image copies Boundary filling is performed so that the target evacuation agent can still obtain a complete local observation area when it is close to the image edge; S222. Self-centered pruning: Deploying agents based on a target i pixel coordinates ( u i , v i Using as the center, crop out a side with a length of from the image after boundary filling. L crop Square local observation area ; When resampling, normalizing, and forming state input for a local observation region, at least the following should be included: S230. Resampling: A bilinear interpolation algorithm is used to resample each evacuation agent. i Corresponding local observation area Resampling to a fixed target size H × W Obtain the state image after uniform size ; S231. Batch normalization processing: For all resampled state images Perform batch normalization processing This ensures that the images of each state satisfy a zero-mean and unit-variance distribution, where μ This is the average of a batch of images. σ For the standard deviation of the batch images, ε To prevent zero from being the smallest positive number; S232. State Tensor Assembly: Assemble all state images after normalization. Assembled according to the evacuation agent dimension, forming a shape corresponding to the preset number of evacuation agents. N agent Status image height H and status image width W 3D state tensor S t ; S233. Status Input / Output: [This part is incomplete and requires further context to translate accurately.] S t Central and various evacuation agents i The corresponding two-dimensional raster image serves as its state input at the current time step. .

5. The method according to claim 4, characterized in that, In step SS3, when constructing a discrete action set for each evacuation agent and completing the action indexing representation, it includes at least the following: S310. Definition of Discrete Action Direction Set: [The set is defined by the first...] i An evacuation agent at the current time step t Current location Construct a discrete action set with the reference center. A i ={ a 0, a 1,…, a 8}, a 0 represents a stationary action. a 1~ a 8 represents actions in the front, back, left, right, and four diagonal directions; S311. Action direction vector association: for a discrete set of actions. A i Every action in a k Pre-associate a direction vector e k and satisfy stillness a 0 corresponds to the zero vector e 0 = (0,0), where k =1,2,…,8, Represents direction vector e k The modulus length; S312. Action Index Creation: Indexing each action... a k Mapped to discrete action index k ∈{0,1,…,8}; When mapping discrete motions to preferred velocity vectors and applying kinematic constraints, at least the following should be included: S320. Preferred velocity vector generation: for evacuated agents i At time step t Selected discrete actions Generate the preferred velocity vector according to the deterministic mapping function. The function representing the mapping from discrete motion to continuous velocity. v i,max Represents evacuation agents i Maximum speed scalar, e k This represents the direction vector corresponding to the selected action. When the selected action is... a 0 o'clock ; S321. Upper limit constraint for single-step displacement: Let the simulation step size be Δ t Then evacuate the intelligent agents i Maximum allowable displacement within a single time step d i,max satisfy d i,max = v i,max Δ t ; When pruning the preferred velocity vector and performing displacement updates based on the RVO algorithm, at least the following should be included: S330. Speed ​​Barrier Construction: Constructing Evacuation Agents i At time step t Based on the set of inaccessible objects around it Speed ​​obstacle set Impassable objects include environmental obstacles and nearby evacuation agents. VO ( i , o ) represents the evacuation agent i relative to objects o Speed ​​obstacle zone; S331. Feasible velocity domain determination: Construct the feasible velocity domain while satisfying the maximum velocity constraint. , v Candidate velocity vector, for v The modulus length; S332. Safe speed selection and displacement update: in the feasible speed domain Internal selection and preferred velocity vector closest safe speed And complete the location update at a safe speed. , Represents evacuation agents i The position vector at the next time step.

6. The method according to claim 5, characterized in that, In step SS4, the nonlinear distance improvement reward based on the optimal objective is calculated, including at least: S410. Construction of Multi-Target Distance Sets: For each evacuation agent i Each time step (cur) and the distance set from the current time step (prev) to the target exit are obtained as follows: ， in, K The target export volume. , Evacuation agents i At the current time step, the previous time step, and the 1st time step k The distance to each target exit, k =1,2,…, K ; S411. Target Importance Weighting: Let the importance weight vector of each target exit be... r important =[ r 1, r 2,…, r K ],in r k Indicates the first k The importance weight of each target export k =1,2,…, K The distance set between the current time step and the previous time step is obtained by weighting each element. , , This indicates an element-wise division operation. , These represent the weighted distance set of the current time step and the distance set of the previous time step, respectively. S412. Distance Improvement Reward Calculation: Calculate the evacuation agent based on the weighted distance set. i Nonlinear distance improvement reward , w 1 indicates the distance improvement reward weighting coefficient. τ d and λ This indicates the parameter for adjusting sensitivity to distance improvement.

7. The method according to claim 6, characterized in that, In step SS4, when blurring the local density variable, motion smoothness variable, and time variable, at least the following should be included: S420. State Variable Acquisition: Acquiring the Evacuation Agent i At time step t Local density variables Motion smoothness variables and the evacuation process time variable T t ; S421. Fuzzy set partitioning: dividing local density variables Divided into three fuzzy sets: low, medium, and high, with motion smoothness variables... Divided into three fuzzy sets: smaller, general, and larger, with time variable... T t It is divided into three fuzzy sets: safe, general, and dangerous. S422. Quantization of motion smoothness: ... Represented as ,in Represents evacuation agents i At time step t The preferred velocity vector, This represents its actual execution speed vector at the previous time step. To prevent extremely small positive numbers with a denominator of zero; S423. Construction of Symmetric Triangular Membership Functions: For each input variable , , T t Constructing symmetric triangular membership functions from corresponding fuzzy sets , a , b , c These represent the left endpoint, vertex, and right endpoint of the trigonometric membership function, respectively. S424. Membership Mapping: Mapping each input variable to multiple membership values, resulting in... Among them, superscript L , M , H They represent low, medium, and high, respectively, with superscript. S , M , L These represent smaller, general, and larger, respectively, and are indicated by superscript. S , M , D These represent safe, normal, and dangerous, respectively.

8. The method according to claim 7, characterized in that, In step SS4, when performing rule activation, aggregation, and defuzzification based on the fuzzy rule base, at least the following are included: S430. Fuzzy rule base construction: A fuzzy rule base based on evacuation safety and passage efficiency criteria is pre-built. Each rule adopts a general form: IF (density state) AND (smoothness state) AND (time state), THEN (reward state); S431. Rule activation strength calculation: at the current time step t For the first m Rule 1 m =1,2,…, M , M The total number of fuzzy rules is represented, and their activation strength is calculated using the minimum operator. , Indicates the first m Rule at time step t activation intensity, These represent the membership degrees of the fuzzy sets corresponding to the local density, motion smoothness, and time variables in the preconditions of the rule, respectively. S432. Reward Fuzzy Set Aggregation: For all activated fuzzy rules, the reward fuzzy set in the rule conclusion is truncated according to its activation strength, and then aggregated using the maximum operator to obtain the comprehensive output fuzzy reward set. , y For discrete reward points in the reward domain, For the first m The fuzzy set of rewards corresponding to the rule conclusions at the reward points y Membership degree of the location; S433. Centroid Defuzzification: The centroid method is used to defuzzify the aggregated output fuzzy reward set to obtain the fuzzy reward value. , Y Let represent the set of the discretized reward universe. ε R This indicates a very small positive number that prevents the denominator from being zero; When integrating fuzzy rewards with distance improvement rewards to form instant reward feedback, it should at least include: S440. Reward Integration: Integrate time steps t Fuzzy reward value Distance Improvement Rewards Integration is performed to obtain evacuation intelligent agents. i At time step t The final instant reward ; S441. Motion Feedback Association: [This will be used in conjunction with other methods] As an evacuation intelligent agent i In state Next action The feedback signal after that, and the state at the next time step. Together they constitute the experience of state transition.

9. The method according to claim 1, characterized in that, In step SS5, the deep reinforcement learning framework uses a CNN network to process the visual state input. The input image size is 84×84. The CNN network includes three convolutional layers, a feature flattening layer, and a fully connected layer. The three convolutional layers use structures of 32 8×8 convolutional kernels with a stride of 4, 64 4×4 convolutional kernels with a stride of 2, and 64 3×3 convolutional kernels with a stride of 1, respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The policy network and target network adopt a Dueling-DQN structure. Convolutional features are flattened and input into the value stream and advantage stream respectively to output state value estimates and advantage values ​​for each discrete action. The fully connected layer adopts a noisy linear layer, where deterministic mean parameters and random noise parameters jointly represent weights and biases. All evacuation agents share the current policy network parameters and construct a unified experience replay pool to store all transfer experiences. The experience replay pool adopts a priority sampling mechanism, and during training, multi-step rewards are calculated, the current policy network parameters are updated based on temporal difference loss, and periodically copied to the target network parameters.

10. The method according to claim 8, characterized in that, In step SS6, when constructing the fuzzy membership degree of evacuation-acceptable actions based on the discrete action values ​​output by the policy network, it includes at least: S610. Action Value Acquisition: For evacuated agents i At time step t Current state Obtain its discrete action set Action value estimation for each action , This represents the total number of discrete actions. Q θ Indicates parameters θ The current policy network; S611. Determining the maximum action value: Determine the maximum action value in the current state based on the action value estimate. , used to characterize the optimal action reference point in the current state; S612. Construction of the fuzzy set of acceptable actions: Define the fuzzy set of evacuation acceptable actions. A acceptable and the first k The degree to which a discrete action belongs to the fuzzy set is denoted as , Represents evacuation agents i At time step t Action a k Acceptable membership degree; S613. Calculation of exponential fuzzy membership: Calculate the fuzzy membership of each discrete action according to the exponential decay function. , δ >0 indicates the diffusion gate parameter; When constructing random diffusion components based on action values ​​and obtaining unnormalized selection weights, at least the following should be included: S620. Construction of Random Diffusion Log-Weights: For Dispersed Agents i At time step t The k For each discrete action, calculate its unnormalized logarithmic weight. , τ >0 indicates a temperature parameter; S621. Numerical stabilization processing: for Standardization process is required to obtain , This represents the standardized logarithmic weights after stabilization. S622. Calculation of random diffusion components: based on Calculate the random diffusion component of each discrete action. , Indicates action a k At time step t The random diffusion component; S623. Unnormalized selection weight calculation: Calculate fuzzy membership degrees With random diffusion components Multiply to get the first k Unnormalized selection weights for discrete actions .

11. The method according to claim 11, characterized in that, In step SS6, when normalizing the unnormalized selection weights and generating action selection probabilities, at least the following is included: S630. Normalized denominator construction: for sparse agents i At time step t All discrete actions are summed using unnormalized weights, and a minimal positive number is introduced. ε Constructing a normalized denominator ; S631. Action Selection Probability Calculation: Calculate the final selection probability of each discrete action based on the normalized denominator. , Represents evacuation agents i At time step t Select Action a k The probability of; S632. Probability constraint satisfied: The final selection probability satisfies... and This makes all discrete actions form a complete probability distribution; S633. Probability Distribution Output: Outputs the action selection probability distribution. The output is sent to the action sampling execution unit as the basis for action sampling at the current time step; When sampling an action based on its selection probability and outputting the sampling result, at least the following should be included: S640. Polynomial sampling execution: based on evacuation agent i At time step t Action selection probability distribution The sampling action index for the current time step is generated using a polynomial sampling method. ; S641. Discrete Action Determination: Based on Determine the discrete action at the current time step ; S642. Action Output Association: Associate discrete actions The output is the preferred velocity mapping and safe velocity clipping process in step SS3, which is used to generate the corresponding preferred velocity vector and safe execution velocity. S643. Sampling Mechanism Constraint Effect: The action value orientation term, fuzzy acceptable constraint term, and random diffusion term are applied together to the action sampling process, so that the action selection retains adjustable randomness only within the acceptable range of actions that meet the basic requirements of evacuation rationality.

12. The method according to claim 1, characterized in that, In step SS7, the preset termination condition is that all evacuation agents complete the evacuation or the simulation time reaches the maximum duration. When the preset termination condition is met, the motion trajectory of each evacuation agent, individual evacuation time, total evacuation time, evacuation flow rate curve, local density distribution of the exit neighborhood, and overall evacuation visualization process are uniformly summarized and output.

13. A personnel evacuation simulation system, comprising multiple modules, characterized in that, Used to perform the personnel evacuation simulation method based on multi-agent reinforcement learning as described in any one of claims 1 to 12.

Images