Chemical industrial park dynamic evacuation path planning method based on GPRO reinforcement learning
By using a GPRO reinforcement learning-based method, a dynamic evacuation route planning system for chemical industrial parks was constructed, which solved the problems of dynamic adaptability and multi-objective optimization in evacuation route planning for chemical industrial parks, and achieved efficient and safe evacuation in chemical accidents.
Patent Information
- Application Number
- CN202511406997.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-29
- Publication Date
- 2026-02-06
AI Technical Summary
Existing methods for planning evacuation routes in chemical industrial parks suffer from problems such as static planning lacking dynamic adaptability, computational complexity and poor real-time performance, neglect of multi-objective optimization, and low efficiency of traditional reinforcement learning. These make it difficult to achieve efficient and safe multi-objective optimized evacuation in chemical accidents.
Using a GPRO-based reinforcement learning approach, a deep neural network is trained by constructing a grid map, designing a multi-objective reward function, and guided exploration to generate real-time, multi-objective optimized evacuation path plans. Dynamic path adjustments are then made by combining sensor and monitoring data.
It enables efficient, safe, and rapid multi-objective optimized evacuation route planning in chemical industrial parks, dynamically responds to disaster changes, meets millisecond-level decision-making needs, and improves evacuation efficiency and safety.
Smart Images
Figure CN121480901A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of evacuation route planning, specifically a dynamic evacuation route planning method for chemical industrial parks based on GPRO reinforcement learning. Background Technology
[0002] Chemical industrial parks are areas with concentrated major hazards. In the event of accidents such as fires, explosions, or toxic gas leaks, efficient emergency evacuation of personnel is crucial to ensuring life safety. Existing evacuation route planning methods have the following main shortcomings:
[0003] Static planning lacks dynamic adaptability. Traditional methods, such as Dijkstra's algorithm and A* algorithm, typically plan the shortest path based on a static environmental map. However, the environment at a chemical accident site is highly dynamic and subject to change. For example, the spread of fire, the diffusion of blast shock waves, and the drift of toxic fumes constantly alter the feasibility and safety of paths. Static planning cannot respond to these changes in real time, potentially causing the planned "optimal path" to quickly become invalid, or even guiding evacuees to more dangerous areas.
[0004] The computation is complex and the real-time performance is poor. Some algorithms based on global optimization, such as genetic algorithms and particle swarm optimization, have high computational complexity and long processing time when solving large-scale evacuation problems, making it difficult to meet the requirement of providing decisions within seconds for emergency evacuation.
[0005] The problem has been simplified, but the coupling of multiple objectives has been ignored. Existing methods often reduce path planning to a single-objective problem, such as finding the shortest distance. However, the evacuation of chemical industrial parks is actually a multi-objective optimization problem, requiring a simultaneous trade-off between several conflicting objectives, such as path length, exposure to hazardous concentrations, and crowd congestion. Traditional methods struggle to effectively handle such complex multi-objective trade-offs.
[0006] Traditional reinforcement learning is inefficient. Although some studies have attempted to use reinforcement learning (RL) to solve path planning problems, traditional RL algorithms, such as DQN, suffer from slow convergence, low sampling efficiency, and insufficient policy exploration in complex, high-dimensional state-action spaces, making it difficult to achieve ideal performance in both training and application phases.
[0007] Therefore, it is essential to propose a dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning. Summary of the Invention
[0008] The purpose of this invention is to provide a dynamic evacuation route planning method for chemical industrial parks based on GPRO reinforcement learning. This method can perceive the evolution of disasters in real time and quickly and dynamically generate multi-objective optimal evacuation routes that take into account short paths, low safety risks, and smooth evacuation, thereby solving the existing technical defects and unmet technical requirements.
[0009] To achieve the above objectives, the present invention provides the following technical solution: a dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning, comprising the following steps:
[0010] I. Construction of Environmental State Space
[0011] 1.1) Digitize the chemical industrial park into a raster map, with each raster containing dynamically updated environmental information;
[0012] 1.2) The state space S_t that constitutes the agent's perception;
[0013] II. Motion Space Design
[0014] 2.1) Define the set of actions A that the agent can execute;
[0015] 2.2) After each step t, the agent selects an action a_t from set A and executes it;
[0016] III. Multi-objective reward function design: R = Rs + Ra + Rf
[0017] 3.1) Position reward Rs: Rs = ROUGE - β*(exp(α*C) - 1);
[0018] 3.2), action reward Ra; Ra=r_step+α*Δd-γ*ρ-δ*(a_t-a_{t-1})^2;
[0019] 3.3) Guiding Reward Rf: A lightweight and fast A* path planning algorithm is introduced as a "prophet". When the short-term path it plans is consistent with the agent's action direction, a small positive reward is given.
[0020] IV. Training the Optimal Policy Network Based on the GPRO Reinforcement Learning Algorithm
[0021] V. Model Deployment and Dynamic Path Generation.
[0022] Preferably, in step 1.1), the state information includes: agent location, target point location, disaster information field, and crowd density field.
[0023] Preferably, in step 1.1), the location of the intelligent agent includes the current coordinates of the evacuated personnel or group, the location of the target point includes the coordinates of the safety exit or assembly point, the disaster information field is the danger intensity on each grid obtained in real time through a sensor network or disaster simulation model, and its gradient over time is calculated, and the crowd density field is the crowd density of each grid and the surrounding area estimated in real time based on surveillance video or IoT device data, used to predict congestion.
[0024] Preferably, step 3.1) specifically includes:
[0025] 3.1.1) Successfully reaching the safe objective point grants a very large positive reward, ROUGE;
[0026] 3.1.2) Entering a high-risk area or dying will result in a corresponding negative reward, -β*(exp(α*C)-1), where C is the normalized risk intensity, 0 represents safety, 1 represents the maximum tolerable risk, β is the penalty coefficient, which controls the overall penalty range, and α is the growth coefficient. The larger α is, the more rapidly the penalty increases. The penalty value increases non-linearly with the risk intensity, encouraging people to avoid dangerous areas.
[0027] Preferably, step 3.2) specifically includes:
[0028] 3.2.1) Time step r_step: For each additional time step, a small negative reward is given to encourage a rapid evacuation;
[0029] 3.2.2) The agent moves closer to the nearest safe exit and is given a positive reward α*Δd, where Δd is the amount of distance reduction;
[0030] 3.2.3) The crowd density of the grid ahead in the direction of the agent's movement is given a negative reward of -γ*ρ to encourage rapid evacuation and the selection of smoother paths. Here, ρ represents the "crowd density of the grid ahead in the direction of the agent's movement". Its value is estimated in real time through monitoring videos, IoT device data or park diffusion models in the chemical industrial park, which directly quantifies the degree of congestion ahead of the path (the higher the density, the larger the ρ value). γ represents the "congestion penalty coefficient", which is a hyperparameter set by humans. Its function is to control the overall magnitude of the congestion penalty (the larger the γ value, the stronger the negative reward corresponding to the same ρ, and the stricter the guidance to avoid congested paths).
[0031] 3.2.4) The agent's actions need to consider the movement path and avoid passing through impassable areas such as walls. For unreasonable movement paths, a negative reward of -δ*(a_t-a_{t-1})^2 is given, where δ represents the penalty coefficient for unreasonable movement. This is a manually set penalty coefficient that controls the overall magnitude of the penalty for "unreasonable movement by the agent." The larger the δ value, the greater the penalty for the same unreasonable movement behavior (i.e., (a_t-a_{t-1})). 2 The larger the value, the stronger the negative reward and the stricter the constraint on unreasonable moves.
[0032] Preferably, in step four), training the optimal policy network using the GPRO-based reinforcement learning algorithm includes: establishing the policy network, guided exploration, fusing reward functions, and in-group evaluation optimization.
[0033] Preferably, the specific content of step four) is as follows:
[0034] 4.1) Use deep neural networks to construct and train policy functions that can output an optimal movement decision based on complex real-time environmental information (state);
[0035] 4.2) In the early stages of training, the agent has a certain probability of following the short-term suggested actions given by the "prophet" algorithm;
[0036] 4.3) Instead of building a separate reward model, the carefully designed multi-objective reward function is used to measure the agent's generated strategy from multiple perspectives, guiding the agent to converge towards higher rewards. Even if its behavior is completely random at the beginning, the penalty term can be used to know whether a certain action is good or bad, thus accelerating the learning process.
[0037] 4.4) In the scoring mechanism, instead of giving an absolute score, the rewards within each group are normalized to encourage the model to generate more solutions similar to successful cases, while avoiding methods with poor performance. KL divergence penalty is used as a safety mechanism.
[0038] Preferably, in step five, the content is: deploying the trained optimal strategy network into the emergency command system of the chemical industrial park.
[0039] Preferably, in step five), when an accident occurs and triggers an evacuation, the system executes the following loop:
[0040] 5.1) Receive disaster data and monitoring data transmitted from sensors, and update the environmental state S_t;
[0041] 5.2) Input the current state S_t into the policy network, and the network outputs the optimal action (movement direction) a_t in the current state;
[0042] 5.3) Update personnel locations based on action a_t and display recommended dynamic evacuation routes on the electronic map.
[0043] 5.4) Wait for the next time step and repeat steps 5.1)-5.3) to achieve real-time, online and dynamic path planning until all personnel are safely evacuated.
[0044] Compared with the prior art, the beneficial effects of the present invention are:
[0045] 1. High dynamic adaptability: This method makes decisions based on real-time environmental data (disasters, people), and can respond immediately to emergencies such as fire spread and toxic gas diffusion, dynamically adjusting evacuation routes and completely avoiding the risk of static planning route failure.
[0046] 2. True multi-objective optimization: Through a carefully designed reward function, the agent's learned behavioral strategy naturally balances path length, safety risk, and congestion level. The generated path is the comprehensive optimal solution, rather than the optimal solution for a single objective, which greatly improves the overall safety and efficiency of evacuation.
[0047] 3. Excellent real-time performance: The training process is completed offline, and the decision can be made with only one forward propagation of the neural network when applied online. The computational overhead is minimal and can meet the millisecond-level response requirements of emergency evacuation.
[0048] 4. Faster convergence and higher sampling efficiency: Thanks to the "guided exploration" and "reward function" mechanism in the GPRO framework, the agent can obtain effective feedback in the early stage of training, which greatly speeds up the convergence speed, reduces the requirement for the number of interactions with the simulation environment, and solves the problems of difficult and slow training in traditional RL.
[0049] 5. Stronger robustness: Deep neural networks have a certain generalization ability and can handle similar disaster scenarios that have not been seen during training, thus improving the reliability of the system when facing different accidents. Attached Figure Description
[0050] Figure 1 This is a schematic diagram of the overall logic of the present invention; Detailed Implementation
[0051] The following will refer to the appendices in the embodiments of the present invention. Figure 1 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0052] Example:
[0053] like Figure 1 As shown: A dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning includes the following steps:
[0054] I. Construction of Environmental State Space
[0055] 1.1) Digitize the chemical industrial park into a raster map, with each raster containing the following dynamically updated environmental information;
[0056] 1.1.1) Agent Position: Each person requiring evacuation is considered an independent agent. The agent position is the individual's current coordinates.
[0057] 1.1.2) Target point location: The target point location includes the coordinates of the safety exit or assembly point;
[0058] 1.1.3) Disaster Information Field: The disaster information field consists of the hazard intensity (such as temperature, toxic gas concentration, explosion overpressure), obstacle information, and available exit direction on each grid obtained through the sensor network, and its gradient over time is calculated.
[0059] 1.1.4) Crowd density field: The crowd density field is based on surveillance video, IoT device data and park diffusion model to estimate the crowd density of each grid and the surrounding area in real time, and is used to predict congestion.
[0060] 1.2) The state space S_t constitutes the perception of the intelligent agent. The state space includes: the current state of the intelligent agent: current position coordinates, health value (whether injured, poisoned, etc.), movement speed, etc.; disaster information field state; and crowd density field state.
[0061] II. Motion Space Design
[0062] 2.1) Define a two-dimensional vector set A(Δx,Δy) to represent the agent's possible direction of movement and velocity in the next time step;
[0063] 2.2) After each step t, the agent selects an action a_t from set A and executes it;
[0064] 3. With the goal of rapid and safe evacuation of intelligent agents, a multi-objective reward function is designed, which is: R = Rs + Ra + Rf.
[0065] 3.1) Set location reward Rs:
[0066] Rs = ROUGE - β*(exp(α*C) - 1)
[0067] 3.1.1) Successfully reaching the safe objective point grants a very large positive reward, ROUGE;
[0068] 3.1.2) Entering a high-risk area or dying will result in a corresponding negative reward: -β*(exp(α*C)-1). Here, C is the normalized risk intensity, where 0 represents safety and 1 represents the maximum tolerable risk. β is the penalty coefficient, controlling the overall penalty magnitude; α is the growth coefficient, with a larger α resulting in a more rapid increase in penalty. The penalty value increases non-linearly with risk intensity, encouraging avoidance of dangerous areas.
[0069] 3.2) Set the action reward Ra:
[0070] Ra=r_step+α*Δd-γ*ρ-δ*(a_t-a_{t-1})^2
[0071] 3.2.1) Time step r_step: For each additional time step, a small negative reward is given to encourage a rapid evacuation;
[0072] 3.2.2) The agent moves closer to the nearest safe exit and is given a positive reward α*Δd (Δd is the reduction in distance);
[0073] 3.2.3) The density of the crowd in the grid in front of the agent's movement direction is given a negative reward of -γ*ρ to encourage rapid evacuation and to encourage the selection of smoother paths;
[0074] 3.2.4) The agent's actions need to consider the movement path and avoid passing through impassable areas such as walls. For unreasonable movement paths, a negative reward of -δ*(a_t-a_{t-1})^2 is given.
[0075] 3.3) Setting a guiding reward Rf: Introducing a lightweight, fast A* path planning algorithm as a "prophet" to give a small positive reward when the short-term path it plans is consistent with the agent's action direction.
[0076] IV. Training the optimal policy network based on the GPRO reinforcement learning algorithm: An improved policy optimization algorithm, namely GPRO, is used as the core of the optimal policy network training.
[0077] 4.1) Establish a policy network: Use deep neural networks to approximate the policy function, which can output an optimal movement decision based on complex real-time environmental information (state);
[0078] 4.2) Guided Exploration: In the early stages of training, the agent has a certain probability of following the short-term suggested actions given by the "prophet" algorithm, rather than relying entirely on its own strategy, so as to quickly obtain high-quality experience samples and solve the problem of low exploration efficiency in traditional RL.
[0079] 4.3) Fusion reward function: Instead of building a separate reward model, the carefully designed multi-objective reward function is used to measure the agent's generated strategy from multiple perspectives. This explicitly guides the agent to learn a strategy that balances safety, efficiency, and smoothness, and guides the agent to converge towards higher rewards. Even if the agent's behavior is completely random at the beginning, the penalty term can be used to determine whether a certain action is good or bad, thus accelerating the learning process.
[0080] 4.4) Intra-group evaluation optimization: In the scoring mechanism, instead of giving an absolute score, the rewards within each group are normalized to encourage the model to generate more solutions similar to successful cases, while avoiding methods with poor performance. KL divergence penalty is used as a safety mechanism to ensure the stability and reliability of training.
[0081] V. Model Deployment and Dynamic Path Generation: The trained optimal strategy network is deployed to the emergency command system of the chemical industrial park. When an accident occurs and triggers evacuation, the system executes the following loop:
[0082] 5.1) Receive disaster data and monitoring data transmitted from sensors, and update the environmental state S_t;
[0083] 5.2) Input the current state S_t into the policy network, and the network outputs the optimal action (movement direction) a_t in the current state;
[0084] 5.3) Update personnel locations based on action a_t and display recommended dynamic evacuation routes on the electronic map.
[0085] 5.4) Wait for the next time step and repeat steps 5.1)-5.3) to achieve real-time, online and dynamic path planning until all personnel are safely evacuated.
[0086] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0087] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning, characterized by the following steps: I. Construction of Environmental State Space 1.1) Digitize the chemical industrial park into a raster map, with each raster containing dynamically updated environmental information; 1.2) The state space S_t that constitutes the agent's perception; II. Motion Space Design 2.1) Define a two-dimensional vector set A(Δx,Δy); 2.2) After each step t, the agent selects an action a_t from set A and executes it; III. Design of Multi-Objective Reward Functions 3.1) Position reward Rs: Rs = ROUGE - β*(exp(α*C) - 1); 3.2), action reward Ra; Ra=r_step+α*Δd-γ*ρ-δ*(a_t-a_{t-1})^2; 3.3) Guiding Reward Rf: A lightweight and fast A* path planning algorithm is introduced as a "prophet". When the short-term path it plans is consistent with the agent's action direction, a small positive reward is given. IV. Training the Optimal Policy Network Based on the GPRO Reinforcement Learning Algorithm V. Model Deployment and Dynamic Path Generation.
2. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, In step 1.1), the environmental information includes: agent location, target point location, disaster information field, and population density field.
3. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, Specifically, in step 1.1), the location of the intelligent agent includes the current coordinates of the evacuated personnel or groups, the location of the target point includes the coordinates of the safety exit or assembly point, the disaster information field is the danger intensity on each grid obtained in real time through sensor networks or disaster simulation models, and its gradient over time is calculated, and the crowd density field is the crowd density of each grid and the surrounding area estimated in real time based on surveillance video or IoT device data, used to predict congestion.
4. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, The reward function in step 3.1) is: R = Rs + Ra + Rf.
5. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, Specifically, in step 3.1): 3.1.1) Successfully reaching the safe objective point grants a very large positive reward, ROUGE; 3.1.2) Entering a high-risk area or dying will result in a corresponding negative reward, -β*(exp(α*C)-1), where C is the normalized risk intensity, 0 represents safety, 1 represents the maximum tolerable risk, β is the penalty coefficient, which controls the overall penalty range, and α is the growth coefficient. The larger α is, the more rapidly the penalty increases. The penalty value increases non-linearly with the risk intensity, encouraging people to avoid dangerous areas.
6. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, Step 3.2) specifically includes: 3.2.1) Time step r_step: For each additional time step, a small negative reward is given to encourage a rapid evacuation; 3.2.2) The agent moves closer to the nearest safe exit and is given a positive reward α*Δd, where Δd is the amount of distance reduction; 3.2.3) The crowd density of the grid in front of the agent in the direction of movement is given a negative reward of -γ*ρ to encourage rapid evacuation and to encourage the selection of smoother paths. Here, ρ represents the crowd density of the grid in front of the agent in the direction of movement, and γ represents the congestion penalty coefficient. 3.2.4) The agent's actions need to consider the movement path and avoid passing through impassable areas such as walls. For unreasonable movement paths, a negative reward of -δ*(a_t-a_{t-1})^2 is given, where δ represents the penalty coefficient for unreasonable movement.
7. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, In step four), training the optimal policy network based on the GPRO reinforcement learning algorithm includes: establishing the policy network, guided exploration, fusing reward functions, and in-group evaluation optimization.
8. The method for dynamic evacuation path planning in chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, The specific content of step four is as follows: 4.1) A policy function is constructed and trained using a deep neural network, which can output an optimal movement decision based on complex real-time environmental information; 4.2) In the early stages of training, the agent has a certain probability of following the short-term suggested actions given by the "prophet" algorithm; 4.3) Instead of building a separate reward model, the carefully designed multi-objective reward function is used to measure the agent's generated strategy from multiple perspectives, guiding the agent to converge towards higher rewards. Even if its behavior is completely random at the beginning, the penalty term can be used to know whether a certain action is good or bad, thus accelerating the learning process. 4.4) In the scoring mechanism, instead of giving an absolute score, the rewards within each group are normalized to encourage the model to generate more solutions similar to successful cases, while avoiding methods with poor performance. KL divergence penalty is used as a safety mechanism.
9. A dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, Step five involves deploying the trained optimal strategy network into the emergency command system of the chemical industrial park.
10. A dynamic evacuation path planning method for chemical industrial parks based on GPRO reinforcement learning according to claim 1, characterized in that, In step five), when an accident occurs and triggers an evacuation, the system executes the following loop: 5.1) Receive disaster data and monitoring data transmitted from sensors, and update the environmental state S_t; 5.2) Input the current state S_t into the policy network, and the network outputs the optimal action (movement direction) a_t in the current state; 5.3) Update personnel locations based on action a_t and display recommended dynamic evacuation routes on the electronic map. 5.4) Wait for the next time step and repeat steps 5.1)-5.3) to achieve real-time, online and dynamic path planning until all personnel are safely evacuated.