An evacuation path planning method and device, computer equipment and storage medium

Abstract

This invention provides an evacuation route planning method, apparatus, computer equipment, and storage medium, applied to an evacuation route planning system. The system includes several intelligent agents configured on target objects, each agent corresponding to a specific target object. The method includes: when an emergency is detected, acquiring the current position and exit position of each intelligent agent; based on the current and exit positions of each agent, generating a planned evacuation route for each agent using a path planning reinforcement learning model incorporating a spatial discount factor; and sending the generated planned evacuation routes to the corresponding agents, enabling the target objects corresponding to each agent to evacuate according to the planned routes. By using the evacuation route planning method, apparatus, computer equipment, and storage medium provided by this invention, the target objects corresponding to the intelligent agents can be evacuated, reducing casualties caused by emergencies and ensuring personnel safety.

Description

Technical Field

[0001] This invention relates to the field of route planning, and more specifically to an evacuation route planning method, apparatus, computer equipment, and storage medium. Background Technology

[0002] During underground coal mine operations, accidents such as gas explosions, floods, fires, and roof falls are prone to occur. These accidents damage systems, causing monitoring and control systems to fail, thus endangering the lives of underground coal mine workers. Therefore, the evacuation of personnel from underground coal mines is of paramount importance in the event of an accident.

[0003] In existing technologies, global path planning methods, represented by the A* algorithm, and local path planning methods, represented by the potential field method, are used for crowd dispersal. However, traditional path planning algorithms are inefficient when faced with dense crowds and complex structural scenarios. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the problem that traditional path planning algorithms are inefficient when facing dense crowds and complex structural scenarios, thereby providing an evacuation path planning method, device, computer equipment and storage medium.

[0005] According to a first aspect, the present invention provides an evacuation route planning method, applied to an evacuation route planning system, the evacuation route planning system comprising a plurality of intelligent agents configured on target objects, each intelligent agent being configured in a one-to-one correspondence with each target object, the method comprising:

[0006] When a sudden event is detected, the current position and exit position of each agent are obtained;

[0007] Based on the current position of each agent and the exit position, a path planning reinforcement learning model with a spatial discount factor is used to generate the planned evacuation path for each agent.

[0008] The generated planned evacuation routes are sent to the corresponding agents so that the target objects of the agents can be evacuated according to the planned evacuation routes.

[0009] In one embodiment, based on the current position of each agent and the exit position, a path planning reinforcement learning model incorporating a spatial discount factor is used to generate the planned evacuation path for each agent, including:

[0010] Calculate the distance between each agent based on their current position.

[0011] Based on the distance between the agents, the agents are divided into multiple agent groups;

[0012] The agent with the shortest distance to reach among all agents in the same agent group is identified as the leader agent.

[0013] Based on the current position of each agent and the exit position, a path planning reinforcement learning model that incorporates a spatial discount factor is used to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent.

[0014] In one embodiment, the step of generating planned evacuation paths for each agent in the agent group corresponding to the leader agent, based on the current position of each agent and the exit position, using a path planning reinforcement learning model that incorporates a spatial discount factor, includes:

[0015] Based on the current position of each agent, calculate the distance between the current leader agent and all other agents;

[0016] Calculate the global reward corresponding to the current leading agent according to the global reward calculation formula, the spatial discount factor, and the distance;

[0017] The current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent are input into a pre-trained path planning reinforcement learning model to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent.

[0018] In one embodiment, the step of inputting the current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent into a pre-trained path planning reinforcement learning model to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent includes:

[0019] The current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent are input into a pre-trained path planning reinforcement learning model to obtain several alternative evacuation paths.

[0020] The reward value corresponding to each alternative planned evacuation path is calculated based on at least one of the following: the distance between each agent in the agent group corresponding to the leader agent, the distance between the current leader agent and the exit location, the distance between the current leader agent and the obstacle, and the distance between the current leader agent and the location where the emergency occurred.

[0021] The planned evacuation routes are determined based on the ranking of the reward values ​​corresponding to each alternative planned evacuation route.

[0022] In one embodiment, the method further includes:

[0023] Obtain the actual evacuation path of each agent and the corresponding feedback result of the actual evacuation path;

[0024] The path planning reinforcement learning model is trained based on the differences between the actual evacuation paths and the planned evacuation paths of each agent and the corresponding feedback results.

[0025] In one embodiment, sending the generated planned evacuation paths to the corresponding agents, so that the target objects corresponding to the agents are evacuated according to the planned evacuation paths, includes:

[0026] Based on the generated planned evacuation routes, corresponding reminder messages are generated;

[0027] Based on the reminder information, the target objects corresponding to each intelligent agent are reminded so that the target objects corresponding to each intelligent agent are evacuated according to the reminder information.

[0028] In one embodiment, the method further includes:

[0029] Obtain the distance between the positions of each agent and the exit position;

[0030] The distances between each agent's location and the exit location are sorted, the priority of each agent is set according to the sorting results, and the target objects corresponding to each agent are controlled to evacuate according to the corresponding planned evacuation path according to the priority order of each agent.

[0031] According to a second aspect, the present invention provides an evacuation route planning device applied to an evacuation route planning system, the evacuation route planning system comprising a plurality of intelligent agents disposed on target objects, each intelligent agent being disposed in a one-to-one correspondence with each target object, the device comprising:

[0032] The acquisition module is used to acquire the current position and exit position of each intelligent agent when a sudden event is detected.

[0033] The generation module is used to generate the planned evacuation path for each agent based on the current position of each agent and the exit position, using a path planning reinforcement learning model that introduces a spatial discount factor.

[0034] The sending module is used to send the generated planned evacuation routes to the corresponding agents, so that the target objects corresponding to the agents can be evacuated according to the planned evacuation routes.

[0035] According to a third aspect, the present invention provides a computer device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the evacuation route planning method as described in any one of the first aspect and its optional embodiments.

[0036] According to a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to perform the evacuation path planning method of any one of the first aspect and its alternative embodiments.

[0037] The technical solution of this invention has the following advantages:

[0038] This invention provides an evacuation route planning method. When an emergency occurs, the current location and exit location of an agent are input into a path planning reinforcement learning model, which outputs evacuation routes for each agent to evacuate the target objects corresponding to those agents. By introducing a spatial discount factor into the path planning reinforcement learning model, the influence of agents farther away from the current agent is reduced, enabling the generation of evacuation routes even in high-density crowds. The generated planned evacuation routes are then sent to the corresponding agents to evacuate the target objects, thereby reducing casualties caused by the emergency and ensuring the safety of staff. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a flowchart of an evacuation route planning method proposed in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the movement sequence of the intelligent agents proposed in an embodiment of the present invention;

[0042] Figure 3 This is a structural block diagram of an evacuation route planning device proposed in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the hardware structure of a computer device proposed in an embodiment of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] In the event of an emergency, timely and reasonable evacuation of people can reduce casualties. The key to evacuation is reasonable route planning, and the quality of route planning directly affects the efficiency of evacuation.

[0046] Currently, there are two main methods for path planning in crowd evacuation simulation: swarm intelligence algorithms and reinforcement learning-based methods.

[0047] The collective behavior of animals has long been a focus of research in the field of planning. These behaviors follow specific rules and exhibit collective intelligence. Problems that collective intelligence can handle cannot be solved by individual individuals. For example, bee colonies find their nests through communication and cooperation, and ants use pheromones to collaboratively decide the optimal path to food sources. Therefore, collective intelligence algorithms are widely used in crowd evacuation. These collective intelligence algorithms mainly include artificial bee colony optimization (ABC), ant colony optimization (ACO), and particle swarm optimization (PSO). The Artificial Bee Colony (ABC) algorithm, inspired by bee colonies, is characterized by few parameters, simplicity, and flexibility, and can demonstrate good planning performance even without the constraints of a test function. The Ant Colony Optimization (ACO) algorithm is inspired by the way ants search for food paths. The increased retention of pheromones in areas where ants repeatedly appear leads to continuous convergence in the food search process, gradually reaching the optimal solution. Researchers are increasingly applying the ACO algorithm to crowd evacuation. The Particle Swarm Optimization (PSO) algorithm is inspired by the way flocks of birds search for food. Bird flocks don't perceive the location of food; they can only be aware of the distance between their current position and the food. In contrast, for particle swarm optimization, each optimization problem is a particle in the search space, and the optimal solution is found through extreme value updates.

[0048] Reinforcement learning is primarily used to solve problems where agents learn strategies to maximize rewards or achieve specific goals during interactions with their environment. Reinforcement learning algorithms have been widely applied to path planning tasks across various fields, such as flight path planning for drones, path planning for autonomous vehicles, and crowd simulation path planning. Currently, reinforcement learning-based path planning methods are mainly divided into traditional reinforcement learning methods and more popular methods. Regardless of the specific method, solving crowd evacuation problems requires extending the single-agent environment to a multi-agent environment.

[0049] To facilitate path planning for crowd evacuation, this invention provides an evacuation path planning method applied to an evacuation path planning system. The system includes several intelligent agents configured on target objects, each agent corresponding one-to-one with a target object. This method is used to evacuate crowds from mines during emergencies. Figure 1 As shown, the method includes the following steps S101 to S103.

[0050] Step S101: When a sudden event is detected, obtain the current position and exit position of each agent.

[0051] In this embodiment of the invention, the staff pre-installed a monitoring system at the work site. The monitoring system includes an emergency event acquisition module, which is used to collect emergency events in the mine.

[0052] For example, the monitoring system may include gas sensors to monitor gas explosions; the monitoring system may include pressure sensors to monitor collapses; the monitoring system may also include sensors for gas, temperature, smoke, etc., to monitor various parameters at the work site and issue alarms when the values ​​of each parameter exceed preset ranges, without limitation.

[0053] Each intelligent agent wears a smart helmet or smart bracelet, or other intelligent device with communication, information collection, and computing capabilities. The current location of each agent is obtained through these devices. Additionally, devices equipped with Ultra Wide Band (UWB) positioning modules can be used to acquire the location information of each agent when an emergency event is detected by the emergency event acquisition module. Exit locations can be determined by acquiring information such as maps of the work site.

[0054] Step S102: Based on the current position and exit position of each agent, generate the planned evacuation path for each agent using a path planning reinforcement learning model that introduces a spatial discount factor.

[0055] In this embodiment of the invention, the path planning reinforcement learning model is a pre-trained learning model. It can be trained based on the S-MADDPG algorithm or a neural network algorithm; no limitation is made here. The S-MADDPG algorithm introduces a spatial discount factor into the Critic network of the MADDPG algorithm to reduce the signal from distant agents, thus paying more attention to the local information of the target object corresponding to each agent when generating planned evacuation paths.

[0056] When training a path planning reinforcement learning model, the state space, action space, and reward function can be determined based on the Markov game model, and the path planning reinforcement learning model can be constructed based on the state space, action space, and reward function.

[0057] The state space consists of the position coordinates of the agents, the position coordinates of the obstacles, the position coordinates of the exits, and the distance between the agents and the event occurrence point. Let P represent the set of position coordinates of all agents, B represent the set of position coordinates of all obstacles, G represent the set of position coordinates of all exits, and E represent the set of position coordinates of the event occurrence point. The state space is represented by the following formula:

[0058]

[0059] Among them, (x i y i Let (x) be the position coordinates of agent i, and (x) be the position coordinates of agent i. e y e (x) represents the location coordinates of the point where the event occurs within the scene. i y i )∈P,(x e y e )∈E.

[0060] The action space is a discrete action space, which includes east, west, south, north, southeast, northeast, west, and northwest. It is represented by the following formula: Action = {North, South, West, East, Northwest, Northeast, Southwest, Southeast}

[0061] The reward function is set according to the tasks that the agent needs to complete during the evacuation process. The tasks that the agent needs to complete are: moving away from the event point, avoiding collisions with obstacles, avoiding collisions with other agents, and reaching the exit. The overall reward function is calculated based on the reward functions of the above four tasks.

[0062] When calculating the reward function far from the event occurrence point, first calculate the relative distance between the agent and the event occurrence point according to the following formula:

[0063]

[0064] Among them, (x i y i )∈P,(x e y e )∈E,

[0065] Reward function R far from the event occurrence pointe =d event ;

[0066] When calculating the reward function for avoiding collisions with obstacles, the distance between the agent and the obstacle is first calculated using the following formula:

[0067]

[0068] in,

[0069] Then calculate the reward function for avoiding collisions with obstacles using the following formula:

[0070]

[0071] Where R is the radius of the region where the agent is located;

[0072] When calculating the reward function for avoiding collisions with other agents, the distance between agents is first calculated using the following formula:

[0073]

[0074] Among them, (x i ,y i ), (x j ,y j )∈P, i≠j, (x j y j () represents the position coordinates of all agents other than agent i.

[0075] Then calculate the reward function for avoiding collisions with other agents using the following formula:

[0076]

[0077] When d agent When R ≤ 2R, a collision is considered to have occurred between the agents. col =-5,

[0078] When d agent When R > 2R, it is considered that there is no collision between agents. col =0;

[0079] When calculating the reward function for reaching the exit, first calculate the distance of the agent from the exit using the following formula:

[0080]

[0081] Among them, (x i ,y i )∈P,(x goal_j ,y goal_j )∈G,

[0082] Then calculate the reward function for reaching the exit using the following formula:

[0083]

[0084] Where, when d i_goal When d ≠ 0, the agent's reward is the negative value of the distance to the nearest exit; when d i_goal When R = 0, it is considered that the agent has reached the exit. goal =5.

[0085] The overall reward function is calculated using the following formula:

[0086] R total =k1(R goal +R bl +R col )+k2(R e (9)

[0087] Where k1+k2=1, k1 and k2 are designed to balance the weights among the various tasks of the agent, R goal R is the reward function for the agent reaching the exit. bl R is the reward function for the agent to avoid collisions with obstacles. col R is the reward function for an agent to avoid collisions with other agents. e The reward function is for the agent to move away from the point where the event occurred.

[0088] Step S103: Send the generated planned evacuation paths to the corresponding agents so that the target objects corresponding to the agents can be evacuated according to the planned evacuation paths.

[0089] In this embodiment of the invention, the planned evacuation path generated by the path planning reinforcement learning model is sent to the corresponding agent, so that the target object corresponding to the agent performs the evacuation task. The evacuation path planning system includes a path planning module, which is used to plan paths according to the trained path planning reinforcement learning model.

[0090] Each agent can be assigned a number, and the agent's location information can be assigned a corresponding number. After the agent's location information and exit information are input into the pre-trained path planning reinforcement learning model, the output planned evacuation path also carries the agent's number, so that the planned evacuation path can be sent to the corresponding target object, thereby reducing the possibility of sending the evacuation path incorrectly.

[0091] Through the above embodiments, when an emergency occurs, the current location and exit location of the agent are input into the path planning reinforcement learning model, and the evacuation path of each agent is output to evacuate the target objects corresponding to the agent. By introducing a spatial discount factor into the path planning reinforcement learning model, the influence of agents farther away from the current agent is reduced, so that evacuation paths can still be generated even when the crowd density is high. The generated planned evacuation paths are sent to the corresponding agents to evacuate the target objects corresponding to the agents, thereby reducing casualties caused by the emergency and ensuring the safety of the staff.

[0092] Specifically, in one embodiment, step S102 above generates planned evacuation paths for each agent based on their current position and exit position using a path planning reinforcement learning model that incorporates a spatial discount factor. This specifically includes the following steps:

[0093] Step S1021: Calculate the distance between each agent based on their current position.

[0094] Step S1022: Divide the agents into multiple agent groups based on the distance between them.

[0095] Step S1023: Determine the leader agent as the agent with the shortest total distance to all agents in the same agent group.

[0096] Step S1024: Based on the current position and exit position of each agent, use a path planning reinforcement learning model with spatial discount factor to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent.

[0097] In this embodiment of the invention, in the multi-agent evacuation path planning, since each agent communicates with each other, it is necessary to consider the distance difference between the agents. The distance difference between the agents will lead to different influences of each agent. Therefore, the agents are grouped according to the distance between them.

[0098] Based on the distances between agents, the agents are divided into K clusters using the K-Medoids algorithm. The agent with the shortest total distance to all agents within the same group is designated as the cluster center, which is also the leader agent. Grouping rules can be pre-defined; for example, based on the number of agents, 50 agents can be grouped into groups of 50. Other clustering algorithms can also be used for grouping agents, and this is not a limitation here.

[0099] Using a path planning reinforcement learning model to generate planned evacuation routes can accelerate the efficiency of generating planned evacuation routes. Furthermore, by grouping agents and generating planned evacuation routes for each agent in the leader's agent group, other agents can follow the leader's evacuation route. This reduces the computational load of the evacuation route generation process, speeds up the efficiency of evacuation route generation, and avoids casualties caused by low evacuation route efficiency due to excessive computation.

[0100] After training the path planning reinforcement learning model, the Prioritized Planning algorithm can be used to optimize the local evacuation path of the global evacuation path. The method of using reinforcement algorithms to strengthen the evacuation path is an existing technology and will not be elaborated here.

[0101] Specifically, in one embodiment, step S1024 above, based on the current position and exit position of each agent, uses a path planning reinforcement learning model that incorporates a spatial discount factor to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent, specifically including the following steps:

[0102] Step S10241: Based on the current position of each agent, calculate the distance between the current leader agent and each other agent.

[0103] Step S10242: Calculate the global reward corresponding to the current leader agent according to the global reward calculation formula, spatial discount factor and distance.

[0104] Step S10243: Input the current position, exit position, and global reward corresponding to the current leader agent into the pre-trained path planning reinforcement learning model to generate the planned evacuation path corresponding to each agent in the agent group corresponding to the leader agent.

[0105] In this embodiment of the invention, in the multi-agent path planning reinforcement learning model, the global reward is calculated according to the following formula:

[0106] r t =∑ i∈N r t,i (10)

[0107] Where N is the set of all agents, r t For global rewards, r t，i The reward for agent i.

[0108] Introducing a spatial discount factor α to adjust the global reward of the agents, taking the leader agent i and other agents j within the same agent group as an example, first calculate the distance between the leader agent i and agent j, denoted as d(i,j), and then calculate d(i,j) according to the following formula:

[0109]

[0110] The global reward for agent i is:

[0111]

[0112] in, This is the adjusted global reward.

[0113] Introducing a spatial discount factor can adjust the agent's global reward to reduce the signal from distant agents, making the adjusted global reward more focused on the agent's current local results, thereby reducing the influence of other distant agents on the current agent's path finding.

[0114] Specifically, in one embodiment, step S10243 above involves inputting the current position, exit position, and global reward corresponding to the current leader agent into a pre-trained path planning reinforcement learning model to generate planned evacuation paths for each agent in the leader agent's agent group. This specifically includes the following steps:

[0115] Step S102431: Input the current position, exit position and global reward corresponding to the current leader agent into the pre-trained path planning reinforcement learning model to obtain several alternative evacuation paths.

[0116] Step S102432: Calculate the reward value corresponding to each alternative planned evacuation path based on at least one of the following: the distance between each agent in the agent group corresponding to the leader agent, the distance between the current leader agent and the exit location, the distance between the current leader agent and the obstacle, and the distance between the current leader agent and the location where the emergency occurred.

[0117] Step S102433: Determine the planned evacuation routes according to the ranking results of the reward values ​​corresponding to each alternative planned evacuation route.

[0118] In this embodiment of the invention, the current position, exit position, and global reward corresponding to the current leader agent are input into the path planning reinforcement learning model to generate several candidate evacuation paths. The planned paths are then selected according to the ranking of the reward values ​​corresponding to each candidate evacuation path.

[0119] When calculating the reward value for each alternative planned evacuation route, it can be calculated based on at least one or more of the following: the distance between each agent in the agent group corresponding to the leader agent, the distance between the current leader agent and the exit location, the distance between the current leader agent and the obstacle, and the distance between the current leader agent and the location where the emergency occurred.

[0120] The evacuation path with the highest reward value can be determined as the optimal evacuation path. Alternatively, a pre-set filtering ratio can be used to identify multiple evacuation paths that meet the criteria as optimal. Based on the optimal evacuation path, the target objects corresponding to each agent are evacuated; no specific limitations are imposed here. Filtering multiple candidate evacuation paths according to the ranking of reward values ​​can eliminate poorly performing evacuation paths, thereby reducing computational load and improving evacuation efficiency.

[0121] The method for calculating the reward value corresponding to the alternative planned evacuation route is the same as the method for calculating the reward value in step S102, and will not be repeated here.

[0122] The reward value of evacuation paths can also be calculated by saving the coordinates of the positions moved by each leader agent when it takes a corresponding action to move to a certain position. The positions moved by the leader agents in each round form a path, and the paths of all leader agents form a set of paths. The reward r of the environmental feedback in each round is accumulated to form the reward value R. The set of paths with the largest reward value R of all agents is selected as the optimal evacuation path.

[0123] Specifically, in one embodiment, the evacuation route planning method provided by the present invention further includes the following steps:

[0124] Step S104: Obtain the actual evacuation path of each agent and the corresponding feedback result.

[0125] Step S105: Train the path planning reinforcement learning model based on the differences between the actual evacuation paths and the planned evacuation paths of each agent and the corresponding feedback results.

[0126] In this embodiment of the invention, in order to improve the expected return of the path planning reinforcement learning model during the training process, the Actor network and Critic network are used to train the model.

[0127] In the Actor network, the gradient of the expected reward of agent i is calculated according to the following formula:

[0128]

[0129] Where, θ i Let J(·) be the parameters of the Actor network in agent i, J(·) be the expected reward of agent i, a be the action set of the agent, D be the experience replay pool, and μ be the parameters of the Actor network in agent i. i For the policy of agent i, a i For the action of agent i, o i Let i be the observation space of agent i. Given the Q-function under policy μ′, the Q-function of agent μ is calculated according to the following formula:

[0130]

[0131] In the Critic network, the loss function is calculated using the following formula:

[0132]

[0133] Where y represents the expected return of the target network computation.

[0134] Calculate the expected return y of the target network computation using the following formula:

[0135]

[0136] Where x' represents the environmental state in the target network, (a1', a'2, ..., a') N ) represents the joint action space in the target network.

[0137] The Actor network and Critic network are updated as follows:

[0138] θ i '←τθ i +(1-τ)θ i '

[0139]

[0140] Where θ represents the parameters of the Actor network. Here are the parameters of the Critic network, and τ is the update coefficient.

[0141] During training, after the leader agent selects action 'a', it moves from the current state 'x' to the next state 'x', receiving a reward 'r' from the environment as feedback. The (x, a, r, x') values ​​at each step are stored in the experience replay pool 'D', and the current state is updated. Then, the objective function is set based on the expected reward calculated by the target network. The Actor and Critic networks for each leader agent are trained based on the gradient of the expected reward of the Actor network agent and the loss function of the Critic network. Finally, the parameters of the Actor and Critic networks are updated, and training proceeds to the next round. Training ends when the preset number of training rounds is reached, resulting in a trained path planning reinforcement learning model.

[0142] The path planning reinforcement learning model is optimized by training the model using Actor and Critic networks. The expected return of the target network is used as the objective, and rewarding the actions of the intelligent agent can make the optimized path planning reinforcement learning model more in line with the needs of path decoupling.

[0143] Specifically, in one embodiment, step S103 above, which involves sending the generated planned evacuation paths to the corresponding intelligent agents, so that the target objects corresponding to the intelligent agents can be evacuated according to the planned evacuation paths, specifically includes the following steps:

[0144] Step S1031: Based on the generated planned evacuation routes, generate corresponding reminder information.

[0145] Step S1032: Based on the reminder information, remind the target objects corresponding to each intelligent agent so that the target objects corresponding to each intelligent agent can be evacuated according to the reminder information.

[0146] In this embodiment of the invention, each target object wears a smart device such as a safety helmet. The device is equipped with the Raspberry Pi official system Raspbian with interface board, which is burned using an SD card. After burning is completed, a wireless connection is established between the computer system Win10 and the Raspberry Pi system through a Virtual Network Console (VNC) to display the burning interface.

[0147] Set up the necessary Python and PyTorch library environment on the Raspberry Pi. Open the command line window on the Raspberry Pi and install torch and torchvision using the pip3 command. Install the S-MADDPG algorithm and the necessary Gym library environment on the Raspberry Pi. Transfer the path planning reinforcement learning model trained on the computer to the Raspberry Pi through a virtual network console. Run the algorithm model on the Raspberry Pi to verify the successful setup of the algorithm environment and check whether the model porting was successful.

[0148] Once the planned evacuation routes are generated, the corresponding reminder information for each route will be sent to the target individuals using devices worn by them. These devices may be equipped with speakers to play pre-defined audio files for the corresponding tags, or displays to show pre-defined visual files for the corresponding tags; however, this is not a limitation.

[0149] The alert information is used to remind the target objects corresponding to each intelligent agent so that the target objects can evacuate according to the evacuation path reminded by the alert information, thereby speeding up the evacuation efficiency and ensuring the safety of the target objects.

[0150] Specifically, in one embodiment, the evacuation path planning method provided by the embodiments of the present invention further includes the following steps:

[0151] Step S106: Obtain the distances between the positions of each agent and the exit position.

[0152] Step S107: Sort the distances between the positions of each agent and the exit position, set the priorities of each agent according to the sorting result, and control the target objects corresponding to each agent to evacuate according to the corresponding planned evacuation path in the order of the priorities of each agent.

[0153] In the embodiments of the present invention, the priorities of each agent are set according to the distances between the positions of each agent and the exit position. For example, Figure 2 As shown, calculate the distances D(A) and D(B) from the position of agent A and the position of agent B to the exit position respectively.

[0154] For example, when D(A) < D(B), set the priority of agent B to be higher than that of agent A. That is, the target object corresponding to agent B passes through the collision point first, and this target object evacuates according to the planned evacuation path first. By setting the priorities of each agent according to the distances between the positions of each agent and the exit position, the target objects corresponding to the agents with longer distances are evacuated first, and then the target objects corresponding to the agents with shorter distances are evacuated, so as to evacuate the target objects with higher risks first under limited time and conditions, thereby avoiding more serious casualties and ensuring the safety of most target objects.

[0155] During the evacuation path planning process, with the expansion and refinement of the area, a very large state space will inevitably be brought, that is, the so-called "curse of dimensionality". Therefore, from the perspective of the action space, the continuous action space to which the MADDPG algorithm is applied is converted into a discrete action space, reducing the exploration dimension of the agent, improving the evacuation efficiency of the agent, and adding a certain amount of noise to improve the generalization ability of the model. The sampling method of Gumbel-Softmax is added to the output layer of the Actor network of MADDPG to output discrete actions.

[0156] Based on the same inventive concept, the present invention also provides an evacuation path planning device.

[0157] Figure 3 It is a structural block diagram of an evacuation path planning device proposed according to an exemplary embodiment. As Figure 3 shown, the device includes:

[0158] An obtaining module 101, configured to obtain the current positions of each agent and the exit position when detecting the occurrence of an emergency. For specific content, refer to the relevant description of step S101 above, and details will not be repeated here.

[0159] The generation module 102 is used to generate planned evacuation paths for each agent based on their current position and exit position, using a path planning reinforcement learning model that incorporates a spatial discount factor. For details, please refer to the relevant description of step S102 above, which will not be repeated here.

[0160] The sending module 103 is used to send the generated planned evacuation paths to the corresponding intelligent agents, so that the target objects corresponding to the intelligent agents can be evacuated according to the planned evacuation paths. For details, please refer to the relevant description of step S103 above, which will not be repeated here.

[0161] The evacuation route planning device provided in this invention, when an emergency occurs, inputs the current position and exit position of the agent into the path planning reinforcement learning model, outputs the evacuation path of each agent, and evacuates the target objects corresponding to the agent. By introducing a spatial discount factor into the path planning reinforcement learning model, the influence of agents far away from the current agent is reduced, so that evacuation paths can still be generated even when the crowd density is high. The generated planned evacuation paths are sent to the corresponding agents to evacuate the target objects corresponding to the agents, thereby reducing casualties caused by the emergency and ensuring the safety of the staff.

[0162] The specific limitations and beneficial effects of the aforementioned evacuation route planning device can be found in the limitations of the modeling section selection method for urban rail transit power supply systems mentioned above, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0163] Figure 4 This is a schematic diagram of the hardware structure of a computer device according to an exemplary embodiment. For example... Figure 4 As shown, the device includes one or more processors 1310 and a memory 1320, the memory 1320 including persistent memory, volatile memory, and a hard disk. Figure 4 Taking a processor 1310 as an example, the device may also include an input device 1330 and an output device 1340.

[0164] The processor 1310, memory 1320, input device 1330, and output device 1340 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0165] Processor 1310 can be a Central Processing Unit (CPU). Processor 1310 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0166] The memory 1320, as a non-transitory computer-readable storage medium, includes persistent memory, volatile memory, and a hard disk. It can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the business management method in this embodiment. The processor 1310 executes various server functions and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 1320, thereby implementing any of the above-mentioned evacuation path planning methods.

[0167] The memory 1320 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data that is needed and required. Furthermore, the memory 1320 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1320 may optionally include memory remotely located relative to the processor 1310, and these remote memories may be connected to the data processing device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] Input device 1330 can receive input digital or character information, and generate key signal inputs related to user settings and function control. Output device 1340 may include display devices such as a display screen.

[0169] One or more modules are stored in memory 1320, and when executed by one or more processors 1310, they perform actions such as... Figure 1 The evacuation route planning method is shown.

[0170] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in [reference 1]. Figure 1 The relevant descriptions in the illustrated embodiments.

[0171] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the methods described in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0172] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An evacuation route planning method, characterized in that, The method is applied to an evacuation route planning system, which includes several agents configured on target objects, each agent corresponding to a specific target object. When a sudden event is detected, the current position and exit position of each agent are obtained; Based on the current position of each agent and the exit position, a path planning reinforcement learning model with a spatial discount factor is used to generate the planned evacuation path for each agent. In the path planning reinforcement learning model, the global reward is calculated according to the following formula: in, For global rewards, N For the set of all intelligent agents, For intelligent agents The reward; Introducing a spatial discount factor to adjust the global reward of agents, the leader agent within the same agent group is calculated according to the following formula. i Other intelligent agents j Distance: ； The agent is calculated according to the following formula. i Global rewards: in, The adjusted global rewards, Spatial discount factor; The generated planned evacuation routes are sent to the corresponding agents so that the target objects of the agents can be evacuated according to the planned evacuation routes.

2. The method according to claim 1, characterized in that, Based on the current position of each agent and the exit position, a path planning reinforcement learning model incorporating a spatial discount factor is used to generate the planned evacuation path for each agent, including: Calculate the distance between each agent based on their current position. Based on the distance between the agents, the agents are divided into multiple agent groups; The agent with the shortest distance to reach among all agents in the same agent group is identified as the leader agent. Based on the current position of each agent and the exit position, a path planning reinforcement learning model that incorporates a spatial discount factor is used to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent.

3. The method according to claim 2, characterized in that, Based on the current position of each agent and the exit position, a path planning reinforcement learning model incorporating a spatial discount factor is used to generate planned evacuation paths for each agent in the agent group corresponding to the leader agent, including: Based on the current position of each agent, calculate the distance between the current leader agent and all other agents; Calculate the global reward corresponding to the current leading agent according to the global reward calculation formula, the spatial discount factor, and the distance; The current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent are input into a pre-trained path planning reinforcement learning model to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent.

4. The method according to claim 3, characterized in that, The step of inputting the current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent into a pre-trained path planning reinforcement learning model to generate the planned evacuation path for each agent in the agent group corresponding to the leader agent includes: The current position of the current leader agent, the exit position, and the global reward corresponding to the current leader agent are input into a pre-trained path planning reinforcement learning model to obtain several alternative evacuation paths. The reward value corresponding to each alternative planned evacuation path is calculated based on at least one of the following: the distance between each agent in the agent group corresponding to the leader agent, the distance between the current leader agent and the exit location, the distance between the current leader agent and the obstacle, and the distance between the current leader agent and the location where the emergency occurred. The planned evacuation routes are determined based on the ranking of the reward values ​​corresponding to each alternative planned evacuation route.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the actual evacuation path of each agent and the corresponding feedback result of the actual evacuation path; The path planning reinforcement learning model is trained based on the differences between the actual evacuation paths and the planned evacuation paths of each agent and the corresponding feedback results.

6. The method according to claim 1, characterized in that, The step of sending the generated planned evacuation paths to the corresponding agents, so that the target objects corresponding to the agents can be evacuated according to the planned evacuation paths, includes: Based on the generated planned evacuation routes, corresponding reminder messages are generated; Based on the reminder information, the target objects corresponding to each intelligent agent are reminded so that the target objects corresponding to each intelligent agent are evacuated according to the reminder information.

7. An evacuation route planning device, characterized in that, An evacuation route planning system is applied to an evacuation route planning system, which includes several intelligent agents configured on target objects, each agent corresponding to a specific target object. The device includes: The acquisition module is used to acquire the current position and exit position of each intelligent agent when a sudden event is detected. The generation module is used to generate the planned evacuation path for each agent based on the current position of each agent and the exit position, using a path planning reinforcement learning model that introduces a spatial discount factor. In the path planning reinforcement learning model, the global reward is calculated according to the following formula: in, For global rewards, N For the set of all intelligent agents, For intelligent agents The reward; Introducing a spatial discount factor to adjust the global reward of agents, the leader agent within the same agent group is calculated according to the following formula. i Other intelligent agents j Distance: ； The agent is calculated according to the following formula. i Global rewards: in, The adjusted global rewards, Spatial discount factor; The sending module is used to send the generated planned evacuation routes to the corresponding agents, so that the target objects corresponding to the agents can be evacuated according to the planned evacuation routes.

8. A computer device, characterized in that, The method includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the evacuation route planning method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the evacuation route planning method according to any one of claims 1-6.

Images