Multi-agent reinforcement learning-based cruise ship personnel evacuation path optimization method

Abstract

The invention discloses a multi-agent reinforcement learning-based cruise ship personnel evacuation path optimization method. The method comprises the following steps of 1, model construction: establishing a cruise ship evacuation model according to a cruise ship design drawing and setting an evacuation exit; step 2, multi-agent construction: constructing multiple groups of agents according to the cruise ship evacuation model; 3, fire information perception decision making; step 4, evacuation action execution: the personnel intelligent agent is divided into a normal motion state and an evacuation state, and the personnel intelligent agent is converted from the normal motion state to the evacuation state to execute the evacuation action after sensing the fire; the personnel agent, the fire source agent, the flue gas agent and the sensor agent serve as a model framework of multiple agents, the human perception ability is simulated, the agents can obtain environment information, behaviors between personnel are presented through dynamic interaction between the agents, the fire spreading behavior is simulated through the fire source agent, and the fire spreading behavior is simulated through the sensor agent. The influence on the environment and other intelligent agents is caused, and the fire situation is fully simulated.

Description

Technical Field

[0001] The present invention relates to the technical field of crowd evacuation simulation, and in particular to a cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning. Background Art

[0002] The evacuation simulation study of cruise ships under fire scenarios is an important topic to ensure the life safety of cruise ship passengers and crew members. By simulating the evacuation process of personnel under fire scenarios, we can find out the bottlenecks and problems of evacuation, formulate more scientific and effective evacuation plans, and improve evacuation efficiency. Based on the simulation results, we can evaluate the safety of cruise ship evacuation under different fire scenarios, provide a basis for improving the fire protection design and safety facility configuration of cruise ships, help crew members become familiar with the evacuation process and their own responsibilities, improve their organization and guidance capabilities in fire emergencies, and enhance the safety awareness and self-rescue ability of passengers. At present, the research on cruise ship evacuation simulation under fire scenarios often ignores the impact of the dynamic spread of fire on the evacuation path of personnel, which easily leads to deviations between the simulation results and the actual situation. When the dynamic spread of fire is not considered, the planned evacuation path may be blocked during the actual spread of the fire, and the evacuation path that cannot adapt to the changes of the fire may cause personnel to gather in dangerous areas. Because personnel will move along the simulated planned path, and the spread of fire may change the direction of safety, causing personnel to gather on the wrong path, increasing the risk of casualties. Therefore, the present invention proposes a cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning to solve the problems existing in the prior art. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to propose a cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning. The cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning simulates human perception ability by using personnel agents, fire source agents, smoke agents and sensor agents as a multi-agent model framework, so that the agents can obtain environmental information, present the behavior between personnel through dynamic interaction between agents, simulate the fire spread behavior through the fire source agent, affect the environment and other agents, and more fully simulate the fire situation.

[0004] To achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: a cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning, comprising the following steps: Step 1: Model construction: Initialize the environment and set training parameters according to the cruise ship design drawings, build a cruise ship evacuation model and set evacuation exits; Step 2: Multi-agent construction: Based on the cruise ship evacuation model, multiple groups of agents are constructed, including personnel agents, fire source agents, smoke agents, and sensor agents. The behaviors between personnel are presented through dynamic interactions between agents. Step 3: Fire information perception and decision-making: The human agent continuously detects fire-related dynamics within the monitoring range, and plans the evacuation route based on the deep reinforcement learning algorithm; Step 4: Execution of evacuation action: The human agent is divided into a normal motion state and an evacuation state. After sensing the occurrence of a fire, it changes from the normal motion state to the evacuation state to execute the evacuation action.

[0005] A further improvement is that in step 2, the human agent moves according to the speed calculation formula, which is as follows:

[0006] in, Represents the current personnel speed, represents normal walking speed, Represents the evacuation speed under the influence of fire, and the unit is m / s. Re represents whether the personnel are aware of the fire.

[0007] A further improvement is that the fire source agent attributes in step 2 include the fire source location, fire source condition check and fire source spread speed, and the fire source location formula is as follows:

[0008] in, Represents the current grid, represents randomly selected neighboring grids; The condition check formula is as follows:

[0009] in, Representative grid Neighboring grid Whether the state meets the conditions, when the value is 0, it means the grid The location cannot spread the fire source. When the value is 1, it means the grid Location can spread fire, , and Represents the current grid Neighboring grid Whether it is an obstacle, fire source, or exit. When the value is 0, it means that the corresponding conditions are not met. Only when all three conditions are met can the fire source spread to the grid position; The spreading speed formula is as follows:

[0010] in, Represents the average speed of fire spread, in m / s. Represents the maximum distance that the fire source spreads, in meters. It represents the shortest time for fire products within the range of fire spread to reach the critical value of personnel danger point, in seconds.

[0011] A further improvement is that: in the smoke intelligent agent in step 2, the smoke diffusion behavior is simulated according to the smoke diffusion speed formula, which is as follows:

[0012] in, Represents the average speed of smoke diffusion, in m / s; Represents the maximum distance of smoke diffusion, in meters; Represents the time for smoke diffusion, in seconds.

[0013] A further improvement is that the sensor agent in step 2 plays the role of monitoring and responding to environmental changes in the evacuation simulation, using the monitoring range and information notification scope To describe the working principle of the sensor agent, the formula is as follows:

[0014]

[0015] in, represents the coordinates of the location to be monitored, Represents the position coordinates of the human agent, represents the position coordinates of the sensor agent, Represents the monitoring range of the sensor agent, in meters, Represents the information notification range of the sensor agent, in meters.

[0016] A further improvement is that in step three, the fire information perception is performed by the human agent through a variety of ways to obtain the fire information, one way is to obtain it indirectly through information interaction with other personnel, and the other is to directly receive the alarm sent by the sensor agent.

[0017] A further improvement is that after the human agent detects the occurrence of a fire, the evacuation decision in step three will plan an evacuation path based on a deep reinforcement learning algorithm, and the calculated path will be stored in its evacuation path list.

[0018] A further improvement is that in step 4, the strategy generated in step 3 is converted into specific action instructions to ensure that the intelligent agent can respond quickly according to the perceived environmental information.

[0019] The beneficial effects of the present invention are as follows: the present invention uses human agents, fire source agents, smoke agents and sensor agents as a multi-agent model framework to simulate human perception capabilities, enable agents to obtain environmental information, present behaviors between people through dynamic interactions between agents, simulate fire spread behaviors through fire source agents, affect the environment and other agents, and more fully simulate fire conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the method of the present invention; Figure 2 The multi-agent cruise evacuation model of the present invention; Figure 3 It is a logic flow chart of the present invention. DETAILED DESCRIPTION

[0021] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0022] according to Figure 1 , Figure 2 , Figure 3 As shown, this embodiment provides a cruise ship personnel evacuation path optimization method based on multi-agent reinforcement learning, including the following steps: Step 1: Model construction: Initialize the environment and set training parameters according to the cruise ship design drawings, build a cruise ship evacuation model and set evacuation exits; Step 2: Multi-agent construction: Based on the cruise ship evacuation model, multiple groups of agents are constructed, including personnel agents, fire source agents, smoke agents, and sensor agents. The behaviors between personnel are presented through dynamic interactions between agents. The human agent moves according to the speed calculation formula, which is as follows:

[0023] in, Represents the current personnel speed, represents normal walking speed, Represents the evacuation speed under the influence of fire, the unit is m / s, Re Delegate whether personnel were aware of the fire.

[0024] The health status of personnel will affect the current status of personnel. This article uses numerical values ​​to represent the health status of personnel. The formula is as follows:

[0025] in, Represents the damage value of fire to personnel; Representatives in Health value at the moment; Representatives in The health value at the moment. If the value is greater than zero, it means the person is alive. The formula is as follows:

[0026] in, Representative Personnel in The survival status at the moment.

[0027] In the human agent, the panic level of the personnel will affect the efficiency of their decision-making and behavior under pressure. The higher the panic level, the more likely they are to avoid the fire. The risk level describes the tendency of the human agent to take risky behaviors when facing a fire. The tolerance value is used to measure the risk level of the personnel. The higher the tolerance value, the higher the risk level, and the more likely they are to take risky behaviors such as crossing the fire. To this end, the initial tolerance value of the personnel is calculated by combining the personnel's health value and risk level. The calculation formula is as follows:

[0028] in, Represents the initial personnel health value; Represents the adjustment factor, the value range is [0,1]; Represents the risk level of personnel, with a value range of [0,1]; Represents the initial tolerance value; Represents the damage value caused by fire to personnel. Represents the current tolerance value of personnel, which will be continuously attenuated by the fire. The attenuation formula is as follows:

[0029] in, is the adjustment coefficient, which is used to control the decay speed of the tolerance value, and its value range is [0,1]; Represents the current tolerance value of the personnel. is the number of grids passed.

[0030] In order to reflect the overall impact of fire on the evacuation process, the following indicators are used for measurement, and the formula is as follows:

[0031] in, represents the total number of evacuees, express time Personal health value, Represents the average health value of the evacuees.

[0032] During the evacuation process, the position of personnel in the environment changes dynamically, affecting the interaction with other agents. The perception range determines the environmental features that personnel can perceive, including other agents and environmental obstacles. This paper uses Euclidean distance as the perception range function, and the formula is as follows:

[0033] in, Represents the position coordinates of the human agent, is the location where the person is to be tested; Represents the Euclidean distance between human agents, in m.

[0034] In order to represent the perception range of the human agent, the perception range attribute is set. Only when the position to be detected is within this range can the person perceive the state of the position. The formula is as follows:

[0035] in, Represents the Whether a person can sense the position of the person to be detected Status; The representative constant is 3 and the unit is m.

[0036] The fire source agent attributes in step 2 include fire source location, fire source condition check and fire source spread speed. The fire source location formula is as follows:

[0037] in, Represents the current grid, represents randomly selected neighboring grids; Perform a condition check on the selected adjacent position to determine whether the fire source can spread at this position. The conditions mainly include three parts: the adjacent position cannot be an obstacle to ensure that the fire source does not pass through the obstacle; the adjacent position does not have a fire source to prevent the fire source from overlapping and spreading at the same position; the adjacent position cannot be an exit to ensure that the fire source does not spread to the area outside the exit. The condition check formula is as follows:

[0038] in, Representative grid Neighboring grid Whether the state meets the conditions, when the value is 0, it means the grid The location cannot spread the fire source. When the value is 1, it means the grid Location can spread fire, , and Represents the current grid Neighboring grid Whether it is an obstacle, fire source, or exit. When the value is 0, it means that the corresponding conditions are not met. Only when all three conditions are met can the fire source spread to the grid position; The formula for the spread rate is as follows:

[0039] in, Represents the average speed of fire spread, in m / s. Represents the maximum distance that the fire source spreads, in meters. It represents the shortest time for fire products within the range of fire spread to reach the critical value of personnel danger point, in seconds.

[0040] In step 2, the smoke agent simulates the smoke diffusion behavior according to the smoke diffusion velocity formula, which is as follows:

[0041] in, Represents the average speed of smoke diffusion, in m / s; Represents the maximum distance of smoke diffusion, in meters; Represents the time for smoke diffusion, in seconds.

[0042] The sensor agent in step 2 plays the role of monitoring and responding to environmental changes in the evacuation simulation, using the monitoring range and information notification scope To describe the working principle of the sensor agent, the formula is as follows:

[0043]

[0044] in, represents the coordinates of the location to be monitored, Represents the position coordinates of the human agent, represents the position coordinates of the sensor agent, Represents the monitoring range of the sensor agent, in meters, Represents the information notification range of the sensor agent, in meters.

[0045] Step 3: Fire information perception and decision-making: The human agent continuously detects fire-related dynamics within the monitoring range, and plans the evacuation route based on the deep reinforcement learning algorithm; Fire information perception is obtained by human agents through a variety of ways. One way is to obtain fire information indirectly through information interaction with other personnel, and the other is to directly receive alarms sent by sensor agents.

[0046] Evacuation Decision After the human agent detects the occurrence of a fire, it plans the evacuation path based on the deep reinforcement learning algorithm and stores the calculated path in its evacuation path list.

[0047] In the fire evacuation scenario, the evacuation guidance agent perceives the surrounding attributes from the environment, such as the current coordinates and the location of the fire, as the state of the agent. The objective function formula of the agent is as follows:

[0048] in represents the conversion parameter; S represents the state distribution; represents the reward function obtained by the agent from the environment at time t; E represents the expected value.

[0049] The value function is introduced to estimate the value of the current paired action state of the evacuation agent, as follows:

[0050] in: represents the value function; represents the conversion parameter; Represents the state of the agent at time t; represents the action taken by the agent at time t.

[0051] During the training process, in order to balance exploration and development, the multi-agent adopts the ε-greedy strategy to select actions. The formula is as follows:

[0052] Among them: a represents a randomly selected action; The value is 0.5.

[0053] Step 4: Execution of evacuation action: The human agent is divided into a normal motion state and an evacuation state. After sensing the occurrence of a fire, it changes from the normal motion state to the evacuation state to execute the evacuation action; The artificial potential field method is integrated into the deep reinforcement learning model to optimize the evacuation path and reduce the impact of fire on the evacuation efficiency of personnel. Among them, the attraction field in the artificial potential field is used to characterize the attraction strength of each exit, which comprehensively considers multiple factors such as exit distance, safety and fire location:

[0054] in: Represents the repulsive force of fire and obstacles in the evacuation scenario; The value of the representation factor is 0.8; x Indicates the position of the human agent, including horizontal and vertical coordinate information, Indicates the exit location, including horizontal and vertical coordinate information.

[0055] Compared with the gravitational field, the repulsive field also expresses the strength of the repulsive force of obstacles or fire points. The repulsive field formula is expressed as follows:

[0056] in, The repulsive forces created by fire and obstacles in evacuation scenarios; Indicates that the hyperparameter value is 2; Indicates the location of the fire source agent, including horizontal and vertical coordinates; Represents the distance from the human agent to the fire agent.

[0057] The reward function of the artificial potential field is designed as shown in the formula to more accurately describe the behavioral dynamics and path selection strategy of individuals in fire evacuation. By setting a reasonable reward function, the model can effectively guide individuals to move in the optimal evacuation direction while avoiding potential dangerous areas to the greatest extent.

[0058]

[0059] in, represents the agent’s reward at a given time step; represents the attractive potential field at time t+1; represents the attractive potential field at time t; represents the repulsive potential field at time t+1; represents the repulsive potential field at time t; and are constants, and their values ​​are 1.2 and 3.4 respectively.

[0060] Convert the strategy generated in step 3 into specific action instructions to ensure that the agent can respond quickly based on the perceived environmental information.

[0061] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning, characterized in that: The following steps are involved: Step 1: Model construction: Initialize the environment and set training parameters according to the cruise ship design drawings, build a cruise ship evacuation model and set evacuation exits; Step 2: Multi-agent construction: Based on the cruise ship evacuation model, multiple groups of agents are constructed, including personnel agents, fire source agents, smoke agents, and sensor agents. The behaviors between personnel are presented through dynamic interactions between agents. Step 3: Fire information perception and decision-making: The human agent continuously detects fire-related dynamics within the monitoring range, and plans the evacuation route based on the deep reinforcement learning algorithm; Step 4: Execution of evacuation action: The human agent is divided into a normal motion state and an evacuation state. After sensing the occurrence of a fire, it changes from the normal motion state to the evacuation state to execute the evacuation action.

2. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: In step 2, the human agent moves according to the speed calculation formula, which is as follows: in, Represents the current personnel speed, represents normal walking speed, Represents the evacuation speed under the influence of fire, and the unit is m / s. Re represents whether the personnel are aware of the fire.

3. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: The fire source agent attributes in step 2 include fire source location, fire source condition check and fire source spread speed. The fire source location formula is as follows: in, Represents the current grid, represents randomly selected neighboring grids; The condition check formula is as follows: in, Representative grid Neighboring grid Whether the state meets the conditions, when the value is 0, it means the grid The location cannot spread the fire source. When the value is 1, it means the grid Location can spread fire, , and Represents the current grid Neighboring grid Whether it is an obstacle, fire source, or exit. When the value is 0, it means that the corresponding conditions are not met. Only when all three conditions are met can the fire source spread to the grid position; The spreading speed formula is as follows: in, Represents the average speed of fire spread, in m / s. Represents the maximum distance that the fire source spreads, in meters. It represents the shortest time for fire products within the range of fire spread to reach the critical value of personnel danger point, in seconds.

4. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: The smoke agent in step 2 simulates the smoke diffusion behavior according to the smoke diffusion velocity formula, which is as follows: in, Represents the average speed of smoke diffusion, in m / s; Represents the maximum distance of smoke diffusion, in meters; Represents the time for smoke diffusion, in seconds.

5. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: The sensor agent in step 2 plays the role of monitoring and responding to environmental changes in the evacuation simulation, using the monitoring range and information notification scope To describe the working principle of the sensor agent, the formula is as follows: in, represents the coordinates of the location to be monitored, Represents the position coordinates of the human agent, represents the position coordinates of the sensor agent, Represents the monitoring range of the sensor agent, in meters, Represents the information notification range of the sensor agent, in meters.

6. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: In the step three, the fire information is perceived by the human agent through a variety of ways to obtain fire information, one way is to obtain it indirectly through information interaction with other personnel, and the other is to directly receive the alarm sent by the sensor agent.

7. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: In the step three, after the human agent detects the occurrence of a fire, the evacuation decision is made, and an evacuation path is planned based on a deep reinforcement learning algorithm, and the calculated path is stored in its evacuation path list.

8. The cruise ship crew evacuation path optimization method based on multi-agent reinforcement learning according to claim 1, characterized in that: In the step 4, the strategy generated in the step 3 is converted into specific action instructions to ensure that the intelligent agent can respond quickly according to the perceived environmental information.

Images