Passenger evacuation method for comprehensive passenger transportation hub based on passenger flow data and reinforcement learning

Abstract

The present invention discloses a passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, including: collecting real-time status data of a certain station and performing data preprocessing; describing the passenger evacuation problem of the comprehensive passenger transportation hub as a Markov game process; including defining a global state space, defining a local state space, defining an action space and setting a reward function; constructing a passenger evacuation model for a transportation hub, wherein the passenger evacuation model for the transportation hub is composed of an Actor network and a Critic network; introducing an asynchronous action coordination mechanism into the passenger evacuation model for the transportation hub and obtaining a stable passenger evacuation model for the transportation hub through training; and realizing passenger evacuation for station A based on the stable passenger evacuation model for the transportation hub. The method of the present invention has the characteristics of flexibility and intelligence, can effectively cope with the challenges during peak passenger flow periods, and improve the operation efficiency of transportation hubs and the travel experience of passengers.

Description

Technical Field

[0001] The present invention relates to the technical field of comprehensive passenger transport hub operation management, and in particular to an intelligent passenger flow evacuation method based on real-time passenger flow data and multi-agent reinforcement learning. Background Art

[0002] In modern cities, comprehensive passenger transportation hubs play an important role and are key nodes in urban transportation systems. These transportation hubs are not only important hubs for passenger travel, but also bear the important task of urban passenger evacuation. However, with the growth of urban population and the increasing demand for transportation, these passenger transportation hubs face serious challenges during peak passenger flow periods. The traditional fixed departure time interval is difficult to adapt to the changing needs of passenger flow, resulting in congestion and long waiting time during peak hours, affecting passengers' travel experience and safety. In addition, fixed departure time intervals also have the problem of resource waste, because during off-peak hours, the intelligent body may be in an empty or low-load state, resulting in a waste of transportation resources. Therefore, an intelligent method is needed to dynamically adjust the departure time interval to achieve a balance between bus and subway departure time intervals to improve passenger evacuation efficiency and operational efficiency.

[0003] At present, there are some defects in the passenger evacuation method of comprehensive passenger transportation hub:

[0004] (1) Lack of flexibility: The traditional fixed departure time interval cannot be adjusted according to the real-time passenger flow, resulting in the inability to effectively evacuate passengers during peak hours. In addition, empty or low-load situations occur during off-peak hours, resulting in a waste of transportation resources. Existing methods lack adaptability and cannot adjust passenger evacuation strategies in real time according to factors such as traffic conditions and special events, resulting in inefficient use of transportation resources.

[0005] (2) Lack of integrity: Passenger evacuation methods are usually limited to a single transportation mode and lack overall planning and coordination, resulting in poor connection between different transportation modes and affecting the smooth flow of passenger flow.

[0006] (3) Poor passenger experience: Crowding and long waiting times during peak hours will affect passengers’ travel experience and reduce the service quality of transportation hubs. Summary of the invention

[0007] The purpose of the present invention is to overcome the deficiencies in the prior art, solve the problems of the lack of flexibility, adaptability, integrity and low efficiency of the traditional passenger evacuation method of the comprehensive passenger transportation hub, and provide a passenger evacuation method for the comprehensive passenger transportation hub based on passenger flow data and reinforcement learning. The method combines the real-time monitored passenger flow data with the multi-agent reinforcement learning algorithm, which can intelligently optimize the operation plans of buses and subways, effectively respond to passenger flow fluctuations, reduce congestion in the station, and improve passengers' travel experience and safety. At the same time, it also improves the operating efficiency, flexibility and convenience of the comprehensive passenger transportation hub. The innovation of the present invention lies in the use of real-time passenger flow data combined with multi-agent reinforcement learning to realize the intelligent coordinated scheduling of the comprehensive passenger transportation hub and the urban public transportation system, bringing a new solution to the station-city integration management.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] A passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, for a certain station A, includes:

[0010] S1 collects real-time status data of site A and performs data preprocessing, real-time status data includes real-time passenger flow data, relevant site data and relevant agent data, the agent includes buses and subways;

[0011] S2. Describe the passenger evacuation problem of integrated passenger transportation hub as a Markov game process, including defining the global state space, defining the local state space, defining the action space and setting the reward function;

[0012] The global state space includes passenger states, related station states and related agent operation states;

[0013] The local state space includes the passenger congestion of the station, the current departure time interval and its remaining capacity;

[0014] The action space is defined as the product of a continuous range [0,1] and the maximum departure time interval;

[0015] S3. Constructing a passenger flow evacuation model for a transportation hub, wherein the passenger flow evacuation model for the transportation hub is composed of an Actor network and a Critic network;

[0016] S4. Introduce an asynchronous action coordination mechanism into the transportation hub passenger flow evacuation model and obtain a stable transportation hub passenger flow evacuation model through training;

[0017] S5. Realize passenger evacuation of station A based on the stable passenger evacuation model of the transportation hub.

[0018] Furthermore, the real-time passenger flow data in step S1 includes the timetable of trains arriving at station A, the number of passengers getting off at station A, and the time they stay at station A;

[0019] The relevant station data include the maximum passenger capacity of the bus and subway stations related to station A, and the number of waiting passengers at the bus and subway stations related to station A;

[0020] The relevant intelligent agent operation data includes the arrival time, departure time and remaining available capacity of each relevant bus and subway at station A; if station A is the starting station, the capacity is equal to the maximum carrying capacity of station A.

[0021] Further, in step S2,

[0022] The global state space is represented by the vector S all =[ s p , s s , s b ] It represents the state space shared by all agents in the environment involved in the evacuation process;

[0023] Passenger status vector indicates; among them is the time when train h arrives at station A within the time of adjacent decision points, is the number of passengers who get off after train h arrives at station A within the time of adjacent decision points; is the retention time of passenger j at station A; the time of adjacent decision points = the end service time of the previous vehicle - the start service time of the current vehicle;

[0024] Related site status vectors s s =[ s zcd ] indicates; among them is the passenger congestion of the relevant station z within the time of adjacent decision points; is the number of waiting passengers at the relevant station z within the time of adjacent decision points, is the maximum passenger capacity of the relevant station z;

[0025] The vector of the running state of the relevant intelligent agent is s b =[ b ieb , b id ] indicates; among them is the departure time interval of agent i within the adjacent decision points; is the remaining capacity of agent i within the time between adjacent decision points; and The departure time and arrival time of agent i respectively;

[0026] The local state space of each agent represents the environmental information it perceives. The local state space of each agent is composed of vectors. s i =[ s zcd , b ieb , b id ] express;

[0027] The action space is defined as: , i=1,2,3…I, a i ∈[0,1] Each of these actions represents the proportion of the departure time interval selected by agent i; the departure time interval of agent i is expressed as: ,in represents the maximum possible departure time interval of agent i;

[0028] Assuming that all agents are fully cooperative, the reward function for agent i is defined as:

[0029] (1);

[0030] (2);

[0031] in, Represents the global state space Perform actions on agent i Rewards received after , the reward value is 1 minus The average passenger detention time, the smaller the average passenger detention time, the greater the reward value; is the total number of passengers riding agent i in a decision cycle; is a positive integer; is the total reward of I agents; Formula (1) is used to evaluate the action of an agent i Contribution to the passenger evacuation efficiency of the entire comprehensive passenger transportation hub; is the time passenger j spends at the station.

[0032] Furthermore, in step S3, the Actor network includes:

[0033] The input layer takes the local state space of the agent as input;

[0034] Hidden layer, using multi-layer perceptron MLP and Self-Attention mechanism as the hidden layer structure;

[0035] Activation function, ReLU activation function is used after each hidden layer;

[0036] Output layer: The output layer corresponds to the action space of the agent and uses the sigmoid activation function;

[0037] The Critic network includes:

[0038] The input layer takes the agent’s global state space and action space as input;

[0039] Hidden layer, using multi-layer perceptron MLP and Self-Attention mechanism as the hidden layer structure;

[0040] Activation function, ReLU activation function is usually used after each hidden layer;

[0041] Output layer, the output layer is a value function, which represents the Q value of the corresponding state and action.

[0042] Furthermore, in step S4, the centralized training distributed execution CTDE framework is used to train the traffic hub passenger evacuation model. The agent optimizes its local strategy by obtaining the global state space of other agents. During the distributed execution process, the agent only uses its own local state space to make decisions. Each agent makes decisions based on the Actor network. On the other hand, the value function of the evaluation state of each agent's Critic network is used to guide the update and optimization of the Actor network. The specific training process is as follows:

[0043] S401. Randomly initialize all states; initialize all parameters of the Actor network and Critic network; initialize hyperparameters: learning rate , Discount Factor , Soft Update Parameters and randomly sample experience sample size b; initialize experience replay pool D;

[0044] S402. Each agent interacts with the environment and transforms the currently observed local state space Input online Actor network and get action , The corresponding parameters in the Actor network are , as shown in formula (3); execute the selected action , observe the reward of environmental feedback and the next state ; After n iterations , and Together they constitute the experience sample B;

[0045] (3);

[0046] in is the noise term, using Gaussian noise;

[0047] S403. The experience samples are stored in the experience playback pool. Each experience sample is It is expressed as shown in formula (4);

[0048] (4);

[0049] in represents the observation vector of all agents on the current environment state, Represents other global states; Represents the observation vector of all agents to the next state of the environment; when the number of experience samples reaches the threshold D, a number of experience samples b are randomly sampled from the experience replay pool to train the Actor network and the Critic network; Refers to the rewards obtained by the 1st to the Ith agents; Refers to the actions performed by the 1st to the Ith agents;

[0050] S404. Centralized training:

[0051] Randomly sample a number of experience samples b from the experience replay pool, using and They are used as inputs of the online critic network and the target critic network, and output state value functions respectively. and ,in , The output is calculated by the target Actor network; the global state space and action space of the agent in the environment are used to centrally train the Critic network; the MSE loss function of the Actor network and the Critic network is constructed using the temporal difference error, and then the parameters of the Critic network are updated using gradient descent ; The specific loss function and gradient are shown in formulas (5) and (6):

[0052] (5);

[0053] (6);

[0054] Where b is the size of the random sampling experience sample; is the loss function of the Critic network; Represents the reward obtained by the i-th agent; represents the gradient operator;

[0055] S405. Distributed Execution:

[0056] When calculating the forward propagation of its own Actor network, each agent only uses its own local state space As input to the online Actor network, it outputs a deterministic action ,Right now ; Calculate the MSE loss function of the temporal difference error and calculate the parameters of the Actor network The gradient of , and then use gradient descent to update the parameters; the loss function and gradient are shown in formulas (7) and (9):

[0057] (7);

[0058] in , I is the number of agents;

[0059] = - 1 b ∑ j=1 b [ ∂ Q ϕ i ∂ θ i ] = - 1 b ∑ j=1 b [ Q ϕ i ( s,a ) ∂ μ θ i ( s i ) μ θ i ( s i ) ∂ θ i ] (8);

[0060] (9);

[0061] S406. Soft update the target actor network and the critic network, as shown in formulas (10) and (11):

[0062] (10);

[0063] (11);

[0064] in, is the loss function of the Actor network, are the parameters of the Critic network after soft update; are the parameters of the Actor network after soft update;

[0065] S407. Repeat S404-S406 until the Actor network and Critic network training are stable, and the training of the transportation hub passenger flow evacuation model is completed.

[0066] Furthermore, the asynchronous action coordination mechanism in step S4 means that after each agent completes its own action, it can immediately request data from other agents and immediately update network parameters; the specific process is:

[0067] Assume that the agent The action of the xth decision was not completed and the xth reward was not obtained. , other agents When making decisions at the current time step, only the agent can be requested The complete data of the x-1th decision is used to update its own strategy, as shown in formula (12):

[0068] S agent i ={[ s 1 i , a 1 i , r 1 i ],[ s 2 i , a 2 i , r 2 i ],...,[ s x-1 i , a x-1 i , r x-1 i ]} (12);

[0069] in, represents the local state space of agent i at the x-1th time, represents the x-1th action of agent i, represents the reward of agent i for the x-1th time.

[0070] Furthermore, the specific evacuation process in step S5 is as follows:

[0071] S501. Collecting real-time status data of the comprehensive passenger transportation hub, performing data preprocessing, and forming a standardized data sample vector;

[0072] S502. Extract the passenger, train and station related data within the current decision cycle t to generate a global state space and a local state space;

[0073] S503. Extract the number of agents within the current decision cycle t;

[0074] S504. All global state space data and local state space data in the current decision cycle t are transmitted to the decision model of the stable traffic hub passenger flow evacuation model, and the optimal departure time interval and action of each agent are output. Collect feedback data after execution;

[0075] S505. Repeat S501-S504 until the corresponding decision cycle T ends.

[0076] The present invention also provides a comprehensive passenger transportation hub passenger evacuation system based on passenger flow data and reinforcement learning, including:

[0077] A data collection module is used to collect the real-time status data of station A and perform data preprocessing, the real-time status data includes real-time passenger flow data, relevant station data and relevant intelligent agent data, and the intelligent agents include buses and subways;

[0078] The Markov game module is used to describe the passenger evacuation problem of the comprehensive passenger transportation hub as a Markov game process; it includes defining a global state space, defining a local state space, defining an action space and setting a reward function; the global state space includes passenger states, related station states and related agent operation states; the local state space includes the passenger congestion of the station, the current departure time interval and its remaining capacity; the action space is defined as the product of a continuous range [0,1] and the maximum departure time interval;

[0079] A model building module, used to build a passenger flow evacuation model for a transportation hub, wherein the passenger flow evacuation model for a transportation hub is composed of an Actor network and a Critic network;

[0080] Model training module, which is used to introduce asynchronous action coordination mechanism into the transportation hub passenger flow evacuation model and obtain a stable transportation hub passenger flow evacuation model through training;

[0081] The evacuation module is used to realize the passenger evacuation of station A through a stable transportation hub passenger evacuation model.

[0082] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the passenger evacuation method for an integrated passenger transportation hub based on passenger flow data and reinforcement learning are implemented.

[0083] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the passenger flow evacuation method for an integrated passenger transportation hub based on passenger flow data and reinforcement learning.

[0084] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0085] 1. Improve passenger evacuation efficiency and operational efficiency: Through the combination of real-time passenger flow data and multi-agent reinforcement learning algorithms, the present invention can dynamically adjust the departure time intervals of buses and subways according to the current passenger flow situation and traffic demand. This flexibility enables transportation hubs to better adapt to changes in passenger flow in different time periods and emergencies, thereby improving the operational efficiency and passenger evacuation efficiency of comprehensive passenger transportation hubs and reducing the operating costs of urban transportation systems.

[0086] 2. Optimize overall traffic coordination: This invention takes into account a variety of urban transportation tools, such as buses and subways, and optimizes the overall passenger evacuation by coordinating the dispatch of these transportation tools. The coordinated actions between multiple agents make the coordination between the various parts of the transportation hub smoother, improve the overall performance, and provide an effective paradigm for the passenger evacuation of comprehensive passenger transportation hubs.

[0087] 3. Enhance model robustness and decision-making efficiency: Under the centralized training and distributed execution framework of multi-agent reinforcement learning, the present invention designs an asynchronous action coordination mechanism. This mechanism improves the robustness and parallelism of the model, enabling it to operate stably even when some agents fail or are delayed, and is suitable for decision-making of agents of different scales. At the same time, this asynchrony and parallelism speeds up the decision-making process, further improving passenger evacuation efficiency and operational efficiency.

[0088] 4. Improve passenger travel experience and safety: By dynamically adjusting departure time intervals and optimizing passenger evacuation strategies, the present invention can effectively reduce station congestion and passenger waiting time during peak hours, and improve passenger travel experience and safety. This not only improves the service quality of transportation hubs, but also increases passenger satisfaction with the public transportation system.

[0089] 5. Improved resource utilization efficiency: Traditional fixed departure time intervals may cause vehicles to be empty or underloaded during off-peak hours, resulting in waste of transportation resources. The present invention avoids waste of resources, improves the utilization efficiency of transportation resources, and optimizes the operation of urban public transportation systems by intelligently adjusting the departure time intervals.

[0090] In summary, the present invention realizes the intelligent scheduling and optimization of comprehensive passenger transportation hubs through the combination of real-time passenger flow data and multi-agent reinforcement learning, which not only improves the passenger evacuation efficiency and operational efficiency, but also significantly improves the travel experience and safety of passengers, and has significant economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 is a specific flow chart of an embodiment of the present invention;

[0092] Figure 2 This is a schematic diagram of the Actor-Critic model network structure;

[0093] Figure 3 This is a diagram of Actor-Critic network training;

[0094] Figure 4 It is a schematic diagram of the asynchronous action coordination mechanism;

[0095] Figure 5 It is a real-time decision-making flowchart based on the trained stable passenger evacuation model. DETAILED DESCRIPTION

[0096] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0097] The present invention provides a passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning. For a certain station A, the specific flow chart of the implementation is as shown in the attached figure. Figure 1 As shown, the following steps are included:

[0098] S1. Data collection; specifically including:

[0099] S101 collects real-time passenger flow data; real-time passenger flow data includes the train arrival schedule at station A, the number of passengers getting off at station A and their stay time at station A;

[0100] S102. Collect relevant station data: including the maximum passenger capacity of the bus and subway stations related to station A, and the number of waiting passengers at the bus and subway stations related to station A;

[0101] S103. Collect relevant intelligent agent operation data: including the arrival time, departure time and remaining available capacity of each relevant bus and subway at station A; if station A is the departure station, the capacity is equal to the maximum carrying capacity of station A.

[0102] The intelligent agents include buses and subways. Based on the collected data, standard sample data for subsequent analysis, calculation and modeling are constructed and data preprocessing is performed.

[0103] S2. The passenger evacuation problem in an integrated passenger transport hub is described as a Markov game process. The Markov game process model is used to model the decision-making and interaction of multiple agents in the passenger evacuation process. The following are the definitions of the key elements:

[0104] S201: Global state space definition:

[0105] According to the characteristics and needs of passenger evacuation in comprehensive passenger transportation and hubs, its global state space mainly includes three elements: passenger state, related station state and related intelligent agent operation state, which are represented by vector S all =[ s p , s s , s b ] Denotes , which represents the state space shared by all agents (buses and subways) in the environment during the evacuation process;

[0106] Passenger status vector indicates; among them is the time when train h arrives at station A within the time of adjacent decision points, is the number of passengers who get off after train h arrives at station A within the time of adjacent decision points; is the retention time of passenger j at station A; the time of adjacent decision points = the end service time of the previous vehicle - the start service time of the current vehicle;

[0107] Related site status vectors s s =[ s zcd ] indicates; among them is the passenger congestion of the relevant station z within the time of adjacent decision points; is the number of waiting passengers at the relevant station z within the time of adjacent decision points, is the maximum passenger capacity of the relevant station z;

[0108] The vector of the running state of the relevant intelligent agent is s b =[ b ieb , b id ] indicates; among them is the departure time interval of agent i within the adjacent decision points; is the remaining capacity of agent i within the time between adjacent decision points; and The departure time and arrival time of agent i respectively;

[0109] S202: local state space definition;

[0110] The local state of each agent (bus and subway) is the environmental information it perceives (also called observation space), which mainly includes the passenger congestion of the station, the current departure time interval and its remaining capacity, represented by the vector s i = [ s icd , b ieb , b id ] express;

[0111] S203: Action space definition;

[0112] In order to better model the decision-making and interaction of intelligent agents in the process of passenger evacuation, optimize the departure time interval of buses and subway trains, and improve the passenger evacuation efficiency and passenger travel experience of transportation hubs. The action space is defined as a continuous range [0,1], and the product of the maximum departure time interval is the departure time interval of the intelligent agent.

[0113] The action space is defined as: A all =[ a 1 , a 2 , a 3 ,..., a i ], i =1,2,3… I ， a i ∈[0,1] , where each action represents the proportion of the departure time interval selected by agent i; the departure time interval of agent i is expressed as: ,in represents the maximum possible departure time interval of agent i;

[0114] S204: Reward function design

[0115] In multi-agent reinforcement learning, considering full cooperation, it is necessary to balance the performance of the overall system and the contribution of individual agents when designing the reward function to promote cooperation and coordination between agents. This embodiment aims to improve the passenger evacuation efficiency and operational efficiency of the comprehensive passenger transportation hub by dynamically adjusting the departure time intervals of buses and subways. Therefore, in the model of passenger evacuation in the comprehensive passenger transportation hub, there is full cooperation between agents (buses and subways). Improving passenger evacuation efficiency and operational efficiency means reducing the time passengers stay at the station. Therefore, the reward function is directly related to the passenger stay time. The reward function for agent i is defined as:

[0116] (1);

[0117] (2);

[0118] in, Status Next, agent i performs an action Rewards received after , the reward value is 1 minus The average passenger detention time, the smaller the average passenger detention time, the greater the reward value; is the total number of passengers riding agent i in a decision cycle; is a large positive integer; is the total reward of I agents; Formula (1) can be used to evaluate the action of an agent Contribution to the passenger evacuation efficiency of the entire integrated passenger transportation hub;

[0119] S3: Construct a passenger evacuation model for a transportation hub. The passenger evacuation model for a transportation hub consists of an Actor network and a Critic network.

[0120] The Actor-Critic (AC) framework is used to establish a multi-agent decision-making model, and the multi-layer perceptron (MLP) and Self-Attention mechanism are used to establish the function approximator of the Actor network and the Critic network. The model can handle the passenger evacuation problem of comprehensive passenger transportation hubs of different scales; Figure 2 It contains the following key components:

[0121] (1) Actor network design:

[0122] Input layer: The observed state (local state space) of the agent is used as input;

[0123] Hidden layer: Multilayer Perceptron (MLP) and Self-Attention mechanism are used as the hidden layer structure. The specific parameters are as follows:

[0124] - MLP hidden layer 1: 256 neurons, activation function is ReLU;

[0125] - Self-Attention layer: 128 heads, each with a dimension of 64;

[0126] - MLP hidden layer 2: 128 neurons, activation function is ReLU;

[0127] Output layer: The output layer corresponds to the action space of the agent and uses the sigmoid activation function (suitable for continuous action space).

[0128] (2) Critic network design:

[0129] Input layer: The agent’s global state space and action space are used as input;

[0130] Hidden layer: Multilayer Perceptron (MLP) and Self-Attention mechanism are also used as the hidden layer structure. The specific parameters are as follows:

[0131] - MLP hidden layer 1: 256 neurons, activation function is ReLU;

[0132] - Self-Attention layer: 128 heads, each with a dimension of 64;

[0133] - MLP hidden layer 2: 128 neurons, activation function is ReLU;

[0134] Output layer: The output layer is a value function, which represents the Q value of the corresponding state and action.

[0135] Please note that this is just an example and the parameter settings in actual applications may be adjusted and optimized according to the specific task and set of scenarios.

[0136] The network structure is as follows Figure 2 As shown: Each agent has its own online Actor network with parameters and the target Actor network, ; Each agent has its own online critic network with parameters and target critic network ; Therefore, if there are N agents, the transportation hub passenger flow evacuation model has a total of 4N network structures, and the detailed network parameters are:

[0137] Online Actor Network Parameters: ;

[0138] Online Critic Network Parameters: ;

[0139] Target Actor Network Parameters: ;

[0140] Target Critic network parameters: ;

[0141] S4. Introduce an asynchronous action coordination mechanism into the transportation hub passenger flow evacuation model and obtain a stable transportation hub passenger flow evacuation model through training;

[0142] The model is trained using the centralized training distributed execution (CTDE) framework. The agent optimizes its local strategy by obtaining information from other agents, namely the global state space. During the distributed execution process, the agent only uses its own local state space to make decisions. Each agent makes decisions based on the Actor network. On the other hand, the value function of the evaluation state of each agent's Critic network is used to guide the update and optimization of the Actor network.

[0143] The specific training process is as follows Figure 3 As shown:

[0144] S401. Randomly initialize all states; initialize all parameters of the Actor network and Critic network; initialize hyperparameters: learning rate , Discount Factor , Soft Update Parameters and randomly sample experience sample size b; initialize experience replay pool D;

[0145] S402. Each agent interacts with the environment and transforms the currently observed local state space Input online Actor network and get action , The corresponding parameters in the Actor network are , as shown in formula (3); execute the selected action , observe the reward of environmental feedback and the next state ; After n iterations , and Together they constitute the experience sample B;

[0146] (3);

[0147] in is the noise term (using Gaussian noise);

[0148] S403. The experience samples are stored in the experience playback pool. Each experience sample is The composition is as shown in formula (4);

[0149] (4);

[0150] in represents the observation vector of all agents on the current environment state, Represents other global states; Represents the observation vector of all agents to the next state of the environment; when the number of experience samples reaches the threshold D, a number of experience samples b are randomly sampled from the experience replay pool to train the Actor network and the Critic network; Refers to the rewards obtained by the 1st to the Ith agents; Refers to the actions performed by the 1st to the Ith agents;

[0151] S404. “Centralized training”: randomly sample a batch of experience samples b from the experience replay pool, using and They are used as inputs of the online critic network and the target critic network, and output state value functions respectively. and ,in represents all agent actions, Output calculated by the target Actor network; use the global state space and action space of the agent in the environment to centrally train its own Critic network; construct the MSE loss function of the two with the time difference error, and then use gradient descent to update the parameters The specific loss function and gradient are shown in formulas (5) and (6):

[0152] (5);

[0153] (6);

[0154] Where b is the size of the sampled experience sample, α is the learning rate, represents the gradient operation;

[0155] S405. “Distributed Execution”: When computing the forward propagation of its own Actor network, each agent only uses its own local state space As input to the online Actor network, it outputs a deterministic action ,Right now ; Calculate the MSE loss function of the time series difference error and calculate the parameters The gradient of , and then use gradient descent to update the parameters. The loss function and gradient are shown in formulas (7) and (9):

[0156] (7);

[0157] in , I is the number of agents;

[0158] = - 1 b ∑ j=1 b [ ∂ Q ϕ i ∂ θ i ] = - 1 b ∑ j=1 b [ Q ϕ i ( s,a ) ∂ μ θ i ( s i ) μ θ i ( s i ) ∂ θ i ] (8);

[0159] (9);

[0160] S406. Soft update the target actor network and the critic network, as shown in formulas (10) and (11):

[0161] (10);

[0162] (11);

[0163] in, is the loss function of the Actor network, are the parameters of the Critic network after soft update; are the parameters of the Actor network after soft update;

[0164] S407. Repeat S404-S406 until the Actor network and Critic network training are stable, and the training of the transportation hub passenger flow evacuation model is completed.

[0165] Specific:

[0166] In the passenger evacuation model of a transportation hub based on multi-agent reinforcement learning, the different carrying capacities and frequencies of buses and subways, as well as the different number of passengers at different stations, lead to different departure time intervals for each agent. In a synchronous setting, data can only be transmitted and network parameters updated after all agents have completed an action. The model execution speed is limited by the agent with the longest action execution time. Therefore, the time steps of decisions made by different agents are inconsistent, and an asynchronous action coordination mechanism needs to be designed.

[0167] In the asynchronous action coordination mechanism, after each agent completes its own action, it can immediately request data from other agents and update network parameters immediately. This asynchronous setting is more time-saving for multi-agent exploration tasks, and will not be blocked by agents being offline. Asynchronous operation can improve the training efficiency and parallelism of the model; the specific process is as follows:

[0168] Assume that the agent The action of the xth decision was not completed and the xth reward was not obtained. , other agents When making decisions at the current time step, only the agent can be requested The complete data of the x-1th decision is used to update its own strategy, as shown in formula (12):

[0169] S agent i ={[ s 1 i , a 1 i , r 1 i ],[ s 2 i , a 2 i , r 2 i ],...,[ s x-1 i , a x-1 i , r x-1 i ]} (12);

[0170] in, represents the local state space of agent i at the x-1th time, represents the x-1th action of agent i, represents the reward of agent i for the x-1th time.

[0171] Take two agents as an example. Figure 4 As shown: In synchronized action, and Each action must be completed before data can be transmitted and new decisions can be made. The data in its playback pool is in a peer state. In asynchronous actions, and After completing its own actions, it can request data from other agents and make new decisions immediately. The data in its playback pool is in an unequal state. Only 2 groups can be requested data, and You can request 3 groups This asynchronous setting is more time-saving for multi-agent exploration tasks and will not be blocked by agents going offline.

[0172] S5. Realize passenger evacuation of station A based on the stable passenger evacuation model of the transportation hub.

[0173] After the passenger evacuation model based on multi-agent reinforcement learning is trained and stabilized, the passenger evacuation efficiency and operational efficiency of the comprehensive passenger transportation hub can be improved by dynamically adjusting the departure time intervals of buses and subways. Figure 5 As shown:

[0174] S501: Collecting real-time status data of the comprehensive passenger transportation hub, performing data preprocessing, and forming a standardized data sample vector;

[0175] S502: extracting passenger, train and station related data within the current decision cycle t, and generating a global state space and a local state space;

[0176] S503: extracting the number of agents in the current decision cycle t;

[0177] S504: All global / local state space data and action state data in the current decision cycle t are transmitted to the stable passenger flow evacuation model, and the optimal departure time interval of each agent is output, and feedback data is collected after the action is executed;

[0178] S505: Repeat S501-S504 until the corresponding decision cycle T ends.

[0179] An embodiment of the present invention further provides a passenger flow evacuation system for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, comprising:

[0180] A data collection module is used to collect the real-time status data of station A and perform data preprocessing, the real-time status data includes real-time passenger flow data, relevant station data and relevant intelligent agent data, and the intelligent agents include buses and subways;

[0181] The Markov game module is used to describe the passenger evacuation problem of the comprehensive passenger transportation hub as a Markov game process; it includes defining the global state space, defining the local state space, defining the action space and setting the reward function; the global state space includes the passenger state, the relevant station state and the relevant agent operation state; the local state space includes the passenger congestion of the station, the current departure time interval and its remaining capacity; the action space is defined as the product of a continuous range [0,1] and the maximum departure time interval

[0182] A model building module, used to build a passenger flow evacuation model for a transportation hub, wherein the passenger flow evacuation model for a transportation hub is composed of an Actor network and a Critic network;

[0183] Model training module, which is used to introduce asynchronous action coordination mechanism into the transportation hub passenger flow evacuation model and obtain a stable transportation hub passenger flow evacuation model through training;

[0184] The evacuation module is used to realize the passenger evacuation of station A through a stable transportation hub passenger evacuation model.

[0185] Preferably, the embodiments of the present application also provide a specific implementation of an electronic device capable of implementing all steps in the passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning in the above embodiments, and the electronic device specifically includes the following contents:

[0186] Processor, memory, communications interface and bus;

[0187] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.

[0188] The processor is used to call the computer program in the memory, and when the processor executes the computer program, it implements all the steps in the passenger evacuation method for the comprehensive passenger transportation hub based on passenger flow data and reinforcement learning in the above embodiment.

[0189] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method for evacuating passenger flow in a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning in the above embodiments. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all the steps of the method for evacuating passenger flow in a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning in the above embodiments.

[0190] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0191] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0192] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or client product in practice is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0193] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0196] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solution of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, a person of ordinary skill in the art can also make many forms of specific changes under the guidance of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, for a certain station A, characterized in that: include: S1. Collect the real-time status data of site A and perform data preprocessing. The real-time status data includes real-time passenger flow data, relevant site data and relevant agent data, and the agent includes buses and subways; S2. Describe the passenger evacuation problem of integrated passenger transportation hub as a Markov game process; Including defining the global state space, defining the local state space, defining the action space and setting the reward function; The global state space includes passenger states, related station states and related agent operation states; The local state space includes the passenger congestion of the station, the current departure time interval and its remaining capacity; The action space is defined as the product of a continuous range [0, 1] and the maximum departure time interval; The global state space is represented by the vector S all =[s p ,s s ,s b ] represents the state space shared by all agents in the environment during the evacuation process; The passenger state is represented by vector s p =[p ht , p htp , p jwt ] indicates; where p ht is the time for train h to arrive at station A within the adjacent decision point time, p htp is the number of passengers getting off after train h arrives at station A within the time between adjacent decision points; p jwt is the time that passenger j stays at station A; the time of adjacent decision points = the end time of the previous vehicle - the start time of the current vehicle; The relevant site status is represented by vector s s =[s zcd ] indicates; where s zcd =s zwp / s zpc is the passenger congestion degree of the relevant station z within the time of adjacent decision points; s zwp is the number of waiting passengers at the relevant station z within the time between adjacent decision points, s zpc is the maximum passenger capacity of the relevant station z; The running state of the relevant intelligent agent is represented by vector s b =[b ieb , b id ] indicates; where b ieb =b idb -b ide is the departure time interval of agent i within the adjacent decision points; b id is the remaining capacity of agent i within the time between adjacent decision points; b idb and b ide The departure time and arrival time of agent i respectively; The local state space of each agent represents the environmental information it perceives. The local state space of each agent is represented by the vector s i =[s zcd , b ieb , b id ]express; The action space is defined as: A all =[a1, a2, a3, ..., a i ], i = 1, 2, 3...I, a i ∈[0, 1]; where each action a i represents the proportion of the departure time interval selected by agent i; the departure time interval of agent i is expressed as: T i =a i ·T imax , where T imax represents the maximum possible departure time interval of agent i; Assuming that all agents are fully cooperative, the reward function for agent i is defined as: in, Denotes the global state space S all Perform action a on agent i i Rewards received after The reward value is 1 minus The average passenger detention time, the smaller the average passenger detention time, the greater the reward value; J is the total number of passengers riding agent i in a decision cycle; is a positive integer; R t (S all , A all ) is the total reward of I agents; Formula (1) is used to evaluate the action a of an agent i i Contribution to the passenger evacuation efficiency of the entire comprehensive passenger transportation hub; jwt is the time passenger j spends at the station; S3. Constructing a passenger flow evacuation model for a transportation hub, wherein the passenger flow evacuation model for the transportation hub is composed of an Actor network and a Critic network; S4. Introduce an asynchronous action coordination mechanism into the transportation hub passenger flow evacuation model and obtain a stable transportation hub passenger flow evacuation model through training; The centralized training and distributed execution CTDE framework is used to train the traffic hub passenger evacuation model. The agent optimizes its local strategy by obtaining the global state space of other agents. During the distributed execution process, the agent only uses its own local state space to make decisions. Each agent makes decisions based on the Actor network. On the other hand, the value function of the evaluation state of each agent's Critic network is used to guide the update and optimization of the Actor network. The specific training process is as follows: S401. Randomly initialize all states; initialize all parameters of the Actor network and the Critic network; initialize hyperparameters: learning rate α, discount factor γ, soft update parameter τ and random sampling experience sample size b; initialize the experience replay pool D; S402. Each agent interacts with the environment and transforms the currently observed local state space s i Input the online Actor network and get action a i , a i The corresponding parameters in the Actor network are As shown in formula (3), the selected action a is executed. i , observe the reward of environmental feedback and the next state s′ i ; After n iterations a i , and s′ i Together they constitute the experience sample B; Where ∈ is the noise term, using Gaussian noise; S403. Store the experience samples in the experience replay pool, where each experience sample is represented by S, as shown in formula (4); S={s,s′,a1,a2,…,a I ,r1,r2,…,r I } (4); where s={s1,s2,...s n , S g } represents the observation vector of all agents on the current environment state, S g represents other global states; s′ represents the observation vector of all agents on the next state of the environment; when the number of experience samples reaches the threshold D, a number of experience samples b are randomly sampled from the experience replay pool to train the Actor network and the Critic network; r1, r2, ..., r I Refers to the rewards obtained by the 1st to the Ith agents; a1, a2, ..., a I Refers to the actions performed by the 1st to the Ith agents; S404. Centralized training: Randomly sample several experience samples b from the experience replay pool, use (s, a) and (s′, a′) as the input of the online critic network and the target critic network respectively, and output the state value function and where a={a1,a2,...,a I }, a′ is calculated and output by the target Actor network; the global state space and action space of the agent in the environment are used to centrally train the Critic network; the MSE loss function of the Actor network and the Critic network is constructed using the temporal difference error, and then the parameters φ of the Critic network are updated using gradient descent i ; The specific loss function and gradient are shown in formulas (5) and (6): Where b is the size of the random sampling experience sample; L iCritic is the loss function of the Critic network; r i Represents the reward obtained by the i-th agent; represents the gradient operator; S405. Distributed execution: When calculating the forward propagation of its own Actor network, each agent only uses its own local state space s i As input to the online Actor network, it outputs a deterministic action a i ,Right now Calculate the MSE loss function of the temporal difference error and calculate the parameters θ about the Actor network i The gradient of , and then use gradient descent to update the parameters; the loss function and gradient are shown in formulas (7) and (9): in I is the number of agents; S406. Soft update the target Actor network and Critic network, as shown in formulas (10) and (11): Among them, L iActor is the loss function of the Actor network, are the parameters of the Critic network after soft update; are the parameters of the Actor network after soft update; S407. Repeat S404-S406 until the Actor network and the Critic network training are stable, and the training of the transportation hub passenger flow evacuation model is completed; The asynchronous action coordination mechanism means that after each agent completes its own action, it can immediately request data from other agents and immediately update network parameters. The specific process is as follows: Assume that the agent i The action of the xth decision was not completed and the xth reward was not obtained. Other intelligent agents -i When making a decision at the current time step, only the agent can be requested i The complete data of the x-1th decision is used to update its own strategy, as shown in formula (12): in, represents the local state space of agent i at the x-1th time, represents the x-1th action of agent i, represents the reward of agent i for the x-1th time; S5. Realize passenger evacuation of station A based on the stable passenger evacuation model of the transportation hub.

2. The passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning according to claim 1, characterized in that: The real-time passenger flow data in step S1 includes the timetable of trains arriving at station A, the number of passengers getting off at station A, and the time they stay at station A; The relevant station data include the maximum passenger capacity of the bus and subway stations related to station A, and the number of waiting passengers at the bus and subway stations related to station A; The relevant intelligent agent operation data includes the arrival time, departure time and remaining available capacity of each relevant bus and subway at station A; if station A is the starting station, the capacity is equal to the maximum carrying capacity of station A.

3. The passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning according to claim 1, characterized in that: The Actor network in step S3 includes: The input layer takes the local state space of the agent as input; Hidden layer, using multi-layer perceptron MLP and Self-Attention mechanism as the hidden layer structure; Activation function, ReLU activation function is used after each hidden layer; Output layer: The output layer corresponds to the action space of the agent and uses the sigmoid activation function; The Critic network includes: The input layer takes the agent’s global state space and action space as input; Hidden layer, using multi-layer perceptron MLP and Self-Attention mechanism as the hidden layer structure; Activation function, ReLU activation function is usually used after each hidden layer; Output layer, the output layer is a value function, which represents the Q value of the corresponding state and action.

4. The passenger evacuation method for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning according to claim 1, characterized in that: The specific evacuation process in step S5 is as follows: S501. Collect real-time status data of the comprehensive passenger transportation hub, perform data preprocessing, and form a standardized data sample vector; S502. Extract the passenger, train and station related data within the current decision cycle t to generate a global state space and a local state space; S503. Extract the number of agents in the current decision cycle t; S504. All global state space data and local state space data in the current decision cycle t are transmitted to the decision model of the stable traffic hub passenger flow evacuation model, and the optimal departure time interval of each agent is output, and action a i Collect feedback data after execution; S505. Repeat S501-S504 until the corresponding decision cycle T ends.

5. A passenger evacuation system for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, characterized in that: include: A data collection module is used to collect the real-time status data of station A and perform data preprocessing, the real-time status data includes real-time passenger flow data, relevant station data and relevant intelligent agent data, and the intelligent agents include buses and subways; Markov game module, which is used to describe the passenger evacuation problem of integrated passenger transportation hub as a Markov game process; It includes defining a global state space, defining a local state space, defining an action space and setting a reward function; the global state space includes passenger states, related station states and related agent operation states; the local state space includes station passenger congestion, current departure time interval and its remaining capacity; the action space is defined as the product of a continuous range [0, 1] and the maximum departure time interval; The global state space is represented by the vector S all =[s p ,s s ,s b ] represents the state space shared by all agents in the environment during the evacuation process; The passenger state is represented by vector s p =[p ht , p htp , p jwt ] indicates; where p ht is the time for train h to arrive at station A within the adjacent decision point time, p htp is the number of passengers getting off after train h arrives at station A within the time between adjacent decision points; p jwt is the time that passenger j stays at station A; the time of adjacent decision points = the end time of the previous vehicle - the start time of the current vehicle; The relevant site status is represented by vector s s =[s zcd ] indicates; where s zcd =s zwp / s zpc is the passenger congestion degree of the relevant station z within the time of adjacent decision points; s zwp is the number of waiting passengers at the relevant station z within the time between adjacent decision points, s zpc is the maximum passenger capacity of the relevant station z; The running state of the relevant intelligent agent is represented by vector s b =[b ieb , b id ] indicates; where b ieb =b idb -b ide is the departure time interval of agent i within the adjacent decision points; b id is the remaining capacity of agent i within the time between adjacent decision points; b idb and b ide The departure time and arrival time of agent i respectively; The local state space of each agent represents the environmental information it perceives. The local state space of each agent is represented by the vector s i =[s zcd , b ieb , b id ]express; The action space is defined as: A all =[a1, a2, a3, ..., a i ], i = 1, 2, 3...I, a i ∈[0, 1]; where each action a i represents the proportion of the departure time interval selected by agent i; the departure time interval of agent i is expressed as: T i =a i ·T imax , where T imax represents the maximum possible departure time interval of agent i; Assuming that all agents are fully cooperative, the reward function for agent i is defined as: in, Denotes the global state space S all Perform action a on agent i i Rewards received after The reward value is 1 minus The average passenger detention time, the smaller the average passenger detention time, the greater the reward value; J is the total number of passengers riding agent i in a decision cycle; is a positive integer; R t (S all , A all ) is the total reward of I agents; Formula (1) is used to evaluate the action a of an agent i i Contribution to the passenger evacuation efficiency of the entire comprehensive passenger transportation hub; jwt is the time passenger j spends at the station; Model building module, used to build a transportation hub passenger flow evacuation model, the transportation hub passenger flow evacuation model consists of Actor network and Criti c Network composition; The model training module is used to introduce an asynchronous action coordination mechanism into the traffic hub passenger evacuation model and obtain a stable traffic hub passenger evacuation model through training; the centralized training distributed execution CTDE framework is used to train the traffic hub passenger evacuation model, and the agent optimizes its local strategy by obtaining the global state space of other agents; during the distributed execution process, the agent only uses its own local state space to make decisions; each agent makes decisions based on the Actor network; on the other hand, the value function of the evaluation state of each agent's Critic network is used to guide the update and optimization of the Actor network; the specific training process is as follows: S401. Randomly initialize all states; initialize all parameters of the Actor network and the Critic network; initialize hyperparameters: learning rate α, discount factor γ, soft update parameter τ and random sampling experience sample size b; initialize the experience replay pool D; S402. Each agent interacts with the environment and transforms the currently observed local state space s i Input the online Actor network and get action a i , a i The corresponding parameters in the Actor network are As shown in formula (3), the selected action a is executed. i , observe the reward of environmental feedback and the next state s′ i ; After n iterations a i , and s′ i Together they constitute the experience sample B; Where ∈ is the noise term, using Gaussian noise; S403. Store the experience samples in the experience playback pool, where each experience sample is represented by S, as shown in formula (4); S={s,s′,a1,a2,…,a I ,r1,r2,…,r I } (4); where s={s1,s2,...s n , S g } represents the observation vector of all agents on the current environment state, S g represents other global states; s′ represents the observation vector of all agents on the next state of the environment; when the number of experience samples reaches the threshold D, a number of experience samples b are randomly sampled from the experience replay pool to train the Actor network and the Critic network; r1, r2, ..., r I Refers to the rewards obtained by the 1st to the Ith agents; a1, a2, ..., a I Refers to the actions performed by the 1st to the Ith agents; S404. Centralized training: Randomly sample several experience samples b from the experience replay pool, use (s, a) and (s′, a′) as the input of the online critic network and the target critic network respectively, and output the state value function and where a={a1,a2,...,a I }, a′ is calculated and output by the target Actor network; the global state space and action space of the agent in the environment are used to centrally train the Critic network; the MSE loss function of the Actor network and the Critic network is constructed using the temporal difference error, and then the parameters φ of the Critic network are updated using gradient descent i ; The specific loss function and gradient are shown in formulas (5) and (6): Where b is the size of the random sampling experience sample; L iCritic is the loss function of the Critic network; r i Represents the reward obtained by the i-th agent; represents the gradient operator; S405. Distributed execution: When calculating the forward propagation of its own Actor network, each agent only uses its own local state space s i As input to the online Actor network, it outputs a deterministic action a i ,Right now Calculate the MSE loss function of the temporal difference error and calculate the parameters θ about the Actor network i The gradient of , and then use gradient descent to update the parameters; the loss function and gradient are shown in formulas (7) and (9): in I is the number of agents; S406. Soft update the target Actor network and Critic network, as shown in formulas (10) and (11): Among them, L iActor is the loss function of the Actor network, are the parameters of the Critic network after soft update; are the parameters of the Actor network after soft update; S407. Repeat S404-S406 until the Actor network and the Critic network training are stable, and the training of the transportation hub passenger flow evacuation model is completed; The asynchronous action coordination mechanism means that after each agent completes its own action, it can immediately request data from other agents and immediately update network parameters. The specific process is as follows: Assume that the agent i The action of the xth decision was not completed and the xth reward was not obtained. Other agents -i When making a decision at the current time step, only the agent can be requested i The complete data of the x-1th decision is used to update its own strategy, as shown in formula (12): in, represents the local state space of agent i at the x-1th time, represents the x-1th action of agent i, represents the reward of agent i for the x-1th time; The evacuation module is used to realize the passenger evacuation of station A through a stable transportation hub passenger evacuation model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the passenger evacuation method for an integrated passenger transportation hub based on passenger flow data and reinforcement learning as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the passenger evacuation method for an integrated passenger transportation hub based on passenger flow data and reinforcement learning as described in any one of claims 1 to 4 are implemented.

Images