Collaborative optimization method of passenger flow control and stop plan for urban rail transit based on deep reinforcement learning

Abstract

The present invention discloses a method for collaborative optimization of passenger flow control and stop plan of urban rail transit based on deep reinforcement learning, including: S1. Collecting real-time status data of urban rail transit system; S2. Data preprocessing; including data cleaning, data format standardization and data normalization; S3. Describing the collaborative optimization process of passenger flow control and stop plan of urban rail transit as a Markov game process; S4. Constructing a collaborative optimization model of passenger flow control and stop plan; S5. Training the collaborative optimization model of passenger flow control and stop plan; S6. Realizing real-time decision-making on passenger flow control and stop plan of urban rail transit based on the trained collaborative optimization model of passenger flow control and stop plan. The method of the present invention is adaptive and flexible, and can dynamically adjust the passenger flow control rate and train skipping mode of each O‑D (Origin‑Destination, O‑D) according to the real-time passenger flow situation and train capacity, so as to improve the operation efficiency of the urban rail transit system and the safety of passengers' travel.

Description

Technical Field

[0001] The present invention relates to the technical field of urban rail transit operation management, and in particular to a method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning. Background Art

[0002] As an efficient, fast and environmentally friendly mode of transportation, urban rail transit systems have become an indispensable and important part of modern cities. However, with the continuous growth of urban population and transportation demand, rail transit systems are facing increasing challenges, especially congestion during peak passenger flow periods, passenger safety hazards and uneven utilization of train capacity. The passenger flow control strategy for entering the station can help alleviate the congestion of urban rail transit stations and ensure the safety of passengers, while the flexible adjustment of train stop plans (i.e., skip stop mode) can help balance and improve the utilization of the existing train capacity. Therefore, how to coordinately optimize the passenger flow control strategy and train stop plan of urban rail transit to improve the operating efficiency of the urban rail transit system and the travel safety and experience of passengers has become an urgent problem to be solved.

[0003] At present, the optimization methods of passenger flow control strategy and train stop plan for urban rail transit system have the following defects:

[0004] (1) Traditional passenger flow control and stop plans are often formulated based on experience and rules. It is difficult to take all factors and variables into account, resulting in the formulation of passenger flow control and stop plans being unscientific and inaccurate, and lacking flexibility and adaptability.

[0005] (2) Traditional passenger flow control and stop plan adjustment are usually implemented separately, lacking comprehensive consideration of the overall performance and dynamic performance of the urban rail transit system;

[0006] (3) The primary passenger flow control and stop plan coupling optimization method only considers the simple combination of passenger flow control intensity and stop plan, lacks precise control of passenger flow during peak hours, and limits the further balance and improvement of train capacity utilization. In summary, in order to ensure the safety of passengers and further improve the efficiency of train operation, it is necessary to design a new method for the coordinated optimization of passenger flow control strategy and train stop plan. Summary of the invention

[0007] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for collaborative optimization of passenger flow control and stop plans for urban rail transit based on deep reinforcement learning. The method uses deep reinforcement learning technology to comprehensively consider the coupling relationship between each OD (Origin-Destination, OD) on the passenger flow control rate and the train skipping mode, and self-learns the optimal strategy through iterative training to achieve flexible adaptive collaborative optimization. Through this method, the operating efficiency and service quality of the urban rail transit system can be improved while considering the overall performance and dynamic performance, providing passengers with a safer travel experience.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] A method for collaborative optimization of passenger flow control and stop plan for urban rail transit based on deep reinforcement learning, comprising:

[0010] S1. Collect real-time status data of urban rail transit system, including train operation data, passenger flow data and station platform data;

[0011] S2. Data preprocessing, including data cleaning, data format standardization and data normalization;

[0012] S3. Describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process, including defining the state space, defining the action space, setting the reward function and setting the strategy function; the state space includes the train running state, passenger state and station platform state;

[0013] S4. Constructing a collaborative optimization model of passenger flow control and stop plan, wherein the collaborative optimization model of passenger flow control and stop plan consists of an Actor network and a Critic network;

[0014] S5. Train the passenger flow control and stop plan collaborative optimization model;

[0015] S6. Realize real-time decision-making on passenger flow control and stop plan of urban rail transit based on the trained collaborative optimization model of passenger flow control and stop plan.

[0016] Furthermore, the train operation data in step S1 includes the remaining available capacity of each carriage upon arrival at each station, the arrival time and the departure time;

[0017] Passenger flow data include the total number of OD pairs on a rail transit line, the number of passengers entering each station for all OD pairs, the number of passengers transferring for all OD pairs at each station, and the number of times each passenger stays at the platform;

[0018] Station platform data includes the maximum passenger capacity of each platform on a rail transit line and the number of passengers waiting for all OD pairs at each platform.

[0019] Further, in step S2,

[0020] Data cleaning is to deal with missing values, outliers and duplicate values. For missing values, column mean is used to fill in the data to ensure data integrity. For outliers, the standard deviation of the data is calculated to identify outliers, and column mean is used to replace the data to avoid affecting subsequent analysis. For duplicate values, data deduplication is performed to ensure data uniqueness.

[0021] Data format standardization includes data format conversion, unit conversion, and encoding conversion;

[0022] Data normalization uses Max-Min deviation standardization to normalize the data set and map all attribute data to the [0,1] interval to avoid training bias in subsequent training.

[0023] Further, in step S3,

[0024] S301. Define the state space:

[0025] The state space of the urban rail transit system includes three elements: train running state, passenger state and station platform state, which are represented by vector S=[ s t , s p , s s ] express;

[0026] Train running status vector s t =[ t id , t ide , t idb ] express;

[0027] in is the average remaining available capacity when train d arrives at station i, is the remaining available capacity of the jth carriage of train d, and n means that train d has n carriages; is the time when train d arrives at station i; is the time when train d leaves station i;

[0028] Passenger status vector s p =[ p iodp , p iodt , p cwf ] express;

[0029] in is the set of the number of passengers entering the station for each OD pair at station i within the time of adjacent decision points; is the set of transfer numbers of passengers for each OD pair at station i within the time of adjacent decision points; is the cumulative number of times passenger c stayed on the platform;

[0030] Station platform status vector s s =[ s iodd ] express;

[0031] in is the passenger density set of each OD pair of stations i within the time of adjacent decision points, is the number of passengers waiting on the platform for each OD pair at station i within the time between adjacent decision points, is the platform capacity of station i;

[0032] S302. Define action space:

[0033] The action space of the urban rail transit system is formed by the combination of passenger flow control rate and skipping mode of each OD, which is represented by the vector express: a=[( p iod1 cr im , p iod2 cr im , p iod3 cr im ,... , p iodn cr im ), st id ] ;

[0034] in, For the passenger flow of the nth OD pair at station i, the OD pair passenger flow is grouped, and the grouping function is shown in formula (1); cr im =[100%,75%,50%,25%,0] is the passenger flow control rate of station i, 100% means no passenger flow control is performed during the adjacent decision point time, 75% means the number of passengers entering station i during the adjacent decision point time is 75% of the original, ..., 0 means passengers are prohibited from entering station i during the adjacent decision point time; is the stop mode of train d at station i, 0 means no skipping, 1 means skipping;

[0035] (1);

[0036] (2);

[0037] in, is the function that divides the original OD pairs of station i into m groups, is the number of passengers of the kth OD pair at station i within the time of adjacent decision points; is a binary variable. If the kth group of OD pairs has been grouped, then ,otherwise ; is the OD pair with the maximum number of passengers at station i within the time of adjacent decision points; Formula (2) is the grouping constraint; K is the total number of OD pairs;

[0038] S303. Set reward function;

[0039] The reward function is directly related to the number of times a passenger stays at the platform. For station i, the reward function is defined as:

[0040] (3);

[0041] in, is the state of the urban rail transit system at time t Next action The reward obtained by the latter station i is the inverse of the total number of passengers’ detentions. The smaller the total number of detentions, the greater the reward value. t is the latest decision time. is a binary variable. If passenger c is stranded at station i at decision time t, then ,otherwise ; C is the maximum number of passengers at station i; is the cumulative number of times passenger c stayed on the platform;

[0042] S304. Set the strategy function;

[0043] Policy Neural Network It is a neural network that maps the state of the urban rail transit system at time t to an action. are the parameters of the neural network; use the strategy neural network Approximate policy function ; And its loss function The definition is as follows:

[0044] (4);

[0045] L i (θ)=E[ ∑ an=1 an max -1 log π iθ ( S t , a t )∙( Q i ( S t , a t )- V i ( S t )) ] (5);

[0046] in, Represents the state of the urban rail transit system at time t Next action space The probability distribution function of Represents the state of the urban rail transit system at time t Take action The action value function of for The state value function of It is expressed as the maximum number of actions, E represents the mean square error, log represents the natural logarithm; an represents the number of actions;

[0047] According to formula (5), two neural networks are needed to estimate and , according to the Bellman equation:

[0048] Q i ( S t , a t )=E[ R i ( S t , a t )+γ V i ( S t+1 )] (6);

[0049] in, Represents the state of the urban rail transit system at time t+1 The state value function, γ is the discount factor, and E represents the mean square error; is the state of the urban rail transit system at time t Next action The reward obtained by the next station i; Substituting formula (6) into (5) we get The final loss function :

[0050] L i (θ) = E[ ∑ an=1 an max -1 log π iθ ( S t , a t )∙( R i ( S t , a t )+γ V i ( S t+1 )- V i ( S t )) ] (7);

[0051] In the case of formula (7), only one neural network is needed to estimate and .

[0052] Further, in step S4,

[0053] Use the pointer network structure to establish an Actor network to generate the probability distribution of actions , the Actor network includes:

[0054] Input layer, including action space data and all state space data;

[0055] Encoder: The encoder uses a recurrent neural network (RNN) to encode the input data into a fixed-length vector representation.

[0056] Attention mechanism, which is used to calculate the attention weight of each input vector through the attention mechanism;

[0057] The decoder uses a recurrent neural network (RNN) combined with an attention mechanism to generate the output sequence, and uses a mask layer and a softmax activation function to generate the final output;

[0058] The Critic network is based on a recurrent neural network (RNN) and is used to estimate the current state of the urban rail transit system. The state value function , which is further used to update the Actor network gradient.

[0059] Furthermore, in step S5, the Actor network in the passenger flow control and stop plan collaborative optimization model is used to select actions, and the Critic network is used to evaluate the value function of the urban rail transit system state; the Actor network selects actions according to the state of the urban rail transit system at time t; the Critic network estimates its value function according to the state of the urban rail transit system at time t, that is, predicts the cumulative reward of the corresponding state; the specific training process is as follows:

[0060] S501. Randomly initialize all states; initialize the experience replay pool D; initialize all parameters in the Actor network and Critic network and ;

[0061] S502. The Actor network interacts with the environment to transform the state of the urban rail transit system at time t Input Actor network, output action probability distribution ;

[0062] S503. Action probability distribution according to Actor network output , use the mask layer and softmax function to classify the action tensor and output the combined action with the highest probability , execute the selected action , observe the reward of environmental feedback and the state of the urban rail transit system at time t+1 ; The state of the urban rail transit system at time t ,action ,award , the state of the urban rail transit system at time t+1 Together they form the experience sample;

[0063] S504. The experience samples are stored in the experience replay pool; when the number of experience samples reaches the threshold D, a number of experience samples are randomly sampled from the experience replay pool for training the Actor network and the Critic network;

[0064] S505. Used in the Critic network and As input, the output state value function and ;

[0065] S506. Calculate the Critic network loss function, as shown in formula (8); use the gradient descent method to update the parameters of the Critic network by minimizing the loss function , the update method is shown in formula (9);

[0066] L i (ω)= E[ R i ( S t , a t )+γ V i ( S t+1 )- V i ( S t )] 2 (8);

[0067] (9);

[0068] in, is the critic network update parameter The learning rate, γ is the discount factor, ∇ represents the gradient operator, and E represents the mean square error;

[0069] S507. State value function calculated using the Critic network and and the probability distribution of the selected action Calculate the loss function of the Actor network; use the gradient ascent method to update the parameters of the Critic network by maximizing the expected return , the update method is shown in formula (10);

[0070] (10);

[0071] S508. Repeat S502-S507 until the Actor network and the Critic network training are stable; complete the training of the passenger flow control and stop plan collaborative optimization model; wherein, is the Actor network update parameter The learning rate is , and ∇ represents the gradient operator.

[0072] Furthermore, the specific decision-making process in step S6 is as follows:

[0073] S601. Collecting real-time status data of urban rail transit system, performing data preprocessing to form a standardized data vector;

[0074] S602. Extract passengers, trains and stations within a single decision cycle t to generate a state space; the decision cycle is the interval between train d and train d-1 arriving at station i;

[0075] S603 extracts passenger flow data of each station OD within a single decision cycle t, and performs combined classification of OD pairs;

[0076] S604. Determine whether the number of passenger flows of various types of OD after grouping meets the constraints. If so, generate the corresponding action space, otherwise go to S603;

[0077] S605. All state space data and action space data are transmitted to the stable passenger flow control and stop plan collaborative optimization model, and the optimal passenger flow control and stop plan collaborative action is output, and feedback data is collected after the action is executed;

[0078] S606. Repeat S601-S605 until the corresponding decision cycle T ends.

[0079] The present invention also provides a comprehensive passenger transportation hub passenger evacuation system based on passenger flow data and reinforcement learning, comprising:

[0080] Data acquisition module, used to collect real-time status data of urban rail transit system, including train operation data, passenger flow data and station platform data;

[0081] Data preprocessing module, used for data cleaning, data format standardization and data normalization;

[0082] The Markov game module is used to describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process; it includes defining the state space, defining the action space, setting the reward function and setting the strategy function; the state space includes the train running state, passenger state and station platform state;

[0083] A model building module, used to build a passenger flow control and stop plan collaborative optimization model, wherein the passenger flow control and stop plan collaborative optimization model is composed of an Actor network and a Critic network;

[0084] Model training module, used to train the passenger flow control and stop plan collaborative optimization model and obtain a stable passenger flow control and stop plan collaborative optimization model;

[0085] The real-time decision-making module is used to make real-time decisions on passenger flow control and stop plans for urban rail transit based on the trained passenger flow control and stop plan collaborative optimization model.

[0086] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for collaborative optimization of passenger flow control and stop plans for urban rail transit based on deep reinforcement learning are implemented.

[0087] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning.

[0088] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0089] 1. Improve operational efficiency and passenger safety:

[0090] Through the coordinated optimization of passenger flow control and stop plan, the present invention can effectively alleviate the congestion of urban rail transit system during peak hours. The optimized stop plan and passenger flow control strategy can balance the capacity of the train, enable more passengers to be transported effectively, reduce passenger waiting time, and improve overall operational efficiency.

[0091] By controlling the boarding ratio of each OD pair, the crowding of platforms and carriages can be reduced, thereby improving passenger safety. Passengers can obtain more fair and balanced public transportation services, which improves the satisfaction and convenience of the urban rail transit system.

[0092] 2. Real-time adaptive decision making:

[0093] The present invention combines deep reinforcement learning methods with pointer network structures, giving full play to the advantages of both. Reinforcement learning can achieve real-time adaptive decision-making in complex rail transit systems by learning optimal decision-making strategies through interaction with the environment. The pointer network structure can model and process input sequences and adapt to different input and output sequences, so that the method of the present invention has strong adaptability and scalability. Therefore, the method of the present invention can be applied to the collaborative optimization problem of passenger flow control and stop schemes in different urban rail transit systems, providing a general paradigm framework for solving the collaborative optimization problem of passenger flow control and stop schemes in urban rail transit systems;

[0094] This method can be dynamically adjusted according to the real-time collected data, realize real-time decision-making and output of the optimal solution, and improve the resilience and efficiency of the urban rail transit system.

[0095] 3. Improve optimization performance and efficiency:

[0096] The present invention designs a mask layer for the encoder output position of the actor network, transforming the action space from the original m n The order of magnitude has been reduced to the m*n order of magnitude, which greatly reduces the complexity of calculation and search and improves the training efficiency and decision-making efficiency of the method.

[0097] The action space is classified according to the constraints in tensor form, and then the optimal action combination is quickly determined through the mask layer (0-1 mask mechanism) and softmax function to improve the accuracy and efficiency of decision-making. This design can effectively optimize the action selection process and improve the performance of the overall optimization method.

[0098] In summary, the method of the present invention can improve the operating efficiency and service quality of the urban rail transit system while taking into account the overall performance and dynamic performance, providing passengers with a safer, more comfortable and efficient travel experience, which has significant practical application value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 It is a specific flow chart of an embodiment of the present invention.

[0100] Figure 2 It is a schematic diagram of OD grouping and action space establishment.

[0101] Figure 3 This is a schematic diagram of the combined action of the mask layer and the softmax function output.

[0102] Figure 4 It is a diagram of the training of the Actor network and the Critic network.

[0103] Figure 5 It is a real-time decision-making flowchart based on a stable collaborative optimization model. DETAILED DESCRIPTION

[0104] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0105] The present invention provides a method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning. The specific flow chart of the implementation is shown in the attached figure. Figure 1 As shown, the following steps are included:

[0106] S1. Data collection: collect real-time status data of urban rail transit system, including train operation data, passenger flow data and station platform data;

[0107] The train operation data includes the remaining available capacity of each car when arriving at each station, the arrival time and the departure time; the passenger flow data includes the total number of OD pairs on a rail transit line, the number of passengers entering the station for all OD pairs at each station, the number of passengers transferring for all OD pairs at each station, and the number of times each passenger stays on the platform; the station platform data includes the maximum passenger capacity of each platform on a rail transit line and the number of passengers waiting for all OD pairs at each platform; based on the above collected data, standard sample data is constructed for subsequent analysis, calculation and modeling.

[0108] S2. Data Preprocessing

[0109] The original sample data of the urban rail transit system contains a lot of noise. In order to ensure the quality and reliability of the data, so that the subsequent analysis and modeling can obtain accurate and reliable results, the following are the steps for data preprocessing:

[0110] S201. Data cleaning;

[0111] Handle missing values, outliers, and duplicate values. For missing values, use the column mean to fill in the data to ensure data integrity. For outliers, calculate the standard deviation of the data to identify outliers and use the column mean to replace them to avoid affecting subsequent analysis. For duplicate values, perform data deduplication to ensure data uniqueness.

[0112] S202. Data format standardization;

[0113] Perform data format conversion, unit conversion, and coding conversion to improve data consistency and comparability, making data easier to process and analyze.

[0114] S203. Data normalization;

[0115] Max-Min deviation standardization is used to normalize the data set and map all attribute data to the interval [0,1] to avoid training bias caused by the dominance of certain dimensions in subsequent training.

[0116] S3. Describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process, including defining the state space, defining the action space, setting the reward function and setting the strategy function; the state space includes the train running state, passenger state and station platform state;

[0117] S301. State space definition;

[0118] According to the characteristics and requirements of the urban rail transit system, its state space mainly includes three elements: train running state, passenger state and station platform state, which are represented by vector S=[ s t , s p , s s ] express;

[0119] Train running status vector s t =[ t id , t ide , t idb ] express;

[0120] in is the average remaining available capacity when train d arrives at station i, is the remaining available capacity of the jth carriage of train d, and n means that train d has n carriages; is the time when train d arrives at station i; is the time when train d leaves station i;

[0121] Passenger status vector s p =[ p iodp , p iodt , p cwf ] express;

[0122] in is the set of the number of passengers entering the station for each OD pair at station i within the time of adjacent decision points; is the set of transfer numbers of passengers for each OD pair at station i within the time of adjacent decision points; is the cumulative number of times passenger c stayed on the platform;

[0123] Station platform status vector s s =[ s iodd ] express;

[0124] in is the passenger density set of each OD pair of stations i within the time of adjacent decision points, is the number of passengers waiting on the platform for each OD pair at station i within the time between adjacent decision points, is the platform capacity of station i;

[0125] S302. Action space definition;

[0126] The action space of the urban rail transit system is formed by the combination of passenger flow control rate and skipping mode of each OD, which is represented by the vector express: a=[( p iod1 cr im , p iod2 cr im , p iod3 cr im ,... , p iodn cr im ), st id ] ;

[0127] in, For the passenger flow of the nth OD pair at station i, the OD pair passenger flow is grouped, and the grouping function is shown in formula (1); cr im =[100%,75%,50%,25%,0] is the passenger flow control rate of station i, 100% means no passenger flow control is performed during the adjacent decision point time, 75% means the number of passengers entering station i during the adjacent decision point time is 75% of the original, ..., 0 means passengers are prohibited from entering station i during the adjacent decision point time; is the stop mode of train d at station i, 0 means no skipping, 1 means skipping;

[0128] (1);

[0129] (2);

[0130] in, is the function that divides the original OD pairs of station i into m groups, is the number of passengers of the kth OD pair at station i within the time of adjacent decision points; is a binary variable. If the kth group of OD pairs has been grouped, then ,otherwise ; is the OD pair with the maximum number of passengers at station i within the time of adjacent decision points; Formula (2) is the grouping constraint; K is the total number of OD pairs;

[0131] like Figure 2 As shown in Figure 2, there were originally 12 OD pairs within the time of a certain adjacent decision point at station i. After grouping by formulas (1) and (2), the number was reduced to 5 OD pairs. This ensures that the passenger flow of each OD pair is basically maintained at the same level, thereby ensuring the fairness of the passenger flow of each OD pair. After grouping, the action space is reduced from the original The order of magnitude has dropped to The order of magnitude reduces the cardinality of the action space and increases operability; the grouped action space can be expressed as: A=[(11,21,31,41,51)*(12,22,32,42,52)*(13,23,33,43,53)*(14,24,34,44,54)*(15,25,35,45,55)*(0,1)] , where 11 means that the first group of ODs do not perform passenger flow control on passengers, 51 means that the first group of ODs prohibit passengers from entering the station, 15 means that the fifth group of ODs do not perform passenger flow control on passengers..., 0 means that the train does not skip a stop, and 1 means that the train skips a stop;

[0132] S303. Set reward function;

[0133] In reinforcement learning, the reward function must consider both the agent (in this embodiment, the agent represents the passenger flow control and skip-stop mode collaborative optimization control center) maximizing its own reward and achieving the corresponding optimization goal. This embodiment aims to improve the operating efficiency of the urban rail transit system and the safety of passenger travel. First, the passenger detention on the platform will increase the passenger density of the platform, and the goal of the passenger flow control strategy is to ensure that the passenger density on the platform is within a safe range and improve the safety of passenger travel; in addition, without increasing the train capacity, the flexible adjustment of the train stop plan helps to balance the utilization of the existing train capacity, improve the operating efficiency of the urban rail transit system, and can reduce the number of passenger detentions on the platform to a certain extent, and improve the safety of passenger travel; therefore, the reward function is directly related to the number of times passengers are detained on the platform. For station i, the reward function is defined as:

[0134] (3);

[0135] in, is the state of the urban rail transit system at time t Next action The reward obtained by the latter station i is the inverse of the total number of passengers’ detentions. The smaller the total number of detentions, the greater the reward value. t is the latest decision time. is a binary variable. If passenger c is stranded at station i at decision time t, then ,otherwise ; C is the maximum number of passengers at station i; is the cumulative number of times passenger c stayed on the platform;

[0136] S304. Set the strategy function;

[0137] In deep reinforcement learning, policy neural network is a function that maps the current state to an action, are the parameters of the neural network. It is the core part of the deep reinforcement learning algorithm, which determines what actions the agent should take in the current state. Through continuous interaction and updating with the environment, the agent gradually optimizes through the policy gradient , thereby selecting a better action strategy to make it perform better in future tasks. This embodiment uses a policy neural network Approximate policy function ; And its loss function The definition is as follows:

[0138] (4);

[0139] L i (θ)=E[ ∑ an=1 an max -1 log π iθ ( S t , a t )∙( Q i ( S t , a t )- V i ( S t )) ] (5);

[0140] in, Represents the state of the urban rail transit system at time t Next action space The probability distribution function of Represents the state of the urban rail transit system at time t Take action The action value function of for The state value function of It is expressed as the maximum number of actions, E represents the mean square error, log represents the natural logarithm; an represents the number of actions;

[0141] According to formula (5), two neural networks are needed to estimate and , according to the Bellman equation:

[0142] Q i ( S t , a t )=E[ R i ( S t , a t )+γ V i ( S t+1 )] (6);

[0143] in, Represents the state of the urban rail transit system at time t+1 The state value function, γ is the discount factor, and E represents the mean square error; is the state of the urban rail transit system at time t Next action The reward obtained by the next station i; Substituting formula (6) into (5) we get The final loss function :

[0144] L i (θ)=E[ ∑ an=1 an max -1 log π iθ ( S t , a t )∙( R i ( S t , a t )+γ V i ( S t+1 )- V i ( S t )) ] (7);

[0145] In the case of formula (7), only one neural network is needed to estimate and .

[0146] S4. Construct a collaborative optimization model of passenger flow control and stop plan, which consists of an Actor network and a Critic network.

[0147] Considering the number of stations and different scenarios of OD classification, an actor network is established using a recurrent neural network (RNN) and an attention mechanism to generate the probability distribution of actions. The model can handle passenger flow control and stop mode collaborative optimization problems in urban rail transit systems of different scales; specifically, it includes the following key components:

[0148] (1) Input layer: includes action space data and all state space data

[0149] (2) Encoder: The encoder uses a recurrent neural network (RNN) to encode the input data into a fixed-length vector representation;

[0150] (3) Attention mechanism: The attention mechanism calculates the attention weight of each input vector;

[0151] (4) Decoder: The decoder uses a recurrent neural network (RNN) combined with an attention mechanism (the importance of each input position to the current output position) to generate the output sequence, and uses a mask layer and a softmax activation function to generate the final output;

[0152] like Figure 3 As shown in the figure, the action space is first classified according to the constraints in the form of tensors to obtain the probability distribution of each group of actions. Then, the mask layer and softmax function are used to output the sub-actions with the highest probability in each group. The mask layer uses a 0-1 mask mechanism to mask other actions with non-maximum probability in this group to 0, thereby eliminating unreasonable action options. Finally, the sub-actions with the highest probability in each group are integrated into an optimal combined action. Such a design can improve the training efficiency and decision-making efficiency of the method. Through classification and masking mechanisms, the number of action spaces can be reduced, thereby reducing the complexity of calculation and search. At the same time, using the softmax function and probability maximization method, the optimal action combination can be quickly determined to improve the accuracy and efficiency of decision-making.

[0153] In addition, a recurrent neural network (RNN) is used to establish a critic network to estimate the current state of the urban rail transit system. The state value function , which is further used to update the policy gradient of the actor network. The detailed parameters of the above neural network are as follows:

[0154] Actor network;

[0155] (1) Input layer: Representation of the input sequence: each position has a vector representation.

[0156] (2) Encoder layer:

[0157] - RNN type: LSTM;

[0158] - RNN hidden state size: 256;

[0159] - Number of layers of RNN: 2;

[0160] - Bidirectionality of RNN: Use bidirectional RNN;

[0161] (3) Attention Mechanism:

[0162] - Type of attention mechanism: additive attention;

[0163] - Calculation method of attention weight: Attention weight is obtained by calculating the inner product of two vectors;

[0164] (4) Decoder layer:

[0165] - RNN type: LSTM;

[0166] - Hidden state size of RNN: 256;

[0167] - Number of RNN layers: 2;

[0168] - Bidirectionality of RNN: Bidirectional RNN is not used;

[0169] Critic Network:

[0170] - RNN type: LSTM;

[0171] - Hidden state size of RNN: 256;

[0172] - Number of RNN layers: 2;

[0173] - Bidirectionality of RNN: Bidirectional RNN is not used;

[0174] This embodiment is only an example, and the parameter settings in actual applications may be adjusted and optimized according to specific tasks and scene sets.

[0175] S5. Train the passenger flow control and stop plan collaborative optimization model;

[0176] The actor network is responsible for selecting actions, and the critic network is responsible for evaluating the value function of the state of the urban rail transit system. The actor network selects actions based on the state of the urban rail transit system at time t, and determines the action based on the probability distribution of the selected action. The critic network estimates its value function based on the state of the urban rail transit system at time t, that is, predicts the cumulative reward of the state. The specific training process of the actor-critic algorithm is as follows Figure 4 As shown:

[0177] S501. Randomly initialize all states; initialize the experience replay pool D; initialize all parameters of the Actor-Critic network and ;

[0178] S502. The actor network interacts with the environment to transform the state of the urban rail transit system at time t Input the actor network and get the probability distribution of the output action ;

[0179] S503. Output the probability distribution of actions based on the actor network , use the mask layer and softmax function to classify the action tensor and output the combined action with the highest probability , execute the selected action , observe the reward of environmental feedback and the state of the urban rail transit system at time t+1 ; The current status of the urban rail transit system ,action ,award , the state of the urban rail transit system at time t+1 Together they form the experience sample;

[0180] S504. Store the experience samples in the experience replay pool; when the number of experience samples reaches a certain threshold D, randomly sample a batch of experience samples from the experience replay pool for training the actor network and the critic network;

[0181] S505. Use in critic networks and As input, the output state value function and ;

[0182] S506. Calculate the critic network loss function, as shown in formula (8); use the gradient descent method to update the parameters of the critic network by minimizing the loss function , the update method is shown in formula (9);

[0183] L i (ω) = E[ R i ( S t , a t )+γ V i ( S t+1 )- V i ( S t )] 2 (8);

[0184] (9);

[0185] in, is the critic network update parameter The learning rate, γ is the discount factor, ∇ represents the gradient operator, and E represents the mean square error;

[0186] S507. State-value function using critic network and and the probability distribution of the selected action Calculate the loss function of the actor network as shown in formula (7); use the gradient ascent method to update the parameters of the critic network by maximizing the expected return , the update method is shown in formula (10);

[0187] (10);

[0188] S508. Repeat S502- S507 until the two neural network training is stable; complete the training of the passenger flow control and stop plan collaborative optimization model; wherein, is the Actor network update parameter The learning rate is , and ∇ represents the gradient operator.

[0189] S6. Real-time decision making of collaborative optimization model based on deep reinforcement learning

[0190] After the collaborative optimization model based on deep reinforcement learning is trained and stabilized, it guides the urban rail transit system to make the best coordinated actions of passenger flow control and stop plan in real time, so as to improve the operation efficiency of the urban rail transit system and the safety of passengers. Figure 5 As shown:

[0191] S601. Collecting real-time status data of urban rail transit system, performing data preprocessing to form a standardized data vector;

[0192] S602: extracting passenger, train and station related data within a single decision cycle t (the decision cycle is the interval between train d and train d-1 arriving at station i) to generate a state space;

[0193] S603: extracting passenger flow data of OD pairs of each station within a single decision cycle t, and performing combined classification of OD pairs in accordance with formula (1);

[0194] S604: Determine whether the number of passenger flows of each type of OD after grouping meets the constraint according to formula (2). If yes, generate a suitable action space; otherwise, go to S603;

[0195] S605: All key state space data and action space data are transmitted to the stable collaborative optimization model, and the coordinated actions of the optimal passenger flow control and stop plan are output, and feedback data is collected after the action is executed;

[0196] S606: Repeat S601-S605 until the corresponding decision cycle T ends.

[0197] An embodiment of the present invention further provides a passenger flow evacuation system for a comprehensive passenger transportation hub based on passenger flow data and reinforcement learning, comprising:

[0198] Data acquisition module, used to collect real-time status data of urban rail transit system, including train operation data, passenger flow data and station platform data;

[0199] Data preprocessing module, used for data cleaning, data format standardization and data normalization;

[0200] The Markov game module is used to describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process; it includes defining the state space, defining the action space, setting the reward function and setting the strategy function; the state space includes the train running state, passenger state and station platform state;

[0201] A model building module, used to build a passenger flow control and stop plan collaborative optimization model, wherein the passenger flow control and stop plan collaborative optimization model is composed of an Actor network and a Critic network;

[0202] Model training module, used to train the passenger flow control and stop plan collaborative optimization model and obtain a stable passenger flow control and stop plan collaborative optimization model;

[0203] The real-time decision-making module is used to make real-time decisions on passenger flow control and stop plans for urban rail transit based on the trained passenger flow control and stop plan collaborative optimization model.

[0204] Preferably, the embodiments of the present application also provide a specific implementation of an electronic device capable of implementing all steps in the method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning in the above embodiment, and the electronic device specifically includes the following contents:

[0205] Processor, memory, communications interface and bus;

[0206] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.

[0207] The processor is used to call the computer program in the memory, and when the processor executes the computer program, it implements all the steps in the method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning in the above embodiment.

[0208] An embodiment of the present application also provides a computer-readable storage medium capable of implementing all steps of the method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning in the above-mentioned embodiment. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, all steps of the method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning in the above-mentioned embodiment are implemented.

[0209] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0210] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0211] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or client product in practice is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0212] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0213] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0214] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0215] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solution of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, a person of ordinary skill in the art can also make many forms of specific changes under the guidance of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A method for collaborative optimization of passenger flow control and stop plan for urban rail transit based on deep reinforcement learning, characterized in that: include: S1. Collect real-time status data of urban rail transit system; Including train operation data, passenger flow data and station platform data; S2. Data preprocessing, including data cleaning, data format standardization and data normalization; S3. Describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process; Including defining state space, defining action space, setting reward function and setting strategy function; state space includes train running state, passenger state and station platform state; The action space is formed by the combination of passenger flow control rate and skipping pattern of each OD; The reward function is directly related to the number of times a passenger stays at the platform. For station i, the reward function is defined as: Among them, R i (S t , a t ) is the state S of the urban rail transit system at time t t Next, perform action a t The reward obtained by the latter station i is the inverse of the total number of passengers’ detention times. The smaller the total number of detention times, the greater the reward value. t is the latest decision time. ict is a binary variable. If passenger c is stranded at station i at decision time t, then o ict =1, otherwise o ict =0; C is the maximum number of passengers at station i; p cwf is the cumulative number of times passenger c stayed on the platform; Policy Neural Network π iθ (S t , a t ) is a neural network that maps the state of the urban rail transit system at time t to an action, θ is the parameter of the neural network; the strategy neural network π is used iθ (S t , a t ) approximates the policy function π i (a t |S t );π iθ (S t , a t ) and its loss function L i (θ) is defined as follows: p iθ (S t ,a t )=p i (a|S t ,θ)≈π i (a t |S t ); Among them, p i (a|S t , θ) represents the state S of the urban rail transit system at time t t The probability distribution function of the action space a; S4. Constructing a collaborative optimization model of passenger flow control and stop plan, wherein the collaborative optimization model of passenger flow control and stop plan consists of an Actor network and a Critic network; Use the pointer network structure to establish an Actor network to generate the probability distribution p of the action i (a|S t ,θ), the Actor network includes: Input layer, including action space data and all state space data; Encoder: The encoder uses a recurrent neural network (RNN) to encode the input data into a fixed-length vector representation. Attention mechanism, which is used to calculate the attention weight of each input vector through the attention mechanism; The decoder uses a recurrent neural network (RNN) combined with an attention mechanism to generate the output sequence, and uses a mask layer and a softmax activation function to generate the final output; The Critic network is based on a recurrent neural network (RNN) and is used to estimate the current state S of the urban rail transit system. t The state value function V i (S t ), which is further used to update the Actor network gradient; S5. Train the passenger flow control and stop plan collaborative optimization model; S6. Real-time decision-making on passenger flow control and stop plan of urban rail transit is realized based on the trained collaborative optimization model of passenger flow control and stop plan.

2. According to claim 1, a method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning, characterized in that: The train operation data in step S1 includes the remaining available capacity of each carriage when arriving at each station, the arrival time and the departure time; Passenger flow data include the total number of OD pairs on a rail transit line, the number of passengers entering each station for all OD pairs, the number of passengers transferring for all OD pairs at each station, and the number of times each passenger stays at the platform; Station platform data includes the maximum passenger capacity of each platform on a rail transit line and the number of passengers waiting for all OD pairs at each platform.

3. According to claim 1, a method for collaborative optimization of passenger flow control and stop plan for urban rail transit based on deep reinforcement learning, characterized in that: In step S2, Data cleaning is to deal with missing values, outliers and duplicate values. For missing values, column mean is used to fill in the data to ensure data integrity. For outliers, the standard deviation of the data is calculated to identify outliers, and column mean is used to replace the data to avoid affecting subsequent analysis. For duplicate values, data deduplication is performed to ensure data uniqueness. Data format standardization includes data format conversion, unit conversion, and encoding conversion; Data normalization uses Max-Min deviation standardization to normalize the data set and map all attribute data to the interval [0, 1] to avoid training deviation in subsequent training.

4. According to claim 1, a method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning, characterized in that: In step S3, S301. Define the state space: The state space of the urban rail transit system includes three elements: train running state, passenger state and station platform state, which is represented by the vector S = [s t ,s p ,s s ]express; The train running status is represented by vector s t =[t id , t ide , t idb ]express; in is the average remaining available capacity when train d arrives at station i, t idj is the remaining available capacity of the i-th carriage of train d, n means that train d has n carriages; t ide is the time when train d arrives at station i; t idb is the time when train d leaves station i; The passenger state is represented by vector s p =[p iodp , p iodt , p cwf ]express; where p iodp is the set of the number of passengers entering the station for each OD pair at station i within the time of adjacent decision points; p iodt is the set of transfer numbers of passengers for each OD pair at station i within the time of adjacent decision points; pcw f is the cumulative number of times passenger c stayed on the platform; The station platform status is represented by vector s s =[s iodd ]express; where s iodd =s iodw / s ipc is the passenger density set of each OD pair of platforms at station i within the time of adjacent decision points, s iodw is the number of passengers waiting on the platform for each OD pair at station i within the time between adjacent decision points, s ipc is the platform capacity of station i; S302. Define action space: The action space of the urban rail transit system is represented by the vector a: a = [(p iod1 cr im , p iod2 cr im , p iod3 cr im , ..., p iodn cr im ), st id ]; Among them, p iodn The passenger flow of the nth OD pair at station i is grouped into OD pairs. The grouping function is shown in formula (1); m =[100%, 75%, 50%, 25%, 0] is the passenger flow control rate of station i, 100% means no passenger flow control is performed during the adjacent decision point period, 75% means the number of passengers entering station i during the adjacent decision point period is 75% of the original number, ..., 0 means passengers are prohibited from entering station i during the adjacent decision point period; id is the stop mode of train d at station i, 0 means no skipping, 1 means skipping; Among them, f i (c m ) is the function that divides the original OD pairs of station i into m groups, p ikod is the number of passengers of the kth OD pair at station i within the time between adjacent decision points; ik is a binary variable. If the kth group OD pair has been grouped, then O ik =0, otherwise O ik =1; is the OD pair with the maximum number of passengers at station i within the time of adjacent decision points; Formula (2) is the grouping constraint; K is the total number of OD pairs; S303. Set reward function; The reward function is directly related to the number of times a passenger stays at the platform. For station i, the reward function is defined as: Among them, R i (S t , a t ) is the state S of the urban rail transit system at time t t Next, perform action a t The reward obtained by the latter station i is the inverse of the total number of passengers’ detention times. The smaller the total number of detention times, the greater the reward value. t is the latest decision time. ict is a binary variable. If passenger c is stranded at station i at decision time t, then o ict =1, otherwise o ict =0; C is the maximum number of passengers at station i; p cwf is the cumulative number of times passenger c stayed on the platform; S304. Set the strategy function; Policy Neural Network π iθ (S t , a t ) is a neural network that maps the state of the urban rail transit system at time t to an action, θ is the parameter of the neural network; the strategy neural network π is used iθ (S t , a t ) approximates the policy function π i (a t |S t );π iθ (S t , a t ) and its loss function L i (θ) is defined as follows: p iθ (S t ,a t )=p i (a|S t ,θ)≈π i (a t |S t ) (4); Among them, p i (a|S t , θ) represents the state S of the urban rail transit system at time t t The probability distribution function of the action space a; Q i (S t , a t ) represents the state S of the urban rail transit system at time t t Take action a t The action value function, V i (S t ) is S t The state value function of max It is expressed as the maximum number of actions, E represents the mean square error, log represents the natural logarithm; an represents the number of actions; According to formula (5), two neural networks are needed to estimate Q i (S t , a t ) and V i (S t ), according to the Bellman equation: Q i (S t ,a t )=E[R i (S t ,a t )+γV i (S t+1 )] (6); Among them, V i (S t+1 ) represents the state S of the urban rail transit system at time t+1 t+1 The state value function, γ is the discount factor, E represents the mean square error; R i (S t , a t ) is the state S of the urban rail transit system at time t t Next, perform action a t The reward obtained by the next station i; Substituting formula (6) into (5) to obtain π iθ (S t , a t )’s final loss function L i (θ): In the case of formula (7), only one neural network is needed to estimate V i (S t ) and V i (S t+1 ).

5. According to claim 1, a method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning, characterized in that: In step S5, the Actor network in the passenger flow control and stop plan collaborative optimization model is used to select actions, and the Critic network is used to evaluate the value function of the urban rail transit system state; the Actor network selects actions according to the state of the urban rail transit system at time t; the Critic network estimates its value function according to the state of the urban rail transit system at time t, that is, predicts the cumulative reward of the corresponding state; the specific training process is as follows: S501. Randomly initialize all states; initialize the experience replay pool D; initialize all parameters θ and ω in the Actor network and Critic network; S502. The Actor network interacts with the environment and converts the state S of the urban rail transit system at time t into t Input Actor network, output action probability distribution p i (a|S t ,θ); S503. Based on the action probability distribution p output by the Actor network i (a|S t , θ), use the mask layer and softmax function to classify the action tensor and output the combined action a with the highest probability t , execute the selected action a t , the reward R of observing environmental feedback i (S t , a t ) and the state S of the urban rail transit system at time t+1 t+1 ; The state S of the urban rail transit system at time t t , action a t , reward R i (S t , a t ), the state S of the urban rail transit system at time t+1 t+1 Together they form the experience sample; S504. Storing the experience sample in the experience replay pool; When the number of experience samples reaches the threshold D, a number of experience samples are randomly sampled from the experience replay pool to train the Actor network and the Critic network; S505. Use S in the Critic network t and S t+1 As input, the output state value function V i (S t ) and V i (S t+1 ); S506. Calculate the loss function of the Critic network, as shown in formula (8); use the gradient descent method to update the parameter ω of the Critic network by minimizing the loss function, and the updating method is shown in formula (9); 50 i (ω)=E[R i (S t ,a t )+γV i (S t+1 )-V i (S t )] 2 (8); Among them, α1 is the learning rate of the Critic network update parameter ω, γ is the discount factor, represents the gradient operator, E represents the mean square error; S507. State value function V calculated using the Critic network i (S t ) and V i (S t+1 ) and the probability distribution p of the selected action i (a|S t , θ) calculates the loss function of the Actor network; uses the gradient ascent method to update the parameters θ of the Actor network by maximizing the expected return, and the update method is shown in formula (10); S508. Repeat S502-S507 until the Actor network and the Critic network training are stable; the training of the passenger flow control and stop plan collaborative optimization model is completed; where α2 is the learning rate of the Actor network update parameter θ, Represents the gradient operator.

6. According to claim 1, a method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning, characterized in that: The specific decision-making process in step S6 is as follows: S601. Collect real-time status data of the urban rail transit system, perform data preprocessing, and form a standardized data vector; S602. Extract the data related to passengers, trains and stations within a single decision cycle t to generate a state space; the decision cycle is the interval between train d and train d-1 arriving at station i; S603. Extract the passenger flow data of OD pairs of each station within a single decision cycle t, and perform combined classification of OD pairs; S604. Determine whether the number of passenger flows of various ODs after grouping meets the constraints. If yes, generate the corresponding action space, otherwise go to S603; S605. All state space data and action space data are transmitted to the stable passenger flow control and stop plan collaborative optimization model, and the optimal passenger flow control and stop plan collaborative action is output, and feedback data is collected after the action is executed; S606. Repeat S601-S605 until the corresponding decision cycle T ends.

7. A system for collaborative optimization of passenger flow control and stop plans for urban rail transit based on deep reinforcement learning, characterized in that: include: Data acquisition module, used to collect real-time status data of urban rail transit system; Including train operation data, passenger flow data and station platform data; Data preprocessing module, used for data cleaning, data format standardization and data normalization; Markov game module, which is used to describe the coordinated optimization process of urban rail transit passenger flow control and stop plan as a Markov game process; Including defining state space, defining action space, setting reward function and setting strategy function; the state space includes the current state of the urban rail transit system, passenger state and station platform state; The action space is formed by the combination of passenger flow control rate and skipping pattern of each OD; The reward function is directly related to the number of times a passenger stays at the platform. For station i, the reward function is defined as: Among them, R i (S t , a t ) is the state S of the urban rail transit system at time t t Next, perform action a t The reward obtained by the latter station i is the inverse of the total number of passengers’ detention times. The smaller the total number of detention times, the greater the reward value. t is the latest decision time. ict is a binary variable. If passenger c is stranded at station i at decision time t, then o ict =1, otherwise o ict =0; C is the maximum number of passengers at station i; p cwf is the cumulative number of times passenger c stayed on the platform; Policy Neural Network π iθ (S t , a t ) is a neural network that maps the state of the urban rail transit system at time t to an action, θ is the parameter of the neural network; the strategy neural network π is used iθ (S t , a t ) approximates the policy function π i (a t |S t );π iθ (S t , a t ) and its loss function L i (θ) is defined as follows: p iθ (S t ,a t )=p i (a|S t ,θ)≈π i (a t |S t ); Among them, p i (a|S t , θ) represents the state S of the urban rail transit system at time t t The probability distribution function of the action space a; The model building module is used to build a passenger flow control and stop plan collaborative optimization model, which consists of an Actor network and a Critic network. The Actor network is established using a pointer network structure to generate the probability distribution p of the action. i (a|S t ,θ), the Actor network includes: Input layer, including action space data and all state space data; Encoder: The encoder uses a recurrent neural network (RNN) to encode the input data into a fixed-length vector representation. Attention mechanism, which is used to calculate the attention weight of each input vector through the attention mechanism; The decoder uses a recurrent neural network (RNN) combined with an attention mechanism to generate the output sequence, and uses a mask layer and a softmax activation function to generate the final output; The Critic network is based on a recurrent neural network (RNN) and is used to estimate the current state S of the urban rail transit system. t The state value function V i (S t ), which is further used to update the Actor network gradient; Model training module, used to train the passenger flow control and stop plan collaborative optimization model and obtain a stable passenger flow control and stop plan collaborative optimization model; The real-time decision-making module is used to make real-time decisions on passenger flow control and stop plans for urban rail transit based on the trained passenger flow control and stop plan collaborative optimization model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for collaborative optimization of urban rail transit passenger flow control and stop plan based on deep reinforcement learning as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for collaborative optimization of urban rail transit passenger flow control and stop plans based on deep reinforcement learning as described in any one of claims 1 to 6 are implemented.

Images