Urban rail passenger flow and train coordination organization optimization method based on deep reinforcement learning

Abstract

Based on deep reinforcement learning, the urban rail passenger flow and train coordination organization optimization method is proposed. Firstly, based on the AFC data of urban rail transit, the passenger arrival law and travel chain are analyzed, and the OD matrix of urban rail transit network is generated by using the diffusion model. Secondly, a simulation model reflecting the train operation and passenger flow state is constructed to provide a verification environment for the optimization process. For train operation scheme and passenger flow scheduling, a DQN model is designed and trained and learned by using the simulation model. This method can reasonably match passenger demand and transport resources, effectively shorten passenger waiting time, and reduce train operating costs.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of urban rail transit transportation scheduling, and specifically relates to a method for coordinated organization and optimization of urban rail passenger flow and trains based on deep reinforcement learning. BACKGROUND

[0002] With the acceleration of urbanization, the passenger flow of urban rail transit systems is increasing year by year. How to efficiently optimize the passenger flow and train operation plan has become a key issue for improving the service quality of rail transit, improving operational efficiency and reducing energy consumption.

[0003] Traditional methods for optimizing rail transit operation plans mainly rely on mathematical modeling, heuristic algorithms and empirical rules. However, with the increasing complexity of passenger flow distribution, passenger demand and train operation, traditional methods face the following challenges: (1) complex passenger flow changes; (2) difficulty in optimizing train operation scheduling; (3) balancing energy consumption and efficiency; (4) lack of adaptability to real-time data.

[0004] Most existing researches currently focus on single-sided optimization, such as train scheduling, station passenger flow management or energy consumption optimization, while comprehensive optimization methods considering multiple factors such as passenger flow, train operation and energy efficiency are relatively scarce. With the rapid development of deep learning technology, methods based on deep reinforcement learning have gradually become an effective way to solve these problems. Reinforcement learning can automatically learn the optimal strategy from historical data through interaction with the environment, while deep neural networks can handle large amounts of nonlinear complex data and improve the performance of the model.

[0005] The present application is distinguished from the prior art as follows:

[0006] Comparison with the technology of patent CN115352502A "Train operation plan adjustment method and device, electronic equipment and storage medium"

[0007] I. Patent CN115352502A adopts an adversarial neural network, which receives normalized time input through a generator and outputs a passenger flow OD matrix. The discriminator is used to determine the authenticity of the input OD matrix and also judges the time information, so that the generated OD matrix is as close as possible to the real matrix in statistical characteristics. This study uses a diffusion spatio-temporal graph model to describe the connection properties between stations using a graph structure and the topological structure between stations using an adjacency matrix. With historical OD matrix as input, combined with the station network structure, the passenger flow distribution at multiple time granularities is predicted. In the original scheme, the generator and discriminator mainly rely on the input time data and the generated OD matrix for adversarial training, and the spatial information is implicitly contained in the OD matrix, lacking explicit modeling of the station network. In the new scheme, the diffusion spatio-temporal graph model can capture the rules of passenger flow between stations, making the capture of passenger flow diffusion and spatio-temporal dynamic characteristics more accurate, thereby helping to improve the overall prediction accuracy.

[0008] II. Patent CN115352502A does not explicitly model the network structure between stations in the process of building a city rail transit network simulation, mainly relying on the pre-set passenger travel path to determine the passenger boarding, transfer and alighting behavior, and cannot truly reflect the asymmetry and individual differences in the process of passenger boarding, transfer and alighting. In this study, after generating passenger initial information using the OD matrix, the passenger travel path is allocated through the Logit model, effectively considering the sensitivity of passengers to travel cost, familiarity and guidance information, and taking into account the randomness and decision-making rules of passenger behavior. Therefore, the city rail transit network simulation model designed in this study can reflect the dynamic changes of train operation, station congestion and passenger behavior in real time, and is more suitable for passenger flow and train collaborative optimization research of complex city rail transit network, laying a foundation for subsequent passenger flow and train collaborative organization optimization.

[0009] III. Patent CN115352502A focuses on train timetable adjustment, adopts an Advantage Actor-Critic (A2C) method to construct a deep reinforcement learning model composed of a policy network and a value network, and a reward function is calculated by the waiting time of inbound passengers and passenger transfers. This reward function can intuitively reflect the comprehensive performance of passenger waiting time under a specific timetable, thereby guiding the policy network to adjust the train departure time to improve passenger experience. In this study, the passenger flow scheduling problem and train operation optimization problem are modeled as a Markov decision process in reinforcement learning, and a Deep Q-Network (DQN) is used to find the optimal strategy. Mechanisms such as experience pool and target network are introduced to greatly improve the stability of training and reduce the risk caused by frequent changes in target values. The reward function is based on train operating costs and passenger waiting time, combining operating costs with passenger experience to enable the training process to dynamically respond to changes in passenger waiting time and congestion while considering economic benefits, thereby achieving coordinated optimization of train schedules and passenger flow scheduling. The original scheme is suitable for continuous control problems based on policy gradient, with a simple structure that facilitates preliminary verification of the impact of train schedule adjustment on passenger waiting time. The new scheme uses the DQN method based on value function approximation, which is more detailed in state representation, action design, and reward construction, and can balance operating costs and passenger experience, providing a more comprehensive solution for the coordinated optimization of complex urban rail transit networks.

[0010] Therefore, in combination with deep reinforcement learning technology, a method is proposed that can comprehensively optimize passenger flow and train operation schemes, which not only improves the efficiency of the system, but also reduces energy consumption, and has important application prospects. SUMMARY

[0011] To solve the above problems, the present application proposes a method for coordinated organization and optimization of urban rail passenger flow and trains based on deep reinforcement learning, which not only improves the efficiency of the system, but also reduces energy consumption, and has important application prospects.

[0012] To achieve the above-mentioned purposes, the technical solutions adopted by the present application are as follows:

[0013] The method for coordinated organization and optimization of urban rail passenger flow and trains based on deep reinforcement learning comprises the following steps:

[0014] Step 1: Use AFC data of rail transit to analyze passenger arrival rules and travel chains, and construct a passenger flow OD matrix;

[0015] Step 2: Use the passenger flow OD matrix constructed in step 1 to train the diffusion model and generate passenger flow OD matrices in different time periods to provide rich data for train operation and passenger flow scheduling optimization;

[0016] Step 3, input the OD matrix generated by the diffusion model in step 2 as the passenger flow, design the interaction events of passenger, train and station elements according to the operation characteristics of urban rail transit network system, simulate the state change of urban rail system, and build a discrete event-based urban rail transit network simulation model;

[0017] Step 4, based on the urban rail transit network simulation model, study the DQN-based urban rail passenger flow and train collaborative scheduling optimization method, construct the system state, action and reward function of DQN under the constraint of urban rail operation rules, and minimize the train traction energy consumption and passenger travel time;

[0018] Step 5, design the training process for the DQN-based urban rail passenger flow and train collaborative scheduling optimization method, obtain the optimized scheduling strategy through training, and apply it to the train operation and passenger flow scheduling of urban rail transit system, so as to improve the passenger travel experience and service level of urban rail transit.

[0019] As a further improvement of the application, step 1 is specifically:

[0020] Using the processed rail transit AFC data, the passenger's entry time, exit time, departure station and destination station information are obtained, the passenger's arrival law and travel chain are analyzed, and the passenger flow OD matrix is generated by taking ten minutes or half an hour as the time granularity Δt, which provides data support for passenger flow generation, wherein the OD matrix of time t is represented as X t ∈R N×N , wherein N is the number of stations, and all OD matrices under Δt are combined into a time sequence three-dimensional matrix X h =[X Δt ,...,X hΔt ], wherein j represents the number of historical time granularity, and the dimension is h×N×N.

[0021] As a further improvement of the application, step 2 is specifically:

[0022] The diffusion space-time graph model is used to predict the passenger flow OD matrix, G=(V, E, A) represents the graph structure, V is the station set, corresponding to the starting station i and the terminal station h of the OD matrix; E represents the connection attribute between stations; A is an adjacency matrix, which is used to describe the topological structure between stations, X t [i,j] represents the passenger flow from station i to station j at time t;

[0023] The target of passenger flow prediction is to input the historical OD matrix X H =[X Δt ,...,X HΔt ], H≤h and the graph structure G, and predict the future OD matrix X F =[X H+Δt ,...,XH+FΔt ], wherein F represents the number of prediction time granularities:

[0024] (X H ; G) → [X H+Δt , …, X H+FΔt ] : = X F .

[0025] As a further improvement of the application, step 3 is specifically:

[0026] 3.1 In the discrete event-based urban rail transit network simulation model, three main bodies are involved, namely rail transit trains, rail transit stations and passengers taking rail transit trips;

[0027] Trains are carriers carrying passengers and need to run on rail lines according to the requirements of the timetable. Train attributes include number, running direction, line number, starting station, stopover station and terminal station;

[0028] Stations are the main places for passenger distribution. Station attributes include station name, line, whether it is a transfer station, train number and current platform passenger number;

[0029] Passengers are the service objects in the rail transit system. Passenger attributes include number, departure station, destination station, travel path and arrival time;

[0030] 3.2 In the discrete event-based urban rail transit network passenger flow simulation model, trains need to trigger a series of events to realize state update and interact with stations and passengers. Events include train initialization, train operation and stopover;

[0031] 3.3 In the simulation model of train operation and passenger flow state, passenger events include initialization, arrival, departure, waiting, transfer and boarding and alighting.

[0032] As a further improvement of the application, the 3.3 simulation model of train operation and passenger flow state, passenger events are specifically described as follows:

[0033] Passenger initialization: at the beginning of the simulation, generate the initial information of passengers according to the OD matrix generated in step 2, including number, departure station, destination station, travel path and arrival time, represented by λ = X nΔt [i, j] represents the passenger flow from station i to station j at the nth time granularity; the arrival time of each passenger p obeys an exponential distribution:

[0034]

[0035] This represents the entry time of passenger p from station i to station j during the time interval from nΔt to (n+1)Δt.

[0036] Passenger p's arrival time for:

[0037]

[0038] Based on each passenger's arrival time, the relevant times during the passenger's journey are generated using a log-normal distribution. The log-normal distribution is:

[0039]

[0040] In the formula, μ is the logarithmic mean of the generation time, σ is the standard deviation of the generation time, and... This indicates that the log-normal distribution is in The probability density at the location is given, assuming that the passenger's walking time to the station follows a log-normal distribution. Passenger transfer times follow a log-normal distribution Passengers' walking time to exit the station follows a log-normal distribution.

[0041] After generating passenger arrival times, the Logit model is used to allocate passenger travel routes:

[0042]

[0043] in K represents the probability that a passenger chooses path k within the path ij. ij This represents the OD matrix representing all possible paths within path ij. Let OD matrix represent the travel cost of path k, θ represent the passenger's familiarity with the urban rail transit network, d represent the passenger's acceptance of guidance information, and β represent the impact of guidance information on path selection.

[0044] Passenger entry, exit, waiting, transfer, boarding / alighting: Passengers arrive at the station according to the generated entry time, and the train arrives at the corresponding station within the specified time according to the set timetable. This will trigger passenger boarding / alighting events. Passenger disembarking needs to be judged in combination with route selection. If the disembarking conditions are not met, the passenger will continue to stay on the train. If the conditions are met, the passenger's transfer or exit event will be triggered. Passenger boarding is constrained by the train direction, the current passenger capacity of the train, and the passenger's route. If the boarding conditions are not met, the passenger will continue to wait for the train.

[0045] As a further improvement to the present invention, step 4 specifically includes:

[0046] The passenger flow scheduling problem and the train operation optimization problem are modeled as Markov decision problems in reinforcement learning, and DQN is used to find the optimal strategy, wherein the learning process comprises the following:

[0047] System state: the state comprises platform congestion, section congestion and train position;

[0048] Action: train departure interval Δt train , section running time Station stop time Section recommendation a m , a m is a 0-1 variable, 1 when recommending the m section, otherwise 0, m ∈ N;

[0049] Reward: based on the optimization objective design, the specific description is as follows:

[0050] Train operation cost:

[0051] C op =C f D t

[0052] C op represents the train operation cost, C f represents the fuel cost per unit distance, D t represents the train running distance;

[0053] Waiting time of passenger p:

[0054]

[0055] In the formula, represents the waiting time of passenger p, represents the arrival time of train x at platform y, which is obtained by a city rail transit network simulation model;

[0056] In order to integrate these objectives, a weighted combination is made into a total reward function R(t), and the goal is to maximize the function:

[0057]

[0058] α1 is the weight of train operation cost, α2 is the weight of passenger waiting time, and η is the overall scaling factor, and the values of α1, α2 and η can be adjusted according to different requirements in practice, so as to balance the influence between different objectives.

[0059] As a further improvement of the application, step 5 is specifically:

[0060] 5.1 The discrete event-based urban rail transit network simulation model is used as a simulation environment for deep reinforcement learning. By generating different passenger flow and train operation scenarios, the simulation environment simulates the situation of passengers between platforms and carriages and the operation of trains. In the simulation environment, the DQN network is trained iteratively to optimize train operation schemes and passenger flow scheduling strategies, thereby improving the overall operation efficiency and passenger experience of the system.

[0061] 5.2 The specific steps in the training process include:

[0062] Initialize network parameters: initialize the experience pool D and set the maximum number of samples M that the experience pool can store: initialize the parameters w of the prediction network and the parameters w of the target network - Initialize, let w = w - . The prediction network outputs the Q value of each possible action by inputting the current state s z , i.e. Q(s z , a z ; w); the target network and the prediction network have the same structure but different parameters, and the role of the target network is to provide stable target values;

[0063] Environment interaction: based on the current state s z , select action a z according to the optimal action value function and observe the new state s z+1 and the reward r z , obtain the sample (s z , a z , s z+1 ) and store it in the experience pool D;

[0064] Calculate the target value y z : batch sample the saved samples from the experience pool D and calculate the target value, is the maximum Q value of the target network in the next state s z+1 and all actions a';

[0065]

[0066] Calculate the loss function L(w): the loss function measures the gap between the predicted Q value and the target Q value, and the loss function is defined as:

[0067]

[0068] Calculate the gradient of the loss function L(w) with respect to w: calculate the gradient of the loss function with respect to the network parameters by backpropagation, and use the gradient descent method to update the network parameters;

[0069]

[0070] Update the target network w- : every fixed step, copy the prediction network parameter w to the target network w - , to stabilize the target value calculation, avoid instability due to rapid changes in the target value during the training process;

[0071] 5.3 continuously repeat the above training process, gradually approach the optimal action value function, and the updated train operation scheme and passenger flow scheduling scheme can be obtained; in addition, by comparing the passenger waiting time, car congestion and operation cost under the influence of different parameters, the model performance is evaluated and the weight coefficient is adjusted to optimize the learning rate, discount factor γ and other hyperparameters, so as to improve the effect and stability of strategy learning.

[0072] Compared with the prior art, the beneficial effects of the present application at least include:

[0073] The present application provides a deep reinforcement learning-based urban rail passenger flow and train collaborative organization optimization method. In addition, by comparing the passenger waiting time, car congestion and operation cost under the influence of different parameters, the model performance is evaluated and the weight coefficient is adjusted to optimize the learning rate, discount factor γ and other hyperparameters, so as to improve the effect and stability of strategy learning.

[0074] Through the deep reinforcement learning framework adopted by the present application, combined with dynamic changes in the environment and real-time data, the comprehensive improvement of passenger flow distribution, train operation scheduling and energy consumption optimization can be realized on the basis of multi-objective optimization. This method not only has high adaptability and flexibility, but also can continuously optimize the system strategy according to real-time feedback, and improve the overall operation efficiency of urban rail transit. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 The design flowchart of the deep reinforcement learning-based urban rail passenger flow and train collaborative organization optimization method in the present application;

[0076] Figure 2 The schematic diagram of the time related to passenger travel by rail transit in the present application;

[0077] Figure 3 The system composition block diagram of the deep reinforcement learning-based urban rail passenger flow and train collaborative organization optimization method in the present application. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but not to limit the scope of the present application.

[0079] As a specific embodiment of the present application, the present application provides a deep reinforcement learning-based urban rail passenger flow and train collaborative organization optimization method, and the design flowchart is as follows Figure 1As shown, the relevant time diagram is as follows Figure 2 As shown, the system block diagram is as follows Figure 3 As shown;

[0080] Step 1, using the AFC data of rail transit, analyze the passenger arrival rules and travel chain, and construct the passenger flow OD matrix.

[0081] Step 2, use the passenger flow OD matrix constructed in step 1 to train the diffusion model, generate passenger flow OD matrix in different time periods, and provide rich data for train operation and passenger flow scheduling optimization.

[0082] Step 3, taking the OD matrix generated by the diffusion model in step 2 as the input passenger flow, designing the interaction events of passengers, trains and stations according to the operation characteristics of urban rail transit network system, simulating the state change of urban rail system, and constructing the discrete event-based urban rail transit network simulation model.

[0083] Step 4, based on the urban rail transit network simulation model, study the DQN-based urban rail passenger flow and train collaborative scheduling optimization method, take the minimization of train traction energy consumption and passenger travel time as the goal, and construct the system state, action and reward function of DQN under the constraint of urban rail operation rules.

[0084] Step 5, design the training process for the DQN-based urban rail passenger flow and train collaborative scheduling optimization method, get the optimized scheduling strategy through training, and apply it to the train operation and passenger flow scheduling of urban rail transit system, so as to improve the passenger travel experience and service level of urban rail transit.

[0085] The discrete event-based urban rail transit network simulation model serves as the simulation environment of deep reinforcement learning, which simulates the situation of passengers between platforms and carriages and the operation of trains by generating different passenger flow and train operation scenarios.

[0086] In this simulation environment, the DQN network optimizes the train operation scheme and passenger flow scheduling strategy through continuous iteration training, so as to improve the overall operation efficiency and passenger experience of the system.

[0087] Initialize the experience pool D and set the maximum number of samples M that the experience pool can store: Initialize the parameters w of the prediction network and the parameters w - of the target network. - .

[0088] The prediction network outputs the Q value of each possible action by inputting the current state s z , that is, Q(s z , a z ; w).

[0089] The target network and the prediction network have the same structure but different parameters, and the target network is used to provide stable target values.

[0090] Based on the current state s z Select an action a according to the optimal action value function z And observe the new state s z+1 And the reward r z Get the sample (s z , a z , r z , s z+1 ) and store it in the experience pool D.

[0091] Batch sample the saved samples from the experience pool D, and calculate the target value, is the maximum Q value of the target network in the next state s z+1 And all actions a′.

[0092]

[0093] The loss function measures the gap between the predicted Q value and the target Q value, and the loss function is defined as:

[0094]

[0095] Calculate the gradient of the loss function with respect to the network parameters by backpropagation, and update the network parameters using gradient descent.

[0096]

[0097] Every fixed number of steps, copy the prediction network parameters w to the target network w - To stabilize the target value calculation and avoid instability caused by rapid changes in the target value during training.

[0098] By repeatedly executing the above training process, the action value function is gradually optimized, and the improved train operation scheme and passenger flow scheduling scheme are obtained.

[0099] The above is only a preferred embodiment of the present application, and is not intended to limit the present application in any other form, and any modification or equivalent change made according to the technical essence of the present application still falls within the scope of the present application.

Claims

1. A method for optimizing urban rail passenger flow and train coordination based on deep reinforcement learning, characterized in that: Comprise the following steps: Step 1, using the AFC data of rail transit, the passenger's arrival station rule and travel chain are analyzed, and the passenger flow OD matrix is constructed; Step 2, using the passenger flow OD matrix constructed in step 1 to train the diffusion model, generate passenger flow OD matrix in different time periods, provide rich data for train operation and passenger flow scheduling optimization; Step 3, taking the OD matrix generated by the diffusion model in step 2 as the input passenger flow, according to the operation characteristics of urban rail transit network system, the interaction events of passenger, train and station elements are designed, the state change of urban rail system is simulated, and the urban rail transit network simulation model based on discrete event is constructed; Step 3 is specifically: 3.1 in the urban rail transit network simulation model based on discrete event, three subjects are involved, which are rail transit train, rail transit station and passenger traveling by rail transit; The train is the carrier of the passengers, which needs to run on the rail line according to the requirements of the timetable, and the train attributes include number, running direction, line number, starting station, stopping station and terminal station; The station is the main place for passenger gathering and distribution, and the station attributes include station name, belonging line, whether it is a transfer station, the number of trains stopping and the current number of passengers on the platform; The passenger is the service object in the rail transit system, and the passenger attributes include number, departure station, destination station, travel path and arrival time; 3.2 in the urban rail transit network passenger flow simulation model based on discrete event, the train needs to trigger a series of events to realize state update and interact with the station and the passenger, and the events include train initialization, train operation and stopping; 3.3 in the simulation model of train operation and passenger flow state, the events of passengers include initialization, arrival, departure, waiting, transfer and getting on and off the train; The 3.3 in the simulation model of train operation and passenger flow state is specifically described as follows: Passenger Initialization: At the start of the simulation, based on the OD matrix generated in step 2, initial passenger information is generated, including passenger number, departure station, destination station, travel route, and arrival time. Indicates the first From the site at the time granularity to station Passenger flow; each passenger Arrival time Follows an exponential distribution: ； Indicates the first to From the site within the time period to station Passengers Arrival time; Passenger the entry time is: ； According to the in- station time of each passenger, the relevant time in the passenger travel process is generated by using a lognormal distribution The lognormal distribution is: ； wherein is the log mean of the generation time, is the standard deviation of the generation time, and denotes the probability density of the log normal distribution at the passenger's walk-in time is assumed to follow a log normal distribution the passenger's transfer time is assumed to follow a log normal distribution the passenger's walk-out time is assumed to follow a log normal distribution ; After generating the passenger arrival time, the Logit model is used to allocate the passenger travel path: ； in Indicates that passengers are Select path within path The probability, Represents the OD matrix pair All optional paths within the path, Represents the path in the OD matrix Travel costs This indicates the passenger's familiarity with the urban rail transit network. This indicates the degree to which passengers accept the guiding information. This indicates the impact of guidance information on route selection; Passenger arrival, departure, waiting, transfer, getting on and off the train: passengers arrive at the station according to the generated arrival time, and trains arrive at the corresponding station according to the set timetable within the specified time, which triggers the getting on and off the train event of passengers, and passengers need to be judged according to the generated path selection if they do not meet the getting off the train condition, otherwise the transfer event or the departure event of passengers will be triggered, and the getting on the train of passengers is constrained by the train direction, the current number of passengers and the passenger path, if the getting on the train condition is not met, the passenger will continue to wait; Step 4, based on the urban rail transit network simulation model, the DQN based urban rail passenger flow and train collaborative scheduling optimization method is studied, taking the minimization of train traction energy consumption and passenger travel time as the objective, and constructing the system state, action and reward function of DQN under the constraint of urban rail operation rules; Step 4 is specifically: The passenger flow scheduling problem and the train operation optimization problem are modeled as Markov decision problem in reinforcement learning, and DQN is used to find the optimal strategy, wherein the learning process includes the following: System state: the state includes platform congestion degree, section congestion degree and train position; Action: Train Departure Interval , Section Running Time , Station Stopping Time , Section Recommendation , is a 0-1 variable, 1 when the section is recommended, 0 otherwise, ;​ Reward: based on the optimization objective design, the specific description is as follows: Train operation cost: ； denotes a train operation cost, denotes a fuel cost per unit distance, denotes a train running distance; Passenger Waiting time: ； In the formula represents the waiting time of the passenger , represents the arrival time of the train at the platform , which is obtained by the urban rail transit network simulation model; To combine these objectives, a weighted combination into one overall reward function is made with the goal of maximizing this function: ； the weight of train operation cost, the weight of passenger waiting time, the overall scaling factor, , , The values of the above can be adjusted according to different needs in practice, to balance the influence between different goals. Step 5: Design a training process for the DQN-based urban rail passenger flow and train coordination scheduling optimization method. Through training, an optimized scheduling strategy is obtained and applied to the train operation and passenger flow scheduling of the urban rail transit system, thereby improving the passenger travel experience and service level of urban rail transit.

2. The deep reinforcement learning based urban rail passenger flow and train collaborative organization optimization method according to claim 1, characterized in that: Step 1 is specifically: Using processed rail transit AFC data, we obtain passenger arrival and departure times, as well as departure and destination station information. By analyzing passenger arrival patterns and travel chains, we use ten-minute or half-hour time granularities. Generate a passenger flow OD matrix to provide data support for passenger flow generation, including time... The OD matrix is ​​represented as ,in It is the number of stations. All the OD matrices below are combined into a time-series three-dimensional matrix. ,in Indicates the number of historical time granularities, with dimensions of .

3. The deep reinforcement learning based urban rail passenger flow and train collaborative organization optimization method according to claim 1, characterized in that: Step 2 is specifically: The OD matrix of passenger flow is predicted by using a diffusion space-time graph model, and the OD matrix is represented by a graph structure , is a set of stations, and corresponds to the starting station and the terminal station of the OD matrix; represents the connection attribute between stations; is an adjacency matrix, used for describing the topological structure between stations, represents the passenger flow from station to station at time ; The goal of traffic volume prediction is to input historical OD matrices , and graph structure and predict future OD matrices where denotes the number of prediction time granularities: 。 4. The deep reinforcement learning based urban rail passenger flow and train collaborative organization optimization method according to claim 1, characterized in that: Step 5 is specifically: 5.1 The discrete event-based urban rail transit network simulation model serves as the simulation environment for deep reinforcement learning. By generating different passenger flow and train operation scenarios, it simulates the situation of passengers between platforms and carriages and the operation of trains. In this simulation environment, the DQN network is trained iteratively to optimize the train operation scheme and passenger flow scheduling strategy, thereby improving the overall operation efficiency and passenger experience of the system. 5.2 The specific steps in the training process include: Initialize network parameters: for the experience pool Initialize and set the maximum number of samples that the experience pool can hold. : Parameters of the prediction network and the parameters of the target network Initialize, let The prediction network predicts the current state by inputting it. To output each possible action Value, that is The target network and the prediction network have the same structure but different parameters. The role of the target network is to provide a stable target value. Environment interaction: based on current state Select action according to optimal action-value function And observe new state And reward Get sample And store in experience pool ​ Compute target value : Batch sample saved samples from experience pool and compute target value, is the maximum value of target network in next state and all actions ; ； Computing the loss function The loss function measures the gap between the predicted Q value and the target Q value. The loss function is defined as: ； On the loss function Regarding Gradient calculation: Calculate the gradient of the loss function with respect to the network parameters by backpropagation, and update the network parameters using gradient descent. ； ； updating target network : every fixed step, copy the predicted network parameters to the target network , to calculate the stable target value, avoid instability due to rapid changes in the target value during training; 5.3 The above training process is repeated continuously, gradually approaching the optimal action-value function, so that the updated train operation scheme and passenger flow scheduling scheme can be obtained; in addition, the performance of the model needs to be evaluated and the weight coefficient adjustment learning rate and discount factor need to be optimized by comparing the passenger waiting time, car congestion and operating cost under the influence of different parameters hyperparameters.

Images