Train scheduling and passenger flow guiding collaborative optimization method and device

Abstract

The invention relates to a train scheduling and passenger flow guiding collaborative optimization method and device, and the method comprises the steps: obtaining the first passenger flow information of a road network in a preset time period, the road network comprises a plurality of lines, each line is provided with a plurality of stations, and the first passenger flow information indicates the number of passengers taking from a departure station to a destination station; the departure station and the destination station are respectively any one station in the road network; the first passenger flow information is input into an optimization model for train dispatching and passenger flow guiding collaborative optimization, an optimization result is obtained, and the optimization model is determined through optimization target training based on the minimum power supply energy consumption and the minimum passenger waiting time; the optimization result indicates train scheduling information and passenger flow guiding information on each line in the optimized road network. According to the train scheduling and passenger flow guiding collaborative optimization method and device provided by the embodiment of the invention, the energy consumption of a traction power supply system can be effectively reduced, and the average waiting time of passengers is shortened, so that the energy efficiency of urban rail transit is improved, and the service quality is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of urban rail transit, and in particular to a method and device for collaborative optimization of train scheduling and passenger flow guidance. Background Art

[0002] Nowadays, subways play an important role in relieving traffic pressure and facilitating people's livelihood. However, huge energy consumption leads to huge operating costs, and rail transit in most cities is in the red. Passenger demand and train scheduling are also not well matched. During peak hours, passenger flow is too large and train "supply" cannot keep up; during off-peak hours, passenger flow is small and the number of train passengers is low, resulting in a waste of resources.

[0003] At present, the relevant technologies only unilaterally optimize the train timetable or unilaterally optimize the passenger flow distribution, and there is still more room for optimization of the system as a whole. In addition, most of the relevant technologies use integer programming or heuristic algorithms to solve, which are only suitable for offline optimization. When there are disturbances in trains and passenger flows, it is impossible to adjust the timetable and passenger flow guidance strategy online in real time. Summary of the invention

[0004] In view of this, the present invention proposes a method and device for collaborative optimization of train scheduling and passenger flow guidance, which can achieve two-way online matching of passenger flow demand and train scheduling, effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0005] According to one aspect of the present disclosure, a method for coordinated optimization of train scheduling and passenger flow guidance is provided, comprising:

[0006] Acquire first passenger flow information of a road network within a preset time period, wherein the road network includes a plurality of routes, a plurality of stations are arranged on the routes, and the first passenger flow information indicates the number of passengers traveling from a departure station to a destination station, wherein the departure station and the destination station are any stations in the road network;

[0007] The first passenger flow information is input into the optimization model to perform coordinated optimization of train scheduling and passenger flow guidance to obtain an optimization result, wherein the optimization model is determined by training based on the optimization objectives of minimizing power consumption and minimizing passenger waiting time, and the optimization result indicates the train scheduling information and passenger flow guidance information on each line in the optimized road network.

[0008] In this way, by obtaining the first passenger flow information of the road network within a preset time period, and inputting the first passenger flow information into the optimization model determined by the optimization goal training based on minimizing power supply energy consumption and minimizing passenger waiting time, the train scheduling and passenger flow guidance collaborative optimization is carried out, and the optimization results indicating the train scheduling information on each line in the optimized road network and the passenger flow guidance information in each time period are obtained. This can achieve two-way online matching of passenger flow demand and train scheduling, effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0009] In a possible implementation, the total operating time of each of the lines includes a plurality of continuous operating time periods; the method further includes a training step of the optimization model, which includes: for each of the lines, in chronological order, in each of the operating time periods: allowing the optimization model to first interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; then interacting with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, wherein the passenger flow action space indicates the attraction of each of the lines to the passenger flow, the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow guidance result is used to minimize the waiting time of passengers, the train action space indicates the train scheduling information on each of the lines, and the train scheduling result is used to minimize the power consumption and minimize the waiting time of passengers; when the optimization model that completes the interaction process in all operating time periods of all lines meets the preset constraints and reward goals, the optimization model is used as the trained optimization model.

[0010] In this way, by training the optimization model with the optimization objectives of minimizing the energy consumption of the traction power supply system and the average waiting time of passengers, and by appropriately intervening in passenger flow and rationally allocating passenger flow to the road network, an optimization model can be generated that can collaboratively optimize train scheduling and passenger flow guidance, which is conducive to the subsequent real-time collaborative optimization based on the trained optimization model, thereby effectively alleviating the phenomenon of passenger gathering in the station during peak passenger flow periods.

[0011] In a possible implementation, the optimization model is firstly interacted with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then interacted with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, including: allowing the intelligent agent used for the passenger flow guidance in the optimization model to generate passenger flow actions, and passing the passenger flow actions to the simulation environment, so that the simulation environment calculates the passenger flow matrix, passenger flow status, and passenger flow guidance reward of the current operating time period, and passes the passenger flow status and the passenger flow reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to achieve parameter update, wherein the passenger flow guidance result includes: the passenger flow status and the passenger flow reward, the passenger flow action is determined based on a preset passenger flow action space, the passenger flow action indicates the attraction of the line to the passenger flow in the current operating time period, and the passenger flow matrix indicates the passenger flow of each train on the line in the current operating time period. the number of passengers on the train path, the passenger flow state indicates the change of passenger flow in the current running time period, and the passenger flow guidance reward is related to the passenger waiting time; for each train in the current running time period in order of the train number: the intelligent agent used for the train scheduling in the optimization model generates a train action, and transmits the train action to the simulation environment, so that the simulation environment calculates the train scheduling information, train state, and train scheduling reward of the current train number in combination with the passenger flow matrix, and transmits the train state and the train scheduling reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to achieve parameter update, wherein the train scheduling result includes: the train state and the train scheduling reward, the train action is determined based on a preset train action space, the train action indicates the train scheduling information, the train scheduling information includes the train departure time, inter-station running time, and station parking time of the current train number, the train state indicates the train running process, and the train scheduling reward is related to the power supply energy consumption and the passenger waiting time.

[0012] By separately training the intelligent agent for passenger flow guidance and the intelligent agent for train scheduling, it is possible to generate an intelligent agent network that can collaboratively optimize train scheduling and passenger flow guidance. This is conducive to the subsequent generation of subway timetables and passenger flow path allocation that meet the constraints based on the trained intelligent agent network, which can effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby achieving collaborative optimization of train scheduling and passenger flow guidance.

[0013] In a possible implementation, the passenger flow action space includes the attraction factor of each section to the passenger flow in each operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train action space includes multiple train actions, wherein the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stop time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is less than the number of the second station; and / or, the passenger flow status includes the time sequence of the current operating time period, the average passenger waiting time of each station of each section in the current operating time period, and the average full load rate of each section in the current operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train status includes the passenger waiting time, train departure time, station running time, number of people on board, train departure time, and train arrival time related to each train.

[0014] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0015] In a possible implementation, in the training step of the optimization model: after the parameters of the optimization model are updated based on the passenger flow guidance result, when it is determined that the current operating time period is the last operating time period in the total operating time, it is determined that the parameter updating process is finished; and / or, after the parameters of the optimization model are updated based on the train scheduling result, when it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, it is determined that the parameter updating process is finished; and / or, in the process of updating the parameters of the optimization model based on the passenger flow guidance result, the parameters are updated using a simulation advancement method with a fixed step size; and / or, in the process of updating the parameters of the optimization model based on the train scheduling result, the parameters are updated using a simulation advancement method with a variable step size.

[0016] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0017] In one possible implementation, the constraints are related to departure interval time, U-turn time, station parking time, inter-station running time, number of passengers on a train, passenger path length, number of sections passed by the passenger path, path allocation ratio, number of trains, and interval between adjacent train arrivals.

[0018] In this way, by setting a series of specific constraints for the optimization model, the optimization model can be converged and trained to a model that meets the expectations, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0019] In a possible implementation, the reward target is to maximize the cumulative reward, and the cumulative reward includes the passenger flow guidance reward and the train scheduling reward.

[0020] In this way, by training the intelligent agent with maximizing the cumulative reward as the reward goal, the training generates an optimization model that meets the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0021] According to another aspect of the present disclosure, a train scheduling and passenger flow guidance coordinated optimization device is provided, comprising:

[0022] an acquisition module, the acquisition module being used to acquire first passenger flow information of a road network within a preset time period, wherein the road network includes a plurality of routes, a plurality of stations are arranged on the routes, the first passenger flow information indicates the number of passengers traveling from a departure station to a destination station, and the departure station and the destination station are respectively any one of the stations in the road network;

[0023] A collaborative optimization module, wherein the collaborative optimization module is used to input the first passenger flow information into an optimization model to perform collaborative optimization of train scheduling and passenger flow guidance to obtain an optimization result, wherein the optimization model is determined by training based on the optimization objectives of minimizing power consumption and minimizing passenger waiting time, and the optimization result indicates the train scheduling information and passenger flow guidance information on each line in the optimized road network.

[0024] In this way, the first passenger flow information of the road network within a preset time period is obtained through the acquisition module, and the first passenger flow information is input into the optimization model determined by the optimization goal training based on minimizing power supply energy consumption and minimizing passenger waiting time through the collaborative optimization module to perform collaborative optimization of train scheduling and passenger flow guidance, and obtain the optimization results of the train scheduling information on each line in the optimized road network and the passenger flow guidance information in each time period, which can realize two-way online matching of passenger flow demand and train scheduling, effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0025] In a possible implementation, the total operation time of each of the lines includes a plurality of continuous operation time periods; the device also includes a training module, the training module is used to execute the training steps of the optimization model, the training steps of the optimization model include: for each of the lines, in chronological order, in each of the operation time periods: the optimization model is first allowed to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; then, the optimization model is interacted with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, wherein the passenger flow action space indicates the attraction of each of the lines to the passenger flow, the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow guidance result is used to minimize the waiting time of passengers, the train action space indicates the train scheduling information on each of the lines, and the train scheduling result is used to minimize the power consumption and minimize the waiting time of passengers; when the optimization model that completes the interaction process in all the operation time periods of all the lines meets the preset constraints and reward goals, the optimization model is used as the trained optimization model.

[0026] In this way, by training the optimization model with the optimization objectives of minimizing the energy consumption of the traction power supply system and the average waiting time of passengers, and by appropriately intervening in passenger flow and rationally allocating passenger flow to the road network, an optimization model can be generated that can collaboratively optimize train scheduling and passenger flow guidance, which is conducive to the subsequent real-time collaborative optimization based on the trained optimization model, thereby effectively alleviating the phenomenon of passenger gathering in the station during peak passenger flow periods.

[0027] In a possible implementation, the optimization model is firstly interacted with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then interacted with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, including: allowing the intelligent agent used for the passenger flow guidance in the optimization model to generate passenger flow actions, and passing the passenger flow actions to the simulation environment, so that the simulation environment calculates the passenger flow matrix, passenger flow status, and passenger flow guidance reward of the current operating time period, and passes the passenger flow status and the passenger flow reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to achieve parameter update, wherein the passenger flow guidance result includes: the passenger flow status and the passenger flow reward, the passenger flow action is determined based on a preset passenger flow action space, the passenger flow action indicates the attraction of the line to the passenger flow in the current operating time period, and the passenger flow matrix indicates the passenger flow of each train on the line in the current operating time period. the number of passengers on the train path, the passenger flow state indicates the change of passenger flow in the current running time period, and the passenger flow guidance reward is related to the passenger waiting time; for each train in the current running time period in order of the train number: the intelligent agent used for the train scheduling in the optimization model generates a train action, and transmits the train action to the simulation environment, so that the simulation environment calculates the train scheduling information, train state, and train scheduling reward of the current train number in combination with the passenger flow matrix, and transmits the train state and the train scheduling reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to achieve parameter update, wherein the train scheduling result includes: the train state and the train scheduling reward, the train action is determined based on a preset train action space, the train action indicates the train scheduling information, the train scheduling information includes the train departure time, inter-station running time, and station parking time of the current train number, the train state indicates the train running process, and the train scheduling reward is related to the power supply energy consumption and the passenger waiting time.

[0028] By separately training the intelligent agent for passenger flow guidance and the intelligent agent for train scheduling, it is possible to generate an intelligent agent network that can collaboratively optimize train scheduling and passenger flow guidance. This is conducive to the subsequent generation of subway timetables and passenger flow path allocation that meet the constraints based on the trained intelligent agent network, which can effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby achieving collaborative optimization of train scheduling and passenger flow guidance.

[0029] In a possible implementation, the passenger flow action space includes the attraction factor of each section to the passenger flow in each operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train action space includes multiple train actions, wherein the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stop time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is less than the number of the second station; and / or, the passenger flow status includes the time sequence of the current operating time period, the average passenger waiting time of each station of each section in the current operating time period, and the average full load rate of each section in the current operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train status includes the passenger waiting time, train departure time, station running time, number of people on board, train departure time, and train arrival time related to each train.

[0030] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0031] In a possible implementation, in the training step of the optimization model executed by the training module: after the parameters of the optimization model are updated based on the passenger flow guidance result, when it is determined that the current operating time period is the last operating time period in the total operating time, it is determined that the parameter updating process is finished; and / or, after the parameters of the optimization model are updated based on the train scheduling result, when it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, it is determined that the parameter updating process is finished; and / or, in the process of updating the parameters of the optimization model based on the passenger flow guidance result, the parameters are updated using a simulation advancement method with a fixed step size; and / or, in the process of updating the parameters of the optimization model based on the train scheduling result, the parameters are updated using a simulation advancement method with a variable step size.

[0032] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0033] In one possible implementation, the constraints are related to departure interval time, U-turn time, station parking time, inter-station running time, number of passengers on a train, passenger path length, number of sections passed by the passenger path, path allocation ratio, number of trains, and interval between adjacent train arrivals.

[0034] In this way, by setting a series of specific constraints for the optimization model, the optimization model can be converged and trained to a model that meets the expectations, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0035] In a possible implementation, the reward target is to maximize the cumulative reward, and the cumulative reward includes the passenger flow guidance reward and the train scheduling reward.

[0036] In this way, by training the intelligent agent with maximizing the cumulative reward as the reward goal, the training generates an optimization model that meets the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0039] Figure 1 A flow chart of a method for collaborative optimization of train scheduling and passenger flow guidance provided according to an embodiment of the present disclosure is shown.

[0040] Figure 2 A schematic diagram of a road network provided according to an embodiment of the present disclosure is shown.

[0041] Figure 3 A schematic diagram of a single line provided according to an embodiment of the present disclosure is shown.

[0042] Figure 4A schematic diagram of a transfer station provided according to an embodiment of the present disclosure is shown.

[0043] Figure 5 A schematic diagram of a transfer station provided according to an embodiment of the present disclosure is shown.

[0044] Figure 6 A schematic diagram of a train scheduling optimization framework based on multi-agent reinforcement learning provided according to an embodiment of the present disclosure is shown.

[0045] Figure 7 A schematic diagram showing an optimization model training process provided according to an embodiment of the present disclosure.

[0046] Figure 8 A schematic diagram showing a total reward curve of a training process provided according to an embodiment of the present disclosure.

[0047] Fig. 9 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 1 provided according to an embodiment of the present disclosure is shown.

[0048] Fig.10 A schematic diagram showing the passenger flow demand and departure interval curve in the downlink direction of Line 1 provided according to an embodiment of the present disclosure is shown.

[0049] Fig.11 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 2 provided according to an embodiment of the present disclosure is shown.

[0050] Fig.12 A schematic diagram showing the passenger flow demand and departure interval curve in the downlink direction of Line 2 provided according to an embodiment of the present disclosure is shown.

[0051] Fig.13 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 3 provided according to an embodiment of the present disclosure is shown.

[0052] Fig.14 A schematic diagram showing the passenger flow demand and departure interval curve in the downlink direction of Line 3 provided according to an embodiment of the present disclosure is shown.

[0053] Fig.15 A schematic diagram showing the optimized uplink timetable of Line 1 provided according to an embodiment of the present disclosure is shown.

[0054] Fig.16 A schematic diagram showing the departure interval and passenger flow demand curve under the original passenger flow and sudden passenger flow in the upward direction of Line 3 provided according to an embodiment of the present disclosure is shown.

[0055] Fig.17 A schematic diagram showing the departure interval and passenger flow demand curve under the original passenger flow and sudden passenger flow in the down direction of Line 3 provided according to an embodiment of the present disclosure is shown.

[0056] Fig.18 A block diagram of a train scheduling and passenger flow guidance collaborative optimization device provided according to an embodiment of the present disclosure is shown.

[0057] Fig.19 It is a block diagram of an apparatus for executing a method for collaborative optimization of train scheduling and passenger flow guidance provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0059] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0060] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0061] In order to facilitate those skilled in the art to understand the technical solution provided by the embodiments of the present disclosure, the technical environment in which the technical solution is implemented is described below.

[0062] From the perspective of the flow of materials, energy and information, vehicle flow, passenger flow and energy flow are the main bodies of the dynamic operation of urban rail transit systems. The three do not exist independently, but interact with each other. In terms of vehicle flow, the optimization of train operation curves and scheduling will directly affect the train power and energy consumption, thereby affecting the energy flow. From the perspective of passenger flow, train scheduling will affect passenger waiting time and passenger service quality. Therefore, how to match passenger flow demand and train scheduling while ensuring passenger service quality and facilitating passenger travel, so as to improve the energy efficiency of urban rail transit, is a problem currently faced.

[0063] The disclosed embodiment provides a method for collaborative optimization of train scheduling and passenger flow guidance, which obtains first passenger flow information of a road network within a preset time period, inputs the first passenger flow information into an optimization model determined by training based on the optimization objectives of minimizing power supply energy consumption and minimizing passenger waiting time, and performs collaborative optimization of train scheduling and passenger flow guidance, thereby obtaining optimization results indicating train scheduling information on each line in the optimized road network and passenger flow guidance information in each time period. In this way, two-way online matching of passenger flow demand and train scheduling can be achieved, effectively reducing the energy consumption of the traction power supply system and shortening the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0064] Figure 1 A flow chart of a method for collaborative optimization of train scheduling and passenger flow guidance provided according to an embodiment of the present disclosure is shown. Figure 1 As shown, the collaborative optimization method may include:

[0065] S101. Acquire first passenger flow information of a road network within a preset time period.

[0066] S102: Input the first passenger flow information into the optimization model to perform coordinated optimization of train scheduling and passenger flow guidance to obtain an optimization result.

[0067] Through step S101, the first passenger flow information of the road network within a preset time period can be obtained to determine the input data of the subsequent optimization model.

[0068] The road network may include multiple lines. Each line may be provided with multiple stations. A corresponding total operating time may be set for each line, and the total operating time of each line may include multiple continuous operating time periods. Figure 2 FIG. 1 is a schematic diagram of a road network provided according to an embodiment of the present disclosure. Figure 2 As shown, the road network includes line l1 (i.e., Metro Line 1), line l2 (i.e., Metro Line 2), and line l3 (i.e., Metro Line 3), and each line is provided with a plurality of stations.

[0069] The first passenger flow information may indicate the number of passengers traveling from the departure station to the destination station, wherein the departure station and the destination station may be any station in the road network. Figure 2 As shown, the departure station may be the MJY station in Metro Line 1, and the destination station may be the DXC station in Metro Line 1. Riding from the MJY station to the DXC station is a ride path. The first passenger flow information may be the number of passengers in all possible ride paths within a preset time period. In this case, the first passenger flow information may be regarded as a passenger flow matrix (or OD matrix, i.e., the number of passengers in each ride path on the line within the operating time period, see below for details). Alternatively, the first passenger flow information may also select the number of passengers in some ride paths according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0070] Through step S102, the first passenger flow information can be input into the optimization model to perform coordinated optimization of train scheduling and passenger flow guidance, and an optimization result can be obtained to deal with train scheduling and passenger flow guidance problems based on the optimization result.

[0071] The optimization model can be determined by training based on the optimization objectives of minimizing power supply energy consumption and minimizing passenger waiting time. The power supply energy consumption is the sum of the output energy of the traction substations in each power supply partition in the DC side traction power supply system, that is, ∑ j E j (t), E j (t) is shown in the following formula (3). Passenger waiting time is the total waiting time of passengers at the station divided by the number of people arriving at the station. By using the trained optimization model, the optimization result can be directly obtained, which is convenient and efficient.

[0072] The optimization result may indicate the train dispatching information and passenger flow guidance information on each line in the optimized road network. The train dispatching information may include the train schedule, that is, it may include the train departure time, station running time, and station parking time of each train. The train departure time may refer to the initial departure time of each train, the station running time may refer to the running time of the train between each station, and the station parking time may refer to the parking time of the train at each station. The passenger flow guidance information may include the proportion of each OD selecting different paths in each operating time period. It should be noted that although the train dispatching information is introduced with the train departure time, train running time, and train arrival time as examples, those skilled in the art can understand that the embodiments of the present disclosure should not be limited to this. In fact, the user can flexibly set the train dispatching information according to the actual application scenario, as long as the determined train dispatching information can cope with the problems of train dispatching and passenger flow guidance.

[0073] In this way, through step S101, the first passenger flow information of the road network within a preset time period can be obtained, and through step S102, the first passenger flow information can be input into the optimization model determined by the optimization goal training based on minimizing power supply energy consumption and minimizing passenger waiting time to perform coordinated optimization of train scheduling and passenger flow guidance, and obtain the optimization results indicating the train scheduling information on each line in the optimized road network and the passenger flow guidance information in each time period, which can achieve two-way online matching of passenger flow demand and train scheduling, effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0074] In a possible implementation, during or before training the optimization model, the collaborative optimization method may further include: constructing a mathematical model, thereby training the optimization model based on a series of constructed models. The mathematical model may include a road network model, a vehicle flow calculation model, and a passenger flow calculation model.

[0075] The road network model indicates each line and station in the road network. is a set of lines and A line set that does not distinguish between uplink and downlink. L is the number of the Lth line and can indicate the line set. The number of lines in is the number of the lth line and is the line number that does not distinguish between up and down directions. Since each line has an up direction and a down direction, the up and down stations are the same physical entity, but they can have different parameters and states in the passenger flow and vehicle flow calculations. In order to distinguish, this paper uses To represent the set of lines in different directions, To distinguish the uplink and downlink line sets, To distinguish the uplink and downlink line numbers, For the uplink direction, For the downlink direction line.

[0076] Figure 3 A schematic diagram showing station numbers on a single line and in different directions provided according to an embodiment of the present disclosure is shown. Figure 3 As shown, in a single line l, Gather for the station, where The corresponding value is the total number of stations on line l, The numbering of each station includes the station number in the upward direction and the station number in the downward direction. The starting station number in the upward direction is 1. l The terminal is R l , the starting station number in the downlink direction is 1 -l The terminal is R -l .

[0077] The traffic flow calculation model can be used for train scheduling. For any line, in the train energy consumption calculation, the optimal speed curve of "maximum acceleration traction-coasting-maximum deceleration braking" is used for inter-station train operation, that is, the train uses the maximum traction gear to reach the predetermined speed during traction, then coasts in neutral, and uses the maximum braking gear until it stops during braking. The air and friction resistance are approximately calculated using the Davis equation, and then the mechanical power P of train i is obtained based on the dynamic characteristics of the train. i (t).

[0078] The traction power supply system of a single line is composed of multiple power supply sections, each of which contains multiple traction substations, trains and overhead lines. There can be multiple stations in a power supply section, and all trains passing through the station can obtain energy output from the power supply section. Obviously, a power supply section can provide energy for multiple trains at the same time. The traction substations in the same power supply section are connected together, and the traction and braking energy of the trains in the same power supply section can be used by each other.

[0079] In the traffic flow calculation model, the sum of the motor power of the trains in the traction state in the power supply section j at time t can be determined by the following formula (1):

[0080]

[0081] In the formula, represents the sum of the electric power of the train motors in the traction state in the power supply section j at time t, δ i，j Indicates whether train i is in the power supply section j. If it is in the power supply section j, then δ i，j =1, otherwise δ i，j =0;P i (t) represents the mechanical power of train i; η1 represents the conversion efficiency of the train from electrical energy to mechanical energy; max function represents the maximum value function.

[0082] Similarly, in the traffic flow calculation model, the sum of the motor power of the trains in the braking state in the power supply section j at time t can be determined by the following formula (2):

[0083]

[0084] In the formula, represents the sum of the motor power of the train in braking state in the power supply section j at time t, δ i，j Indicates whether train i is in the power supply section j. If in δ i，j =1, otherwise δ i，j =0; η2 represents the conversion efficiency of train mechanical energy to electrical energy; P i (t) represents the mechanical power of train i; min function represents the function for finding the minimum value.

[0085] Therefore, in the traffic flow calculation model, the output energy and E of the DC traction substation in the power supply section j from the start of simulation (i.e., the interaction with the simulation environment begins in the following text) to time t can be determined by the following formula (3): j (t):

[0086]

[0087] In the formula, E j(t) represents the output energy and the energy of the DC traction substation in the power supply section j at time t, represents the sum of the electric power of the train motors in the traction state in the power supply section j at time u; r Indicates the efficiency of the train's regenerative braking energy being directly utilized by other trains; It represents the sum of the electric powers of the train motors in the braking state in the power supply section j at time u; the max function represents the maximum value function.

[0088] In the traffic flow calculation model, the number of trains running on line l can be determined as k by the following formula (4): l The train is at the station Departure time

[0089]

[0090] In the formula, Indicates that the train number is k l The train is at the station The departure time, For train number k l The departure time of the train, For train number k l The train is at the station parking time, For train number k l The train is at the station and Station It should be noted that the running time between train number k l Trains are numbered according to their departure times. For example, the first train to depart on a certain line (i.e. the first train or the first train) is the first train.

[0091] Similarly, in the traffic flow calculation model, the train number k can be determined by the following formula (5): l The train is at the station Arrival time

[0092]

[0093] In the formula, Indicates that the train number is k l The train is at the station Arrival time, Indicates that the train number is k l The train is at the station (Right now The departure time of the last stop of Indicates that the train number is k l The train is at the station and Station The running time between.

[0094] The passenger flow calculation model can be used for passenger flow guidance. The passenger flow calculation model can include OD demand, passenger waiting time, passenger flow distribution characteristics, and transfer paths.

[0095] OD demand can be used to describe the passenger flow from each departure station to each destination station in the road network per unit time. OD demand will change over time.

[0096] Let φ be the sampling period, T begin is the starting departure time of the road network every day, T end is the earliest time for the last bus of the road network to arrive at the terminal every day, B = T end -T begin is the total time period, then the OD demand of line l in the (τ-φ, τ] period can be obtained by the OD matrix OD l (τ) represents, where τ∈{T begin +φ,...,T begin +Bφ} (that is, starting from the earliest departure time of the road network every day, the OD demand is determined once every sampling period), the OD matrix OD l The rows and columns of (τ) represent the departure station and the destination station respectively. Indicates that there are different departure stations during this period. To different destinations of passengers.

[0097] Regarding the waiting time of passengers, we can first model the passenger entry and exit and boarding and exiting behaviors.

[0098] In terms of passengers entering and leaving the station, the OD matrix can be used to obtain the number of passengers leaving the station at time t. To the station The passenger arrival rate is (Unit: person / unit time), then within the time period [t1, t2], line l starts from the station To the station The number of passengers entering the station is Determined by integration over time.

[0099] In terms of boarding and alighting behavior, If all the passengers waiting in the train can board without causing the train to be overloaded, the train will be determined at the station. All passengers waiting in the station must board the bus. Otherwise, A preset proportion of waiting passengers are allowed to board the train so that the train is just full.

[0100] The station at time t can be determined by the above passenger entry and exit and boarding and exiting behaviors. The number of passengers waiting in The time from the departure of the i-1th train to the arrival of the i-th train at the station can be determined by the following formula (6): Passenger waiting time

[0101]

[0102] In the formula, It means from the departure of the i-1th train to the arrival of the i-th train at the station Passenger waiting time Indicates that the i-th train is at the station The departure time, Indicates that the i-1th train is at the station The departure time, Indicates the station at time t The number of passengers waiting inside.

[0103] Therefore, the sum of the waiting time of all trains at all stations can be determined by the following formula (7), that is, the total passenger waiting time w of line l l :

[0104]

[0105] In the formula, w l represents the total passenger waiting time of line l, Indicates the upstream direction of line l or downlink direction The number of train departures, It means from the departure of the i-1th train to the arrival of the i-th train at the station Passenger waiting time.

[0106] In view of the characteristics of passenger flow distribution, in order to simplify the road network, the departure station, transfer station and terminal station on each line in the road network can be used as the dividing points to divide it into different sections, and the passenger flow distribution index of the section can be used to represent the passenger flow distribution of the road network. For a line, the line has an upward direction and a downward direction, so the section also has an upward direction and a downward direction. Figure 4 FIG. 1 is a schematic diagram of a transfer station provided according to an embodiment of the present disclosure. Figure 4 As shown, the segmentation points in the road network are Figure 4The 8 stations selected in the red circle can get 10 sections (for one direction, there are 20 sections in two directions), namely, section one from station JBJCT2HZL to station NJT, section two from station NJT to station DP, section three from station NJT to station LLK, section four from station NJT to station JCK, section five from station DP to station BS, section six from station DP to station YD, section seven from station DP to station LLK, section eight from station LLK to station JCK, section nine from station JCK to station XSZ, and section ten from station LLK to station YD.

[0107] Let the total number of sections in the road network be D, d = 1, 2, ..., D, J d is the set of stations included in section d, R d is the total number of stations included in section d, then the following formula (8) can be used to determine the section d in (T begin +(j-1)φ,T begin +jφ| Average passenger waiting time per station in time period (j=1, 2, ..., B)

[0108]

[0109] In the formula, Indicates that segment d is in (T begin +(j-1)φ,T begin The average waiting time of passengers at each station in the time period (j = 1, 2, ..., B), T begin Indicates the daily starting departure time of the road network. Indicates the station at time t The number of passengers waiting inside, R d Indicates the total number of stations contained in section d.

[0110] The segment d can be determined by the following formula (9): begin +(j-1)φ,T begin +jφ] average full load factor during the period

[0111]

[0112] In the formula, Indicates that segment d is in (T begin +(j-1)φ,T begin +jφ] the average full load factor during the period, represents the total number of passengers on the corresponding train, For Line The i-th train leaves the station The number of passengers on the bus at the time, Rd represents the total number of stations contained in section d, C max is the maximum number of passengers on a single train, Indicates that all (T begin +(j-1)φ,T begin +jφ] period, the number of trains passing through section d, The calculation of the average load factor is for trains leaving the station. The time when (T begin +(j-1)φ,T begin +jφ] all trains within the time period.

[0113] For transfer routes, the transfer passenger flow can be equated with the non-transfer passenger flow to provide a basis for calculating the passenger waiting time. Figure 5 A schematic diagram of a transfer station provided according to an embodiment of the present disclosure is shown. Figure 5 As shown in the figure, an equivalent method is used. A passenger chooses the path "abcghlm" to take a bus from station a to station m. Then, the path can be split into three paths: "abc", "cgh", and "hlm". Then, three passengers are generated to choose the three paths "abc", "cgh", and "hlm" respectively and added to the OD matrix. Therefore, when calculating the waiting time of passengers, the waiting time of the three passengers is taken.

[0114] When calculating the total number of passengers, the total number of passengers before equivalence can be used, that is, there is no need to perform equivalent conversion of transfer passenger flow.

[0115] The collaborative optimization method provided by the embodiments of the present disclosure models the train scheduling and passenger flow guidance problems into Markov decision processes respectively, that is, multiple agents can be divided to be responsible for train scheduling on multiple lines and one agent can be divided to be responsible for passenger flow guidance, and a simulation environment for train operation and passenger flow can be built according to the mathematical model constructed above, so as to enable the intelligent agent to interact with the simulation environment, perceive the environmental state and select better passenger flow actions and train actions, thereby realizing collaborative optimization of train scheduling and passenger flow guidance based on multi-agent reinforcement learning.

[0116] For the train scheduling problem, multiple agents for train scheduling optimization can be set up in the optimization model, so that each of these agents is responsible for optimizing the train scheduling information (i.e., timetable) of all trains in one direction on one line. Figure 2There are six agents A1, A2, A3, A4, A5 and A6 set for the three lines in the road network shown, among which agent A1 is used for optimizing the timetables of all trains in the upward direction of line l1, agent A2 is used for optimizing the timetables of all trains in the downward direction of line l1, agent A3 is used for optimizing the timetables of all trains in the upward direction of line l2, agent A4 is used for optimizing the timetables of all trains in the downward direction of line l2, agent A5 is used for optimizing the timetables of all trains in the upward direction of line l3, and agent A6 is used for optimizing the timetables of all trains in the downward direction of line l3.

[0117] Each agent used for train scheduling optimization may correspond to multiple stages, and each stage or the state of each stage (i.e., the train state) describes the process of a train running from the starting station to the terminal station. Taking the above-mentioned agent A1 as an example, the first stage of agent A1 represents the process of the first train (or the first trip) in the upward direction of line l1 running from the starting station of line l1 to the terminal station of line l1, and the second stage of agent A1 represents the process of the second train (or the second trip) in the upward direction of line l1 running from the starting station of line l1 to the terminal station of line l1. By analogy, the nth stage of agent A1 represents the process of the nth train (or the nth trip) in the upward direction of line l1 running from the starting station of line l1 to the terminal station of line l1.

[0118] Each agent used for train scheduling optimization can have different states at each stage. The responsible line can be determined by the following formula (10): The state of the train dispatching agent at stage n (i.e. train status):

[0119]

[0120] In the formula, Indicates responsible line The train status of the train dispatching agent at the nth stage, Indicates line The n-1th train leaves the station After the nth train arrives at the previous station The passenger waiting time (can be determined by the above formula (6)), Indicates line The departure time of the n-1th train in the opposite direction, Indicates line The n-1th train in the opposite direction is at the station and Station The running time between Indicates line The nth train arrives at the station The number of people boarding the bus later, Indicates line The nth train is at the station The departure time (i.e. the time when the train leaves the station), Indicates line The nth train is at the station Thus, in one possible implementation, the train status includes the passenger waiting time, train departure time, inter-station running time, number of people on board, train departure time, and train arrival time associated with each train number.

[0121] In formula (10), n can be 0, 1, 2, ..., N. When n = 0, the initial train state can be set as When n=N, the final train state for That is, when the time a bus takes to reach the terminal is greater than the earliest time T that the last bus of the network reaches the terminal every day, end , then the optimization of the agent ends (it can also be called the end of this round of training of the agent). Therefore, in a possible implementation, the training step of the optimization model may also include: after updating the parameters of the optimization model based on the train scheduling result, if it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, determining that the parameter update process ends.

[0122] For responsible lines The train dispatching agent defines the state transfer function Thus there is in, Can be regarded as relative to The next train state, Indicates responsible line The train action space of the train scheduling agent, It can be determined by the following formula (11):

[0123]

[0124] In the formula, Indicates responsible line The train action space of the train scheduling agent, Indicates line The departure interval between the n+1th train and the nth train, Indicates line The n+1th train is at the station and Station The running time between Indicates line The n+1th train is at the station Thus, in a possible implementation, the train action space includes a plurality of the train actions, and the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stopping time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is smaller than the number of the second station.

[0125] When the initial train state is set After that, responsible for the line The train dispatching agent (hereinafter referred to as the agent) generates train actions To the simulation environment, the simulation environment can be based on the state transfer function Generate next train state Give it to the agent, and repeat this cycle. The train scheduling agent, if the departure time of the nth train is The time to arrive at the terminal is but When Pass it to the agent, and the agent will move the train To the simulation environment. At this time, time t goes back to time, because usually the departure time of this vehicle is less than the time when the previous vehicle arrives at the terminal, the simulation environment simulates to Time, generate the next train state Repeat this cycle.

[0126] Responsible for the line The train dispatching agent (hereinafter referred to as the agent) can generate the next train action based on the train status of the previous step. The parameters of the agent network are randomly set at first, and then they are continuously optimized according to the training rewards (i.e., train dispatching rewards) and train status transmitted by the simulation environment, so as to train the agent that meets the optimization goal. Among them, when conducting targeted training in train dispatching, the simulation advancement method with variable step size can be used to update the parameters of the agent network.

[0127] The optimization goal set in the embodiment of the present disclosure for dealing with the train scheduling problem is to minimize the power consumption and minimize the waiting time of passengers. By designing a reward related to energy consumption and a reward related to waiting time, and taking the weighted sum of these two rewards as the train scheduling reward, the train scheduling reward at the nth stage can be determined by the following formula (12):

[0128]

[0129] In the formula, represents the train dispatch reward in the nth stage, b1 and b2 represent weighted coefficients, represents the reward related to energy consumption, Represents the reward associated with the waiting time, reward and rewards This can be determined through the following process:

[0130] make for At each station It is the total waiting time of passengers at each station from the time when the n-1th train leaves to the time when the nth train arrives at each station. for At each station is the total number of passengers waiting at the platform when the nth train arrives at each station, and w0 is the known benchmark average waiting time. The difference between the waiting time of the new train scheduling plan and the benchmark can be obtained by To portray, that is: Let b3 and b4 be weighted coefficients and sgn(·) be the sign function, then the reward It can be determined by the following formula (13):

[0131]

[0132] In the formula, represents the train scheduling reward, sgn(·) represents the sign function, represents the difference between the waiting time of the new train scheduling scheme and the benchmark, express At each station The sum of Indicates line The nth train arrives at the station is the number of passengers waiting at the platform, b3 and b4 represent weighting coefficients.

[0133] award It can be determined by the following formula (14):

[0134]

[0135] In the formula, represents the reward related to energy consumption, Indicates that the simulation starts at The DC traction substation outputs energy at all times. represents the time when the nth train arrives at the terminal station, Indicates the time from the start of simulation to the end of simulation under the benchmark scheme Output energy of DC traction substation at each moment.

[0136] Therefore, in one possible manner, the collaborative optimization method may also include a training step for the optimization model, and the training step for the optimization model may include a training process for processing train scheduling problems: for each line, in chronological order and in each operating time period: let the optimization model interact with the simulation environment based on the train action space to perform train scheduling, so as to update the parameters of the optimization model based on the train scheduling results, wherein the simulation environment is used to provide an interactive environment for the optimization model, the train action space indicates the train scheduling information on each line, and the train scheduling results are used to minimize power consumption.

[0137] For the passenger flow guidance problem, multiple agents for passenger flow guidance optimization can be set up in the optimization model, so that each of these agents is responsible for optimizing the passenger flow information on a route. Figure 2 The three routes in the road network shown are equipped with three agents A1', A2', and A3', wherein agent A1' is used to optimize the passenger flow information on route l1, agent A2' is used to optimize the passenger flow information on route l2, and agent A3' is used to optimize the passenger flow information on route l3.

[0138] Each intelligent agent used for passenger flow optimization can correspond to multiple stages, and each stage or the state of each stage (i.e., passenger flow state) describes the passenger flow in (T begin +(n-1)φ,T begin +nφ] time period, n = 1, 2, ..., B, B is the total time period. Taking the above agent A1' as an example, the first stage of agent A1' is represented by (T begin , T begin +φ], the passenger flow changes on route l1 during the period, the second stage of agent A1' is shown in (T begin +φ,T begin +2φ] period on route l1, and so on. The nth stage of agent A1' is represented by (T begin +(n-1)φ,T begin +nφ] changes in passenger flow on line l1 during the period.

[0139] Each intelligent agent used for passenger flow guidance optimization can have different states at each stage. The responsible route can be determined by the following formula (15): The state of the passenger flow guiding agent at the nth stage (i.e. passenger flow status):

[0140]

[0141] In the formula, Indicates responsible line The passenger flow state of the passenger flow guiding agent at the nth stage, where n represents the time period. represents the average passenger waiting time at the station in each section during the period, Indicates the average full load rate of each section in the time period, and the section division method is detailed above. Therefore, in a possible implementation, the passenger flow state includes the time sequence of the current operation time period, the average passenger waiting time of each station in each section in the current operation time period, and the average full load rate of each section in the current operation time period, wherein the sections are obtained by dividing the multiple lines.

[0142] In formula (15), n can be 0, 1, 2, ... B. When n = 0, the initial passenger flow state can be set When n = B, the final passenger flow state for That is, when n=B, the optimization of the agent ends (it can also be called the end of this round of training of the agent). Therefore, in a possible implementation, the training step of the optimization model may further include: after updating the parameters of the optimization model based on the passenger flow guidance result, when it is determined that the current operating time period is the last operating time period in the total operating time, determining that the parameter updating process ends.

[0143] For responsible lines The intelligent agent for passenger flow guidance is defined as the state transfer function P psg , so there is in, Can be regarded as relative to The next passenger flow state, Indicates responsible line The passenger flow action space of the intelligent agent guided by the passenger flow, It can be determined by the following formula (16):

[0144]

[0145] In the formula, Indicates responsible line The passenger flow action space of the intelligent agent guided by the passenger flow, ω n，d Indicates (T begin +nφ,T begin Thus, in a possible implementation, the passenger flow action space includes the attraction factor of each segment to the passenger flow in each operating time period, wherein the segments are obtained by dividing the multiple routes.

[0146] In addition, it is necessary to convert the passenger flow attraction factor of each section into the distribution ratio of each OD demand on each path. The specific process is as follows: Let M ij is the total number of all paths of OD pairs from station i to station j, is the set of intervals that the mth path of the OD pair from station i to station j passes through, is (T begin +nφ,T begin +(n+1)φ] period, the proportion of passengers from station i to station j choosing the mth path needs to satisfy but

[0147] When the initial passenger flow state is set After that, responsible for the line The passenger flow guided agent (hereinafter referred to as the agent) generates passenger flow actions Give the simulation environment, the simulation environment can be based on the state transfer function P psg Generate the next passenger flow state For the agent, and so on t = T begin +nφ, the state of the simulation environment is The intelligent agent will move the passenger flow Pass it to the simulation environment, and the simulation environment performs (T begin +nφ,T begin +(n+1)φ] time period, t = T begin +(n+1)φ generates the next passenger flow state Give the agent.

[0148] Responsible for the line The passenger flow guiding agent (hereinafter referred to as the agent) can generate the next passenger flow action according to the passenger flow status of the previous step. The parameters of the agent network are randomly set at first, and then they are continuously optimized according to the training rewards (i.e. passenger flow guiding rewards) and passenger flow status transmitted by the simulation environment, so as to train the agent that meets the optimization goal. Among them, when conducting targeted training in passenger flow guidance, the fixed-step simulation advancement method can be used to update the parameters of the agent network.

[0149] The optimization goal set in the embodiment of the present disclosure to deal with the passenger flow guidance problem is to minimize the passenger waiting time. The passenger flow guidance reward in the nth stage can be determined by the following formula (17):

[0150]

[0151] In the formula, represents the passenger flow guidance reward in the nth stage, b5 represents the weighted coefficient, is the waiting time for all passengers to take the shortest path in the benchmark scenario, Indicates that segment d is in (T begin +(i-1)φ,T begin +iφ] average station passenger waiting time during the period, δ represents the length of the period considered. Describes the begin +(n-δ-1)φ,T begin +nφ] period, the difference between the passenger waiting time of the passenger flow guidance scheme and the benchmark scheme.

[0152] Therefore, in one possible manner, the collaborative optimization method may also include a training step of the optimization model, and the training step of the optimization model may include a training process for handling the passenger flow guidance problem: for each line, in chronological order and in each operating time period: let the optimization model interact with the simulation environment based on the passenger flow action space to guide the passenger flow, and then update the parameters of the optimization model based on the passenger flow guidance results, wherein the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow action space indicates the attractiveness of each of the lines to the passenger flow, and the passenger flow guidance results are used to minimize the waiting time of passengers.

[0153] In a possible implementation, the optimization target provided by the embodiment of the present disclosure can be achieved by maximizing the cumulative reward, that is, the reward target can be to maximize the cumulative reward, and the cumulative reward includes the passenger flow guidance reward and the train scheduling reward.

[0154] Cumulative rewards can be for responsible lines The train scheduling agent obtains the train scheduling reward at all stages and is responsible for the line The sum of the passenger flow guidance rewards received by the passenger flow guidance agent in all its stages, that is, the cumulative reward Among them, r n include and The optimization goal of reinforcement learning is to maximize the cumulative reward R.

[0155] In this way, by training the intelligent agent with maximizing the cumulative reward as the reward goal, the training generates an optimization model that meets the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0156] In a possible implementation, the collaborative optimization method may further include a training step of the optimization model, and the training step of the optimization model may include a training process for processing train scheduling problems and a training process for processing passenger flow guidance problems, namely:

[0157] For each of the lines, in chronological order and in each of the operating time periods: the optimization model is first allowed to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then the optimization model is allowed to interact with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, wherein the passenger flow action space indicates the attraction of each of the lines to the passenger flow, the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow guidance result is used to minimize the waiting time of passengers, the train action space indicates the train scheduling information on each of the lines, and the train scheduling result is used to minimize the power consumption and minimize the waiting time of passengers;

[0158] When the optimization model of the interaction process in all operating time periods of all lines satisfies the preset constraints and reward targets, the optimization model is used as the trained optimization model.

[0159] In this way, by training the optimization model with the optimization objectives of minimizing the energy consumption of the traction power supply system and the average waiting time of passengers, and by appropriately intervening in passenger flow and rationally allocating passenger flow to the road network, an optimization model can be generated that can collaboratively optimize train scheduling and passenger flow guidance, which is conducive to the subsequent real-time collaborative optimization based on the trained optimization model, thereby effectively alleviating the phenomenon of passenger gathering in the station during peak passenger flow periods.

[0160] In a possible implementation, the training step of the optimization model includes firstly allowing the optimization model to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then interacting with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, which may include:

[0161] The intelligent agent used for the passenger flow guidance in the optimization model generates a passenger flow action, and transmits the passenger flow action to the simulation environment, so that the simulation environment calculates the passenger flow matrix, passenger flow state, and passenger flow guidance reward of the current operation time period, and transmits the passenger flow state and the passenger flow reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to realize parameter update,

[0162] The passenger flow guidance result includes: the passenger flow state and the passenger flow reward; the passenger flow action is determined based on a preset passenger flow action space; the passenger flow action indicates the attraction of the route to the passenger flow in the current operation time period; the passenger flow matrix indicates the number of passengers on each riding path on the route in the current operation time period; the passenger flow state indicates the change of passenger flow in the current operation time period; and the passenger flow guidance reward is related to the passenger waiting time;

[0163] For each train in the current running time period, in order of train sequence: the agent used for train scheduling in the optimization model generates a train action, and transmits the train action to the simulation environment, so that the simulation environment calculates the train scheduling information, train status, and train scheduling reward of the current train in combination with the passenger flow matrix, and transmits the train status and the train scheduling reward to the agent, so that the agent adjusts its own strategy to achieve parameter update,

[0164] Among them, the train scheduling result includes: the train status and the train scheduling reward, the train action is determined based on a preset train action space, the train action indicates the train scheduling information, the train scheduling information includes the train departure time, inter-station running time, and station parking time of the current train, the train status indicates the train operation process, and the train scheduling reward is related to the power supply energy consumption and the passenger waiting time.

[0165] In this way, by separately training the intelligent agent for passenger flow guidance and the intelligent agent for train scheduling, it is possible to generate an intelligent agent network that can collaboratively optimize train scheduling and passenger flow guidance. This is conducive to the subsequent generation of subway timetables and passenger flow path allocation that meet the constraints based on the trained intelligent agent network, which can effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby achieving collaborative optimization of train scheduling and passenger flow guidance.

[0166] In a possible implementation, the passenger flow action space includes the attraction factor of each section to the passenger flow in each operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train action space includes multiple train actions, wherein the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stop time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is less than the number of the second station; and / or, the passenger flow status includes the time sequence of the current operating time period, the average passenger waiting time of each station of each section in the current operating time period, and the average full load rate of each section in the current operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train status includes the passenger waiting time, train departure time, station running time, number of people on board, train departure time, and train arrival time related to each train.

[0167] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0168] In a possible implementation, in the training step of the optimization model: after the parameters of the optimization model are updated based on the passenger flow guidance result, when it is determined that the current operating time period is the last operating time period in the total operating time, it is determined that the parameter updating process is finished; and / or, after the parameters of the optimization model are updated based on the train scheduling result, when it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, it is determined that the parameter updating process is finished; and / or, in the process of updating the parameters of the optimization model based on the passenger flow guidance result, the parameters are updated using a simulation advancement method with a fixed step size; and / or, in the process of updating the parameters of the optimization model based on the train scheduling result, the parameters are updated using a simulation advancement method with a variable step size.

[0169] In this way, by setting the termination conditions and simulation advancement methods to conduct the optimization model training process, it is beneficial to train and generate an optimization model that meets the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0170] The disclosed embodiment provides a series of constraint conditions for the optimization model. In a possible implementation, the constraint conditions may be related to the departure interval, turn-around time, station stop time, inter-station running time, number of passengers on a train, passenger path length, number of sections a passenger path passes through, path allocation, number of trains, and train interval ratio.

[0171] In terms of departure interval, each line has a minimum departure interval h min and the maximum departure interval h max , that is, the following formula (18):

[0172]

[0173] In the formula, h min It indicates the minimum departure interval between two adjacent trains on each line. Indicates that the train number is k′ l The train is at the station Arrival time, Indicates that the train number is k l The train is at the station Departure time, h max represents the maximum departure interval between two adjacent trains on each line, k l represents the train number on line l, k′ l Indicates the train number on route l and the train number is later than k l .

[0174] Since there are a limited number of vehicles in the road network, a constraint is imposed on the maximum number of vehicles that can be used on each route. l is the maximum number of trains that can be dispatched on line l, q l is the number of each train, then q l <Q l .

[0175] Usually, after a train arrives at the terminal, it needs to turn around and perform related maintenance work before it can be put back into the network to continue running. Therefore, there is a minimum turning time t turn Constraint, that is, the following formula (19):

[0176]

[0177] In the formula, Indicates that the train number is q l , the train number is k′ l Located at the station (which can be regarded as the starting station in the opposite direction) departure time, Indicates that the train number is q l , train number is k l Located at the terminal Arrival time, final destination and Station For the same physical station, t turn It represents the minimum turning time required for a train to turn around and continue running, k′ l Indicates the train number on route l and the train number is later than k l , k l Indicates the train number on line l.

[0178] For safety and service quality considerations, there is a maximum interval u between the departure of the previous train and the arrival of the next train at the same station. max and the minimum interval u min Constraint, that is, the following formula (20):

[0179]

[0180] In the formula, u min Indicates the minimum interval between the last train leaving and the next train arriving at the same station. Indicates that the train number is k′ lThe train is at the station Arrival time, Indicates that the train number is k l The train is at the station The departure time, u max represents the maximum interval between the last train leaving and the next train arriving at the same station, k′ l Indicates the train number on route l and the train number is later than k l , k l Indicates the train number on line l.

[0181] The train's station stop time has a maximum g max and minimum g min Constraint, that is, the following formula (21):

[0182]

[0183] In the formula, g min Indicates the minimum stopping time of the train (i.e. the minimum time the train stops in the station). Indicates that the train number is k l The train is at the station Parking time, g max Indicates the maximum stopping time (i.e. the maximum time the train stays in the station).

[0184] The train's inter-station running time has a maximum and minimum Constraint, that is, the following formula (22):

[0185]

[0186] In the formula, Indicates that the train is at the station and Station The minimum running time between Indicates that the train number is k l The train is at the station and Station The running time between Indicates that the train is at the station and Station The maximum running time between

[0187] The number of passengers per train has a maximum capacity C max Constraint, that is, the following formula (23):

[0188] C ql ≤C max (twenty three)

[0189] In the formula, C ql Indicates that the train number is ql The actual number of passengers on the train, C max Indicates the maximum number of passengers a train can carry.

[0190] In terms of passenger flow guidance, first of all, the longest path of each OD feasible path (i.e. passenger path) cannot be too different from the shortest path. On the one hand, the number of stations passed by each feasible path minus the number of stations on the shortest path cannot exceed three stations (or other set numbers); on the other hand, the number of passenger transfers minus the number of transfers on the shortest path cannot exceed one. At the same time, the passenger flow demand distribution ratio of each OD path is Needs to be greater than or equal to zero and is the sum of all path allocation ratios If the If the denominator in is 0, all passenger flows are assigned to the shortest path.

[0191] The disclosed embodiment also designs another set of action spaces for the intelligent agent used for train scheduling: The dimension and same, The range of is [-1, 1], which is the action directly generated by the reinforcement learning agent, so that the constraints on the decision space can be satisfied, that is, the constraints described in the above equations (18) to (22). Under the action of the constraints described in the above equation (23), the optimization model determined by the training can achieve the following in terms of passenger flow guidance: if all the passengers waiting in the station board the train without causing the train to be overloaded, then all the passengers waiting in the station are determined to board the train; otherwise, a preset proportion of the passengers waiting in the station are determined to board the train so that the train is just fully loaded. After that, In One-to-one mapping to the original action space of make The constraints (i.e., the constraints described in equations (24) to (26) below) are satisfied. The specific process is as follows:

[0192] Define a linear map f, which takes values ​​of [-1, 1] Mapping to the constraints of equations (18), (21), and (22) above, we get equation (24):

[0193]

[0194] In the formula, Indicates line The nth train is at the station and Station The running time between It is through linear mapping Lieutenant General Mapping to the original action space Confirm in g min Indicates the minimum stopping time of the train, g max Indicates the maximum parking time.

[0195] According to the constraint of equation (19), for train q l The interval between the vehicles ahead Let the departure time of the preceding vehicle be Train q l The time when the previous trip arrived at the terminal from the opposite direction Satisfies the following formula (25):

[0196]

[0197] In the formula, Indicates the train number is Train q l The interval between the train and the preceding train, Represents train q l The number of trains running in the opposite direction before turning around is Arrived at the terminal Time, t turn It indicates the minimum turning time required for a train to turn around and continue running. Indicates that on train q l The train number of the train that departs before is q′ l , the train number is From the station Departure time, final destination and Station For the same physical station, Represents another set of action spaces.

[0198] According to the constraint of equation (20), let the last train be at the station The departure time is This train is at the previous station The departure time is make In satisfying Under the premise of , we have the following formula (26):

[0199]

[0200] In this way, by setting a series of specific constraints for the optimization model, the optimization model can be converged and trained to a model that meets the expectations, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0201] Due to the complexity and dimensionality explosion of train scheduling and passenger flow guidance problems, it is difficult to find an optimal solution. Therefore, the disclosed embodiment provides a train scheduling optimization framework based on multi-agent reinforcement learning and a passenger flow guidance optimization framework based on a deep deterministic policy gradient algorithm to optimize the two problems of train scheduling optimization and passenger flow guidance optimization respectively. They share the same train passenger flow simulation environment and influence each other to achieve the coordinated optimization of train scheduling and passenger flow guidance.

[0202] Figure 6 A schematic diagram of a train scheduling optimization framework based on multi-agent reinforcement learning provided according to an embodiment of the present disclosure is shown. Figure 6 As shown, the disclosed embodiment can use the multi-agent actor-evaluator reinforcement learning algorithm (MAA2C) to achieve train scheduling optimization. For a single line, two decentralized strategy networks responsible for uplink train scheduling and downlink train scheduling are divided (for example, for Figure 6 As shown in line 1, the two policy networks are Figure 6 The two are in a fully cooperative relationship with the same reward and value function Q π (S, A) and V π (S), share a centralized value network (e.g., for Figure 6 As shown in line 1, this value network is Figure 6 Line 1Critic network in). During training, the decentralized policy network receives local states from the environment and maps them to local actions. The goal is to find the optimal strategy to maximize expected future returns. The centralized value network is used to estimate the state-action value function, with the goal of minimizing the temporal difference error between the estimated value and the target value. It evaluates the value of behavior from a global perspective, which is conducive to maintaining the connection between intelligent agents and stabilizing system transitions. The environment and policy network obtain states, actions, and rewards through interaction, and jointly improve the value network parameters and the two policy network parameters. When making decisions, only the policy network is needed to generate actions, and the value network is no longer needed. The use of centralized training and decentralized decision-making can not only maintain the connection and synergy between the two intelligent agents, but also speed up the decision-making process, which is conducive to real-time decision-making. The disclosed embodiment expands the MAA2C ​​network of the above-mentioned single line to multiple lines to form a train scheduling optimization network for the road network. The overall framework is as follows Figure 6 As shown in the figure, since trains between different lines are not shared, the train scheduling of a single line has little impact on the energy consumption and passenger waiting time of other lines. Therefore, different lines are regarded as independent optimization tasks, and the MAA2C ​​networks between different lines are trained and decided independently, without sharing parameters, states and actions, and interacting with the same environment.

[0203] The disclosed embodiments may use a deep deterministic policy gradient algorithm (DDPG) to optimize passenger flow guidance. The DDPG algorithm combines the heterogeneous deterministic actor-evaluator algorithm and the deep Q-learning algorithm, uses a deep neural network to approximate the action-value function, and returns a definite action value. It uses a target network in addition to the conventional policy network and value network. Compared with the policy network and the value network, its parameter updates will not be too frequent and the amplitude will not be large, making the calculation more stable. In addition, it uses experience replay technology to collect each experience (S, A, R, S′), and use batch update when the parameters are updated, which further reduces the instability of training, where S and S′ are two adjacent states of the agent (that is, S can be the above And the corresponding S' can be the above Alternatively, S can be And the corresponding S' can be the above ). To improve the ability to explore the action space, the DDPG algorithm adds noise when selecting actions in the continuous space. The DDPG network for passenger flow guidance is independent of the MAA2C ​​network for train scheduling, and does not share parameters, states, and actions. However, they both interact with the same simulation environment and obtain corresponding states from the same passenger flow distribution and timetable determined by the passenger waiting time and number of passengers getting on and off the train, thereby achieving coordinated optimization of train scheduling and passenger flow guidance.

[0204] In this way, by using a train scheduling optimization framework based on multi-agent reinforcement learning and a passenger flow guidance optimization framework based on a deep deterministic policy gradient algorithm and combining the simulation environment to perform collaborative optimization of train scheduling and passenger flow guidance, new solutions can be quickly generated to cope with the online collaborative optimization of train scheduling and passenger flow guidance.

[0205] In one example, Chongqing Metro Line 1, Line 2, and Line 3 are used as the scenario. The road network structure is as follows: Figure 2 As shown in the figure, there are 84 stations and 20 sections in both directions. The line and passenger flow OD data are based on real scenarios and are obtained after desensitization. The passenger flow sampling frequency is 15 minutes and the operating hours are from 6:00 am to 23:00 pm.

[0206] In terms of train parameters, this paper sets the train to have 6 carriages and a maximum passenger capacity of C. max is 1440, the train's unloaded mass is 210,000 kg, and its maximum traction The maximum braking force is 344kN The efficiency of the train regenerative braking energy directly used by other trains is λ ris 0.9, the train electrical energy to mechanical energy conversion efficiency η1 is 0.93, the train mechanical energy to electrical energy conversion efficiency η2 is 0.9, and the Davis equation calculates the empirical parameters a1, a2, and a3 of air and friction resistance, respectively, which are 8.9×10 -3 , 2.89×10 -5 , 1.06×10 -6 .

[0207] In terms of train scheduling, set the maximum number of trains that can be scheduled on each line Q l is 50, the minimum departure interval is h min is 170s, the maximum departure interval is h max The minimum time u between the departure of the first train and the arrival of the second train at the same station is 480s. min is 120s, the maximum time u max The minimum stopping time of the train is 720s. min The maximum parking time is 20s. max The maximum cruising speed between stations is 25m / s, and the minimum is 20m / s. The minimum and maximum inter-station running time are determined according to the distance between each station. The shortest time tturn required for a train to turn around at the terminal is 218s.

[0208] This example is performed on a Windows machine with 32.0GB of memory, an Intel Xeon E5-2678v3 CPU with 12 cores and 24 logical processors. The GPU is an NVIDIA GeForce RTX3090 with 10496 CUDA cores. The algorithm and program are written in Python 3.8 and Tensorflow 2.4.0.

[0209] Figure 7 A schematic diagram of the optimization model training process provided according to an embodiment of the present disclosure is shown. Figure 7 As shown, firstly, the passenger flow guidance optimization training is carried out, and the passenger flow guidance agent generates the action when n=0 and will Pass it to the simulation environment so that the simulation environment calculates (T begin , T begin +φ] period OD matrix and and will Then, the train dispatching optimization is started. Taking line l as an example, the train dispatching agent in the up direction of line l generates the action when n=0. and will Passed to the simulation environment, so that the simulation environment calculates the timetable of the first train in the upward direction and leaves the starting station Arrival time at the starting and ending stations Build Status and will The maximum time of the train dispatching agent in the up direction of line l and the down direction is greater than T begin +φ, it means that the train schedule has reached the last moment of the current completed passenger flow guidance, and the next period (T begin +φ,T begin +2φ] period. After the passenger flow guidance, train scheduling optimization is continued, and this cycle continues. It can be seen that at any time during the simulation, the line The stage of the train dispatching agent and the stage n of the passenger flow guidance agent psg There is the following relationship Among them, T begin Indicates the start time of daily operation of the road network, n psg represents the stage of the passenger flow guidance agent, Indicates line The nth train is at the terminal Each time passenger flow guidance or train dispatch obtains the status of the next stage, it will determine whether the round termination condition is met. For train dispatch, the termination condition is that the time when the last train arrives at the destination is greater than the end time of the daily operation of the road network T end For passenger flow guidance, the termination condition is that stage n is greater than the total time period B of the OD matrix. If an agent reaches the termination condition, the simulation interaction stops. If all agents meet the round termination condition, the current round ends.

[0210] Figure 8 A schematic diagram showing a total reward curve of a training process provided according to an embodiment of the present disclosure is shown. Figure 8 As shown in the figure, a total of 400 rounds were trained. The vertical axis represents the cumulative reward of each round during training, and the horizontal axis represents the number of rounds. In each round, the agent responsible for train scheduling in one direction of a line will perform about 120-170 train schedulings. The total number of times is related to the departure interval. Each scheduling will generate the arrival time and departure time of the next train at all stations. The agent responsible for passenger flow guidance will perform 68 passenger flow guidances, and each time it will generate the passenger flow attraction factor of each section within the next 15 minutes. It can be seen that although the reward curve may drop in the middle and fall into a worse solution, it will eventually get out of the worse solution due to random exploration and finally find a better solution. After about 150 rounds, the reward tends to be stable overall, but because the exploration strategy of reinforcement learning has a certain degree of randomness, the reward curve fluctuates slightly.

[0211] Fig. 9 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 1 provided according to an embodiment of the present disclosure is shown. Fig.10 A schematic diagram showing the passenger flow demand and departure interval curve in the downlink direction of Line 1 provided according to an embodiment of the present disclosure is shown. Fig.11 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 2 provided according to an embodiment of the present disclosure is shown. Fig.12 A schematic diagram showing the passenger flow demand and departure interval curve in the downlink direction of Line 2 provided according to an embodiment of the present disclosure is shown. Fig.13 A schematic diagram showing the passenger flow demand and departure interval curve in the upward direction of Line 3 provided according to an embodiment of the present disclosure is shown. Fig.14 A schematic diagram showing the passenger flow demand and departure interval curve of Line 3 in the down direction provided according to an embodiment of the present disclosure is shown. Figures 9 to 14 As shown in the figure, we can see that during the morning peak period around 8 o'clock and the evening peak period around 18 o'clock, the passenger flow demand reaches the maximum value, and the departure interval is also reduced accordingly, which effectively reduces the waiting time of passengers during the morning peak period. In other time periods, the intelligent agent reduces the total traction energy consumption of the system by increasing the departure interval.

[0212] Fig.15 FIG. 1 is a schematic diagram showing an optimized uplink timetable for Line 1 provided according to an embodiment of the present disclosure. Fig.15 As shown in the figure, the vertical axis of the optimized 6:00-9:00 timetable for the up direction of Line 1 represents the station, the horizontal axis represents the time, and the number of vehicles actually put into operation is 25. It can be seen that the frequency of train departures from 6:00 to 9:00 shows a trend of first increasing and then decreasing, which is more obvious in the down direction, and the running time is also shortened. The train running speed is increased, which further shortens the waiting time for passengers.

[0213] This example designs five scenarios to discuss the optimization effect of the optimization scheme while keeping the full-turnaround operation time of the train constant. The full-turnaround operation time of the train refers to the time it takes for the train to return to the starting station after running for one week from the starting station. Generally speaking, a longer full-turnaround operation time means that the train's inter-station operation time is longer, the train's cruising speed is lower, and the train's energy consumption is also lower, but it will also increase passenger travel time and waiting time. When the full-turnaround operation time of the train is constant, the total energy consumption of the train and the passenger waiting time on the entire line can be adjusted by adjusting the train's running time between stations. The five scenarios are as follows:

[0214] (I) Benchmark plan: A fixed timetable is adopted, and passenger flow is not guided. Passengers travel by the shortest route. The single train operation adopts the speed curve of "maximum acceleration traction-coasting-maximum deceleration braking", and the fixed station cruising speed is 21.75m / s, that is, the train enters the coasting state when it accelerates to 21.75m / s at the maximum acceleration. The fixed station stop time is 30s, and the departure interval is 418s.

[0215] (II) Passenger flow guidance scheme: Train the passenger flow guidance agent under a fixed timetable, with a fixed stop time of 30 seconds, a departure interval of 418 seconds, and a cruising speed of 21.75 m / s.

[0216] (III) Train dispatching scheme: Train the train dispatching agent alone without guiding passenger flow, and passengers travel along the shortest route.

[0217] (IV) Simple superposition scheme: The passenger flow guidance agent trained under a fixed timetable and the train dispatching agent trained under collaborative optimization are used to generate the train timetable and passenger flow path allocation. That is, the train dispatching and passenger flow guidance agents are not jointly optimized, but optimized separately and then simply superimposed.

[0218] (V) Collaborative optimization plan: joint optimization of train scheduling and passenger flow guidance.

[0219] Table 1 shows the test results of different optimization schemes provided according to the embodiments of the present disclosure. It can be seen from the system traction energy consumption and the average waiting time of passengers obtained after simulation verification shown in Table 1 that, compared with the fixed timetable scheme, the train scheduling optimization scheme after trial and error learning of the intelligent agent has a certain improvement in the total system traction energy consumption and the average waiting time of passengers. The average waiting time of passengers in the passenger flow guidance scheme is 1.4% lower than that of the baseline scheme without passenger flow guidance, and the total energy consumption has only increased by 0.04%, indicating the effectiveness of passenger flow guidance. At the same time, the system traction energy consumption and the average waiting time of passengers in the collaborative optimization scheme are both lower than those in the simple superposition scheme, indicating that the matching of train scheduling and passenger flow guidance has a certain influence on the optimization effect. Under collaborative optimization, train scheduling and passenger flow guidance can be matched in both directions, so that a better solution can be generated than separate optimization and simple superposition.

[0220] Table 1 Test results of different optimization schemes

[0221]

[0222] Compared with the separate optimization of passenger flow guidance and train scheduling, the collaborative optimization method proposed in the embodiment of the present disclosure takes the interaction between vehicle flow and passenger flow into consideration during training, and ultimately achieves better optimization results. This further demonstrates that collaborative optimization is not a simple superposition of optimization of each part, and the dependency between vehicle flow and passenger flow in the rail transit system will affect the optimization effect. There is indeed room for optimization of the system as a whole, which also demonstrates that the multi-agent-based reinforcement learning framework provided in the embodiment of the present disclosure is effective in collaborative optimization.

[0223] The embodiment of the present disclosure trains the reinforcement learning algorithm in three different passenger flow scenarios. Scenario 1 is deterministic passenger flow demand, that is, the original passenger flow OD matrix is ​​used. Scenario 2 assumes that each OD pair obeys a normal distribution with a standard deviation of 15% of its mean value. Scenario 3 assumes that each OD pair obeys a normal distribution with a standard deviation of 30% of its mean value. Table 2 shows the test results of the agent provided by the embodiment of the present disclosure in different passenger flow scenarios. Table 2 shows the test results of the reinforcement learning agent trained under deterministic passenger flow demand in these three scenarios. Among them, the optimization scheme selects the joint optimization of train scheduling and passenger flow guidance. The benchmark scheme adopts a fixed departure interval of 418s, a parking time of 30s and a cruising speed of 21.75m / s between stations, and does not guide passenger flow. The results show that as the randomness of demand increases, the optimization effect of reinforcement learning also fluctuates, but the power supply system energy consumption and the average waiting time of passengers are still better than the benchmark scheme. This shows that when the passenger flow distribution changes, the trained agent will be able to respond to the current demand changes without re-optimization or re-training, indicating that the algorithm has certain robustness and engineering application value.

[0224] Table 2 Test results of the agent in different passenger flow scenarios

[0225]

[0226] The disclosed embodiment further tests the robustness of the reinforcement learning agent by generating sudden passenger flow. It is assumed that the passenger flow at Gongmao Station on Line 3 from 10:30 to 12:00 increases 10 times of the original value, while the passenger flow at other time periods and stations remains unchanged. Fig.16 A schematic diagram showing the departure interval and passenger flow demand curve under the original passenger flow and sudden passenger flow in the upward direction of Line 3 provided according to an embodiment of the present disclosure is shown. Fig.17 A schematic diagram showing the departure interval and passenger flow demand curve of the original passenger flow and sudden passenger flow in the down direction of Line 3 provided by the embodiment of the present disclosure is shown. Fig.16 and Fig.17It can be seen that in order to respond to changes in passenger flow demand, the departure interval under sudden passenger flow has a certain decrease after 10:30, and gradually returns to the original departure interval at 12:00. This shows that the reinforcement learning provided by the embodiment of the present disclosure captures the complex state transitions and related dependencies of vehicle and passenger flows during the training process, so that reliable state estimates and corresponding scheduling controls can be obtained for current demand changes. In terms of real-time performance, it takes an average of 0.06 seconds for the agent responsible for scheduling trains in the up / down direction of the line to generate a set of actions based on the current state, and it takes an average of 0.07 seconds for the agent responsible for passenger flow guidance to generate a set of actions. The reinforcement learning agent provided by the embodiment of the present disclosure can modify the timetable of the next vehicle in real time according to the current state when the passenger flow changes dynamically, meeting the needs of online optimization.

[0227] The disclosed embodiment provides a method for collaborative online optimization of network train scheduling and passenger flow guidance based on a multi-agent deep reinforcement learning framework, which is used for online collaborative optimization of network train scheduling and passenger flow guidance, and online adjustment of timetables and passenger flow guidance strategies when passenger flow changes dynamically, thereby effectively reducing the energy consumption of the traction power supply system and shortening the average waiting time of passengers. First, the disclosed embodiment models the traffic and passenger flow of the subway as a discrete event dynamic system, and takes into account the transfer of passengers on different service lines. Then, the disclosed embodiment models the optimization problem as a multi-agent Markov decision process, aiming to minimize the energy consumption of the traction power supply system and the average waiting time of passengers. Among them, the network train scheduling optimization provided by the embodiment of the present disclosure adopts a multi-agent actor-evaluator algorithm, and the passenger flow guidance optimization adopts a deep deterministic policy gradient algorithm to solve, with the optimization goal of minimizing the traction power supply energy consumption and the average waiting time of passengers, realizing the coordinated optimization of train departure time, station-to-station running time, station parking time and passenger flow path. In the process of multiple agents interacting with the subway vehicle and passenger flow discrete event simulation environment, a subway timetable and passenger flow path allocation that meet the constraints are generated, which can effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers. In addition, the collaborative optimization method provided by the embodiment of the present disclosure is suitable for online optimization, that is, when the train and passenger flow are disturbed or dynamically changed, the next train timetable and passenger flow guidance strategy are adjusted online according to the state of the train and passenger flow after the disturbance. Finally, the embodiment of the present disclosure uses desensitized data based on the real scenes of Chongqing Metro Line 1, Line 2, and Line 3 to test the simulation environment and optimization scheme developed in this article. By comparing with the baseline scheme with fixed departure interval, train cruising speed and no passenger flow guidance, the experimental results verify the effectiveness, robustness and real-time performance of the proposed multi-agent reinforcement learning optimization, which meets the needs of online optimization.

[0228] The embodiment of the present disclosure also provides a device. Fig.18 FIG. 1 is a block diagram of a device provided according to an embodiment of the present disclosure. Fig.18 As shown, the apparatus 1800 may include:

[0229] An acquisition module 1801 is used to acquire first passenger flow information of a road network within a preset time period, wherein the road network includes multiple routes, multiple stations are arranged on the routes, and the first passenger flow information indicates the number of passengers traveling from a departure station to a destination station, and the departure station and the destination station are any stations in the road network;

[0230] Collaborative optimization module 1802, the collaborative optimization module 1802 is used to input the first passenger flow information into the optimization model to perform collaborative optimization of train scheduling and passenger flow guidance to obtain an optimization result, wherein the optimization model is determined by training based on the optimization objectives of minimizing power consumption and minimizing passenger waiting time, and the optimization result indicates the train scheduling information and passenger flow guidance information on each line in the optimized road network.

[0231] In this way, the first passenger flow information of the road network within a preset time period is obtained through the acquisition module, and the first passenger flow information is input into the optimization model determined by the optimization goal training based on minimizing power supply energy consumption and minimizing passenger waiting time through the collaborative optimization module to perform collaborative optimization of train scheduling and passenger flow guidance, and obtain the optimization results of the train scheduling information on each line in the optimized road network and the passenger flow guidance information in each time period, which can realize two-way online matching of passenger flow demand and train scheduling, effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby improving the energy efficiency of urban rail transit and improving service quality.

[0232] In a possible implementation, the total operation time of each of the lines includes a plurality of continuous operation time periods; the device also includes a training module, the training module is used to execute the training steps of the optimization model, the training steps of the optimization model include: for each of the lines, in chronological order, in each of the operation time periods: the optimization model is first allowed to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; then, the optimization model is interacted with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, wherein the passenger flow action space indicates the attraction of each of the lines to the passenger flow, the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow guidance result is used to minimize the waiting time of passengers, the train action space indicates the train scheduling information on each of the lines, and the train scheduling result is used to minimize the power consumption and minimize the waiting time of passengers; when the optimization model that completes the interaction process in all the operation time periods of all the lines meets the preset constraints and reward goals, the optimization model is used as the trained optimization model.

[0233] In this way, by training the optimization model with the optimization objectives of minimizing the energy consumption of the traction power supply system and the average waiting time of passengers, and by appropriately intervening in passenger flow and rationally allocating passenger flow to the road network, an optimization model can be generated that can collaboratively optimize train scheduling and passenger flow guidance, which is conducive to the subsequent real-time collaborative optimization based on the trained optimization model, thereby effectively alleviating the phenomenon of passenger gathering in the station during peak passenger flow periods.

[0234] In a possible implementation, the optimization model is firstly interacted with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then interacted with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result. The parameter update includes: the intelligent agent used for the passenger flow guidance in the optimization model generates a passenger flow action, and transmits the passenger flow action to the simulation environment, so that the simulation environment calculates the passenger flow matrix, passenger flow state, and passenger flow guidance reward of the current operating time period, and transmits the passenger flow state and the passenger flow reward to the intelligent agent, so that the intelligent agent adjusts its own strategy, wherein the passenger flow guidance result includes: the passenger flow state and the passenger flow reward, the passenger flow action is determined based on a preset passenger flow action space, the passenger flow action indicates the attraction of the line to the passenger flow in the current operating time period, and the passenger flow matrix indicates the passenger flow of each train on the line in the current operating time period. the number of passengers on the train path, the passenger flow state indicates the change of passenger flow in the current running time period, and the passenger flow guidance reward is related to the passenger waiting time; for each train in the current running time period in order of the train number: the intelligent agent used for the train scheduling in the optimization model generates a train action, and transmits the train action to the simulation environment, so that the simulation environment calculates the train scheduling information, train state, and train scheduling reward of the current train number in combination with the passenger flow matrix, and transmits the train state and the train scheduling reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to achieve parameter update, wherein the train scheduling result includes: the train state and the train scheduling reward, the train action is determined based on a preset train action space, the train action indicates the train scheduling information, the train scheduling information includes the train departure time, inter-station running time, and station parking time of the current train number, the train state indicates the train running process, and the train scheduling reward is related to the power supply energy consumption and the passenger waiting time.

[0235] By separately training the intelligent agent for passenger flow guidance and the intelligent agent for train scheduling, it is possible to generate an intelligent agent network that can collaboratively optimize train scheduling and passenger flow guidance. This is conducive to the subsequent generation of subway timetables and passenger flow path allocation that meet the constraints based on the trained intelligent agent network, which can effectively reduce the energy consumption of the traction power supply system and shorten the average waiting time of passengers, thereby achieving collaborative optimization of train scheduling and passenger flow guidance.

[0236] In a possible implementation, the passenger flow action space includes the attraction factor of each section to the passenger flow in each operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train action space includes multiple train actions, wherein the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stop time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is less than the number of the second station; and / or, the passenger flow status includes the time sequence of the current operating time period, the average passenger waiting time of each station of each section in the current operating time period, and the average full load rate of each section in the current operating time period, wherein the sections are obtained by dividing the multiple lines; and / or, the train status includes the passenger waiting time, train departure time, station running time, number of people on board, train departure time, and train arrival time related to each train.

[0237] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0238] In a possible implementation, in the training step of the optimization model executed by the training module: after the parameters of the optimization model are updated based on the passenger flow guidance result, when it is determined that the current operating time period is the last operating time period in the total operating time, it is determined that the parameter updating process is finished; and / or, after the parameters of the optimization model are updated based on the train scheduling result, when it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, it is determined that the parameter updating process is finished; and / or, in the process of updating the parameters of the optimization model based on the passenger flow guidance result, the parameters are updated using a simulation advancement method with a fixed step size; and / or, in the process of updating the parameters of the optimization model based on the train scheduling result, the parameters are updated using a simulation advancement method with a variable step size.

[0239] In this way, by setting a specific passenger flow action space and passenger flow state, train action space and train state to train the optimization model, it is conducive to training and generating an optimization model that can meet the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model, and the passenger flow action space and passenger flow state, train action space and train state can be flexibly set according to needs to achieve the desired optimization effect.

[0240] In a possible implementation, the constraints are related to departure interval time, U-turn time, station parking time, passenger path length, number of sections passed by the passenger path, path allocation ratio, number of trains, and interval between adjacent trains arriving at the station.

[0241] In this way, by setting a series of specific constraints for the optimization model, the optimization model can be converged and trained to a model that meets the expectations, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0242] In a possible implementation, the reward target is to maximize the cumulative reward, and the cumulative reward includes the passenger flow guidance reward and the train scheduling reward.

[0243] In this way, by training the intelligent agent with maximizing the cumulative reward as the reward goal, the training generates an optimization model that meets the expectations and can collaboratively optimize train scheduling and passenger flow guidance, thereby facilitating subsequent real-time collaborative optimization based on the trained optimization model.

[0244] In some embodiments, the functions or modules included in the train scheduling and passenger flow guidance collaborative optimization device provided in the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above train scheduling and passenger flow guidance collaborative optimization method embodiment. For the sake of brevity, it will not be repeated here.

[0245] The disclosed embodiment also proposes a train scheduling and passenger flow guidance collaborative optimization device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above-mentioned train scheduling and passenger flow guidance collaborative optimization method when executing the instructions stored in the memory.

[0246] In some embodiments, the functions or modules included in the train scheduling and passenger flow guidance collaborative optimization device provided in the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above train scheduling and passenger flow guidance collaborative optimization method embodiment. For the sake of brevity, it will not be repeated here.

[0247] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned train scheduling and passenger flow guidance coordinated optimization method is implemented. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0248] In some embodiments, the functions or modules contained in the computer-readable storage medium provided in the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation thereof can refer to the description of the above train scheduling and passenger flow guidance collaborative optimization method embodiment, and for the sake of brevity, it will not be repeated here.

[0249] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned train scheduling and passenger flow guidance collaborative optimization method.

[0250] In some embodiments, the functions or modules included in the computer program product provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation thereof can refer to the description of the above train scheduling and passenger flow guidance collaborative optimization method embodiment, and for the sake of brevity, it will not be repeated here.

[0251] Fig.19 1900 is a block diagram of a device for executing a method for collaboratively optimizing train scheduling and passenger flow guidance according to an embodiment of the present disclosure. For example, the device 1900 may be provided as a server or a terminal device. Fig.19, the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0252] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0253] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0254] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0255] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0256] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0257] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0258] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0259] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0260] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0261] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0262] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for collaborative optimization of train scheduling and passenger flow guidance, characterized in that: include: Acquire first passenger flow information of a road network within a preset time period, wherein the road network includes a plurality of routes, a plurality of stations are arranged on the routes, and the first passenger flow information indicates the number of passengers traveling from a departure station to a destination station, wherein the departure station and the destination station are any stations in the road network; The first passenger flow information is input into the optimization model to perform coordinated optimization of train scheduling and passenger flow guidance to obtain an optimization result, wherein the optimization model is determined by training based on the optimization objectives of minimizing power consumption and minimizing passenger waiting time, and the optimization result indicates the train scheduling information and passenger flow guidance information on each line in the optimized road network.

2. The method according to claim 1, characterized in that: The total operation time of each of the lines includes a plurality of continuous operation time periods; the method further includes a training step of the optimization model, and the training step of the optimization model includes: For each of the lines, in chronological order and in each of the operating time periods: the optimization model is first allowed to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; and then the optimization model is allowed to interact with the simulation environment based on the train action space to perform train scheduling, thereby updating the parameters of the optimization model based on the train scheduling result, wherein the passenger flow action space indicates the attraction of each of the lines to the passenger flow, the simulation environment is used to provide an interactive environment for the optimization model, the passenger flow guidance result is used to minimize the waiting time of passengers, the train action space indicates the train scheduling information on each of the lines, and the train scheduling result is used to minimize the power consumption and minimize the waiting time of passengers; When the optimization model of the interaction process in all operating time periods of all lines satisfies the preset constraints and reward targets, the optimization model is used as the trained optimization model.

3. The method according to claim 2, characterized in that The optimization model is first allowed to interact with the simulation environment based on the passenger flow action space to guide the passenger flow, thereby updating the parameters of the optimization model based on the passenger flow guidance result; Then, based on the train action space, the simulation environment is interacted with to perform train scheduling, so as to update the parameters of the optimization model based on the train scheduling result, including: The intelligent agent used for the passenger flow guidance in the optimization model generates a passenger flow action, and transmits the passenger flow action to the simulation environment, so that the simulation environment calculates the passenger flow matrix, passenger flow state, and passenger flow guidance reward of the current operation time period, and transmits the passenger flow state and the passenger flow reward to the intelligent agent, so that the intelligent agent adjusts its own strategy to realize parameter update, The passenger flow guidance result includes: the passenger flow state and the passenger flow reward; the passenger flow action is determined based on a preset passenger flow action space; the passenger flow action indicates the attraction of the route to the passenger flow in the current operation time period; the passenger flow matrix indicates the number of passengers on each riding path on the route in the current operation time period; the passenger flow state indicates the change of passenger flow in the current operation time period; and the passenger flow guidance reward is related to the passenger waiting time; For each train in the current running time period, in order of train sequence: the agent used for train scheduling in the optimization model generates a train action, and transmits the train action to the simulation environment, so that the simulation environment calculates the train scheduling information, train status, and train scheduling reward of the current train in combination with the passenger flow matrix, and transmits the train status and the train scheduling reward to the agent, so that the agent adjusts its own strategy to achieve parameter update, Among them, the train scheduling result includes: the train status and the train scheduling reward, the train action is determined based on a preset train action space, the train action indicates the train scheduling information, the train scheduling information includes the train departure time, inter-station running time, and station parking time of the current train, the train status indicates the train operation process, and the train scheduling reward is related to the power supply energy consumption and the passenger waiting time.

4. The method according to claim 3, characterized in that: The passenger flow action space includes the attraction factor of each section to the passenger flow in each operation time period, wherein the sections are obtained by dividing the multiple routes; and / or, The train action space includes a plurality of train actions, wherein the train actions include the departure interval of two adjacent trains, the running time of the later train of the two trains between the first station and the second station, and the stop time of the later train of the two trains at the first station, wherein the first station and the second station are any two adjacent stations on the line, and the number of the first station is smaller than the number of the second station; and / or, The passenger flow status includes the time sequence of the current operation time period, the average passenger waiting time of each station in each section in the current operation time period, and the average full load rate of each section in the current operation time period, wherein the sections are obtained by dividing the multiple lines; and / or, The train status includes the passenger waiting time, train departure time, inter-station running time, number of people on board, train departure time, and train arrival time related to each train number.

5. The method according to claim 2, characterized in that: In the training step of the optimization model, After the parameters of the optimization model are updated based on the passenger flow guidance result, when it is determined that the current operation time period is the last operation time period in the total operation time, determining that the parameter update process is finished; and / or, After the parameters of the optimization model are updated based on the train scheduling result, when it is determined that the time for the current train to arrive at the terminal station is greater than the time for the last train in the network to arrive at the terminal station, it is determined that the parameter updating process is finished; and / or, In the process of updating the parameters of the optimization model based on the passenger flow guidance result, the parameters are updated by using a simulation advancement method with a fixed step length; and / or, In the process of updating the parameters of the optimization model based on the train scheduling result, the parameters are updated by using a simulation advancement method with a variable step size.

6. The method according to claim 2, characterized in that The constraints are related to the departure interval, U-turn time, station stop time, inter-station running time, number of passengers on a train, passenger path length, number of sections passed by the passenger path, path allocation ratio, number of trains, and intervals between adjacent train arrivals.

7. The method according to claim 2, characterized in that The reward target is to maximize the cumulative reward, and the cumulative reward includes the passenger flow guidance reward and the train scheduling reward.

8. A train dispatching and passenger flow guidance collaborative optimization device, characterized in that: include: an acquisition module, the acquisition module being used to acquire first passenger flow information of a road network within a preset time period, wherein the road network includes a plurality of routes, a plurality of stations are arranged on the routes, the first passenger flow information indicates the number of passengers traveling from a departure station to a destination station, and the departure station and the destination station are respectively any one of the stations in the road network; A collaborative optimization module, wherein the collaborative optimization module is used to input the first passenger flow information into an optimization model to perform collaborative optimization of train scheduling and passenger flow guidance to obtain an optimization result, wherein the optimization model is determined by training based on the optimization objectives of minimizing power supply energy consumption and minimizing passenger waiting time, and the optimization result indicates the train scheduling information and passenger flow guidance information on each line in the optimized road network.

9. A train dispatching and passenger flow guidance collaborative optimization device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images