A bus dynamic scheduling method and system based on vehicle management

Abstract

The application discloses a kind of public transport dynamic scheduling method and system based on vehicle management, by collecting the historical data and real-time data of electric bus operation, construct dispatching environment simulation system, based on bus route data and historical passenger boarding data automatically generate passenger travel information, bus operation is modeled as dynamic discrete process, and introduce vehicle availability and charging restriction and other constraints, build close to actual bus operation simulation environment;Based on the scheduling simulation environment, a Markov decision process is established, and the vehicle availability management parameters are clearly defined in the state space and the reward function;The Rainbow DQN model is used for training, the scheduling strategy is continuously optimized according to the environment feedback, and the trained model is used to infer and generate a dynamic scheduling scheme based on real-time data, taking into account passenger travel demand and bus company operational efficiency while meeting vehicle availability, providing intelligent scheduling decision support for management personnel.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of public transport scheduling and reinforcement learning, and particularly relates to a public transport dynamic scheduling method and system based on vehicle management. BACKGROUND

[0002] The public transport scheduling algorithm based on the exhaustive method and the genetic algorithm can only generate a public transport timetable in an offline manner according to historical passenger flow, and is difficult to cope with the real-time passenger flow that changes constantly in the actual operation process. In real scenarios, passenger flow often fluctuates suddenly, and the offline public transport timetable is difficult to respond to the demand changes in a timely manner, resulting in prolonged waiting time of passengers and even the phenomenon of passenger retention. Although the scheduling algorithm based on reinforcement learning, such as the DRL-TO method and the timetable dynamic generation method based on DQN, can optimize the departure frequency according to the real-time passenger flow, it assumes that there is always a vehicle available and ignores the charging restrictions and other constraints of the electric bus, and cannot be used in the real running environment. In addition, these methods all use DQN for policy learning, and have problems such as overestimation of Q value, insufficient exploration ability, and low sample utilization efficiency, and it is difficult to obtain a stable and efficient scheduling strategy in a complex environment. SUMMARY

[0003] To solve the problems of the prior art and achieve the purposes of improving the efficiency of public transport operation and reducing the cost of public transport operation, the application adopts the following technical solutions:

[0004] A public transport dynamic scheduling method based on vehicle management, comprising the following steps:

[0005] Step 1: obtaining vehicle historical operation data and real-time operation state;

[0006] Step 2: constructing a multi-element scheduling environment simulation system considering vehicles, stations and passengers;

[0007] Step 3: modeling the scheduling decision process, defining the state space, action space considering vehicle constraints, and reward function considering the interests of all parties involved in public transport;

[0008] Step 4: constructing a public transport dynamic scheduling model, triggering the scheduling decision based on the scheduling environment simulation system, estimating the value distribution of each action using a network structure with noise based on the triggered current state, selecting the optimal action in combination with the minimum and maximum departure interval constraints, applying the selected action to the scheduling environment simulation environment, obtaining the next state and the immediate reward brought by the current action, constructing the public transport dynamic scheduling model, and training based on the priority experience replay pool constructed based on the current state, action, reward and next state, to generate a dynamic departure timetable.

[0009] Further, the step 3 comprises the following steps:

[0010] Step 3.1: Model the dynamic scheduling problem of the vehicle as a Markov scheduling decision process, define the state space S, action space A and reward function R considering the constraints of vehicle management;

[0011] The state space S contains different states obtained by the agent from the environment at m time, the characteristics of the state include time information of different scales, demand pressure coefficient, the sum of waiting time of boarding passengers of all vehicles running on the current line, the utilization rate of carrying capacity of all vehicles running on the current line, the ratio of the number of current idle buses to the total number of buses;

[0012] The action space contains the action of departure or no departure;

[0013] Step 3.2: Define the reward function considering the interests of passengers and bus companies, and ensure that the departure scheduling is not in conflict with the management of the bus; the interests of passengers are measured by the degree of bus congestion and waiting time, and the interests of bus companies are measured by the number of departures, the reward function represents the reward value obtained based on the state and the action at the m time, which provides immediate performance feedback for the deep reinforcement learning model.

[0014] Further, in the step 3.1, the demand pressure coefficient is obtained by dividing the sum of the number of waiting passengers at each station at a certain time by the sum of the load capacity of all vehicles running on the line at that time, and the load capacity is obtained based on the number of seats and the capacity coefficient of the vehicle; the demand pressure coefficient is defined as:

[0015]

[0016] wherein SN represents the total number of line stations, represents the number of waiting passengers at the s-th station at m time, represents the total number of vehicles running on the current line, represents the load capacity of the k-th vehicle, represents the number of seats of the k-th vehicle, is the capacity coefficient, which represents that the load capacity of the vehicle is times the number of seats, The smaller the value is, the more sufficient the overall system capacity is;

[0017] The sum of the waiting time of boarding passengers of all vehicles running on the current line is obtained by subtracting the time when the passenger arrives at the bus station from the time when the passenger gets on the vehicle at the bus station, and based on the total number of vehicles running on the current line, the number of bus stations that the vehicle has passed, and the number of boarding passengers at the boarding station of the vehicle;

[0018] The normalized sum of boarding passenger waiting times for all vehicles currently operating on the line is defined as:

[0019]

[0020] in, This represents the total waiting time for passengers to board all vehicles currently operating on the route. This represents the parameter that normalizes the total waiting time; the calculation formula is:

[0021]

[0022] in, This indicates the total number of vehicles currently operating on the line. This represents the number of bus stops that the k-th bus has passed. This represents the number of passengers who board the k-th bus at the s-th bus stop. This represents the time when the i-th passenger boards the k-th bus at the s-th bus stop. This represents the time when the i-th passenger arrives at the s-th bus stop;

[0023] The carrying capacity utilization rate of all vehicles currently operating on the line is obtained by dividing the actual carrying capacity required by all vehicles on the line by the carrying capacity that all vehicles can provide.

[0024] The carrying capacity utilization rate of all vehicles currently operating on the line It is an important indicator for measuring the match between actual passenger demand and bus carrying capacity, and can help determine whether it is necessary to dispatch a bus at the current decision-making point in time. It is defined as:

[0025]

[0026] in, This indicates the actual carrying capacity required for all vehicles on the line. , and The definition is the same as before. This represents the number of bus stops actually traveled by the i-th passenger who boarded the k-th bus at the s-th bus stop. This indicates the carrying capacity available to all vehicles. , , The definition is the same as before;

[0027] The ratio of the current number of available buses to the total number of buses is obtained by dividing the number of available buses at the starting station by the total number of buses on the current route.

[0028] The ratio of the current number of available buses to the total number of buses Defined as:

[0029]

[0030] in, This indicates the number of available vehicles on the current route. This represents the total number of vehicles on the current route, satisfying the following conditions: , This indicates the total number of vehicles currently operating on the line, and the number of idle vehicles at the starting station. When the value is zero, departure will not be possible.

[0031] Furthermore, the reward function in step 3.2 includes:

[0032] When passenger flow is low, the agent should prioritize not to depart. Based on the capacity utilization rate, the agent is rewarded for not departing. The lower the capacity utilization rate, the lower the passenger demand in the current time period. At the same time, a penalty for waiting time is introduced.

[0033] When passenger flow is high, the agent should prioritize departure actions. Based on the high passenger demand at the current time, the agent should be rewarded for departure actions. If the agent still issues a departure instruction when there are no available vehicles at the origin station, it will be penalized for incorrect departure.

[0034] When passenger delays occur, the agent will be penalized for both non-departure and departure actions based on the number of delayed passengers.

[0035] Specifically, the reward function is defined as follows:

[0036]

[0037] When passenger flow is low, the intelligent agent should prioritize not dispatching the vehicle. In order to reduce the operating costs of bus companies, rewards are given to intelligent agents for not dispatching buses. ,in This represents the utilization rate of transport capacity; a smaller value indicates lower passenger demand during the current time period. However, not dispatching vehicles may increase passenger waiting time, therefore a penalty is introduced. , The penalty factor for waiting time;

[0038] When passenger flow is high, the intelligent agent should prioritize departure actions. Give the agent a reward for the action of starting the vehicle. This indicates high passenger demand during the current time period, encouraging departure at minute m. To comply with the bus company's vehicle management requirements, penalties need to be imposed on agents that erroneously depart, such as when there are no available vehicles at the starting station. If the intelligent agent still issues the departure command, it will be subject to... The punishment, among which Indicates the penalty factor for incorrect departure;

[0039] Furthermore, passenger congestion will severely impact passenger satisfaction. Therefore, regardless of whether the bus has departed, any instance of passenger congestion will result in penalties for both the departure and non-departure actions. , The penalty factor represents the number of stranded passengers. The total number of stranded passengers on the current line at time m is represented by the following formula:

[0040]

[0041] Wherein, SN represents the total number of stations on the line. This represents the number of passengers stranded at station s at time m.

[0042] Furthermore, step 4 includes the following steps:

[0043] Step 4.1: Train the bus dynamic scheduling model using a deep reinforcement learning network;

[0044] First, a bus is forcibly dispatched at the first departure time, and the environmental state is updated. Then, the scheduling environment simulation system triggers a scheduling decision every certain period of time. At each decision time, the vehicle state is input. Based on the vehicle state, the deep reinforcement learning network estimates the value distribution of the action of dispatching or not dispatching using a noisy network structure, and selects the optimal action by combining the minimum and maximum departure interval constraints. The selected action is applied to the simulation environment to obtain the new vehicle state and the immediate reward brought by the current action. The generated interactive data containing state, action, reward, and next state is stored in the priority experience replay pool. After the priority experience replay pool is full, the deep reinforcement learning network begins training. Batch samples are periodically sampled from the pool for gradient updates to optimize the network parameters. After a fixed number of steps, the target network is updated synchronously and the model state is saved. Finally, the bus dynamic scheduling model is obtained.

[0045] Step 4.2: Generate a dynamic departure timetable based on the bus dynamic scheduling model;

[0046] Based on real-time arrival, departure, and passenger boarding data, the trained bus dynamic scheduling model is used to reason at each decision point. The final action is selected according to the probability distribution of the action in the current state, thereby dynamically generating the departure timetable.

[0047] Further, step 1 includes the following steps:

[0048] Step 1.1: Obtain historical vehicle operation data and perform preprocessing;

[0049] The system collects historical vehicle operation datasets, including route data tables, arrival / departure data tables, and passenger pick-up data tables. The route data tables record basic information about all vehicle routes, including route name, route direction, names, coordinates, and single-route numbers of each stop in the current direction. The single-route number indicates which stop the bus is located at in the current route direction. The arrival / departure data tables record information about buses arriving at and leaving stops during operation, including route name, vehicle ID, route direction, single-route number of the stop where the vehicle is located, vehicle arrival / departure status, recording time, and remaining battery power. The passenger pick-up data tables record information about all passengers boarding, including route name, route direction, vehicle ID, boarding stop name, and boarding transaction time.

[0050] Preprocess historical operational data, including data cleaning and deletion of invalid data;

[0051] For each vehicle route, the average travel time between stations is calculated based on the vehicle's arrival and departure times; the passenger boarding frequency of each route at different times is statistically analyzed based on the passenger boarding information table; and the original vehicle timetable is extracted as a benchmark reference for subsequent optimization.

[0052] Step 1.2: Obtain the real-time operating status of the vehicle;

[0053] During vehicle operation, real-time arrival and departure data are obtained from the vehicle's GPS positioning device and battery management system, and real-time arrival and departure data and real-time passenger boarding data are obtained from the IC card swiping device and mobile payment terminal. The field settings for these two types of real-time data are consistent with those for historical data.

[0054] Furthermore, in step 2, the scheduling environment simulation system simulates the operation of vehicle routes based on vehicle management, station management, and passenger management. It sets minimum and maximum departure interval scheduling parameters to divide time periods, sets decision time points based on each time period, outputs the current simulation status at each decision time point, and receives scheduling decision instructions from the intelligent agent. After the instructions are executed, it continues to run until the next decision time point, repeating the cycle until the simulation ends.

[0055] Furthermore, regarding vehicle management, the scheduling environment simulation system performs detailed modeling of vehicle scheduling status. Each vehicle includes status parameters such as its current route direction and station, number of passengers, number of available seats, and remaining battery power, which are updated in real time. Vehicles depart from the starting station according to the timetable, stop at each station along the route in sequence to allow passengers to board and alight, and finally arrive at the destination station in the current direction. After a short stop, the vehicle switches directions according to the scheduling arrangement, completes the opposite direction operation, and returns to the starting station. If the remaining battery power can support the next trip, the vehicle will be directly added to the idle queue to wait for subsequent scheduling; otherwise, it will first be added to the charging queue, and then enter the idle queue after charging. Parameters such as the number of vehicles, passenger capacity, and battery capacity are set according to the actual vehicle model configuration.

[0056] Regarding the station management, each route includes a starting station, a terminal station, and several intermediate stations; the starting station is responsible for managing the idle queue and charging queue of the vehicles on this route; each station is responsible for managing the waiting passengers at this station and recording the number of passengers stranded due to the previous vehicle being fully loaded.

[0057] Regarding passenger management, the scheduling environment simulation system dynamically generates passengers based on real passenger flow data patterns; each station continuously generates new passengers during operation, and the number and frequency of generation are set according to the characteristics of the route and time period changes; when a bus arrives at a station, passengers board in order, and if the vehicle capacity is full, the remaining passengers will wait for the next bus; the scheduling environment simulation system records the number of waiting passengers and the number of stranded passengers at each station in real time.

[0058] Furthermore, in step 2, in order to enhance the realism of the simulation, the scheduling environment simulation system introduces random disturbances in the travel time between adjacent stations and simulates various special waiting situations such as severe weather, traffic light waiting, and congestion.

[0059] A vehicle-management-based dynamic bus dispatching system includes a vehicle data and status acquisition module, a dispatching environment simulation module, a dispatching decision module, and a bus dynamic dispatching module. According to the aforementioned vehicle-management-based dynamic bus dispatching method, the system sequentially executes the following steps: acquiring historical vehicle operation data and real-time operation status; constructing the dispatching environment simulation system; modeling the dispatching decision process; and constructing the bus dynamic dispatching model.

[0060] The advantages and beneficial effects of this invention are as follows:

[0061] This invention constructs a scheduling environment simulation system that considers multiple factors such as vehicle management, station management, and passenger management by acquiring historical operation data and real-time operation status of bus routes. It not only models routes based on historical operation characteristics and passenger travel demand but also considers random factors such as weather changes and traffic congestion, as well as charging constraints, achieving a realistic simulation of bus route operation. Based on this, the dynamic scheduling problem is defined as a Markov decision process considering vehicle management constraints, with vehicle availability management parameters explicitly defined in the state space and reward function. Finally, a Rainbow DQN is used to train the bus dynamic scheduling model, and a dynamic departure timetable is generated based on the trained scheduling model. This effectively reduces departure frequency while meeting vehicle management constraints and passenger travel demand, thereby reducing the operating costs of bus companies. Attached Figure Description

[0062] Figure 1 This is a flowchart of the method in an embodiment of the present invention.

[0063] Figure 2 This is a schematic diagram of the operation of the scheduling environment simulation system in an embodiment of the present invention.

[0064] Figure 3 This is a comparison chart of the carrying capacity of the 7 bus timetables generated by different methods in the embodiments of the present invention.

[0065] Figure 4 This is a schematic diagram of the system structure in an embodiment of the present invention. Detailed Implementation

[0066] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0067] like Figure 1 As shown, a dynamic bus scheduling method based on vehicle management includes the following steps:

[0068] Step 1: Obtain historical operating data and real-time operating status of electric buses.

[0069] Step 1.1: Obtain historical operating data of electric buses and perform preprocessing.

[0070] Historical operational datasets for electric buses were collected, including route data tables, arrival / departure data tables, and passenger pick-up data tables. The route data table records basic information for all bus routes, including route name, route direction, names, coordinates, and single-route numbers of each stop in the current direction. The single-route number represents the station's position within the current route direction; for example, a single-route number of 1 indicates that the station is the starting point in the current direction. The arrival / departure data tables record information about buses arriving at and leaving stations during their operation, including route name, vehicle ID, route direction, single-route number of the station where the vehicle is located, vehicle arrival / departure status, recording time, and remaining battery power. The passenger pick-up data table records information about all passengers boarding, including route name, route direction, vehicle ID, boarding station name, and boarding transaction time.

[0071] After obtaining the historical data, the raw data is first cleaned, and invalid data is deleted. Then, for each bus route, the average travel time between each stop is calculated based on the vehicle's arrival and departure times; according to the passenger boarding information table, the passenger boarding frequency of each route at different times is statistically analyzed; and the original bus timetable is extracted as a benchmark reference for subsequent optimization.

[0072] Step 1.2: Obtain the real-time operating status of the electric bus.

[0073] During the operation of electric buses, real-time arrival and departure data are obtained from the on-board GPS positioning device and battery management system, and real-time arrival and departure data and real-time passenger boarding data are obtained from the IC card swiping device and mobile payment terminal. The field settings of these two types of real-time data are consistent with the historical data.

[0074] Step 2: Construct a scheduling environment simulation system that considers multiple factors such as vehicle management, station management, and passenger management, such as... Figure 2 As shown.

[0075] The scheduling environment simulation system comprehensively considers vehicle management, station management, and passenger management to simulate the daily operation of a bus route. The system sets scheduling parameters such as minimum and maximum departure intervals, dividing time into two-minute intervals, with the start time of each time period designated as a decision point. At each decision point, the system outputs the current simulation environment state and receives scheduling decision instructions from the agent. After executing the instructions, the system continues to run until the next decision point, repeating this cycle until the simulation ends.

[0076] In terms of vehicle management, the simulation system provides a detailed model of the dispatch status of electric buses. Each bus includes status parameters such as its current route direction and stops, number of passengers on board, number of available seats, and remaining battery power, which are updated in real time. Buses depart from the starting station according to the timetable, stopping at each stop along the route to allow passengers to board and alight, ultimately reaching the destination station in the current direction. After a short stop, the bus switches directions according to the dispatch schedule, completes its inbound journey, and returns to the starting station. If the remaining battery power is sufficient for the next trip, the bus will be directly added to the idle queue, awaiting further dispatch; otherwise, it must first be added to the charging queue to complete charging before entering the idle queue. Parameters such as the number of buses, passenger capacity, and battery capacity are set according to the actual vehicle model configuration.

[0077] In terms of station management, each route includes a starting station, a terminal station, and several intermediate stations. The starting station is responsible for managing the idle queue and charging queue of vehicles on its route. Each station is responsible for managing the waiting passengers at its station and recording the number of passengers stranded due to the previous vehicle being full.

[0078] In terms of passenger management, the simulation system dynamically generates passengers based on real passenger flow data patterns. New passengers are continuously generated at each station during operation, with the number and frequency determined according to route characteristics and time of day. When a bus arrives at a station, passengers board in order; if the bus is full, remaining passengers will wait for the next bus. The system records the number of waiting and stranded passengers at each station in real time.

[0079] To enhance the realism of the simulation, the system also introduces random disturbances into the travel time between adjacent stations and simulates special situations such as severe weather, traffic light waiting, and congestion.

[0080] Step 3: Model the Markov decision process, defining the state space, action space, and reward function that takes into account vehicle management constraints and balances the interests of passengers and the bus company.

[0081] Step 3.1: Model the Markov decision process and define the state space and action space considering vehicle management constraints.

[0082] The dynamic scheduling problem of urban electric buses is modeled as a Markov decision process, defining the corresponding state space S, action space A, and reward function R.

[0083] The state space S contains different states. , The state characteristics that the agent obtains from the environment at time m are represented as follows:

[0084] (1)

[0085] The meaning of each parameter is explained in detail below:

[0086] (1) and This represents the normalized time information, where , , Represents the number of hours in the current environment. This represents the number of minutes in the current environment.

[0087] (2) The demand pressure coefficient is defined as follows:

[0088] (2)

[0089] Where SN represents the total number of stations on the line. This represents the number of people waiting at the s-th station at time m. This indicates the total number of vehicles currently operating on the line. This represents the load capacity of the k-th vehicle. This represents the number of seats in the k-th car. This is the capacity coefficient, representing the vehicle's load capacity as a percentage of its seating capacity. times. The smaller the value, the more sufficient the overall system capacity.

[0090] (3) Let represent the sum of boarding passenger waiting times for all vehicles currently operating on the route after normalization, defined as:

[0091] (3)

[0092] in, The total waiting time for passengers boarding in all vehicles currently operating on the route is calculated using the following formula:

[0093] (4)

[0094] in, This indicates the total number of vehicles currently operating on the line. This represents the number of bus stops that the k-th bus has passed. This represents the number of passengers who board the k-th bus at the s-th bus stop. This represents the time when the i-th passenger boards the k-th bus at the s-th bus stop. This represents the time when the i-th passenger arrives at the s-th bus stop. It is a parameter that normalizes the total waiting time.

[0095] (4) This represents the utilization rate of the carrying capacity of all vehicles currently operating on the route. It is an important indicator for measuring the match between actual passenger demand and the carrying capacity of buses, and can help determine whether it is necessary to dispatch a bus at the current decision-making point in time. Its definition is:

[0096] (5)

[0097] in, This reflects the actual carrying capacity required by all vehicles on the line. , and The definition is the same as before. This represents the number of bus stops actually traveled by the i-th passenger who boarded the k-th bus at the s-th bus stop. This reflects the load-bearing capacity available to all vehicles. , , The definition is the same as before.

[0098] (5) This represents the ratio of the number of currently available buses to the total number of buses, and is defined as follows:

[0099] (6)

[0100] in, This represents the number of available vehicles on the current route. Represents the total number of vehicles on the current route, satisfying , This represents the total number of vehicles currently operating on the line. It also includes the number of idle vehicles at the originating station. When the value is zero, departure will not be possible.

[0101] Action space contains two types of actions: actions Indicates departure, action This indicates that the train will not depart.

[0102] Step 3.2: Define a reward function that balances the interests of passengers and the bus company.

[0103] The reward function balances the interests of both passengers and the bus company, ensuring that departure scheduling and bus management do not conflict. Passenger interests are measured by bus crowding levels and waiting times, while the bus company's interests are measured by the number of departures. Reward Function Indicates that at time m, based on state Execute action The reward value obtained provides immediate performance feedback for the deep reinforcement learning model, and is defined as follows:

[0104] (7)

[0105] When passenger flow is low, the agent should prioritize action 0 (not dispatching the bus) to reduce the bus company's operating costs. Therefore, action 0 should be rewarded to the agent. ,in This represents the utilization rate of transport capacity; a smaller value indicates lower passenger demand during the current time period. However, not dispatching a vehicle may increase passenger waiting time, therefore a penalty is introduced. , This is a penalty factor for waiting time.

[0106] When passenger flow is high, the agent should prioritize action 1 (departure), therefore, the agent should be rewarded for action 1. This indicates that passenger demand is high during the current time period, and departure at minute m should be encouraged. To comply with the bus company's vehicle management requirements, penalties need to be imposed on agents that erroneously depart. When there are no available vehicles at the starting station ( If the agent still issues the departure command, it will be subject to... The punishment, among which This is a penalty factor for incorrect vehicle departure.

[0107] Furthermore, passenger congestion will severely impact passenger satisfaction. Therefore, regardless of whether the bus has departed, any instance of passenger congestion will result in penalties for actions 0 and 1. , The penalty factor for the number of stranded passengers, The total number of stranded passengers on the current line at time m is represented by the following formula:

[0108] (8)

[0109] Where SN represents the total number of stations on the line. This represents the number of passengers stranded at station s at time m.

[0110] Step 4: Train the bus dynamic scheduling model using Rainbow DQN and infer the dynamic departure timetable.

[0111] Step 4.1: Train the bus dynamic scheduling model using Rainbow DQN.

[0112] A dynamic bus scheduling model was constructed using the Rainbow DQN algorithm to learn an optimized dynamic departure strategy. First, a bus was forcibly dispatched at the first departure time, and the environmental state was updated. Then, the scheduling environment simulation system triggered a scheduling decision every two minutes. At each decision point, the model input was a 6-dimensional state vector containing vehicle management information. Based on this state, Rainbow DQN estimated the value distribution of each action using a noisy network structure and selected the optimal action by combining minimum and maximum departure interval constraints. The selected action was applied to the simulation environment, and the system obtained a new state vector and the immediate reward for the current action. The interaction data <state, action, reward, next state> generated during this process was stored in a priority experience replay pool. Once the priority experience replay pool was full, Rainbow DQN began training, periodically sampling batches of samples from the pool for gradient updates to optimize the Q-network parameters. After a fixed number of steps, the target network was updated synchronously, and the model state was saved, ultimately yielding the dynamic bus scheduling model.

[0113] Step 4.2: Generate a dynamic departure timetable based on the bus scheduling model.

[0114] Based on real-time arrival and departure data and passenger boarding data, the trained bus scheduling model is used to reason at each decision point, and the final action is selected according to the probability distribution of the action in the current state, thereby dynamically generating the departure timetable.

[0115] In this embodiment of the invention, as shown in Table 1, the performance of the original bus timetable, the genetic algorithm, DRL-TO, and the bus timetable generated by the method of the present invention are compared.

[0116] Table 1 Evaluation results of different methods

[0117]

[0118] For three bus routes in Hangzhou (routes 7, 10, and 156), two deep reinforcement learning-based methods (DRL-TO and the method of this invention) showed significant advantages compared to the original bus timetable and the results of the genetic algorithm. While the original timetable ensured no incorrect departures, it resulted in a higher number of departures and a higher number of stranded passengers. Compared to the original timetable, both the genetic algorithm and the DRL-TO method improved in terms of the number of departures, average waiting time, and number of stranded passengers, with the DRL-TO method outperforming the genetic algorithm. However, both the timetables generated by the genetic algorithm and DRL-TO exhibited incorrect departures. Furthermore, although the genetic algorithm achieved the lowest number of departures on route 156 in the southbound direction, its average waiting time and number of stranded passengers were the highest among all methods. In contrast, the method of this invention showed a more balanced performance across all metrics, maintaining low levels in the number of departures, average waiting time, and number of stranded passengers, while completely avoiding incorrect departures, demonstrating excellent overall scheduling performance.

[0119] like Figure 3 The figure shows a comparison between the carrying capacity provided by the timetables for seven bus routes generated by different methods and the actual passenger demand. Each marker in the figure represents the route's carrying capacity within 30 minutes from that time point. As can be seen from the figure, the original bus timetable provided far more capacity than actually needed between 9:30 and 13:00, resulting in a significant waste of resources, while experiencing short-term capacity shortages during the evening peak. The genetic algorithm is insensitive to changes in passenger demand, and its capacity adjustment is lagging. The DRL-TO method provides redundant capacity during the morning peak, ensuring service levels but increasing operating costs. In contrast, the method of this invention better matches carrying capacity with actual passenger demand, dynamically adjusting departures based on real-time passenger demand and maintaining a certain level of capacity redundancy to cope with emergencies.

[0120] like Figure 4 As shown, a vehicle management-based dynamic bus dispatching system includes a vehicle data and status acquisition module, a dispatching environment simulation module, a dispatching decision module, and a bus dynamic dispatching module. According to the vehicle management-based dynamic bus dispatching method, the system sequentially performs the following steps: acquiring historical vehicle operation data and real-time operation status, constructing the dispatching environment simulation system, modeling the dispatching decision process, and constructing the bus dynamic dispatching model.

[0121] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic bus scheduling method based on vehicle management, characterized in that... Includes the following steps: Step 1: Obtain historical vehicle operation data and real-time operating status; Step 2: Construct a multi-factor scheduling environment simulation system that considers vehicles, stations, and passengers; Step 3: Model the scheduling decision-making process, defining the state space and action space considering vehicle constraints, as well as the reward function that takes into account the interests of public transport stakeholders; specifically including the following steps: Step 3.1: Model the dynamic vehicle scheduling problem as a Markov scheduling decision process, and define the state space considering vehicle management constraints. S、 Action space A and reward function R ; state space S Including intelligent agents m The different states obtained from the environment at all times, the characteristics of which include time information at different scales, demand pressure coefficient, the sum of passenger waiting times for all vehicles currently running on the line, the capacity utilization rate of all vehicles currently running on the line, and the ratio of the number of currently available buses to the total number of buses. The action space includes the action of starting the vehicle or not starting the vehicle; Step 3.2: Define a reward function that balances the interests of passengers and the bus company, ensuring that departure scheduling and bus management do not conflict. Passenger interests are measured by bus congestion and waiting time, while the bus company's interests are measured by the number of departures. The reward function represents the reward value obtained by performing an action based on a specific state at a given time. The reward function is as follows: When passenger flow is low, the agent should prioritize not to depart. Based on the capacity utilization rate, the agent is rewarded for not departing. The lower the capacity utilization rate, the lower the passenger demand in the current time period. At the same time, a penalty for waiting time is introduced. When passenger flow is high, the agent should prioritize departure actions. Based on the high passenger demand at the current time, the agent should be rewarded for departure actions. If the agent still issues a departure instruction when there are no available vehicles at the origin station, it will be penalized for incorrect departure. When passenger congestion occurs, the agent will be penalized for both non-departure and departure actions based on the number of congested passengers. Step 4: Construct a dynamic bus scheduling model. Based on the scheduling environment simulation system, trigger the scheduling decision. Using the current state of the trigger, estimate the value distribution of each action and select the optimal action by combining the minimum and maximum departure interval constraints. The selected action is applied to the scheduling environment simulation environment to obtain the immediate reward brought by the next state and the current action. Construct a dynamic bus scheduling model and train it based on the priority experience replay pool constructed from the current state, action, reward, and next state to generate dynamic departure times.

2. The method for dynamic bus scheduling based on vehicle management according to claim 1, characterized in that: In step 3.1, the demand pressure coefficient is the sum of the number of people waiting at each station at a certain moment, divided by the sum of the load capacity of all vehicles running on the line at that moment. The load capacity is obtained based on the number of seats and capacity coefficient of the vehicle. The sum of the waiting times for passengers boarding all vehicles currently operating on the route is obtained by subtracting the time it takes for passengers to arrive at the bus stop from the time it takes for passengers to board the vehicle at the bus stop, and summing the results based on the total number of vehicles currently operating on the route, the number of bus stops the vehicles have passed, and the number of passengers boarding at the passenger boarding station. The carrying capacity utilization rate of all vehicles currently operating on the line is obtained by dividing the actual carrying capacity required by all vehicles on the line by the carrying capacity that all vehicles can provide. The ratio of the current number of available buses to the total number of buses is obtained by dividing the number of available buses at the starting station by the total number of buses on the current route.

3. The method for dynamic bus scheduling based on vehicle management according to claim 1, characterized in that: Step 4 includes the following steps: Step 4.1: Train the bus dynamic scheduling model using a deep reinforcement learning network; First, a bus is forcibly dispatched at the first departure time, and the environmental state is updated. Then, the scheduling environment simulation system triggers a scheduling decision every certain period of time. At each decision time, the vehicle state is input. Based on the vehicle state, the deep reinforcement learning network estimates the value distribution of the action of dispatching or not dispatching using a noisy network structure, and selects the optimal action by combining the minimum and maximum departure interval constraints. The selected action is applied to the simulation environment to obtain the new vehicle state and the immediate reward brought by the current action. The generated interactive data containing state, action, reward, and next state is stored in the priority experience replay pool. After the priority experience replay pool is full, the deep reinforcement learning network begins training. Batch samples are periodically sampled from the pool for gradient updates to optimize the network parameters. After a fixed number of steps, the target network is updated synchronously and the model state is saved. Finally, the bus dynamic scheduling model is obtained. Step 4.2: Generate a dynamic departure timetable based on the bus dynamic scheduling model; Based on real-time arrival, departure, and passenger boarding data, the trained bus dynamic scheduling model is used to reason at each decision point. The final action is selected according to the probability distribution of the action in the current state, thereby dynamically generating the departure timetable.

4. The method for dynamic bus scheduling based on vehicle management according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: Obtain historical vehicle operation data and perform preprocessing; Collect historical vehicle operation datasets, including route data tables, arrival and departure data tables, and passenger pick-up data tables; The route data table records basic information about all vehicle routes; The arrival and departure data table records information about buses arriving at and leaving stations during their operation; The passenger boarding data table records the boarding information of all passengers; For each vehicle route, the average travel time between stations is calculated based on the vehicle's arrival and departure times. Based on the passenger boarding information table, the passenger boarding frequency of each route at different times is statistically analyzed; Extract the original vehicle timetable as a benchmark for subsequent optimization; Step 1.2: Obtain the real-time operating status of the vehicle; During vehicle operation, real-time arrival and departure data and real-time passenger boarding data are acquired.

5. The method for dynamic bus scheduling based on vehicle management according to claim 1, characterized in that: In step 2, the scheduling environment simulation system simulates the operation of vehicle routes based on vehicle management, station management, and passenger management. It sets interval scheduling parameters to divide time periods and sets decision time points based on each time period. At each decision time point, it outputs the current simulation status and receives scheduling decision instructions from the intelligent agent. After the instructions are executed, it continues to run until the next decision time point, repeating the cycle until the simulation ends.

6. The method for dynamic bus scheduling based on vehicle management according to claim 5, characterized in that: Regarding vehicle management, the scheduling environment simulation system performs detailed modeling of vehicle scheduling status; each vehicle includes status parameters such as current route direction and station, number of passengers in the vehicle, number of available seats, and remaining battery power, which are updated in real time; the vehicle departs from the starting station in the order of the timetable, stops at each station along the route in sequence to allow passengers to get on and off, and finally arrives at the destination station in the current direction; after a short stop, the vehicle switches directions according to the scheduling arrangement, completes the opposite operation task, and returns to the starting station; If the remaining battery power is sufficient to support the next trip, the vehicle will be directly added to the idle queue and await subsequent dispatch; otherwise, it will be added to the charging queue first, and will enter the idle queue after charging is completed; vehicle parameters are set according to the actual vehicle model configuration. Regarding the station management, each route includes a starting station, a terminal station, and several intermediate stations; the starting station is responsible for managing the idle queue and charging queue of the vehicles on this route; each station is responsible for managing the waiting passengers at this station and recording the number of passengers stranded due to the previous vehicle being fully loaded. Regarding passenger management, the scheduling environment simulation system dynamically generates passengers based on real passenger flow data patterns; each station continuously generates new passengers during operation, and the number and frequency of generation are set according to the characteristics of the route and time period changes; when a bus arrives at a station, passengers board in order, and if the vehicle capacity is full, the remaining passengers will wait for the next bus; the scheduling environment simulation system records the number of waiting passengers and the number of stranded passengers at each station in real time.

7. The method for dynamic bus scheduling based on vehicle management according to claim 1, characterized in that: In step 2, the scheduling environment simulation system introduces random disturbances into the travel time between adjacent stations and simulates various special waiting scenarios.

8. A dynamic bus dispatching system based on vehicle management, comprising a vehicle data and status acquisition module, a dispatching environment simulation module, a dispatching decision module, and a dynamic bus dispatching module, characterized in that: According to any one of claims 1 to 7, a dynamic bus scheduling method based on vehicle management is performed sequentially, including acquiring historical vehicle operation data and real-time operation status, constructing a scheduling environment simulation system, modeling the scheduling decision process, and constructing a dynamic bus scheduling model.

Images