DQN-based bus uplink and downlink dynamic equilibrium timetable generation method
Through the deep reinforcement of the interaction between learning and simulation environment, a dynamic balance timetable for buses is generated, which solves the problem that bus timestables cannot dynamically adapt to passenger flow and traffic changes in the existing technology, realizes real-time scheduling and two-way route optimization, and improves the Bus operation efficiency and service quality.
Patent Information
- Application Number
- CN202410915199.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-07-03
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-09
AI Technical Summary
The existing bus schedule optimization method cannot dynamically adapt to sudden passenger flow changes and traffic conditions, and cannot schedule two directions of a bus line at the same time, resulting in the scheduling of shifts that do not meet the actual requirements.
A deep reinforcement learning (DQN)-based method is adopted to interact with the simulation environment to generate a dynamic balanced timetable for buses, adjust the departure time and vehicle scheduling in real time, and generate a new timetable based on passenger flow, itinerary time and site changes.
The dynamic adjustment of the bus schedule has been achieved, and the departure plan can be optimized in emergencies, operational efficiency and service quality can be improved, and the scheduling needs of two-way routes can be met.
Smart Images

Figure CN118940733A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of public transport operation systems, and in particular to a method for generating a dynamic balanced timetable for up and down buses based on deep reinforcement learning (DQN). Background Art
[0002] Optimizing the bus departure schedule can effectively improve the quality of bus services and operational efficiency. In actual applications, the departure time of the bus schedule can be optimized according to passenger flow.
[0003] At present, there are many methods for optimizing bus schedules, such as genetic algorithms, graph heuristic algorithms, and exhaustive methods. All of these methods can generate bus schedules offline based on historical passenger flow. In the actual operation of the bus system, there are various emergencies, including sudden increase in passenger flow due to emergencies, bus travel time due to road congestion, station changes due to road closures and construction, etc. The timetable generated based on historical data cannot meet the travel needs of passengers. In addition, the actual departure of the bus system requires that the line departs in both directions and the vehicles cannot be empty. There are existing methods based on reinforcement learning to optimize the departure timetable of one-way lines. The generated departure timetable cannot dynamically adapt to changes in passenger flow and traffic conditions, and cannot dispatch both directions of a bus line at the same time, resulting in scheduling that does not meet actual requirements and cannot be applied. Summary of the invention
[0004] The purpose of the present invention is to provide a method for generating a dynamic balanced timetable for buses up and down based on DQN. Based on the traffic conditions of bus operation and bus passenger flow information, the method is trained by directly interacting with a simulation environment, and decisions are made in a real-time simulation environment to generate a dynamic balanced timetable for buses up and down. When the passenger flow changes suddenly due to an unexpected event, a new departure timetable can be generated based on the reasoning of the changed passenger flow. When the bus travel time changes suddenly, a new departure timetable can be generated based on the reasoning of the changed travel time. When the bus stop is temporarily adjusted and changed, a new departure timetable can be generated based on the reasoning of the changed stop information, so as to solve the problems raised in the above-mentioned background technology.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A method for generating a dynamic balanced timetable for up and down buses based on DQN includes the following steps:
[0007] S1: Initialize bus route information, including setting route ID, shortest departure interval t min 、The longest departure interval t max 、Number of people allowed to travel by vehicle C max 、First shift time Start time、End time ;
[0008] S2: Obtain historical traffic conditions and passenger flow information for both upstream and downstream routes;
[0009] S3: Initialize the reinforcement learning DQN model parameters, input dimension m input , output dimension m output , the number of hidden layers m hidden , the number of hidden layer neurons n hidden , activation function is ReLU, learning rate lr, batch size batch_size, experience playback capacity C ER ;
[0010] S4: Construct a DQN model that can generate a dynamic equilibrium schedule for up and down buses;
[0011] S5: DQN model reasoning can generate a dynamic balanced schedule for up and down buses. Based on the trained model, the traffic conditions and passenger flow data of the current line are used for reasoning to obtain a dynamic balanced schedule for up and down buses. When the passenger flow changes suddenly, a new departure schedule can be generated based on the reasoning of the changed passenger flow. When the bus travel time changes suddenly, a new departure schedule can be generated based on the reasoning of the changed travel time. When the bus stop is temporarily adjusted and changed, a new departure schedule can be generated based on the reasoning of the changed stop information.
[0012] S6: Output of the dynamic balanced timetable for buses in the up and down directions, outputting each departure time point planned for the up and down directions of the line respectively, and including the interval between the current departure time and the previous departure time, the passenger capacity provided by this departure plan at each specific interval, the total number of departures, the number of passengers who could not get on the bus due to full load, and the average waiting time of passengers.
[0013] Furthermore, the specific method in S2 is:
[0014] S201: Calculate the travel time between two adjacent stops for each bus based on the data of bus entry and exit;
[0015] S202: Calculate the travel time between two adjacent stops every 15 minutes in a day based on the travel time between two adjacent stops of each bus, as historical traffic condition data;
[0016] S203: The data of the passenger's getting-off station is derived based on the bus card swiping data, and then combined with the bus entry and exit data, the passenger's getting-on and getting-off station information and time information are calculated as the passenger flow information.
[0017] Furthermore, the specific method in S4 is:
[0018] S401: Establishing state variables of a DQN model that can generate a dynamic equilibrium timetable for up and down buses;
[0019] S402: Establishing the action state of a DQN model that can generate a dynamic equilibrium timetable for up and down buses;
[0020] S403: Establishing a reward function of a DQN model that can generate a dynamic equilibrium timetable for up and down buses;
[0021] S404: Train the DQN model that can generate a dynamic balanced timetable for up and down buses. During the training process, a complete simulation is performed for each day. First, a bus is forced to be dispatched at the first bus time and the environment state is updated. After that, for every minute, the DQN intelligent body makes a decision and obtains an action. This action has p m The probability is the result generated by the model, 1-p m The probability is the result of random value; if the time from the current moment to the last departure is less than the shortest departure interval t min If the time between the current time and the last time of departure is greater than the longest departure interval t max Then the departure is mandatory;
[0022] S405: Simulate the DQN model that can generate a dynamic balanced timetable for up and down buses. After training in S404, apply this action to the simulation environment to obtain the next state vector and the reward value obtained by making the current action, and store the current state, current reward, current action and next state in the experience replay; when the experience replay is full, the DQN model will be trained once after every 5 simulations, and the target network in the DQN algorithm will be updated once after every 100 training times, and the obtained model will be saved.
[0023] Furthermore, the state variables in S401 will take into account the time state, the uplink and downlink vehicle passenger status, the uplink and downlink passenger flow status, and the uplink and downlink departure status. Among them, m is the different time of the day, X mu and X md Respectively represent the bus up state and down state, definition
[0024] X mu =[x u_h , x u_min , x u_pn , x u_wmin , x u_tpn , x u_vn ],
[0025] X md =[x d_h , x d_min , x d_pn, x d_wmin , x d_tpn , x d_vn ], x u_h and x d_h Respectively represent the hours of the uplink and downlink at time m, x u_min and x d_min Respectively represent the minutes of the uplink and downlink at time m, x u_pn and x d_pn They represent the number of people in the uplink and downlink vehicles at time m, respectively, u_wmin and x d_wmin They represent the waiting time of passengers in the uplink and downlink at time m, respectively. u_tpn and x d_tpn They represent the passenger flow of the uplink and downlink vehicles at time m, respectively, u_vn and x d_vn They respectively represent the number of departures of the uplink and downlink lines at time m.
[0026] Furthermore, the action vector of the action state in S402 defines four action elements: no train is sent up or down, only the train is sent up, only the train is sent down, and both the train is sent up and down. A=[0, 1, 2, 3] is defined. If the agent selects action 0, it means that no train is sent up or down at the current moment; if action 1 is selected, it means that only the train is sent up at the current moment; if action 2 is selected, it means that only the train is sent down at the current moment; if action 3 is selected, it means that both the train is sent up and down at the current moment.
[0027] Furthermore, the reward function in S403 includes two parts: reward = reward up +reward down -reward departure_std ; For the uplink, if no departure reward is given at the current time up =1-(o m / e m )-(ω×W m )-(β×ds m )+γ(c1-c2), where o m is the number of passengers boarding a vehicle if it is dispatched at the mth minute, e m is the number of passengers allowed to travel by the vehicle, ω can take several values. If the value is large, the model tends to dispatch more vehicles, and if the value is small, the model tends to dispatch fewer vehicles. The value can be 1 / 1000; β is a constant, which is 0.2. ds m Indicates how many passengers will be unable to board the bus at the mth minute because the bus is full; γ is a constant, γ = 1e-3 × 200. When the simulation time is less than two hours from the last bus, γ = 5e-3 × 200. If the current time is a departure reward, up =(om / e m )-(β×ds m )-γ(c1-c2), for the downlink, if no reward is given at the current time down =1-(o m / e m )-(ω×W m )-(β×ds m )-γ(c1-c2); if the current time is the departure reward down =(o m / e m )-(β×ds m )+γ(c1-c2); reward departure_std is the standard deviation of the departure interval.
[0028] Compared with the prior art, the present invention has the following beneficial effects:
[0029] The method for generating a dynamic balanced timetable for up and down buses based on DQN of the present invention designs a model for generating a dynamic balanced timetable for up and down buses based on DQN based on the traffic conditions of bus operation and bus passenger flow information, including designing state variables, action states, and reward functions of a DQN model that can generate a dynamic balanced timetable for up and down buses, training by directly interacting with a simulation environment, making decisions in a real-time simulation environment, and realizing real-time generation of a dynamic balanced timetable for up and down buses. When an unexpected event occurs in the passenger flow, a new departure timetable can be generated based on the reasoning of the changed passenger flow. When the bus travel time suddenly changes, a new departure timetable can be generated based on the reasoning of the changed travel time. When a bus stop is temporarily adjusted and changed, a new departure timetable can be generated based on the reasoning of the changed stop information. The problem that the departure timetable generated in the prior art cannot dynamically adapt to changes in passenger flow and traffic conditions, and cannot simultaneously dispatch two directions of a bus line, resulting in scheduling that does not meet actual requirements and cannot be applied, is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is the overall framework diagram of the present invention;
[0031] Figure 2 is a method flow chart of the second step of the present invention;
[0032] Figure 3 This is a method flow chart of the fourth step of the present invention. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] See also Figure 1-3 In an embodiment of the present invention, a method for generating a dynamic balanced timetable for up and down buses based on DQN is provided, comprising the following steps:
[0035] Step 1: Initialize bus route information, including setting route ID, shortest departure interval t min 、The longest departure interval t max 、Number of people allowed to travel by vehicle C max 、First shift time Start time 、End time Specific: Set the route ID to 211 and the shortest departure interval of the vehicle to t min The longest departure interval is 10 minutes. max 30 minutes, vehicle capacity C max 84 people, first shift starts time 6:30, the last shift ends time 22:30;
[0036] Step 2: Obtain historical traffic conditions and passenger flow information for both upstream and downstream routes; the specific method is as follows:
[0037] Step 2-1: Calculate the travel time between two adjacent stops for each bus based on the data of bus entry and exit;
[0038] Step 2-2: Based on the travel time between each bus's two adjacent stops, calculate the travel time between two adjacent stops every 15 minutes in a day as historical traffic condition data;
[0039] Step 2-3: Derived from the data of the bus card swiping, the data of the passenger's getting off station is combined with the bus entry and exit data to calculate the passenger's getting on and off station information and time information as passenger flow information;
[0040] Step 3: Initialize the reinforcement learning DQN model parameters and input dimension m input , output dimension m output , the number of hidden layers m hidden , the number of hidden layer neurons n hidden , activation function is ReLU, learning rate lr, batch size batch_size, experience playback capacity C ER; Specifically, set the input dimension m input is 12, output dimension m output is 4, the number of hidden layers is m hidden is 12, the number of hidden layer neurons n hidden is 500, the activation function is ReLU, the learning rate lr is 0.001, the batch size batch_size is 64, and the experience playback capacity is C ER is 3000;
[0041] Step 4: Construct a DQN model that can realize the generation of a dynamic equilibrium timetable for up and down buses; the specific method is as follows:
[0042] Step 4-1: Design the state variables of the DQN model that can generate the dynamic equilibrium timetable for up and down buses. The state variables will take into account the time status, up and down vehicle passenger status, up and down passenger flow status, and up and down departure status. Definition Among them, m is the different time of the day, X mu and X md Respectively represent the bus up state and down state, definition
[0043] X mu =[x u_h , x u_min , x u_pn , x u_wmin , x u_tpn , x u_vn ],
[0044] X md =[x d_h , x d_min , x d_pn , x d_wmin , x d_tpn , x d_vn ], x u_h and x d_h Respectively represent the hours of the uplink and downlink at time m, x u_min and x d_min Respectively represent the minutes of the uplink and downlink at time m, x u_pn and x d_pn They represent the number of people in the uplink and downlink vehicles at time m, respectively, u_wmin and x d_wmin They represent the waiting time of passengers in the uplink and downlink at time m, respectively. u_tpn and x d_tpn They represent the passenger flow of the uplink and downlink vehicles at time m, respectively, u_vn and x d_vn Respectively represent the number of departures of the uplink and downlink lines at time m;
[0045] Step 4-2: Design the action state of the DQN model that can generate a dynamic equilibrium schedule for up and down buses. The action vector of the action state defines four action elements: no up and down buses, only up buses, only down buses, and both up and down buses. Define A = [0, 1, 2, 3]. If the agent chooses action 0, it means that no up and down buses are dispatched at the current moment; if action 1 is selected, it means that only up buses are dispatched at the current moment; if action 2 is selected, it means that only down buses are dispatched at the current moment; if action 3 is selected, it means that both up and down buses are dispatched at the current moment;
[0046] Step 4-3: Design a reward function for the DQN model that can generate a dynamic equilibrium schedule for up and down buses. The reward function consists of three parts: reward = reward up +reward down -reward departure_std ; For the uplink, if no departure reward is given at the current time up =1-(o m / e m )-(ω×W m )-(β×ds m )+γ(c1-c2), where o m is the number of passengers boarding a vehicle if it is dispatched at the mth minute, e m is the number of passengers allowed to travel by the vehicle, ω can take several values. If the value is large, the model tends to dispatch more vehicles, and if the value is small, the model tends to dispatch fewer vehicles. The value can be 1 / 1000; β is a constant, which is 0.2. ds m Indicates how many passengers will be unable to board the bus at the mth minute because the bus is full; γ is a constant, γ = 1e-3 × 200. When the simulation time is less than two hours from the last bus, γ = 5e-3 × 200. If the current time is a departure reward, up =(o m / e m )-(β×ds m )-γ(c1-c2), for the downlink, if no reward is given at the current time down =1-(o m / e m )-(ω×W m )-(β×ds m )-γ(c1-c2); if the current time is the departure reward down =(o m / e m )-(β×ds m )+γ(c1-c2); reward departure_std is the standard deviation of the departure interval;
[0047] Step 4-4: DQN model training that can generate a dynamic balanced timetable for up and down buses. During the training process, a complete simulation is performed for each day. First, a bus is forced to be dispatched at the first bus time and the environment state is updated. After that, for every minute, the DQN intelligent body will make a decision and get an action. This action has P m The probability is the result generated by the model, 1-P m The probability of random value is the result of random value, set P m =0.9. If the time from the current time to the last departure is less than the shortest departure interval t min If the time between the current time and the last time of departure is greater than the longest departure interval t max Then the departure is mandatory;
[0048] Step 4-5: A DQN model simulation of a dynamic balanced bus up and down timetable can be generated. After training, this action is applied to the simulation environment to obtain the next state vector and the reward value obtained by making the current action, and the current state, current reward, current action and next state are stored in the experience replay. When the experience replay is full, the DQN model will be trained once after every 5 simulations, and the target network in the DQN algorithm will be updated once after every 100 trainings, and the obtained model will be saved.
[0049] Step 5: DQN model reasoning can generate a dynamic balanced schedule for up and down buses. Based on the trained model, the traffic conditions and passenger flow data of the current line are used for reasoning to obtain a dynamic balanced schedule for up and down buses. When the passenger flow changes suddenly, a new departure schedule can be generated based on the reasoning of the changed passenger flow. When the bus travel time changes suddenly, a new departure schedule can be generated based on the reasoning of the changed travel time. When the bus stop is temporarily adjusted and changed, a new departure schedule can be generated based on the reasoning of the changed stop information.
[0050] Step 6: Generate the result output of the dynamic balanced timetable for buses in the up and down directions, output each departure time point planned for the up and down directions of the line respectively, and include the interval between the current departure time and the previous departure time, the passenger capacity provided by this departure plan every 30 minutes, the total number of departures, the number of passengers who cannot get on the bus due to full loads, and the average waiting time of passengers.
[0051] In summary, the embodiment of the present invention provides a method for generating a dynamic balanced timetable for up and down buses based on DQN. Based on the traffic conditions of bus operation and bus passenger flow information, a DQN-based two-way bus timetable is designed for simultaneous generation of an optimization model, including designing a state matrix, an action state, and a reward function of a DQN model that can generate a dynamic balanced timetable for up and down buses. Training is performed by directly interacting with a simulation environment, and decisions are made in a real-time simulation environment to generate a dynamic balanced timetable for up and down buses in real time. When the passenger flow changes suddenly due to an unexpected event, a new departure timetable can be generated based on the reasoning of the changed passenger flow. When the bus travel time changes suddenly, a new departure timetable can be generated based on the reasoning of the changed travel time. When the bus stop is temporarily adjusted and changed, a new departure timetable can be generated based on the reasoning of the changed stop information. This solves the problem that the departure timetable generated in the prior art cannot dynamically adapt to changes in passenger flow and traffic conditions, and cannot simultaneously dispatch two directions of a bus line, resulting in scheduling that does not meet actual requirements and cannot be applied.
[0052] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for generating a dynamic balanced timetable for up and down buses based on DQN, characterized in that: The following steps are involved: S1: Initialize bus route information, including setting route ID, shortest departure interval t min 、The longest departure interval t max 、Number of people allowed to travel by vehicle C max 、First shift time Start time 、End time ; S2: Obtain historical traffic conditions and passenger flow information for both upstream and downstream routes; S3: Initialize the reinforcement learning DQN model parameters, input dimension m input , output dimension m output , the number of hidden layers m hidden , the number of hidden layer neurons n hidden , activation function is ReLU, learning rate lr, batch size batch_size, experience playback capacity C ER ; S4: Construct a DQN model that can generate a dynamic equilibrium schedule for up and down buses; S5: DQN model reasoning can generate a dynamic balanced schedule for up and down buses. Based on the trained model, the traffic conditions and passenger flow data of the current line are used for reasoning to obtain a dynamic balanced schedule for up and down buses. When the passenger flow changes suddenly, a new departure schedule can be generated based on the reasoning of the changed passenger flow. When the bus travel time changes suddenly, a new departure schedule can be generated based on the reasoning of the changed travel time. When the bus stop is temporarily adjusted and changed, a new departure schedule can be generated based on the reasoning of the changed stop information. S6: Output of the dynamic balanced timetable for buses in the up and down directions, outputting each departure time point planned for the up and down directions of the line respectively, and including the interval between the current departure time and the previous departure time, the passenger capacity provided by this departure plan at each specific interval, the total number of departures, the number of passengers who could not get on the bus due to full load, and the average waiting time of passengers.
2. A method for generating a dynamic balanced timetable for up and down buses based on DQN as claimed in claim 1, characterized in that: The specific method in S2 is: S201: Calculate the travel time between two adjacent stops for each bus based on the data of bus entry and exit; S202: Calculate the travel time between two adjacent stops every 15 minutes in a day based on the travel time between two adjacent stops of each bus, as historical traffic condition data; S203: The data of the passenger's getting-off station is derived based on the bus card swiping data, and then combined with the bus entry and exit data, the passenger's getting-on and getting-off station information and time information are calculated as the passenger flow information.
3. A method for generating a dynamic balanced timetable for up and down buses based on DQN as claimed in claim 1, characterized in that: The specific method in S4 is: S401: Establishing state variables of a DQN model that can generate a dynamic equilibrium timetable for up and down buses; S402: Establishing the action state of a DQN model that can generate a dynamic equilibrium timetable for up and down buses; S403: Establishing a reward function of a DQN model that can generate a dynamic equilibrium timetable for up and down buses; S404: Train the DQN model that can generate a dynamic balanced timetable for up and down buses. During the training process, a complete simulation is performed for each day. First, a bus is forced to be dispatched at the first bus time and the environment state is updated. After that, for every minute, the DQN intelligent body makes a decision and obtains an action. This action has p m The probability is the result generated by the model, 1-p m The probability is the result of random value; if the time from the current moment to the last departure is less than the shortest departure interval t min If the time between the current time and the last time of departure is greater than the longest departure interval t max Then the departure is mandatory; S405: Simulate the DQN model that can generate a dynamic balanced timetable for up and down buses. After training in S404, apply this action to the simulation environment to obtain the next state vector and the reward value obtained by making the current action, and store the current state, current reward, current action and next state in the experience replay; when the experience replay is full, the DQN model will be trained once after every 5 simulations, and the target network in the DQN algorithm will be updated once after every 100 training times, and the obtained model will be saved.
4. A method for generating a dynamic balanced timetable for up and down buses based on DQN as claimed in claim 3, characterized in that: The state variables in S401 will take into account the time state, the uplink and downlink vehicle passenger state, the uplink and downlink passenger flow state, and the uplink and downlink departure state; define Among them, m is the different time of the day, X mu and X md Respectively represent the up state and down state of the bus, define X mu =[x u_h ,x u_min ,x u_pn ,x u_wmin ,x u_tpn ,x u_vn ],X md =[x d_h ,x d_min ,x d_pn ,x d_wmin ,x d_tpn ,x d_vn ],x u_h and x d_h Respectively represent the hours of the uplink and downlink at time m, x u_min and x d_min Respectively represent the minutes of the uplink and downlink at time m, x u_pn and x d_pn They represent the number of people in the uplink and downlink vehicles at time m, respectively, u_wmin and x d_wmin They represent the waiting time of passengers in the uplink and downlink at time m, respectively. u_tpn and x d_tpn They represent the passenger flow of the uplink and downlink vehicles at time m, respectively, u_vn and x d_vn They respectively represent the number of departures on the uplink and downlink lines at time m.
5. A method for generating a dynamic balanced timetable for up and down buses based on DQN as claimed in claim 4, characterized in that: The action vector of the action state in S402 defines four action elements: no train is sent in either the uplink or downlink direction, only the train is sent in the uplink direction, only the train is sent in the downlink direction, and both the uplink and downlink directions are sent. A=[0, 1, 2, 3] is defined. If the agent selects action 0, it means that no train is sent in either the uplink or downlink direction at the current moment; if action 1 is selected, it means that only the uplink direction is sent at the current moment; if action 2 is selected, it means that only the downlink direction is sent at the current moment; if action 3 is selected, it means that both the uplink and downlink directions are sent at the current moment.
6. A method for generating a dynamic balanced timetable for up and down buses based on DQN as claimed in claim 5, characterized in that: The reward function in S403 includes three parts: reward = reward up +reward down -reward departure_std ; For the uplink, if no departure reward is given at the current time up =1-(o m / e m )-(ω×W m )-(β×ds m )+γ(c1-c2), where o m is the number of passengers boarding a vehicle if it is dispatched at the mth minute, e m is the number of passengers allowed to travel by the vehicle, ω can take several values. If the value is large, the model tends to dispatch more vehicles, and if the value is small, the model tends to dispatch fewer vehicles. The value can be 1 / 1000; β is a constant, which is 0.
2. ds m Indicates how many passengers will be unable to board the bus at the mth minute because the bus is full; γ is a constant, γ = 1e-3 × 200. When the simulation time is less than two hours from the last bus, γ = 5e-3 × 200. If the current time is a departure reward, up =(o m / e m )-(β×ds m )-γ(c1-c2), for the downlink, if no reward is given at the current time down =1-(o m / e m )-(ω×W m )-(β×ds m )-γ(c1-c2); if the current time is the departure reward down =(o m / e m )-(β×ds m )+γ(c1-c2); reward departure_std is the standard deviation of the departure interval.
Citation Information
Patent Citations
Train dynamic adjustment method and system in an emergency
CN109740839A
Bus departure timetable dynamic optimization algorithm based on deep reinforcement learning
CN114240002A
Train timetable determination method and device, equipment and medium
CN115782988A
Bus intelligent scheduling method based on multi-agent reinforcement learning
CN116307448A
Cited By
Urban driving strategy generation method and equipment based on AI large model
CN120430920A
Bus dynamic scheduling method and system based on vehicle management
CN120673615A
