Flexible dynamic bus dispatching method considering mixed requests and fairness

Abstract

The application discloses a kind of flexible public transport dynamic scheduling method considering mixed request and fairness, including the following steps: based on the Markov decision process model of rolling time domain framework, the flexible public transport dynamic scheduling problem considering mixed request and fairness is explained by Markov decision process model;Markov decision process model designs temporary request rejection mechanism, prediction error correction mechanism and overage compensation mechanism, solve the problem in flexible public transport dynamic scheduling;Determine the cost of flexible public transport dynamic scheduling, establish the objective function of Markov decision process model, set the constraint condition of Markov decision process model;Based on multi-agent reinforcement learning definition core element, design new multi-agent reinforcement learning framework, solve flexible public transport dynamic scheduling problem.The application has significant advantages in reducing operating costs, improving passenger fairness and improving service quality, has good solving performance, and has strong practical application prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of flexible bus dynamic scheduling, in particular, to a flexible bus dynamic scheduling method considering mixed requests and fairness. BACKGROUND

[0002] Public transportation plays an important role in urban travel and development, which usually needs to make a trade-off between service level and economic feasibility. Traditional fixed-route buses are cost-effective in high-demand areas, but it is difficult to support operation in low-density demand areas. Taxis and online car-hailing services can provide convenient and fast door-to-door services, but their operating costs and environmental costs are high. Therefore, flexible bus service (FBS) emerges as the times require, which provides more flexible and accurate travel services at a relatively low price, and has great development potential in future public transportation systems.

[0003] FBS usually operates in a reservation mode, which requires passengers to specify a time window in advance. However, passengers today have higher expectations for travel and hope to make real-time requests and get on the bus as soon as possible, similar to the temporary requests met by taxis and online car-hailing services. Therefore, it is crucial to improve the acceptance of FBS by passengers. The mixed request mode, which allows both reservation requests and temporary requests, is an effective measure to improve passenger acceptance and improve FBS carpooling services. Unlike traditional reservation requests, temporary requests do not require a specified time window, similar to the temporary services provided by taxis.

[0004] Current research on flexible bus scheduling has some defects and deficiencies. Existing research mainly focuses on FBS optimization problems with reservation requests, ignoring the potential mixed request mode. For FBS with mixed requests, a basic problem is how to balance economic feasibility and service level; secondly, this problem usually needs to handle travel requests over time and also consider the impact of future requests on current scheduling decisions. Therefore, it is necessary to design an efficient method that can maximize long-term revenue and make dynamic decisions.

[0005] Traditional dynamic FBS optimization usually relies on repeated single-period static models and construction insertion heuristic methods, which fail to predict future demand using historical demand data and schedule vehicles in advance, which is not conducive to improving service quality and efficiency. Traditional reinforcement learning (RL) and imitative learning (IL) have certain limitations in solving FBS problems with mixed requests. For reinforcement learning, its training is time-consuming and the solution quality is unstable, and it does not use historical demand data to predict future demand. For imitative learning, its performance will decrease significantly when deployed to unknown environments, and its performance is not consistent under different conditions. SUMMARY

[0006] The present application aims to overcome the shortcomings and deficiencies of the prior art, and provides a flexible public transport dynamic scheduling method considering mixed requests and fairness, which integrates multi-agent reinforcement learning, imitation learning and heuristic methods through a double-target optimization model considering cost and fairness, and considers the trade-off between operating cost and service level in flexible public transport dynamic scheduling with mixed requests, and gives consideration to the efficient solution of multi-agent reinforcement learning.

[0007] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0008] A flexible public transport dynamic scheduling method considering mixed requests and fairness, comprising the following steps:

[0009] S1. A Markov decision process model is established based on a rolling horizon framework, and the flexible public transport dynamic scheduling problem considering mixed requests and fairness is explained through the Markov decision process model;

[0010] S2. The Markov decision process model designs a temporary request rejection mechanism, a prediction error correction mechanism and an overtime compensation mechanism to solve problems in flexible public transport dynamic scheduling;

[0011] S3. The cost of flexible public transport dynamic scheduling is determined, the objective function of the Markov decision process model is established, and the constraint conditions of the Markov decision process model are set;

[0012] S4. Based on multi-agent reinforcement learning, core elements are defined, a new multi-agent reinforcement learning framework is designed, and the flexible public transport dynamic scheduling problem is solved.

[0013] Further, in step S1, the Markov decision process model is established based on the rolling horizon framework, specifically:

[0014] The operating period is divided into |G| periods of length H, and each period g corresponds to a request set N g , including a reservation request set And a temporary request set In the case of demand prediction, temporary requests are expected to be submitted within period g, and in the case of no demand prediction, temporary requests are submitted before period g, but temporary requests are included in the temporary request set In order to establish the Markov decision process model, each period g∈G is divided into |T| time slices, and decisions are made only at the beginning of each time slice, and the maximum number of decisions for each vehicle in a period is |T|, and the buffer time of the optimization execution phase is set as BT, and the next period is planned during the buffer time BT.

[0015] Further, in step S2, a temporary request rejection mechanism is designed, specifically:

[0016] When the temporary request cannot be allocated to any vehicle in the current cycle, it is considered to be included in the next cycle for planning. By comparing the additional cost of serving the request in the next cycle with the penalty cost of rejecting the request, it is evaluated whether it is beneficial to accept the request. If the additional cost of serving the request is lower than the penalty cost of rejecting the request, the request is included in the next cycle planning, otherwise the request is rejected.

[0017] A prediction error correction mechanism is designed, specifically:

[0018] The prediction error of request n includes three types of passenger quantity error, submission time error and station error. The condition for triggering correction is whether the constraint is violated or not. The constraint violation is derived from the existing scheme planning route and the updated information of request n. Once the prediction error does not meet the constraint condition, corresponding correction is needed to ensure the effectiveness of the planning scheme. Assume that the predicted passenger number of request n is The actual passenger number is q n , and vehicle k is planned to be allocated; when Not only when request n is picked up, but also in the subsequent route of the planning scheme, it is necessary to determine whether vehicle k still meets the capacity constraint;

[0019] If vehicle k still meets the capacity constraint in the updated route, the existing scheme can be executed, otherwise vehicle k cannot serve request n as expected, i.e. the existing scheme fails, and corresponding correction is needed to ensure effective scheduling. The correction starts from searching whether there is a running vehicle that can serve request n. If there is no vehicle that can serve request n, it is merged into the request set of the next cycle.

[0020] A overtime compensation mechanism is designed, specifically:

[0021] For vehicle k∈K, initialize a variable AT g,t,k in each cycle, which represents the timestamp of vehicle k completing the current decision at time slice t in cycle g. Therefore, when cycle g ends, AT g,|T|,k is the duration of its route; if AT g,|T|,k >g·H, the duration of vehicle k's route in cycle g is overtime; in this case, the usage time information of vehicle k in the next cycle g+1 needs to be updated, i.e. AT g,|T|,k is added to the initialization of the next cycle.

[0022] Further, in step S3, the cost of flexible bus dynamic scheduling is determined, specifically:

[0023] The cost includes transportation cost and passenger cost, the transportation cost is the product of driving mileage and unit mileage cost, and is expressed as

[0024]

[0025] In the formula, represents the transportation cost of period g; δ f represents the unit mileage cost; i g,t,k represents the station of vehicle k in period g at time slice t; i g,t+1,k represents the station of vehicle k in period g at time slice t+1; DIS(i g,t,k ,i g,t+1,k represents the distance from station i g,t,k to station i g,t+1,k ;

[0026] The passenger cost includes time window penalty cost and rejection penalty cost, and the calculation formula is

[0027]

[0028] In the formula, represents the passenger cost of period g; is a 0-1 variable, represents that temporary request n is successfully responded, represents that temporary request n is rejected; δ e represents the unit early arrival penalty cost; q n represents the number of boarding passengers of request n; represents the earliest departure time of reservation request n or the submission time of temporary request; represents the pickup time of request n; δ l represents the unit late arrival penalty cost; represents the latest departure time of request n; δ b represents the unit rejection response cost; N g represents the request set in period g.

[0029] Further, the objective function of the Markov decision process model is established, and specifically is:

[0030] The first objective function minimizes the total cost to minimize the total cost of each period, and is expressed as

[0031]

[0032] The second objective function optimizes the fairness of temporary request waiting time, and the calculation formula of the waiting time is

[0033]

[0034] In the formula, Wn denotes the waiting time of request n;

[0035] Introducing fairness index to measure the fairness of passenger waiting time, the second objective function considering fairness index is denoted as

[0036]

[0037] wherein, denotes the average waiting time of temporary requests in period g.

[0038] Further, the constraint condition of Markov decision process model is set, specifically:

[0039] Vehicle passenger capacity constraint: the vehicle must meet the passenger capacity constraint when accessing each station, denoted as

[0040]

[0041] wherein, z g,t,k denotes the number of passengers of vehicle k in time slice t in period g; Z denotes the maximum number of passengers of vehicle;

[0042] Running time constraint: for each vehicle k, all requests must be served within the operating time (|G|·H), and the running vehicle must return to the depot before the end of the operating time, denoted as

[0043]

[0044]

[0045] wherein, denotes the waiting time of vehicle k in time slice t in period g, denotes the total waiting time of vehicle in operating time; i |G|,|T|,k denotes the station of vehicle k at the end of operating time;

[0046] In-transit time constraint: the in-transit time of each request must not be greater than the shortest path direct time of the request multiplied by α, denoted as

[0047]

[0048] wherein, denotes the boarding station of request n; denotes the alighting station of request n; vel denotes the average speed of vehicle.

[0049] Further, in step S4, the core elements are defined based on multi-agent reinforcement learning, specifically:

[0050] The global state S is the vehicle state SK and request state S N denote, i.e. S = {S K , S N}, with S g,t denoting the global state at time slice t within period g, then and correspond to the vehicle state and request state at time slice t within period g, respectively, the vehicle state request state o g,t,k denotes the local state of agent vehicle k at time slice t within period g;

[0051] The action space A g,t,k of agent vehicle k is defined as g,t,k ∈ A g,t,k

[0052]

[0053] wherein, denotes the next movement action of vehicle k at time slice t within period g; p n denotes the pickup action of request n; d n denotes the drop-off action of request n; denotes the return-to-depot action; denotes the waiting time of vehicle k before movement action ;

[0054] The four kinds of pruning operations include pruning according to the visiting order, pruning according to the in-transit time constraint, pruning according to the running time constraint, and pruning according to the passenger capacity constraint; the state transition includes intra-period state transition and inter-period state transition; in view of the task of double-objective optimization of the Markov decision process model, two reward functions, i.e. cost reward R C and fairness reward are set, and a weight p is set to calculate the total reward function.

[0055] Further, in step S5, a new multi-agent reinforcement learning framework is designed, including a rolling value decomposition network algorithm combined with demand prediction, enhancing the forward-looking of each period planning, specifically:

[0056] Each Q function is represented by a neural network, before training in period g, for each agent vehicle k ∈ K, a local Q function Q g,k characterized by θ k is initialized; considering that the local state with time-related information is used to represent the state information, the joint state is O = (o1, o2, …, o |K| , and the joint action is A = (a1, a2, …, a |K| ​), the joint network can be obtained by each Q k Linear summation, that is, Therefore, the Q tot and the Q k The objective function of each cycle is converted to

[0057]

[0058] In the formula, Q tot denotes the joint Q function; O g,1 denotes the joint state at time slice t=1 in cycle g; o g,0,k denotes the local state of vehicle k at time slice t=0 in cycle g; θ g,k denotes the parameter of vehicle k in cycle g; o g,0 denotes the local state at time slice t=0 in cycle g; θ g denotes the parameter in cycle g; π g denotes the policy function in cycle g; π g,k denotes the policy function of vehicle k in cycle g; Q k denotes the local Q function of vehicle k;

[0059] Since the rolling horizon framework and demand prediction are introduced, the cycle transition and demand prediction are included in the optimization of each cycle, and the objective function of each cycle is further converted to:

[0060]

[0061] In the formula, θ g,1,k denotes the local state of vehicle k at time slice t=1 in cycle g; denotes the predicted request set of cycle g; o g,1 denotes the local state at time slice t=1 in cycle g; o g-1,|T|,k denotes the local state of vehicle k at time slice t=|T| in cycle g-1.

[0062] Further, in step S5, a new multi-agent reinforcement learning framework is designed, including improving the ∈-greedy action selection strategy through time-related information, and improving the search efficiency in the reinforcement learning training process, specifically:

[0063] In order to combine random search and search based on time-related information, an ∈-greedy greedy algorithm is proposed, two initial search rates ∈1 and ∈2 are set at the beginning of training, wherein ∈1+∈2≤1, and the action selection strategy is

[0064]

[0065] In the formula, greedy action that represents the maximum expected reward at the current time; ∈1(τ) represents the search rate of ∈1 at step τ, and ∈2(τ) represents the search rate of ∈2 at step τ;

[0066] The corresponding search rate calculation formulas of ∈1(τ) and ∈2(τ) under the decay mechanism are

[0067]

[0068] In the formula, is the maximum decay step of ∈1, is the maximum decay step of ∈2.

[0069] Further, in step S5, a new multi-agent reinforcement learning framework is designed, including combining multi-agent reinforcement learning with local search strategy based on variable neighborhood descent and imitation learning, and designing a loss function, which is specifically:

[0070] In order to improve the calculation efficiency, three neighborhood operators are designed, and the concept of "entropy" is combined to guide the application of neighborhood operators. The calculation formulas of the three entropy indexes are

[0071]

[0072] In the formula, ρ represents the target weight; represents the related cost of route ; represents the request set of route ; β represents the weight coefficient of the two types of rewards; represents the duration of route ; c(l) represents the related cost of loop l; N(l) represents the request set of loop l; Δ(l) represents the duration of loop l; c(n) represents the related cost of request n; Δ(n) represents the duration of request n;

[0073] Each entropy index is designed according to the defined reward function. Dividing by the duration indicates the degree of reward increase in unit operating time. First, priority list 1 is constructed in descending order of μ1(k), then priority list 2 is constructed in descending order of μ2(l), and finally priority list 3 is constructed in descending order of μ3(n);

[0074] The local search strategy includes intra-loop movement, inter-loop movement, and insert-reject-request movement. A local search strategy based on variable neighborhood descent algorithm is proposed. Given an initial planning scheme of a period, i.e., the initial solution x0, and the maximum number of iterations, the local search strategy based on variable neighborhood descent algorithm is as follows:

[0075]

[0076]

[0077] The total loss function is equal to the sum of the mean squared error loss function of the TD error and the imitation learning loss, denoted as

[0078] L = L TD + L IL

[0079] In the formula, L TD represents the mean squared error of the TD error; L IL represents the imitation learning loss.

[0080] Compared with the prior art, the present application is based on a rolling horizon framework, establishes a multi-objective Markov decision process model, designs a temporary request rejection mechanism, a prediction error correction mechanism and an overdue compensation mechanism to solve possible problems in dynamic scheduling, establishes a double-objective flexible public transport dynamic scheduling objective function, the first objective is to minimize the total cost, and the second objective is to optimize the fairness of the waiting time of temporary requests, defines the core elements of state, action and reward based on multi-agent reinforcement learning, designs a new multi-agent reinforcement learning framework, and solves the flexible public transport dynamic scheduling problem. The method of the present application can reduce operating costs, improve passenger fairness and improve service quality. BRIEF DESCRIPTION OF DRAWINGS

[0081] Figure 1 It is a flow chart of the flexible public transport dynamic scheduling method considering mixed requests and fairness.

[0082] Figure 2 In the figure, (a) is a schematic diagram of a rolling horizon framework, (b) is a schematic diagram of a temporary request rejection mechanism, and (c) is a schematic diagram of a prediction error correction mechanism.

[0083] Figure 3 It is a schematic diagram of an overdue compensation mechanism.

[0084] Figure 4 It is a schematic diagram of a new multi-agent reinforcement learning framework.

[0085] Figure 5 It is a schematic diagram of three neighborhood movement operators.

[0086] Figure 6 It is a Sioux Falls road network map in the embodiment.

[0087] Figure 7 It is a result diagram of the optimal objective value solved by five algorithms in the embodiment.

[0088] Figure 8 It is a CTDE and decentralized training evolution process in the embodiment. DETAILED DESCRIPTION

[0089] The flexible dynamic bus scheduling method of the present invention, which considers mixed requests and fairness, will be further described below with reference to the accompanying drawings and specific embodiments.

[0090] Please see Figure 1 This invention discloses a flexible dynamic bus scheduling method that considers mixed requests and fairness, comprising the following steps:

[0091] S1. A Markov decision process model is established based on the rolling time domain framework. The Markov decision process model is used to explain the flexible dynamic scheduling problem of public transport considering mixed requests and fairness.

[0092] S2. The Markov decision process model is designed with a temporary request rejection mechanism, a prediction error correction mechanism, and an overdue compensation mechanism to solve the problems that arise in flexible dynamic scheduling of public transportation.

[0093] S3. Determine the cost of flexible public transport dynamic scheduling, establish the objective function of the Markov decision process model, and set the constraints of the Markov decision process model;

[0094] S4. Based on the core elements defined by multi-agent reinforcement learning, a novel multi-agent reinforcement learning framework is designed to solve the problem of flexible bus dynamic scheduling.

[0095] To facilitate the description of the mathematical model later, Table 1 lists the symbolic variables involved in the mathematical model of this invention.

[0096] Table 1 Symbolic Variables

[0097]

[0098]

[0099] In step S1, such as Figure 2 As shown in (a), the rolling time-domain method is used to transform the problem into a series of interdependent subproblems. The operating period is divided into |G| cycles of length H, and the request set corresponding to each cycle g is N. g Including the set of appointment requests and temporary request sets Temporary requests when demand forecasts are available Requests are expected to be submitted within cycle g; however, in the absence of demand forecasts, ad hoc requests may be submitted before cycle g, but incorporated into the planning phase of cycle g. To establish a Markov decision process model, each cycle g∈G is divided into |T| time slices, and decisions are made only at the beginning of each time slice. Therefore, the maximum number of decisions per vehicle in a cycle is |T|. Furthermore, a buffer time of BT is set for the optimization execution phase, during which planning for the next cycle is performed.

[0100] Step S2, as shown in (b) of Figure 2 , a temporary request rejection mechanism is designed. When a temporary request cannot be allocated to any vehicle in the current period, it is considered to be included in the planning of the next period. If the additional cost of the service request is less than the penalty cost of rejecting the request, the request is included in the planning of the next period, otherwise the request is rejected. If the request is rejected, the request is marked as rejected at the beginning of the next period, at which time

[0101] As shown in (c) of Figure 2 , a prediction error correction mechanism is designed. Generally speaking, there are three types of prediction errors for request n, namely passenger quantity error, submission time error and station error. The condition for triggering correction is whether the constraint is violated, assuming that the predicted passenger number of request n is the actual passenger number is q n , and is planned to be allocated to vehicle k. When not only at the time of picking up request n, but also in the subsequent route of the planning scheme, it is necessary to determine whether vehicle k still satisfies the capacity constraint. If vehicle k still satisfies the capacity constraint in the updated route, the existing scheme can be executed, otherwise vehicle k cannot serve request n as expected, i.e. the existing scheme fails, and corresponding correction needs to be made to ensure effective scheduling. The correction starts from searching whether there is a running vehicle that can serve request n, if there is no vehicle that can serve request n, it is merged into the request set of the next period.

[0102] As shown in Figure 3 , an overtime compensation mechanism is designed. For vehicle k∈K, initialize a variable AT g,t,k at each period, which represents the timestamp of vehicle k completing the current decision at time slice t in period g, so when period g ends, AT g,|T|,k is the duration of its route; if AT g,|T|,k >g·H, the duration of the route of vehicle k in period g is overtime; in this case, the usage time information of vehicle k in the next period g+1 needs to be updated, i.e. AT g,|T|,k is added to the initialization of the next period. The route of vehicle k in period 1 ends before H (AT 1,|T|,k <H), so there is no overtime in period 1. However, since the route duration in period 2 exceeds 2H (AT 2,|T|,k >2H), overtime AT 2,|T|,k -2H needs to be added at the beginning of period 3.

[0103] In step S3, the cost of flexible bus dynamic scheduling is determined, including transportation cost and passenger cost. The transportation cost is the driving mileage and the unit mileage cost δ fThe product of the number of passengers and the cost of each passenger is denoted as

[0104]

[0105] The passenger cost includes the time window penalty cost and the rejection response cost, denoted as

[0106]

[0107] The objective function of the Markov decision process model is established, and the first objective function minimizes the total cost to minimize the total cost of each period, denoted as

[0108]

[0109] The second objective function optimizes the temporary request waiting time fairness, and the waiting time W n The calculation formula is

[0110]

[0111] The fairness index is introduced to measure the fairness of the passenger waiting time, and the second objective function considering the fairness index is denoted as

[0112]

[0113] The constraint conditions of the Markov decision process model are set, including the vehicle passenger capacity constraint, the running time constraint and the in-transit time constraint.

[0114] The vehicle passenger capacity constraint: the vehicle must meet the passenger capacity constraint when visiting each station.

[0115]

[0116] The running time constraint: for each vehicle k, all requests must be served within the operating time (|G|·H), and the vehicle must return to the depot before the operating time ends.

[0117]

[0118] The in-transit time constraint: the in-transit time of each request must not be greater than the shortest path direct time of the request multiplied by α.

[0119]

[0120] In step S4, the global state S can be represented by the vehicle state S K and the request state S N , that is, S={S K ,S N}. For a certain intelligent agent vehicle k, the corresponding vehicle state is defined as

[0121]

[0122] Request status is defined as

[0123]

[0124] Use o g,t,k to represent the local state of agent vehicle k at time slice t in period g, o g,t,k has a dimension of 2|N g |+3.

[0125]

[0126] Action space A g,t,k of agent vehicle k is defined as g,t,k ∈A g,t,k

[0127]

[0128] The waiting time is only considered when vehicle k decides to pick up the request, and the optimal waiting time calculation formula is

[0129]

[0130] The dimension of the entire action space is determined by the request set N g of each period, i.e., 2|N g |+|J|+1.

[0131] After pruning the action space, the modified action space of vehicle k containing only valid actions can be obtained , which is called the "valid action space". The four pruning operations include pruning according to the access order, pruning according to the in-transit time constraint, pruning according to the running time constraint, and pruning according to the passenger capacity constraint.

[0132] Pruning according to the access order: define as the in-transit request set of vehicle k at time slice t in period g, therefore, for vehicle k, if request then the arrival request n (i.e., action ) is considered as the valid action of vehicle k at this moment; if request action will be removed from .

[0133] Pruning according to the in-transit time constraint: if i.e., vehicle k has no in-transit request, then there is no need to prune according to the in-transit time constraint; if ​For any unassigned or unfulfilled request, whether vehicle k can provide pickup / drop-off service depends on the current location i of vehicle k g,t,k , the set of en-route requests , and the pick-up time of the relevant request in .

[0134] There are two scenarios to determine whether vehicle k can access request n. Scenario 1: when request n has not been assigned to any vehicle (i.e. ), determine whether is a valid action for vehicle k. Scenario 2: when request n has been assigned to vehicle k but not yet fulfilled (i.e. ), determine whether is a valid action for vehicle k.

[0135] Pruning according to running time constraints: Scenario 1: determine whether vehicle k can pick up request n; Scenario 2: determine whether vehicle k can drop off request n; Scenario 3: determine whether vehicle k can return to the depot.

[0136] Pruning according to passenger capacity constraints: whether vehicle k can pick up unassigned request n depends on the current passenger capacity z g,t,k of vehicle k and the number of reserved passengers q n of request n. If (17) is satisfied, then is a valid action that complies with the passenger capacity constraint, otherwise it is removed from .

[0137] z g,t,k +q n ≤Z (17)

[0138] State transitions include intra-period state transitions and inter-period state transitions.

[0139] Intra-period state transition: given a g,t,k , the vehicle state of vehicle k, can update to by (18)-(20). Then, update according to and a g,t,k , i.e., update v g,t＋1,n = k, if update Moreover, can also update the local state o with time-dependent information according to and g,t,k → p g,t+1,k .

[0140]

[0141] State transition during the cycle: When g→g+1, if there are no overdue requests or routes, it can be easily passed through... and N g+1 Initialize the state variables; otherwise, it is necessary to determine: (1) N g Are there any unassigned requests? (2) Can each vehicle k∈K complete the route before the end of period g? To update AT k In case (1), for each unassigned request n, if It will then be rejected and recorded. and if Then it will be inherited into the request set of the next cycle, i.e., N. g+1 ←N g+1 ∩{n}, and through the updated N g+1 renew In case (2), the update can be performed using equations (21)-(23). If AT g,|T|,k >(g+1)·H, representing the route If it expires, AT will be sent. g,|T|,k Inherited to the next cycle; otherwise, AT will be... g+1,1,k It is initialized to (g+1)·H, which is the beginning of period g+1.

[0142] i g+1,1,k =i g,|T|,k (twenty one)

[0143] z g+1,1,k =z g,|T|,k (twenty two)

[0144] AT g+1,1,k =max{AT g,|T|,k ,(g+1)·H} (23)

[0145] Define reward function

[0146] From S g,t To S g,t+1 Reward R(S) after state transition g,t+1 |S g,t ) represents

[0147]

[0148] The calculation of mileage cost during state transition occurs at each time the vehicle position changes, as expressed by equation (25).

[0149]

[0150] When t < |T|, the passenger cost in time slice t within period g is represented by equation (26).

[0151]

[0152] When t = |T|, the passenger cost incurred due to the rejection of temporary requests is represented by equation (27).

[0153]

[0154] Cost bonus within time slice t in period g The calculation formula is

[0155]

[0156] Fair rewards Primarily through fairness indicators The recursive calculation is implemented across time slices. This metric is determined by the waiting time of temporary requests n in each time slice t within period g, and the calculation formula is as follows:

[0157]

[0158] Average waiting time for temporary requests And fairness indicators within time slice t within period g. The calculation formula is

[0159]

[0160] The formula for calculating fair rewards within time slice t in period g is as follows:

[0161]

[0162] The Bellman equation is key to optimizing a Multiplication Table (MDP), including the definitions of the policy function π and the state-action value function Q (called the Q function). The policy function π represents which action should be chosen given a certain state, i.e. Given S g,t and At that time, the value of the Q function is based on the strategy. Execute A g,t The cumulative rewards obtained later are:

[0163]

[0164]

[0165] Since the model needs to optimize the planning scheme for the entire period g∈G, it is necessary to find an optimal strategy. Maximize the Q-function from t=1 to t=|T|, that is:

[0166]

[0167] In step S4, such as Figure 4 As shown, a novel multi-agent reinforcement learning framework is designed to solve the problem of flexible bus dynamic scheduling.

[0168] The rolling value-decomposition networks (VDN) algorithm, which incorporates demand forecasting, enhances the foresight of planning in each cycle. The VDN algorithm is based on a centralized training and distributed execution approach. A joint Q-function Q is used. tot To represent team rewards, but allow each agent vehicle k∈K to use a local Q function Q. k To make a decision, Q k Q can be obtained through linear summation tot This is decomposed from the above. In this invention, each Q-function is represented by a neural network. Before training period g, for each agent vehicle k∈K, a network is initialized using θ. g,k The local Q-function Q is characterized k Let the joint state be O = (o1, o2, ..., o2). |K| The combined action is A = (a1, a2, ..., a...). |K| ),but Therefore, Q can be obtained for training. tot and Q for each intelligent agent vehicle decision k The objective function for each cycle can be transformed into

[0169]

[0170] By incorporating cycle shifts and demand forecasting into the optimization of each cycle, equation (35) can be further transformed into

[0171]

[0172] Regarding the experience replay mechanism, a transition tuple is recorded for each interaction between the agent and the environment. As an experience transfer, it is stored in a fixed-size buffer called the "RL buffer".

[0173] Improving the ∈-greedy action selection strategy with time-related information enhances search efficiency during reinforcement learning (RL) training. Given o g,t,k This allows us to obtain time-related information for each request from vehicle k, i.e. Γ(o g,t,k ) indicates based on o g,t,k The time-related information determines the action function, defined by equations (37)-(38).

[0174]

[0175]

[0176] This invention proposes an ∈-greedy greedy algorithm, which sets two initial search rates ∈1 and ∈2 at the start of training, where ∈1 + ∈2 ≤ 1. Let τ represent the current training step number, then ∈1(τ) and ∈2(τ) represent the search rates ∈1 and ∈2 at step number τ, respectively. Therefore, the action selection strategy is as follows:

[0177]

[0178] The formulas for calculating the search rates of ∈1(τ) and ∈2(τ) under the decay mechanism are as follows:

[0179]

[0180] Multi-Agent Reinforcement Learning (MARL) is combined with a local search strategy based on Variable Neighborhood Descent (VND) and imitation learning to improve solution quality and training performance, and a corresponding loss function is designed.

[0181] The completed route is divided into several relatively short routes to perform neighborhood movement operations, which is called a "service loop".

[0182] To quantify the potential of each route, loop, and request to improve solution quality, the concept of "entropy" is introduced, denoted by μ1, μ2, and μ3, respectively. The greater the entropy, the greater the potential to improve solution quality. The three entropy indices are calculated as follows.

[0183]

[0184] First, construct priority list 1 in descending order of μ1(k) (k∈K), then construct priority list 2 in descending order of μ2(l), and finally construct priority list 3 in descending order of μ3(n).

[0185] like Figure 5 As shown, three neighborhood movement operators are designed for the local search strategy: in-loop movement, inter-loop movement, and insertion-rejection-request movement.

[0186] Movement within the loop

[0187] Step 1: Select a loop. (Through...) Select a route from priority list 1. Then through Select a route from priority list 2 One of the loops l.

[0188] Step 2: Check the requested moves in the selected loop. This is done using n = argmax. n′∈l μ3(n′) selects a request n from priority list 3 and enumerates all feasible moves for n. For For each feasible move, calculate the route reward. Remove n from priority list 3 and repeat the above steps to process the remaining requests in l.

[0189] Step 3: Confirm The optimal move is determined by selecting the move with the highest reward from all feasible moves in Step 2. Move 1 is removed from priority list 2, and the above steps are repeated until priority list 2 is empty.

[0190] Movement between loops

[0191] A loop index μ4(l,n) is introduced to indicate which loop has the priority to serve a new request n. Given a request n, a new priority list 4 can be constructed by calculating μ4(l,n) for each loop and sorting them.

[0192]

[0193] Step 1: Select a request. (Using n = argmax) n′ μ3(n′) selects a request n from priority list 3.

[0194] Step 2: Select a loop and insert the selected request. (By...) Select a loop l from priority list 4 (excluding loops that previously contained requests for n), and insert n at the end of l.

[0195] Step 3: Check all feasible moves. Move to the end position of l, then execute Step 2 of "moving within the loop," but only for request n. If a feasible move is available, calculate the reward; otherwise, remove l from priority list 4. Repeat the above steps until priority list 4 is empty.

[0196] Step 4: Confirm Find the optimal move. In Step 3, select the move with the largest increase in reward from all feasible moves; otherwise, stop neighborhood moves and output the original solution. Remove n from priority list 3 and repeat the above steps until priority list 3 is empty.

[0197] Insert-Reject-Request Move

[0198] Step 1: Select a request to be rejected. That is, n = argmax n′ q n′ ·δ b .

[0199] Step 2: Select a loop and insert the rejected request. (By...) Select a loop l from priority list 4 and insert n at the end of l.

[0200] Step 3: Check all feasible moves and determine the optimal move plan.

[0201] This invention proposes a local search strategy based on a variable neighborhood descent algorithm, specifically as follows:

[0202]

[0203] The formula for calculating the target value R(x) is as follows:

[0204] R(x)=ρ·(C f +C u )+(1-ρ)·ζ·β (46)

[0205] Similar to the experience replay mechanism of RL, a local search transition is recorded. It is then stored in a fixed-size local search buffer.

[0206] Extract ω from the RL buffer RL Based on batch experience, the mean square error loss function of TD error is calculated using equations (47)-(49), and is expressed as follows:

[0207]

[0208]

[0209] Extract ω from the local search buffer LS The batch experience is used to calculate the imitation learning loss L using equations (50)-(52). IL , represented as

[0210]

[0211] The formula for calculating the total loss function is as follows:

[0212] L = L TD +L IL (53)

[0213] Example

[0214] This embodiment uses the classic Sioux Falls road network. To better evaluate the optimization performance of the proposed algorithm on multi-period problems, four evaluation metrics are designed: Effective Average User Cost (EAUC), Weighted Average Fairness Index (WAFI), Average Delay Arrival Time (ALAT), and Response Rate (RR).

[0215]

[0216]

[0217] like Figure 6 As shown, the Sioux Falls network has 24 nodes, with station 0 being a parking lot and the remaining 23 nodes being service stations. A downtown area is set up in the road network. Figure 6 (Orange shaded area). For a single-cycle problem, consider a medium-sized static problem where 30 requests need to be distributed to 8 buses within 1 hour; for a multi-cycle problem, consider a dynamic scenario where 86 requests need to be distributed to 8 buses within 3 hours.

[0218] Assuming each period's time slice (|T|) is 30, the average vehicle speed (vel) is 30 km / h, and the relevant unit transportation cost and penalty cost δ... f ,δ e ,δ l ,δ b The costs are ¥1.2 / km, ¥0.25 / (min·pax), ¥0.5 / (min·pax), and ¥10 / pax, respectively, with a delay coefficient (α) of 2.5, and the weights of the cost target (ρ) and the weights of the two types of rewards (β) are 0.8 and 20, respectively.

[0219] For the hyperparameter values, a two-layer fully connected RNN network is used as the deep neural network architecture, with each hidden layer consisting of 64 neurons, and the training steps (τ) are... RL The value is 8000, and the number of VND iterations (τ) LS The initial search rates (∈1, ∈2) are 2000, the learning rate is 0.01, the discount rate (γ) is 0.95, the initial search rates (∈1, ∈2) are 0.4 and 0.6, and the corresponding decay steps ( and The values ​​are 4000 and 8000.

[0220] For experience replay, the sizes of the reinforcement learning RL buffer and the local search buffer are 20000 and 2000, respectively, ω RL and ω LS The minimum batch size is 32. The target network update interval is 100.

[0221] Discussion of the single-cycle problem

[0222] To verify the effectiveness of the algorithm in solving single-cycle problems, 10 medium-sized computational examples were randomly generated. Figure 7 As can be seen, the algorithm proposed in this invention outperforms other algorithms in all instances. Compared with "DRL (improved)" and "VND", the objective value of "DRL (improved) + VND" is significantly reduced, with average reductions of 25.71% and 43.39% respectively in all 10 examples, demonstrating the advantages of combining DRL with local search algorithms.

[0223] Furthermore, comparing the "Improved DRL" with the "Standard DRL" reveals an average reduction of 83.32% in the target value, demonstrating the effectiveness of the improved ∈-greedy greedy strategy. Moreover, the "greedy" strategy exhibits the worst performance across all instances. These results demonstrate the superiority of the proposed algorithm compared to other algorithms, showcasing its ability to effectively solve complex static problems.

[0224] Depend on Figure 8 As can be seen, the centralized training and distributed execution method (referred to as "CTDE") outperforms the distributed method (referred to as "distributed") in almost all examples (except example 1). Regarding training stability, regardless of the value of ρ, the CTDE curves for all examples continuously rise and converge to a stable level after 5000 steps; while the distributed method fails to converge in most examples (5 out of 9). This demonstrates that the algorithm proposed in this invention has strong scalability, thus ensuring reliable training performance.

[0225] Regarding convergence, considering the average cumulative reward over the last 500 training steps, when ρ is 0, 0.5, and 1, the convergence value of CTDE is 61.23%, 32.24%, and 32.1% higher than the distributed average, respectively. This indicates that the algorithm proposed in this invention has superior convergence ability when searching for improved solutions. Furthermore, the gap between the two curves widens as the value of ρ decreases, demonstrating that the algorithm of this invention has significant advantages in providing optimal solutions while considering fairness objectives.

[0226] The training evolution process of the algorithm proposed in this invention when solving Example 1 is measured by four evaluation metrics. For EAUC and WAFI, they initially fluctuated in the first 4000 steps, and then rapidly converged to 6.268 and 3.485, respectively, with reductions of 36.66% and 68.32% compared to the initial values. This indicates that the algorithm of this invention has good performance in solving the dual-objective optimization problem of cost and fairness.

[0227] At the same time, the service quality evaluation indicators (i.e., ALAT and RR) were also significantly improved, with ALAT decreasing from a maximum of about 6.059 minutes to 0.142 minutes and RR increasing from a minimum of about 58% to 96%, further confirming the superior performance of the algorithm of the present invention in indicator optimization.

[0228] Discussion of multi-period problems

[0229] Table 2 lists the results of three metrics and the average planning time for each cycle when solving multi-cycle problems with and without demand forecasting. The results show that the operational performance with demand forecasting is consistently better than without, with EAUC, WAFI, and ALAT significantly reduced by 58.42%, 73.4%, and 94.58%, respectively. This indicates that when the forecasting method achieves sufficient accuracy, demand forecasting has significant advantages in reducing costs, improving passenger fairness, and enhancing service quality.

[0230] Compared to no demand forecasting, the planning time per cycle is longer with demand forecasting (average 12.88 seconds). This is because additional computation time is required to correct forecasting errors upon each ad-hoc request submission, including neighborhood moves and feasibility checks. Furthermore, there is no significant difference in response rate (RR) with and without demand forecasting, indicating that demand forecasting can significantly improve the other three metrics while ensuring a high response rate.

[0231] Table 2 Comparison of three indicators with and without demand forecasting.

[0232]

[0233] Sensitivity analysis

[0234] The impact of different cycle lengths H on performance with and without demand forecasting was analyzed. As H increases, although the difference in response time (RR) with and without demand forecasting is small, the differences in efficiency intervals (EAUC), WAFI (Wait and Response Time), and ALAT (Alternative Alternative Targets) are significant. Without considering demand forecasting, when H increases from 5 to 10, EAUC decreases slightly by 1.56%, while when H further increases to 50, EAUC increases by 114.3%. This is because a larger H allows for more requests to be scheduled simultaneously, but at the cost of lower scheduling flexibility. Therefore, when H exceeds a critical value, the negative impact of reduced scheduling frequency outweighs the benefits of centralized planning.

[0235] Conversely, when demand forecasting is considered, EAUC decreases as H increases for two reasons: first, demand forecasting mitigates the negative impact of reduced scheduling rebalancing frequency; second, a longer look-ahead length (H) allows for the consideration of more future requests. Similarly, without demand forecasting, WAFI and ALAT increase significantly with increasing H, while the change is less pronounced when demand forecasting is considered. This is because, with lower scheduling rebalancing frequency, temporary requests cannot be satisfied in a timely manner.

[0236] The impact of the target weight ρ on the four indicators was analyzed, revealing that the differences in RR were small. As ρ increased, EAUC decreased while WAFI increased, indicating a trade-off between cost and fairness. The magnitudes of change in EAUC and WAFI differed. Specifically, when ρ increased from 0 to 1, EAUC decreased by 17%; when ρ increased from 0 to 0.5, WAFI increased slightly by 1.523%; and when ρ increased to 1, WAFI increased significantly by 42.66%.

[0237] Therefore, when using demand forecasting, setting ρ to 0.5 can effectively balance these two metrics, as it can quickly reduce EAUC without significantly increasing WAFI. Without demand forecasting, ALAT decreases significantly with increasing ρ, while the change is less pronounced with demand forecasting. This is mainly because cost optimization helps reduce time window penalty costs, while emphasizing fairness worsens this metric. However, introducing demand forecasting can mitigate this negative effect.

[0238] The impact of demand forecasting error Ψ on performance was analyzed. The performance indicators with demand forecasting (regardless of whether a correction mechanism is in place) were significantly better than without demand forecasting, at the cost of a slight decrease in response rate (RR). This validates the benefits of introducing demand forecasting. With demand forecasting, all indicators appear to deteriorate as Ψ increases, especially when Ψ increases from 0 to 0.1.

[0239] However, EAUC, ALAT, and RR with prediction error correction consistently outperform those without, and the gap widens with increasing Ψ, particularly for EAUC and ALAT. This suggests that the prediction error correction mechanism helps mitigate the negative impact of prediction errors. The differences in WAFI are not significant, indicating that prediction errors have a limited impact on fairness.

[0240] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit disclosed in the present invention should fall within the patent scope covered by the present invention.

Claims

1. A flexible dynamic bus scheduling method considering mixed requests and fairness, characterized in that, Includes the following steps: S1. A Markov decision process model is established based on the rolling time domain framework. The Markov decision process model is used to explain the flexible dynamic scheduling problem of public transport considering mixed requests and fairness. S2. The Markov decision process model is designed with a temporary request rejection mechanism, a prediction error correction mechanism, and an overdue compensation mechanism to solve the problems that arise in flexible dynamic scheduling of public transportation. S3. Determine the cost of flexible public transport dynamic scheduling, establish the objective function of the Markov decision process model, the first objective function minimizes the total cost, the second objective function optimizes the fairness of temporary request waiting time, and set the constraints of the Markov decision process model. S4. Based on the core elements defined by multi-agent reinforcement learning, a novel multi-agent reinforcement learning framework is designed to solve the problem of flexible dynamic scheduling of public transportation. In step S1, a Markov decision process model is established based on the rolling time-domain framework, specifically as follows: The operation period is divided into |G| cycles of length H, and the request set corresponding to each cycle g is N. g Including the set of appointment requests and temporary request sets Temporary requests when demand is forecasted Expected to be submitted within period g, in the absence of demand forecasting, ad hoc requests will be submitted before period g, but will be included in the ad hoc request set during the planning phase of period g. To establish a Markov decision process model, each period g∈G is divided into |T| time slices. Decisions are made only at the beginning of each time slice. The maximum number of decisions per vehicle in a period is |T|. The buffer time for the optimization execution phase is set to BT. During the buffer time BT, the next period is planned. G represents the period set; T represents the time slice set; N g ={n|n=1,2,…,|N g |}, where n represents the request number; In step S2, a temporary request rejection mechanism is designed, specifically as follows: When a temporary request cannot be allocated to any vehicle in the current cycle, it is considered to be included in the planning of the next cycle. The benefit of accepting the temporary request is evaluated by comparing the additional cost of serving the temporary request in the next cycle with the penalty cost of rejecting the temporary request. If the additional cost of serving the temporary request is lower than the penalty cost of rejecting the temporary request, the temporary request is included in the planning of the next cycle; otherwise, the temporary request is rejected. Design a prediction error correction mechanism, specifically as follows: The prediction error of request n includes three types: passenger number error, submission time error, and station error. The condition for triggering correction is whether the constraint is violated. Whether the constraint is violated is derived from the planned route of the existing scheme and the updated information of request n. If the prediction error does not meet the constraints, corresponding corrections are required to ensure the effectiveness of the plan. Assume the predicted number of passengers for request n is The actual number of passengers is q n And plans to allocate it to vehicle k; when Not only when receiving request n, but also in the subsequent routes of the plan, it is necessary to determine whether vehicle k still meets the capacity constraint; If vehicle k still meets the capacity constraint in the updated route, the existing plan can be executed; otherwise, vehicle k cannot serve request n as expected, that is, the existing plan fails and needs to be corrected to ensure effective scheduling. The correction starts by searching for whether there is a running vehicle that can serve request n. If no vehicle can serve request n, it is merged into the request set of the next cycle. Design an overdue compensation mechanism, specifically as follows: For a vehicle k∈K, initialize a variable AT in each cycle. g,t,k , represents the timestamp of vehicle k completing its current decision within time slice t during period g. Therefore, when period g ends, AT g,|T|,k It is the duration of its route; If AT g,|T|,k If > g·H, then the route r of vehicle k within period g g,k The duration is overdue; in this case, it is necessary to update the usage time information of vehicle k in the next cycle g+1, that is, AT g,|T|,k Add to the initialization of the next cycle; K represents the vehicle set; In step S3, the cost of flexible bus dynamic scheduling is determined, specifically as follows: Costs include transportation costs and passenger costs. Transportation costs are the product of mileage traveled and cost per unit mile, expressed as... In the formula, The transportation cost represents the period g; δ f Indicates cost per unit mileage; i g,t,k Indicates the station of vehicle k within time slice t during period g; i g,t+1,k This represents the station of vehicle k in time slice t+1 within period g; DIS(i g,t,k i g,t+1,k ) indicates from site i g,t,k To site i g,t+1,k The distance; Passenger costs include time window penalty costs and refusal penalty costs, calculated using the following formula: In the formula, This represents the passenger cost over period g. For 0-1 variables, This indicates that the temporary request n was successfully responded to. This indicates that the temporary request n was rejected; δ e This indicates the penalty cost for arriving early; q n This indicates a request for the number of passengers, n; This indicates the earliest departure time of reservation request n or the submission time of a temporary request; Indicates the reception time of request n; δ l This indicates the cost of penalties for being late to work. Indicates the latest departure time of request n; δ b Indicates the cost of refusing to respond; N g This represents the set of requests within a period g.

2. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 1, characterized in that, The objective function for establishing the Markov decision process model is as follows: The first objective function minimizes the total cost in each cycle, expressed as: The formula for calculating waiting time is: In the formula, W n Indicates the waiting time for request n; Indicates the rejection time for temporary request n; Introducing fairness indicators To measure the fairness of passenger waiting times, the second objective function considering fairness indicators is expressed as follows: In the formula, This represents the average waiting time for temporary requests within period g.

3. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 2, characterized in that, The constraints for the Markov decision process model are set as follows: Vehicle passenger capacity constraint: A vehicle must meet the passenger capacity constraint when visiting each station, which is represented as follows: In the formula, z g,t,k Z represents the number of passengers carried by vehicle k during time slice t within period g; Z represents the maximum number of passengers a vehicle can carry. Running time constraint: For each vehicle k, all requests must be served within the operating time (|G|·H), and vehicles in operation must return to the depot before the end of the operating time, represented as In the formula, This represents the waiting time of vehicle k within time slice t during period g. Indicates the total waiting time of vehicles during operating hours; i |G|,|T|,k This indicates the station of vehicle k at the end of the operating hours; In-transit time constraint: The in-transit time of each request must not exceed α times the shortest path direct arrival time of that request, expressed as... In the formula, This indicates the boarding station for request n; This indicates the drop-off point for request n; vel represents the average speed of the vehicle. This indicates the delivery time of request n.

4. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 3, characterized in that, In step S4, the core elements are defined based on multi-agent reinforcement learning, specifically as follows: Global state S uses vehicle state S K and request status S N This means that S = {S} K ,S N }, using S g,t Let g represent the global state within time slice t during period g. and These correspond to the vehicle state and request state within time slice t of period g, respectively. Request status o g,t,k This represents the local state of the intelligent agent vehicle k within time slice t during period g; Action space A of intelligent agent vehicle k g,t,k (a g,t,k ∈A g,t,k ) is defined as In the formula, p represents the next movement of vehicle k within time slice t in period g; n Indicates the action of requesting n; d n This indicates the action of delivering request n; This indicates the action of returning to the parking lot; This indicates that vehicle k is moving. Previous waiting time; I represents the station set; J represents the parking lot set; Four pruning operations are included: pruning based on visit order, pruning based on transit time constraints, pruning based on runtime constraints, and pruning based on passenger capacity constraints; state transitions include intra-cycle state transitions and inter-cycle state transitions; given the bi-objective optimization task of the Markov decision process model, two reward functions are defined, namely cost-reward R0. C And fair rewards R ζ And set the weight ρ to calculate the total reward function.

5. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 4, characterized in that, In step S4, a novel multi-agent reinforcement learning framework is designed, including a rolling value decomposition network algorithm that combines demand prediction to enhance the foresight of planning in each cycle, specifically as follows: Each Q-function is represented by a neural network. Before training period g, for each agent vehicle k∈K, a Q-function is initialized by θ. g,k The local Q-function Q is characterized k Considering the use of time-related local states to represent state information, let the joint state be O = (o1, o2, ..., o...). |K| The combined action is A = (a1, a2, ..., a...). |K| If ), then the joint network can be composed of each Q k Linear summation yields, i.e. Therefore, Q can be obtained for training. tot and Q for each intelligent agent vehicle decision k The objective function for each cycle is transformed into In the formula, Q tot Represents the joint Q-function; O g,1 This represents the joint state within period g at time slice t=1; o g,0,k θ represents the local state of vehicle k within period g at time slice t = 0; g,k The parameter representing vehicle k within period g; o g,0 θ represents the local state at time slice t = 0 within period g; g The parameter represents the period g; π g Represents the strategy function within period g; π g,k Q represents the policy function of vehicle k within period g; k Let Q represent the local Q-function of vehicle k; By introducing a rolling time-domain framework and demand forecasting, and incorporating cycle transitions and demand forecasting into the optimization of each cycle, the objective function for each cycle is further transformed into: In the formula, o g,1,k This represents the local state of vehicle k within period g at time slice t=1; Represents the set of prediction requests for period g; o g,1 This represents the local state at time slice t=1 within period g; o g-1,|T|,k This represents the local state of vehicle k within the period g-1 at time slice t = |T|.

6. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 4, characterized in that, In step S4, a novel multi-agent reinforcement learning framework is designed, including improving the ∈-greedy action selection strategy through time-related information to enhance search efficiency during reinforcement learning training. Specifically: To combine random search and time-related search, an ∈-greedy greedy algorithm is proposed. At the start of training, two initial search rates, ∈1 and ∈2, are set, where ∈1 + ∈2 ≤ 1. The action selection strategy is as follows: In the formula, This represents the greedy action that can bring the maximum expected reward at the current moment; ∈1(τ) represents the search rate of ∈1 when the number of steps is τ, and ∈2(τ) represents the search rate of ∈2 when the number of steps is τ. The formulas for calculating the search rates of ∈1(τ) and ∈2(τ) under the decay mechanism are as follows: In the formula, The maximum decay steps ∈1, The maximum decay step number is ∈2.

7. The flexible dynamic bus scheduling method considering mixed requests and fairness according to claim 4, characterized in that, In step S4, a novel multi-agent reinforcement learning framework is designed, including combining multi-agent reinforcement learning with a local search strategy based on variable neighborhood descent and imitation learning, and designing a loss function, specifically: To improve computational efficiency, three neighborhood operators were designed, and the application of these operators was guided by the concept of "entropy." The formulas for calculating the three entropy indices are as follows: In the formula, ρ represents the target weight; Indicate route Related costs; Indicate route The request set; β represents the weighting coefficient of the two types of rewards; Indicate route The duration of loop l; c(l) represents the associated cost of loop l; N(l) represents the request set of loop l; Δ(l) represents the duration of loop l; c(n) represents the associated cost of request n; Δ(n) represents the duration of request n; Each entropy metric is designed based on a defined reward function, divided by the duration to indicate the degree of reward increase per unit of operating time; first, priority list 1 is constructed in descending order of μ1(k), then priority list 2 is constructed in descending order of μ2(l), and finally priority list 3 is constructed in descending order of μ3(n). Local search strategies include in-loop movement, inter-loop movement, and insertion-rejection-request movement. A local search strategy based on the variable neighborhood descent algorithm is proposed. Given an initial planning scheme of one cycle, i.e., the initial solution x0, and the maximum number of iterations, x can be returned based on the variable neighborhood descent algorithm. The total loss function equals the sum of the mean squared error loss function of the TD error and the imitation learning loss, expressed as L = L TD +L IL In the formula, L TD L represents the mean square error of the TD error; IL This indicates the loss from imitation learning.

Images