Automatic driving bus collaborative formation dynamic scheduling method

Abstract

The invention discloses an automatic driving bus collaborative formation dynamic scheduling method, which comprises the following steps of: after introducing a vehicle formation operation, constructing a vehicle energy consumption model based on vehicle specific power, and measuring the vehicle operation energy consumption of an automatic driving bus formation in any formation mode; in an intelligent network connection environment, based on a rolling time domain optimization framework, a Markov decision process model is constructed, and an automatic driving bus collaborative formation dynamic scheduling problem is explained; defining a state variable and a decision variable of the Markov decision process model, and determining constraint conditions of related variables and a target function of the Markov decision process model; the decision space of the Markov decision process model is reduced, and an approximate dynamic programming algorithm is used to solve the automatic driving bus collaborative formation scheduling problem; a multi-step look-ahead strategy based on dynamic planning is provided, the convergence efficiency of an approximate dynamic planning algorithm is improved, and an automatic driving bus collaborative formation optimization scheduling scheme is obtained. And the automatic driving bus capacity utilization rate is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving bus scheduling, in particular, to an automatic driving bus cooperative platoon dynamic scheduling method based on a rolling horizon planning framework. BACKGROUND

[0002] Automatic driving technology has been widely concerned in recent years, and has a wide range of applications, involving port trucks, mine trucks, sanitation trucks and buses. In the field of automatic driving buses, many automatic driving bus pilot projects have been introduced by countries around the world to explore this new transportation mode, such as LTA-ST in Singapore, CityMobil2 and EUREF in Europe, ApolloRobobus and WeRide in China. Automatic driving buses can run on urban roads all day long, effectively solving the problems of driver fatigue driving and low punctuality rate in the process of bus operation, and have broad application prospects, which can provide passengers with more flexible and accurate travel services, and therefore have great development potential in future public transportation systems.

[0003] With the development of car-road cooperation and automatic driving technology, the bus vehicles are transformed from traditional single vehicle intelligence to multi-vehicle and car-road cooperation, greatly improving the global perception and decision-making ability. In the intelligent network environment, automatic driving buses can receive information of adjacent vehicles and lead vehicles through communication technology, thereby having the ability of cooperative platoon and speed guidance, allowing vehicles to combine and split the vehicle fleet, dynamically dispatch, and adjust the vehicle speed during operation. These operations make the vehicle fleet have variable capacity and scheduling flexibility to adapt to changes in passenger demand, thereby improving the overall performance, energy consumption and economy of the system. Although the automatic driving bus equipment is becoming mature, the key element of commercialization landing, the operation organization mode with maximum efficiency, still needs to be explored, and the main problems are as follows:

[0004] First, the scheduling decision is limited to a single non-grouping mode, and the adaptability to passenger flow demand and energy saving potential are limited, and the efficiency of automatic driving cooperative platoon is not fully released. The more mature automatic driving electric buses on the market are small buses, such as Baidu "Apollo" (6.5 meters long) and Wen Yuan Zhi Xing "Mini-Robobus" (5.5 meters long), due to the limited load capacity of single vehicle, in the case of passenger flow fluctuation, the traditional fixed departure interval dispatching method is difficult to match the passenger flow demand, causing queuing, overcrowding or vacancy in the vehicle, resulting in waste of resources. If multiple vehicles can be grouped into a vehicle fleet and the size of the grouped vehicle fleet can be adjusted flexibly during operation, the passenger carrying capacity can be dynamically changed to adapt to the spatial and temporal uneven passenger flow along the way.

[0005] Second, existing researches are mostly oriented to optimal supply-demand matching, and few of them concern the influence of vehicle trajectory on energy consumption from the supply side. Electric vehicles are the mainstream of autonomous driving buses, and their energy consumption is closely related to vehicle speed, trajectory, formation and passenger flow. Due to the limited range of electric buses, unreasonable trajectory will increase energy consumption and charging demand, shorten the range, and increase the size of the vehicle fleet and operating costs to maintain normal operation of the line.

[0006] Although the autonomous driving bus formation operation mode has advantages, there are still some key challenges in the process of popularization and application. First, the formation operation mode has more complex decision-making tasks, such as speed guidance, formation control, supply-demand matching and skip stop, and there is strong coupling between tasks. For example, the vehicle spacing and wind resistance are significantly different under different formation states and running speeds, which will directly affect the energy consumption of vehicle operation. Second, autonomous driving buses constantly adjust the vehicle trajectory according to the time-varying environment and respond to passenger travel demand in real time, which requires a response level of seconds, which poses a challenge to the timeliness of the algorithm. SUMMARY

[0007] The purpose of the present application is to overcome the defects and deficiencies of the prior art, and to provide an autonomous driving bus cooperative formation dynamic scheduling method, which provides an optimal scheduling scheme for passenger matching and vehicle fleet operation control by considering autonomous driving bus cooperative formation, effectively improves the balance between passenger travel demand and transport capacity resources, and improves the capacity utilization rate of autonomous driving buses.

[0008] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0009] An autonomous driving bus cooperative formation dynamic scheduling method, comprising the following steps:

[0010] S1: After introducing vehicle formation operation, a vehicle energy consumption model based on vehicle specific power is constructed to measure the energy consumption of autonomous driving bus formation vehicles under any formation mode;

[0011] S2: Under the intelligent networked environment, a Markov decision process model is constructed based on the rolling horizon optimization framework to explain the autonomous driving bus cooperative formation dynamic scheduling problem;

[0012] S3: Define the state variables and decision variables of the Markov decision process model, determine the constraint conditions of the related variables and the objective function of the Markov decision process model;

[0013] S4: Reduce the decision space of the Markov decision process model, and use the approximate dynamic programming algorithm to solve the autonomous driving bus cooperative formation scheduling problem;

[0014] S5: A multi-step look-ahead strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm, and an automatic driving bus cooperative platoon optimization scheduling scheme is obtained.

[0015] Further, in step S1, a vehicle energy consumption model based on vehicle specific power is constructed to measure the energy consumption of automatic driving bus platoon vehicles under any platoon mode. Specifically,

[0016] Assuming that the acceleration and deceleration of the vehicle are not considered, the vehicle specific power is converted into a formula related only to the vehicle speed, which is expressed as:

[0017] VSP=β1v+β2v 3

[0018] In the formula, β1 and β2 are parameters to be calibrated; v represents the vehicle speed;

[0019] Based on the traditional vehicle specific power calculation method, the platoon air resistance correction factors β3 and β4 of the leading vehicle and the following vehicle are introduced, and the vehicle specific power formula is modified as:

[0020]

[0021] In the formula, Leading Bus represents the leading vehicle; Following Bus represents the following vehicle;

[0022] Based on the vehicle specific power formula in the marshalling scenario, the power output of the vehicle platoon is calculated according to the passenger capacity and the vehicle weight, and the travel time is calculated according to the distance and speed, and the kilowatt-hour consumed by the vehicle platoon is obtained. The energy consumption calculation formula is:

[0023]

[0024] In the formula, is the number of vehicles in the vehicle platoon; m is the total weight of the vehicle platoon, including the weight of the loaded passengers and the weight of the vehicle; s is the distance traveled.

[0025] Further, in step S2, in the rolling horizon optimization framework, the entire operation period is divided into multiple optimization periods with a time length of T, and passenger orders are responded in batches; At the beginning of each optimization period, the operator will import the orders into the scheduling platform as a batch of orders, including the orders matched but not served before and the new orders submitted by passengers; Since the current information in each optimization period is known, it is essentially a deterministic dynamic programming problem, which is constructed as a Markov decision process;

[0026] Based on the updated system state after the execution of the previous optimization cycle's strategy, the scheduling platform searches for and outputs the optimal strategy for the current optimization cycle within a computational time much shorter than T. The platform notifies passengers of the corresponding order-vehicle matching plan and boarding and alighting times, and simultaneously delivers the strategy to the autonomous bus fleet for execution, repeating the above process until the next batch of passenger orders is imported. The first optimization cycle imports static passenger orders, and the resulting scheduling strategy is also called the initial departure plan; subsequent optimization cycles import dynamic passenger orders.

[0027] Furthermore, in step S3, the state variables and decision variables of the Markov decision process model are defined, specifically:

[0028] The decision-making tasks of the coordinated platooning model for autonomous buses include supply and demand matching, speed adjustment, departure time, waiting time at stations, station skipping, platooning, and disbanding operations. The coordinated platooning scheduling problem is modeled as a Markov decision process.

[0029] Define the state variables as:

[0030]

[0031] Where, Indicates the order status. Indicates the boarding status. Indicates the state of getting off the vehicle. Indicates order The boarding time, Indicates order The time of getting off, Represents a collection of orders; Indicates the vehicle status. In the decision-making period vehicle The site where Indicates the vehicle dispatch status. In the decision-making period vehicle The passenger capacity, represents the set of autonomous vehicles; Indicates the vehicle-station status, In the decision-making period vehicle At the bus stop The arrival time, In the decision-making period vehicle At the bus stop The departure time, represents a set of physical bus stops;

[0032] Define decision variables as:

[0033]

[0034] Let be the dispatch decision, let denote the decision period The vehicle platoon at the bus stop, the platoon speed, the waiting time and the decision of whether to skip the stop;

[0035] denote the state variable at the decision period If the vehicle and the vehicle constitute a vehicle platoon dispatched to serve the order then otherwise denote the travel speed of the dispatched vehicle platoon at the bus stop in the next arc of the decision; denote the waiting time of the dispatched vehicle platoon at the bus stop in the next arc of the decision; denote the waiting time of the dispatched vehicle platoon at the bus stop in the next arc of the decision; denote the decision of whether to skip the stop by the dispatched vehicle platoon at the bus stop in the next arc of the decision; if the dispatched vehicle platoon skips the stop at the bus stop otherwise

[0036] Further, determine the constraint conditions of the relevant variables, specifically:

[0037] The constraint conditions of the decision variables are:

[0038]

[0039] wherein, denote the set of decision periods within an optimization period;v max denote the maximum travel speed of the vehicle;v min denote the minimum travel speed of the vehicle; DT denotes the maximum dwell time of the vehicle.

[0040] Further, define the state transition equation as:

[0041]

[0042] wherein, denote the state variable at the decision period S K denote the state transition equation, describing the change process of the system state at any decision;

[0043] The calculation formula of each attribute in the state variable changing with the decision is:

[0044]

[0045] In the formula, denotes the drop-off station of the order ; denotes the pick-up station of the order ; denotes the decision-making period ; denotes the set of vehicles constituting the vehicle formation dispatched; denotes the decision-making period ;

[0046] Further, the objective function of the Markov decision process model is determined, specifically:

[0047] The objective function, i.e., the cost term, includes the operation cost and the order rejection penalty cost , and the expression is:

[0048]

[0049] The operation cost includes the use cost of the bus, the time penalty cost, the empty mileage cost, and the energy consumption cost;

[0050] The calculation formula of the operation cost is:

[0051]

[0052] In the formula, denotes the pick-up time declared by the order ; denotes the number of passengers of the order ; denotes the average weight of the passengers; M denotes the weight of the vehicle; α1, α2, α3, and α4 are the weights of the vehicle use cost, the time penalty cost, the empty mileage cost, and the energy consumption cost, respectively; denotes the distance of the arc passed by the vehicle;

[0053] The calculation formula of the order rejection penalty cost is:

[0054]

[0055] In the formula, α5 is the weight of the order rejection penalty cost; denotes that if the decision-making period rejects the service order, then otherwise

[0056] decision rule prescribes the decision at decision epoch the system is in state from the set of feasible decisions the process of selecting a decision from the set of feasible decisions; a policy π is a sequence of decision rules Given the initial state condition of the current optimization period, find an optimal policy π that minimizes the expected total cost over the entire planning horizon, resulting in: *

[0057]

[0058] where, is the initial state of the optimization period; V is the set of all possible policies for the optimization period; V * (s1) is the optimal state value function for the initial state of the optimization period.

[0059] Further, in step S4, the decision space of the Markov decision process model is reduced, specifically:

[0060] Each sub-problem of dynamic programming decomposition is in state Make the optimal decision based on the optimal value expectation of future downstream decisions Through the Bellman equation expression, the optimal solution of the entire problem is linked to the optimal solution of the decomposed sub-problem:

[0061]

[0062] where, is the optimal value function of state ; is the state after decision, i.e. is replaced by represents the optimal value expectation of future downstream decisions, i.e. The value is equal to the sum of the cost items caused by the optimal decisions from decision epoch * to the final decision epoch

[0063] Let the symbol |v max -v min |, |LT| represent the number of possible speed and waiting time decisions, and the size of the decision space for state is After reducing the speed, waiting, and station hopping part decisions, the size of the decision space is reduced to​​​​​

[0064] Furthermore, an approximate dynamic programming algorithm is used to solve the autonomous bus cooperative platoon scheduling problem, specifically:

[0065] The Markov decision process model is solved using an approximate dynamic programming algorithm. In the approximate dynamic programming algorithm, the recursive structure is decomposed into:

[0066]

[0067] Where, Indicates the state after decision The approximation of the value function of Converges to the optimal value When , the solution of Bellman equation is completed;

[0068] Command Symbol Indicates that in iteration Status after The optimal approximation of the value function; according to the value function approximation strategy, the complexity of solving the problem will be reduced from the original Reduced to acceptable polynomial time

[0069] In the In the iterations, a sample path will be generated; for the decision period in the path The decision cost caused and the value function of the state after the decision are approximately added together to calculate the first The estimated value of the value function in the iteration

[0070]

[0071] Where, Indicates the Status in iteration The approximate value function of ;

[0072] In the In the iteration, the decision period is made based on the approximation of the value function after the previous iteration. Optimal sample decision; the optimal decision will minimize the self-decision period The expected value of future downstream decisions is:

[0073]

[0074] Where, Indicates the Status in iteration the value function approximation of state

[0075] Once the value function approximation of state in the kth iteration is calculated, its value is used to update the value function approximation of the last iteration; using the direct one-step forward update method, the value function approximation of state in the kth iteration is obtained:

[0076]

[0077] wherein, is the step size between 0 and 1 in the kth iteration; represents the value function approximation of state in the kth iteration.

[0078] Further, in step S5, a multi-step forward looking strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm, specifically:

[0079] The multi-step forward looking strategy based on dynamic programming is proposed, and the idea of the strategy is that in the kth iteration, using the dynamic programming algorithm, a specific number of sample paths are selected by multi-step forward decision making from state , and the optimal path is recursively obtained from all reachable paths, and finally the optimal decision at decision making period is found; a maximum multi-step forward looking range H is set, which represents the maximum range of forward rolling decision making at any decision making period, and let represent the optimal decision at decision making period

[0080]

[0081] The formula of the value function approximation of state in the kth iteration is modified into the form of the multi-step forward looking strategy; given state at decision making period , the optimal decision is found in the feasible multi-step forward looking range, which minimizes the cost of the subsequent decision sequence and the value function approximation of the state after the decision, and is represented as:

[0082]

[0083] wherein, represents the cost item caused in the system when the decision is made at decision making period . represents the value function approximation of state​​​​​​ State in sub-iteration approximation of the value function.

[0084] Compared with the prior art, the application provides an optimal ride matching and vehicle fleet operation control scheduling scheme by considering autonomous driving bus cooperative platooning, effectively improves the supply and demand balance between passenger travel demand and transport capacity resources, improves the capacity utilization rate of autonomous driving buses, realizes efficient cooperative planning of autonomous driving bus operation complex tasks, easily forms the scale effect of transportation in a large passenger flow environment, and has strong practical application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 is a flow chart of the autonomous driving bus cooperative platooning dynamic scheduling method.

[0086] Figure 2 is a schematic diagram of the vehicle platoon or the vehicle fleet deplatooning process.

[0087] Figure 3 is a schematic diagram of a rolling horizon optimization framework.

[0088] Figure 4 is a schematic diagram of an autonomous driving bus vehicle.

[0089] Figure 5 is a schematic diagram of a multi-vehicle platoon. DETAILED DESCRIPTION

[0090] The autonomous driving bus cooperative platooning dynamic scheduling method of the application will be further described below in combination with the drawings and specific embodiments. Please refer to Figure 1 The application discloses an autonomous driving bus cooperative platooning dynamic scheduling method, which comprises the following steps:

[0091] S1: After introducing the vehicle platooning operation, a vehicle energy consumption model based on vehicle specific power is constructed to measure the energy consumption of autonomous driving bus platoon vehicles under any platoon mode;

[0092] S2: Under the intelligent networked environment, a Markov decision process model is constructed based on a rolling horizon optimization framework to explain the autonomous driving bus cooperative platooning dynamic scheduling problem;

[0093] S3: The state variables and decision variables of the Markov decision process model are defined, and the constraint conditions of the related variables and the objective function of the Markov decision process model are determined;

[0094] S4: The decision space of the Markov decision process model is reduced, and an approximate dynamic programming algorithm is used to solve the autonomous driving bus cooperative platooning scheduling problem;

[0095] S5: A multi-step look-ahead strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm, and an automatic driving bus cooperative platoon optimization scheduling scheme is obtained.

[0096] First, in order to facilitate the description of the mathematical model in the following, Table 1 lists the symbol variables involved in the present application.

[0097] Table 1 Symbol variables

[0098]

[0099]

[0100]

[0101]

[0102] Step S1, after introducing the vehicle platoon operation, a vehicle energy consumption model based on vehicle specific power is constructed to measure the energy consumption of automatic driving bus platoon vehicles under any platoon mode.

[0103] Vehicle Specific Power (VSP) is the core parameter of the energy consumption measurement method, which was first proposed by Jimenez-Palacios (1998) of Massachusetts Institute of Technology. Its unit is kW / t, which represents the power output of each ton of mass (including vehicle weight) moved. Assuming that the acceleration and deceleration of the vehicle are not considered, the vehicle specific power can be finally converted into a formula related only to the vehicle speed:

[0104] VSP = β1v + β2v 3 (1)

[0105] Where β1 and β2 are parameters to be calibrated.

[0106] Vehicle platoon technology can enable multiple vehicles to form a platoon. When multiple vehicles travel in the form of a platoon, the aerodynamic drag of the front and rear vehicles will change, and the aerodynamic drag will significantly change the value of the vehicle specific power. Based on the traditional vehicle specific power calculation method only applicable to single vehicle energy consumption calculation, the platoon air resistance correction factors β3 and β4 of the lead vehicle and the following vehicle are introduced, and the formula (1) is modified as follows:

[0107]

[0108] Based on the modified vehicle specific power formula in the platoon scenario, the power output of the vehicle platoon is calculated according to the passenger capacity and vehicle weight, and the travel time is calculated according to the distance and speed, and then the kilowatt-hour consumed by the vehicle platoon is obtained. The energy consumption calculation formula is:

[0109]

[0110] wherein, is the number of vehicles that make up the vehicle platoon, m is the total weight of the vehicle platoon (including the weight of the passengers and the vehicles), and s is the distance traveled.

[0111] Step S2, under the intelligent connected environment, a Markov decision process model is constructed based on a rolling horizon optimization framework to explain the automatic driving bus cooperative platoon dynamic scheduling problem.

[0112] Figure 2 (a) is a schematic diagram of an automatic driving bus operation loop. Under the intelligent connected environment, the automatic driving bus can implement speed adjustment, station waiting and skip station strategies. After the automatic driving bus enters any bus station, it can adjust its driving speed to the next station. For bus stations without assigned orders, the automatic driving bus can skip the station. Considering the platoon and disbanding operations, the automatic driving bus platoon can travel together to increase the capacity to load large passenger flow and reduce air resistance to reduce overall energy consumption. After the platoon is disbanded, the transportation unit is increased to execute more scheduling decisions.

[0113] As Figure 2 (b) shows that a full load vehicle and two vehicles each loaded with one person arrive at a bus station with four passengers at the same time. To fully utilize the capacity and save energy, the three vehicles form a new platoon for subsequent travel. Subsequently, affected by the passenger's in-vehicle time and other assigned orders, the automatic driving bus platoon may disband at a certain bus station to improve scheduling flexibility. The research object of the present application is the fixed line automatic driving bus operation scenario.

[0114] In the rolling horizon optimization framework, the entire operation period is divided into multiple optimization periods with a time length of T, so that passenger orders are responded in batches. As Figure 3 shown, at the beginning of each optimization period, the operator will import the orders into the scheduling platform as a batch of orders, including the previously matched but unserved orders and the newly submitted orders of passengers. Since the current information in each optimization period is known, it is essentially a deterministic dynamic programming problem, which is constructed as a Markov decision process. Based on the updated system state after the execution of the strategy in the previous optimization period, the scheduling platform finds and outputs the optimal strategy for the current optimization period within a calculation time much smaller than T. Subsequently, the platform will inform the passengers of the corresponding order matching scheme and boarding and alighting times, and deliver the strategy to the automatic driving bus platoon for execution until the next batch of passenger orders is imported to repeat the above process. The static orders of passengers are imported in the 1st optimization period, and the generated scheduling strategy is also called the initial departure plan; while the dynamic orders of passengers are imported in the subsequent optimization periods.

[0115] In the rolling horizon optimization framework, there are multiple forms of passenger orders, which are defined as follows: new orders submitted by passengers over time and previously matched but unserved orders are called “received orders” after being received by the dispatching platform; the platform assigns results, which are either matching or rejection, corresponding to “matched orders” and “rejected orders”, respectively. For matched orders, they are divided into “served orders” and “matched but unserved orders” according to whether the vehicle has transported the passenger from the pickup station to the drop-off station. Matched but unserved orders need to be reassigned to the same vehicle in subsequent optimization cycles until the vehicle is served, and they are not allowed to be rejected.

[0116] Step S3, define the state variables and decision variables of the Markov decision process model, determine the constraint conditions of the related variables and the objective function of the Markov decision process model.

[0117] The decision tasks of the automatic driving bus cooperative platoon mode include supply-demand matching, speed adjustment, departure time, waiting time at the station, skip station, platoon and release operation; in the present application, the cooperative platoon scheduling problem is modeled as a Markov decision process (MDP), which is as follows:

[0118] Define the state variables:

[0119]

[0120] Among them, the order state contains the pickup and drop-off state and time of the order; the vehicle state records the current station, passenger capacity and dispatch state of the vehicle; the vehicle-station state is composed of the arrival time and departure time of the vehicle at each bus stop. For any time-type variable that is not assigned or updated, such as and , the initial value is and for the rest of the variables, the initial value is 0.

[0121] Define the decision variables:

[0122]

[0123] Among them, let be the dispatch decision, and let represent the decision period vehicle platoon speed, waiting time and whether to skip station decision at the bus stop. Since the decision period of dispatch can be a single vehicle or a platoon, in order to distinguish which vehicle it is, only the dispatched vehicle is marked, not all vehicles in the platoon. For example, in the decision period dispatch is Composition including vehicles Fleet service orders When the decision variables satisfy and Since in this case the passengers will be in the vehicle Ride inside so you won't be marked That is and Only when a single vehicle Dispatched service orders When, only

[0124] Define the constraints:

[0125]

[0126]

[0127]

[0128] Formula (6) ensures that all unserved orders within the optimization cycle are matched; since an order includes both the boarding station and the alighting station, an order needs to be matched twice; Formula (7) requires that for orders that are not rejected, the passengers need to be transported from the boarding station to the alighting station by the assigned vehicle formation; Formula (8) stipulates that the number of vehicles constituting a vehicle formation does not exceed the maximum number of vehicles in a formation; Formula (9) ensures that all vehicles in a vehicle formation are located at the same station before dispatch; Formula (10) stipulates the range of stations that the dispatched vehicle formation can go to within the current decision;

[0129] Formula (11) ensures that the vehicles in the vehicle formation after receiving the order The number of passengers in the bus does not exceed the maximum capacity; Formula (12) ensures that the boarding time of any order is after the boarding time declared by the passenger, and allows the vehicle formation to be late but not exceed the threshold LT; Formula (13) stipulates that the on-board time of any order shall not exceed σ times the driving time from the boarding station to the off-board station at the minimum driving speed; Formula (14) ensures that the station time of the vehicle at any station does not exceed the threshold DT; Formula (15) ensures that the difference between the departure time of each vehicle in the autonomous bus fleet and other vehicles at the depot in any decision period is not less than the minimum departure interval T d ; Equation (16) declares the decision variables relationship to the vehicle serving the order;

[0130] Formula (17) describes the decision variables and vehicles that form a platoon The relationship between the set; Formula (18) specifies the range of vehicle formation speed; Formula (19) declares the decision variables and total waiting time The relationship between the decision variables is described in formula (20). and the order rejection variable; if at the decision time Order If served by a platoon, Must be no less than 1, otherwise, Equal to 0; these two cases can accurately define the variable The numerical value of; Equations (21) to (24) define the decision variables.

[0131] Define the state transition equation:

[0132]

[0133] The calculation equations for each attribute in the state variable as it changes with decision making are:

[0134]

[0135]

[0136] In formula (26) and formula (27), only when the vehicle formation arrives at the boarding station and the alighting station of the assigned order, the boarding and alighting status of the order is updated respectively; in formula (28) and formula (29), only when the vehicle formation arrives at the boarding station and the alighting station of the assigned order, the boarding and alighting time of the order passenger is updated, and its value is equal to the vehicle formation's first The time of the decision period plus the waiting time and driving time of this decision process; in formula (30), when the vehicle When a convoy is dispatched, the positions of all vehicles in the convoy will be the destination. If not dispatched, the position remains unchanged;

[0137] In formula (31), when When the vehicle formation arrives at the order boarding station, the vehicles The load is equal to the current load plus the number of passengers in the order. When it arrives at the order's drop-off station, the vehicle The load is equal to the current load minus the number of ordered passengers minus the number of ordered passengers; in formula (32), when the vehicle When a vehicle formation is dispatched, the dispatch status of all vehicles in the vehicle formation will be updated to 1. If they are not dispatched, they will remain unchanged;

[0138] In formula (33), when the vehicle When dispatched, all vehicles forming a platoon are Station and The arrival time of the station will be updated, and its value is equal to the departure time of the previous station plus the travel time between the two stations in this decision process; in formula (34), the departure time of the vehicle formation at the station is equal to the arrival time plus the waiting time for decision making. If the dispatched vehicle formation is grouped or split, it is necessary to When the vehicle arrives at the order's boarding or disembarking station, the time for the vehicle to be assembled or disembarked is added. The service time at this station must be added.

[0139] The objective function is the cost term, which is composed of the operating cost and penalty costs for rejection It consists of two parts, and the specific formula is as follows:

[0140]

[0141] operating costs Including the use cost of public transportation vehicles, time penalty cost, empty mileage cost and energy consumption cost. Among them, the use cost of the vehicle is related to the number of vehicles in the entire fleet, based on the vehicle dispatch status in the state variable The number of vehicles can be determined by the decision period and the post-decision period The time penalty cost is related to the difference between the actual boarding time and the declared boarding time of the passenger and the number of passengers, that is, the time all passengers wait to board the bus at the bus stop, which can be calculated through the boarding time in the state variable and the time window in the order information. The number of empty seats can be calculated based on the vehicle passenger capacity and the rated passenger capacity of the vehicle at the decision period k, and the empty mileage cost of the vehicle formation can be obtained by multiplying it by the arc distance traveled. Based on the energy consumption estimation equation Energy() in equation (3), the operating energy consumption of the fleet in each decision can be calculated. In summary, the calculation equation for the operating cost is as follows:

[0142]

[0143] in, and M represent the average weight of a passenger and the rated mass of an autonomous bus, respectively. α1, α2, α3, and α4 are the weights of vehicle usage cost, time penalty cost, idle mileage cost, and energy consumption cost, respectively.

[0144] The penalty cost for rejection is only related to the number of passenger orders that are rejected by the system, which can be expressed as the rejection variable If the decision period If the system rejects the service order, This will cause a new order rejection penalty, which is calculated as follows:

[0145]

[0146] Where α5 is the weight of the penalty cost for rejection.

[0147] Decision Rules It stipulates that during the decision-making period The system is in state From the set of feasible decisions The process of choosing a decision, such as It means the choice decision A policy π is a sequence of decision rules Given the initial state conditions of the current optimization cycle, it aims to find an optimal strategy π for the operator * To minimize the expected total cost over the entire planning horizon:

[0148]

[0149] in, To optimize the cycle The set of all possible strategies in V * (s1) is the optimal state value function of the initial state of the optimization cycle.

[0150] In step S4, the decision space of the Markov decision process model is reduced, and the autonomous driving bus cooperative platoon scheduling problem is solved using an approximate dynamic programming algorithm.

[0151] Each sub-problem decomposed by dynamic programming is in state Make optimal decisions based on the expected optimal value of future downstream decisions This recursive form can be expressed through the Bellman equation, which links the optimal solution to the entire problem with the optimal solutions to the decomposed subproblems:

[0152]

[0153] in, is the status The optimal value function of is the post-decision state, that is, use replace Expressed in simplified notation. represents the optimal value expectation of future downstream decisions, that is, Its value is equal to the optimal strategy π * Decision-making period To the final decision period The optimal decision The sum of the cost items caused. Similarly, It is also the optimal value expectation for future downstream decisions. The difference between the two lies in the different benchmark decision-making periods.

[0154] To fine-tune the search for decision variables The optimal speed, waiting time and station skipping decision in the equation need to be discretized. The search accuracy depends on the discretization granularity. max -v min |, |LT| represents the number of possible speed and waiting time decisions. For example, assuming that the speed decision is searched at a granularity of 1 km / h and the speed range of the autonomous bus is 10-40 km / h, there are 31 possible speed decisions. According to the above decision-making ideas, the entire decision space will be expanded to 6 dimensions, namely, dispatching vehicles Dispatch vehicles Order Speed ​​at bus stops along the way Waiting time and whether to skip a station In-depth analysis and exploration of decision-making space and the size of the state space, such as Figure 3 As shown, it shows the status The decision space of and The feasible decision set consists of three parts: order service sequence, dispatch vehicle formation, speed, waiting, and skipping stations. The probability of each part is as follows:

[0155] (1) Order service sequence. For each order received in the optimization cycle, its demand at the boarding station and the alighting station can be converted into a vertex set. For example, Orders with a maximum of Vertex, if the order has not been served. At any decision time, the operator needs to and get-off status Select the vertex to which the current dispatched vehicle or fleet is to go from multiple vertices that have not been served. Therefore, the order service sequence part has possibilities.

[0156] (2) Dispatching vehicle formations. Once the vertex to be traveled to is determined, the dispatching platform needs to dispatch the appropriate vehicle formation to perform the task within the decision. As mentioned above, a vehicle formation can be either a single vehicle or a fleet of multiple vehicles. Based on the constraints, the possibility of all vehicle formations in the current decision period can be calculated. However, since the states of each decision period are different, only the maximum possibility is discussed here, that is, the states of all vehicles are the same. Calculating the possibility of vehicle formations belongs to the mathematical concept of permutation and combination problems, that is, from There are different ways to get a specific number of cars. Therefore, the maximum number of possibilities for this part is kind.

[0157] (3) Speed, waiting and skipping. Based on the first two parts, the bus stop where the dispatched vehicle formation is located can be determined. and the bus stop you want to go to That is, passing by For each bus stop, the dispatching platform needs to make a decision The speed, waiting time and skipping decision of the vehicle formation. According to the previous discretization search idea, there are possibilities.

[0158] state The size of the decision space is The state space refers to the set of all possible states of the entire problem, which describes the various situations the system may be in. The scale of the state space can then be derived: to derive the largest scale, the dispatching of vehicle formations always maintains the greatest possibility. After making a decision in any state, the most direct change is that the number of vertices in the order service sequence will decrease by one. The factorial structure can be used to represent all the states from the initial decision period 1 to the final decision period The sum of all possibilities, the complexity of the state space can be expressed as in is omitted if it is a constant.

[0159] The decision space and state space of the problem studied in this invention are so large that it is impossible to solve it in a short time. Therefore, it is necessary to reduce The size of the state space is further reduced. Since the core part of the decision is the order service sequence and the dispatch of vehicle formations, The search can only be conducted after the two are determined. It can be considered an exogenous decision, and its basic assumption is that there are no orders at any bus stops along the way. The rationality of this idea can be explained as follows: even if there are other orders at the bus stops along the way, the scheduling platform can meet the basic assumption by adjusting the order service order, and because the decision includes all access orders, the optimal solution will not be lost. For example, suppose the boarding station for order B is before the boarding station for order A. In the current decision period, the scheduling platform can make a decision to go to the boarding station A or to the boarding station B; in the next decision period, the scheduling platform will make a decision to go to the other order's boarding station, namely A→B (which does not actually exist because the vehicle formation does not allow reverse operation) and B→A. Based on the objective function, it can be determined which of the two has better performance, thereby completing the search for the optimal strategy.

[0160] Based on this idea, once the vehicle formation to be dispatched and the vertex to be headed to are determined, the vehicle formation moves at a constant speed. Go to the target bus stop At the same time, based on the departure time of the vehicle formation and the expected pickup time of the order, the most appropriate total waiting time can be calculated first. Then, considering the station time constraint, we assign it to each bus station along the way to obtain the waiting time of each station. Based on this method, we can effectively reduce the decision space, where speed The possibility of The number of species was reduced to 1; waiting time The possibility of The number of types is reduced to 1; and the skipping decision can be determined directly without considering various possibilities, that is, all bus stops along the way are skipped. After reducing the speed, waiting and skipping decisions, the size of the decision space is reduced to

[0161] In step S5, a multi-step forward-looking strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm and obtain an optimized scheduling solution for the coordinated formation of autonomous buses.

[0162] Approximate dynamic programming algorithms effectively address the curse of dimensionality by estimating the impact of current decisions on future values, replacing the subsequent heavy recursive search process. In the approximate dynamic programming algorithm, the recursive structure of the dynamic programming algorithm can be decomposed as follows:

[0163]

[0164] in Indicates the state after decision The approximate value of the value function. The approximate dynamic programming algorithm makes the approximate value of the value function close to the optimal value through continuous simulation decision making and updating. Converges to the optimal value The solution to the Bellman equation is completed.

[0165] There are many ways to approximate the value function Converge to the true value, among which the Value Function Approximation (VFA) strategy is a popular way. The Value Function Approximation strategy will In the iteration, starting from the state s1 of the initial decision period, decisions are made along the planning time domain until the state of the final decision period All the decisions made and the costs incurred during the process are collectively referred to as sample paths. Subsequently, the value function is updated along the sample paths. After a sufficient number of iterations, the value function approximation will eventually converge to the optimal value. In general, let the symbol Indicates that in iteration Status after The optimal approximation of the value function. According to the value function approximation strategy, the complexity of solving the problem will be reduced from the original Reduced to acceptable polynomial time

[0166] Then, the specific process of the value function approximation strategy is introduced. In the iterations, a sample path Will be generated. For the decision period in the path The decision cost caused and the value function of the state after the decision can be approximately added to calculate the estimated cost of the decision period

[0167]

[0168] In the In the iteration, the decision period is made based on the approximation of the value function after the previous iteration. Optimal sample decision. The optimal decision will minimize the self-decision period The expected value of future downstream decisions is:

[0169]

[0170] Once the Iteration is calculated, and its value can be used to update the value function approximation of the previous iteration. In the present invention, a direct single-step forward update method is used:

[0171]

[0172] in For the The step size between 0 and 1 per iteration.

[0173] The traditional approximate dynamic programming algorithm uses a single-step forward decision. Since it only considers the approximate value function of the decision period after the decision, rather than multiple decision periods in the future, it is similar to a local optimum and has the defect of short-sighted development. In the approximate dynamic programming algorithm, if the value function is approximate, Without convergence to the optimal value, the obtained decision is not necessarily optimal. Since the Markov decision process is a chain structure, i.e. from back to front, only the state value function approximation of the later decision period has been updated to the optimal value, which can ensure the update of the value function of the previous decision period. Therefore, single-step forward decision cannot guarantee the overall optimality of future decision periods, unless each state value function approximation is accurate. If more consideration is given to the optimality of multiple decision periods in the future when making sample decisions , the overall convergence efficiency of the approximate dynamic programming algorithm can be improved.

[0174] The present application proposes a multi-step lookahead strategy based on dynamic programming. The idea of this strategy is to use the dynamic programming algorithm to make multi-step forward decisions to select a certain number of sample paths from the state in the first iteration, recursively find the optimal path from all reachable paths, and finally find the optimal decision in the decision period . Therefore, a maximum multi-step lookahead range H is set, which represents the maximum range of forward decision for any decision period. At the same time, some constraints need to be given to this range to prevent the decision period near the end of the decision period from being forward. Let represent the state in the decision period

[0175]

[0176] Based on the above principle, formula (42) is modified into the form of multi-step lookahead strategy. In the decision period , given state , find the optimal decision within the feasible multi-step lookahead range to minimize the cost of the subsequent decision sequence and the value function approximation of the state after the decision.

[0177]

[0178] The following describes the dynamic scheduling method of autonomous bus cooperative platoon through a real autonomous bus project, which is the Guangzhou Tower operating line in the autonomous bus demonstration project of Guangzhou, China. As of December 30, 2023, the project has cumulatively dispatched 36,070 times, with a passenger carrying mileage of 213,531 kilometers, and 125,938 online ticket purchases. The line is a circular line with 7 non-uniformly spaced bus stops, with a total length of 8 kilometers, and currently uses fixed departure interval dispatching. For example Figure 4As shown, the operator has put 8 "Wen Yuan Zhi Xing" company production 5.5 meters microcirculation minibus in the line, each vehicle is equipped with laser radar, millimeter wave radar, high-definition camera and other equipment. Based on 5G communication network and V2X vehicle-road cooperation technology, multiple vehicles can form a vehicle fleet as shown. Figure 5

[0179] Table 2 compares the actual operation of the automatic driving bus demonstration project, the non-cooperative formation mode and the cooperative formation mode in ten indicators. It is found that the overall performance of the optimized scheduling scheme in the two modes is significantly better than that of the actual scheduling scheme. Taking the non-cooperative formation operation mode as an example, the response rate is increased by 42.0%. Even though the number of vehicles is increased by 38.2% due to serving more orders, more passenger orders can be served by a single vehicle. Moreover, in terms of service punctuality, the late rate and the average late time are reduced by 49.2% and 28.9%, respectively. In terms of energy consumption of automatic driving buses, compared with the actual operation, the total energy consumption of the vehicle fleet of the optimized scheduling scheme is reduced by 51.1%, and the energy consumption per order is reduced to one fourth on average. This result fully proves that using speed guidance and formation operation can achieve energy-saving driving.

[0180] In summary, using dynamic optimization method to match passenger flow and control operation process will have higher service level and operation efficiency than traditional operation mode. To evaluate the impact of introducing formation operation on overall performance, the scheduling schemes in the two modes are compared. As shown in Table 2, the response rate, late rate and average late time are significantly optimized, and the remaining indicators do not change significantly. Among them, the improvement degree of formation operation on passenger late time and average late time is the largest, which is increased by 66.2% and 69.2% respectively, which can significantly reduce the waiting time of passengers in the station. After introducing formation operation, the energy consumption basically does not change, but the overall operation energy consumption-efficiency ratio of the system is significantly improved.

[0181] Table 2 Comparison of indicators of actual operation and optimized scheduling scheme (monthly average)

[0182]

[0183] The results show that the forward scheduling method and solving algorithm proposed in the present application have good characteristics and can be popularized to practical applications. The present application provides an optimal scheduling scheme for passenger matching and vehicle fleet operation control by considering the cooperative formation of automatic driving buses, effectively improves the balance between passenger travel demand and transport capacity, improves the capacity utilization rate of automatic driving buses, realizes efficient cooperative planning of automatic driving bus operation complex tasks, and easily forms the scale effect of transportation in a large passenger flow environment, which has strong practical application prospect.

[0184] ​The above description is for the preferred embodiment of the present application, but the embodiment is not intended to limit the scope of the patent application of the present application. Any equivalent changes or modifications made under the technical spirit of the present application should be covered by the patent scope of the present application.

Claims

1. A dynamic scheduling method for autonomous driving bus cooperative formation, characterized in that: The following steps are involved: S1: After introducing vehicle platooning operations, a vehicle energy consumption model based on vehicle specific power is constructed to measure the energy consumption of autonomous bus platooning vehicles under any platooning mode; S2: In an intelligent connected environment, a Markov decision process model is constructed based on a rolling horizon optimization framework to explain the dynamic scheduling problem of autonomous bus cooperative platooning. S3: Define the state variables and decision variables of the Markov decision process model, determine the constraints of the relevant variables and the objective function of the Markov decision process model; S4: Reduce the decision space of the Markov decision process model and use the approximate dynamic programming algorithm to solve the autonomous bus cooperative platooning scheduling problem; S5: A multi-step look-ahead strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm and obtain an optimized scheduling solution for the coordinated formation of autonomous buses.

2. The method for dynamic scheduling of autonomous bus cooperative formation according to claim 1, characterized in that: In step S1, a vehicle energy consumption model based on vehicle specific power is constructed to measure the operating energy consumption of autonomous bus platoon vehicles in any platooning mode, specifically: Assuming that the acceleration and deceleration of the vehicle are not considered, the vehicle specific power is converted into a formula related only to the vehicle speed, and the expression is: VSP=β1v+β2v 3 Where β1 and β2 are parameters to be calibrated; v represents the vehicle speed; Based on the traditional vehicle specific power calculation method, the air resistance correction factors β3 and β4 of the leading and following vehicles are introduced to correct the vehicle specific power formula as follows: Where, leading Bus represents the leading bus; Following Bus means following bus; Based on the vehicle specific power formula in the marshaling scenario, the power output of the vehicle formation is calculated according to the passenger capacity and vehicle weight, and the travel time is calculated according to the distance and speed to obtain the kilowatt-hours consumed by the vehicle formation. The specific energy consumption calculation formula is: Where, is the number of vehicles that make up the vehicle formation; m is the total weight of the vehicle formation, including the weight of the loaded passengers and the weight of the vehicles; s is the distance traveled.

3. The method for dynamic scheduling of autonomous bus cooperative formation according to claim 1, characterized in that: In step S2, within the rolling horizon optimization framework, the entire operating period is divided into multiple optimization cycles of duration T, and passenger orders are responded to in batches. At the beginning of each optimization cycle, the operator treats previously matched but unserved orders and newly submitted passenger orders as a batch of orders and imports them into the scheduling platform. Since the current information within each optimization cycle is known, the solution is essentially a deterministic dynamic programming problem, which is constructed as a Markov decision process. Based on the updated system state after the execution of the previous optimization cycle's strategy, the scheduling platform finds and outputs the optimal strategy for the current optimization cycle within a computational time much shorter than T. The platform then notifies passengers of the corresponding order-vehicle matching plan and boarding and alighting times, and simultaneously delivers the strategy to the autonomous bus fleet for execution until the next batch of passenger orders is imported, repeating the above process. The first optimization cycle imports static passenger orders, and the resulting scheduling strategy is also called the initial departure plan. The subsequent optimization cycle imports passengers' dynamic orders.

4. The method for dynamic scheduling of autonomous bus cooperative formations according to claim 1, characterized in that: In step S3, the state variables and decision variables of the Markov decision process model are defined, specifically: The decision-making tasks of the coordinated platooning model for autonomous buses include supply and demand matching, speed adjustment, departure time, waiting time at stations, station skipping, platooning, and disbanding operations. The coordinated platooning scheduling problem is modeled as a Markov decision process. Define the state variables as: Where, Indicates the order status. Indicates the boarding status. Indicates the state of getting off the vehicle. Indicates order The boarding time, Indicates order The time of getting off, Represents a collection of orders; Indicates the vehicle status. In the decision-making period vehicle The site where Indicates the vehicle dispatch status. In the decision-making period vehicle The passenger capacity, represents the set of autonomous vehicles; Indicates the vehicle-station status, In the decision-making period vehicle At the bus stop The arrival time, In the decision-making period vehicle At the bus stop The departure time, represents a set of physical bus stops; Define the decision variables as: make To make deployment decisions, Indicates decision-making period Vehicle formation speed, waiting time and decision-making on whether to skip a bus stop at bus stops; In the decision-making period If the vehicle and vehicles The vehicle fleet is dispatched to serve the order but otherwise In the decision-making period The dispatched vehicle fleet is at the bus station The speed of the next arc to be decided; In the decision-making period The dispatched vehicle fleet is at the bus station Waiting time; In the decision-making period If the dispatched vehicle formation is at the bus stop Jump station, then otherwise 5. The method for dynamic scheduling of autonomous bus cooperative formation according to claim 4, characterized in that: Determine the constraints on the relevant variables, specifically: The constraints on the decision variables are: Where, represents the set of decision periods within an optimization cycle; v max Indicates the maximum speed of the vehicle; v min Indicates the minimum speed of the vehicle; DT indicates the maximum station time of the vehicle.

6. The method for dynamic scheduling of autonomous bus cooperative formations according to claim 4, characterized in that: The state transfer equation is defined as: Where, In the decision-making period State variable; S K () represents the state transition equation, which describes the change process of the system state under any decision; The calculation formula for each attribute in the state variable changing with the decision is: Where, Indicates order The station where you get off; Indicates order The boarding station; Indicates decision-making period Vehicles that make up the dispatch vehicle formation gather; Indicates decision-making period The bus stop where the dispatched vehicle fleet is headed.

7. The method for dynamic scheduling of autonomous bus cooperative formations according to claim 6, characterized in that: Determine the objective function of the Markov decision process model, specifically: The objective function is the cost term, including operating costs and penalty costs for rejection The expression is: operating costs Including the use cost of public transportation vehicles, time penalty cost, idle mileage cost and energy consumption cost; The operating cost calculation formula is: Where, Indicates order Declared boarding time; Indicates order the number of passengers; represents the average weight of passengers; M represents the weight of the vehicle; α1, α2, α3 and α4 are the weights of vehicle use cost, time penalty cost, empty mileage cost and energy consumption cost respectively; Arc representing the vehicle's travel distance; The calculation formula for the penalty cost of order rejection is: Where α5 is the weight of the penalty cost for rejection; If the decision period If you refuse a service order, otherwise Decision Rules It stipulates that during the decision-making period The system is in state From the set of feasible decisions The process of choosing a decision; A policy π is a sequence of decision rules Given the initial state conditions of the current optimization cycle, find an optimal strategy π * To minimize the expected total cost over the entire planning time domain, we get: Where, For the The set of all possible strategies within the optimization period; V * (s1) is the optimal state value function of the initial state of the optimization cycle.

8. The method for dynamic scheduling of autonomous bus cooperative formations according to claim 7, characterized in that: In step S4, the decision space of the Markov decision process model is reduced, specifically: Each sub-problem decomposed by dynamic programming is in state Make optimal decisions based on the expected optimal value of future downstream decisions The Bellman equation is used to express the optimal solution of the entire problem and the optimal solution of the decomposed subproblems: Where, is the status The optimal value function of is the state after decision making, i.e. use replace represents the optimal value expectation of future downstream decisions, that is, Its value is equal to the optimal strategy π * Decision-making period Until the final decision The optimal decision The sum of the cost items incurred; Let symbol |v max -v min |,|LT| represents the number of possible speed and waiting time decisions, state The size of the decision space is After reducing the speed, waiting and skipping decisions, the size of the decision space is reduced to 9. The method for dynamic scheduling of autonomous bus cooperative formations according to claim 8, characterized in that: The approximate dynamic programming algorithm is used to solve the autonomous bus cooperative platoon scheduling problem, specifically: The Markov decision process model is solved using an approximate dynamic programming algorithm. In the approximate dynamic programming algorithm, the recursive structure is decomposed into: Where, Indicates the state after decision The approximation of the value function of Converges to the optimal value When , the solution of Bellman equation is completed; Command Symbol Indicates that in iteration Status after The optimal approximation of the value function; according to the value function approximation strategy, the complexity of solving the problem will be reduced from the original Reduced to acceptable polynomial time In the lth iteration, a sample path ω l will be generated; for the decision period in the path Approximately add the decision cost caused and the value function of the post-decision state to calculate the estimated value function in the lth iteration of the decision period Where, Indicates the state in the lth iteration The approximate value function of ; In the lth iteration, the decision period is made based on the approximation of the value function after the previous iteration. Optimal sample decision; the optimal decision will minimize the self-decision period The expected value of future downstream decisions is: Where, Indicates the state in the l-1th iteration The approximate value function of ; Once the lth iteration is calculated, and its value is used to update the approximate value function of the previous iteration; using the direct single-step forward update method, the state in the lth iteration is obtained The approximate value function of is: Where, α l-1 is the step size between 0 and 1 in the l-1th iteration; Indicates the state in the l-1th iteration An approximation of the value function of .

10. The method for dynamic scheduling of autonomous driving bus cooperative formation according to claim 9, characterized in that: In step S5, a multi-step look-ahead strategy based on dynamic programming is proposed to improve the convergence efficiency of the approximate dynamic programming algorithm, specifically: A multi-step forward strategy based on dynamic programming is proposed. The idea of ​​this strategy is to use the dynamic programming algorithm to Start multi-step forward decision-making to select a specific number of sample paths, recursively find the optimal path from all reachable paths, and finally find the optimal path in the decision period The optimal decision; set a maximum multi-step forward range H, which represents the maximum range of forward rolling decisions in any decision period, and let In the decision-making period The state in the lth iteration The formula for the approximate value function of is modified into the form of a multi-step forward strategy; in the decision period Given state Finding the optimal decision within a feasible multi-step look-ahead range The cost of the subsequent decision sequence and the value function of the post-decision state are approximately minimized, which can be expressed as: Where, Indicates decision-making period Make a decision Cost items incurred in the system when Indicates the state in the l-1th iteration An approximation of the value function of .

Images