Intercity dynamic carpooling scheduling optimization method and device integrating intra-city vehicle relocation

Through deep learning and multi-intelligent image deep Q learning network algorithm, dynamic intercity carpooling scheduling is optimized, and the problem of unbalanced supply and demand of online car-hailing in cities is solved, operational efficiency and service quality are improved, and higher profits and more effective vehicle repositioning.

CN118735233BActive Publication Date: 2024-12-06HUAQIAO UNIVERSITY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411230475.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-12-06
Estimated Expiration
2044-09-04

AI Technical Summary

Technical Problem

The imbalance between supply and demand of online car-hailing in cities has led to a decrease in vehicle utilization and an increase in operating costs, and there is a lack of effective intercity dynamic carpooling scheduling optimization methods.

Method used

Deep learning algorithms are used to predict passenger travel demand, build a double-layer optimization model, combine multi-agent map deep Q learning network algorithm, optimize intercity dynamic carpooling scheduling, and balance supply and demand by relocating idle vehicles.

Benefits of technology

It effectively alleviates the imbalance between supply and demand of online car-hailing, improves operational efficiency and service quality, and increases the total profit of intercity carpooling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118735233B_ABST
    Figure CN118735233B_ABST
Patent Text Reader

Abstract

The inter-city dynamic carpooling scheduling optimization method and device integrated with the relocation of vehicles in the city provided by the present invention relate to the field of vehicle scheduling technology. After obtaining the road network data of two cities on a certain operating route of the inter-city online car-hailing, the present invention divides several areas respectively, and obtains the round-trip carpooling travel data of passengers in each area in real time; at each relocation scheduling moment, the passenger travel demand is predicted by a deep learning algorithm, the number of idle vehicles, the supply and demand balance difference and the total number of profitable paths are calculated, and a two-layer optimization model is constructed; then an independent intelligent agent is constructed, and a multi-agent graph deep Q learning network algorithm is used for training and learning; by traversing each scheduling moment, it is determined whether the relocation interval is reached. If reached, the passenger travel demand is predicted, the relocation decision is made and the dynamic carpooling scheduling is carried out; if not, the dynamic carpooling scheduling is directly carried out. The present invention can improve the total profit of inter-city carpooling and effectively alleviate the problem of imbalance between vehicle supply and demand.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle dispatching, and in particular to an inter-city dynamic carpooling dispatching optimization method and device integrating intra-city vehicle relocation. Background Art

[0002] With the acceleration of urbanization and the diversification of residents' travel needs, the imbalance between supply and demand of online ride-hailing services in cities has become increasingly prominent, which not only affects passengers' travel experience, but also poses a challenge to the operational efficiency of online ride-hailing platforms.

[0003] The imbalance between supply and demand of online ride-hailing vehicles in a city is often manifested in the fact that there is a shortage of vehicles in certain areas during peak hours, while vehicles are idle in other periods or areas. This mismatch between supply and demand not only leads to longer waiting times for passengers, but also leads to a decrease in vehicle utilization and increased operating costs for online ride-hailing platforms.

[0004] As an important measure to solve the imbalance between supply and demand of online ride-hailing vehicles in the city, intra-city vehicle relocation is gradually gaining attention and importance. However, there is no good inter-city dynamic ride-sharing scheduling optimization method that integrates vehicle relocation strategy.

[0005] In view of this, the applicant filed this application after studying the existing technology. Summary of the invention

[0006] The present invention aims to provide an inter-city dynamic carpooling scheduling optimization method and device integrating intra-city vehicle relocation, so as to solve the problem of imbalance between supply and demand of online car-hailing vehicles between urban areas, improve the operating efficiency and service quality of the online car-hailing platform, and alleviate urban traffic pressure.

[0007] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0008] An inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation includes:

[0009] S1, after obtaining the road network data of city A and city B on a certain operation route of an intercity online car-hailing service, divide them into several areas respectively, and obtain the round-trip passenger carpooling travel data in each area in real time;

[0010] S2, given a set of scheduling times and a set of relocation scheduling times, at each relocation scheduling time in the relocation scheduling time set, predict the passenger travel demand of each area of ​​city A and city B in the future period of time through a deep learning algorithm;

[0011] S3, based on the predicted passenger travel demand, calculate the number of idle vehicles, the supply-demand balance difference and the total number of people on the profitable path in each area of ​​City A and City B respectively;

[0012] S4, constructing a two-layer optimization model based on the calculated number of idle vehicles, the supply-demand gap, the total number of people on the profitable path and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after relocation;

[0013] S5, constructing an independent intelligent agent for each area in city A and city B, wherein the intelligent agent includes a state space, an action space, and a reward function;

[0014] S6, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each region to capture the spatial relationship of each region and obtain an approximate optimal Q value function; according to the approximate optimal Q value function, a relocation decision of the idle vehicle is obtained;

[0015] S7, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if reached, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and iterate until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization solution that integrates the relocation of vehicles within the city.

[0016] The present invention also provides an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation, comprising:

[0017] A data acquisition unit is used to obtain the road network data of city A and city B on a certain operation route of an intercity online car-hailing service, divide the data into several areas, and obtain the round-trip passenger carpooling travel data in each area in real time;

[0018] A passenger demand prediction unit is used to predict the passenger travel demand of each area of ​​city A and city B in a future period of time at each relocation scheduling time in the relocation scheduling time set, given a scheduling time set and a relocation scheduling time set;

[0019] A supply and demand index calculation unit is used to calculate the number of idle vehicles, the supply and demand balance difference and the total number of people on the profitable path in each area of ​​City A and City B according to the predicted passenger travel demand;

[0020] A two-layer model building unit is used to build a two-layer optimization model based on the calculated number of idle vehicles, the supply-demand gap, the total number of profitable paths and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after relocation;

[0021] A multi-agent construction unit, used to construct an independent agent for each area in city A and city B; wherein the agent includes a state space, an action space and a reward function;

[0022] A vehicle relocation unit is used to obtain a relocation decision, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each area to capture the spatial relationship of each area and obtain an approximate optimal Q value function; based on the approximate optimal Q value function, a relocation decision for an idle vehicle is obtained;

[0023] The dynamic carpooling scheduling unit is used to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if it reaches it, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and it is iterated until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city.

[0024] The present invention also provides an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation as described above.

[0025] The present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, an inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation as described above is implemented.

[0026] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention is based on a two-layer planning model of inter-city dynamic carpooling and vehicle relocation. The upper model takes maximizing the sum of profits at all scheduling time points as the optimization goal, and the lower model takes maximizing the potential profit after relocation as the optimization goal. The present invention also designs a multi-agent graph deep Q-learning network (Multi-agent Graph Deep Q-Network, MAGDQN) algorithm. The algorithm combines multi-region demand forecasting, considers supply and demand balance, matching path feasibility and mobility cost, regards each region as an independent agent, introduces a multi-head graph attention network MGAT to learn relocation relationships, and realizes multi-agent collaborative decision-making and idle vehicle relocation in bilateral cities.

[0028] The present invention proposes an inter-city dynamic carpooling optimization scheme based on multi-region demand forecasting and vehicle relocation, which further improves the total profit of inter-city carpooling and can effectively alleviate the problem of imbalance between supply and demand of online car-hailing services. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A schematic diagram of an inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation provided in Embodiment 1 of the present invention.

[0031] Figure 2 An example graph of an undirected graph provided in the first embodiment of the present invention;

[0032] Figure 3 A schematic diagram of the training process of the MAGDQN algorithm provided in the first embodiment of the present invention;

[0033] Figure 4 The inter-city dynamic carpooling optimization flow chart provided in the first embodiment of the present invention;

[0034] Figure 5 A schematic diagram of the supply-demand difference comparison results of different methods provided in Example 1 of the present invention on the Anxia round-trip data set;

[0035] Figure 6 Iteration graph of the MAGDQN algorithm provided in Example 1 of the present invention on the Anxia round-trip data set;

[0036] Figure 7 A schematic diagram of an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation provided in the second embodiment of the present invention.

[0037] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Embodiment 1

[0040] Embodiment 1 of the present invention provides an inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation, which can be implemented by an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation (hereinafter referred to as scheduling optimization device), and in particular, is executed by one or more processors in the scheduling optimization device.

[0041] In this embodiment, the scheduling optimization device may be an electronic device equipped with a processor, the processor having a computer program of the inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation and the computer program can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., and the present invention does not make specific limitations.

[0042] In this embodiment, the imbalance between supply and demand of online-hailing vehicles in a city refers to the problem of imbalance between the actual supply of online-hailing vehicles and regional demand due to the allocation and scheduling problems of online-hailing vehicles. For example, during peak hours, there is insufficient supply of vehicles in certain areas, resulting in less supply than demand; while during other periods or in other areas, vehicles are idle, with supply exceeding demand, which increases vehicle operating costs.

[0043] For this reason, Figure 1 As shown, an inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation includes steps S1 to S7.

[0044] S1, after obtaining the road network data of city A and city B on a certain operating route of an intercity online car-hailing service, divide them into several areas respectively, and obtain the round-trip passenger carpooling travel data in each area in real time.

[0045] In this embodiment, the regional node set of city A is , the regional node set of city B is , the real-time round-trip passenger carpooling travel data in each area includes passenger data and vehicle data; among them,

[0046] The passenger data includes: boarding point , Drop-off point , Order time , Boarding time , Service Hours Number of people traveling with you ; Among them, the order time refers to the time when the passenger places a carpooling request order; the boarding time refers to the time the passenger expects to board the car; and the service time refers to the time it takes the passenger to complete the boarding at the boarding point.

[0047] The vehicle data includes: current starting point , Current destination , garage point , Current location , the location of the next trip , Passenger capacity , Number of trips and itinerary path set ;vehicle Number of passes In the itinerary path set Locate your current itinerary path ,vehicle The itinerary path , which starts from , the requested pickup point served to , Departure city exit point , Destination city entry point , the requested drop-off point being served to and end point composition.

[0048] Each requested boarding time has an earliest boarding time window constraint and the latest boarding time window constraint ,vehicle Time window Arrive at the requested pickup point within 10 days.

[0049] Set the maximum number of round trips for the vehicle to , the shortest rest time of the trip is , the maximum waiting time for the return trip is ; Set up vehicle Arrival point The time is , , is the set of all points in the road network of the two cities, and the vehicles At the current starting point The departure time is , before any new requests are inserted into the route to the point The time is , A set of matched requests.

[0050] S2, given a scheduling time set and a relocation scheduling time set, at each relocation scheduling time in the relocation scheduling time set, a deep learning algorithm is used to predict the passenger travel demand in each area of ​​city A and city B in the future.

[0051] In this embodiment, the scheduling time set is , each scheduling time is t, there are The relocation scheduling time set is , each relocation scheduling time is ;

[0052] The historical passenger carpooling travel data is used as input features, and the short-term demand forecasting model is trained through deep learning algorithms. , predict the future through the trained short-term demand forecasting model Passenger travel demand in each area of ​​City A and City B during the time period.

[0053] In this embodiment, there is no limitation on the deep learning algorithm used, which can be any existing deep learning algorithm, such as long short-term memory (LSTM) neural network, spatiotemporal graph convolutional network (T-GCN), historical average method (HA), autoregressive moving average method (ARIMA), etc.

[0054] S3, based on the predicted passenger travel demand, calculate the number of idle vehicles, the supply-demand balance difference and the total number of profitable paths in each area of ​​City A and City B respectively.

[0055] Taking city A as an example, according to the regional node set of city A , then each area of ​​city A Contains: Road network centroids , forecast demand , poor balance between supply and demand , Total number of profit paths , idle vehicle collection and expected passing area A collection of passenger vehicles .

[0056] Among them, the forecast demand The area output by the prediction model exist arrive Demand within a time period; total number of profitable paths , refers to the total number of people in all feasible carpooling route solutions requested in area a; the balance difference between supply and demand , refers to the supply and demand relationship of area a, which is calculated by subtracting the regional forecast demand from the number of empty and non-empty supply spaces. The calculation formula is as follows:

[0057] ;

[0058] in, For vehicles The number of vacancies, For vehicles The expected number of coverage areas.

[0059] S4, constructing a two-layer optimization model based on the calculated number of idle vehicles, the difference between supply and demand, the total number of people on the profitable path and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after repositioning.

[0060] In this embodiment, the upper model includes an upper objective function, an upper decision variable and an upper constraint condition; wherein the upper objective function is:

[0061] ;

[0062] ;

[0063] The upper-level decision variables are:

[0064] Represents a 0-1 decision variable. If the value is 1, it indicates a carpooling request. By vehicle service; otherwise the value is 0; where, ;

[0065] Represents a 0-1 decision variable. If the value is 1, it means that the vehicle From point To point otherwise, the value is 0; ;

[0066] in, is the profit at the scheduling time t, which is the revenue brought by the carpooling service request minus the cost of the vehicle's driving path; is the sum of profits at all scheduling time points;

[0067] is a set of requests to be matched, is a set of matched requests, is a set of vehicles currently available on the road network; For scheduling time , the set of all points;

[0068] Indicates the single fare. represents the cost per kilometer of distance; Indicate point The shortest travel time between roads, Indicate point The shortest driving distance between roads, , ;

[0069] Will The set of pick-up and drop-off points for all carpooling requests in is set to ; The set of pick-up and drop-off points for all carpooling requests in is set to ,point Matching vehicle express;

[0070] Order, at the scheduling time When , the set of all points is:

[0071] ;

[0072] in, The exit point of the departure city; The entry point to the destination city;

[0073] The upper level constraints are:

[0074] Define the decision variable value as binary, then:

[0075] ,

[0076] ;

[0077] Ensure that at most one vehicle serves a request:

[0078] ;

[0079] Make sure each vehicle has a starting point and a destination on its route:

[0080] ;

[0081] The path flow conservation constraint ensures that the demand point served by the vehicle must have a point before and after it:

[0082] ;

[0083] To ensure that when a request is served by a vehicle, both the pickup and drop-off points must be served by the same vehicle:

[0084] ,

[0085] ;

[0086] Make sure the match between the request and the vehicle formed in the previous time period cannot be changed:

[0087] .

[0088] In this embodiment, the lower layer model includes a lower layer objective function, lower layer decision variables and lower layer constraints; wherein the expression of the lower layer objective function is:

[0089] ;

[0090] Among them, the potential profit after repositioning The function consists of three factors: the absolute value of the reduction in the difference between supply and demand in each area, the increase in the number of matching demands, and the cost of moving distance for relocation. The first two factors are multiplied by the single fare to obtain the potential revenue, and the moving cost of the third factor is subtracted from it to obtain the potential profit after relocation.

[0091] Take city A as an example. The calculation formula of the function is as follows:

[0092] ;

[0093] in, The potential revenue is obtained by multiplying the absolute value of the reduction in the difference between supply and demand in area a by the single-person ticket price; The potential revenue is obtained by multiplying the increase in the number of matching demands in area a by the single-person ticket price; is the moving distance cost of relocation of area a;

[0094] Similarly, the potential profit after relocation of city B is obtained:

[0095] ;

[0096] The potential revenue is the reduction in the absolute value of the difference between supply and demand in area b multiplied by the single-person ticket price; =The potential revenue obtained by multiplying the increase in the number of matching demands in area b by the single-person ticket price; is the moving distance cost of relocation of area b;

[0097] The lower-level decision variables are:

[0098] Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate the dispatch from the current starting point to the target area The road network centroid point; otherwise the value is 0. ;

[0099] Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate the dispatch from the current starting point to the target area The road network centroid point; otherwise the value is 0. ;

[0100] Among them, the absolute value of the reduction in the supply-demand gap in each area of ​​City A is The calculation formula is as follows:

[0101] ;

[0102] The growth in the number of matching demands The calculation formula is as follows:

[0103] ;

[0104] The moving distance cost of relocation The calculation formula is as follows:

[0105] ;

[0106] The lower level constraints are:

[0107] Ensure that each idle vehicle can only be repositioned once at each decision:

[0108] ;

[0109] ;

[0110] Make sure that the idle vehicle being relocated has already started its next trip and is not resting:

[0111] ;

[0112] in, Indicates vehicle The first +1 drop-off service point; Indicates vehicle Arrival at the destination time;

[0113] Ensure that the number of idle vehicles to be relocated does not exceed the number of round trips on both sides:

[0114] ;

[0115] Ensure that repositioning cannot be performed when an idle vehicle is returning empty after waiting for the maximum waiting time:

[0116] ;

[0117] Ensure that idle vehicles can travel to the relocation area within the predicted time range:

[0118] .

[0119] S5, constructing an independent intelligent agent for each area in city A and city B respectively; wherein the intelligent agent includes a state space, an action space and a reward function.

[0120] In the inter-city dynamic carpooling scenario, each divided area can be regarded as an independent intelligent agent whose task is to collaboratively manage the relocation of idle vehicles within the area to other areas.

[0121] The state space of the agent includes: the number of adjustable idle vehicles in the area, the difference between supply and demand, the total number of people on the profitable path and the moving cost. The state space of each agent covers the state of all areas, aiming to effectively make decisions to move idle vehicles to other appropriate areas. The action task of the agent is to move idle vehicles to other appropriate areas or not to move them, and the numbers of all areas are used as the action space of the agent. The reward function of the agent is designed from three aspects: supply and demand balance, matching path feasibility and moving cost; among them, the steps of constructing the agent of city A are:

[0122] Step 1: In the state space, assume that city A is The agent is represented as , ; Agent At the time of relocation scheduling The state space is represented as ;

[0123] Then, the region The state is represented as:

[0124] ;

[0125] Among them, the area Adjustable number of idle vehicles , which is equal to the cardinality of the set of idle vehicles in the area ; For Region The gap between supply and demand; For Region Total number of profitable paths;

[0126] Then, any area The state space of contains the states of all regions and can be expressed as , Indicates the first |The status of a region.

[0127] Step 2: In terms of action space, let the number set of all areas in city A be:

[0128] ;

[0129] in, Represents the total number of nodes in all areas of city A;

[0130] Agent Action Space It is expressed as:

[0131] ;

[0132] The value is expressed as the value from the current area Move to the area indicated by the arrow number; if the area indicated by the arrow number is the same as the current area It is the same area, which means no movement.

[0133] Step 3: In terms of the reward function, the agent needs to comprehensively consider the current area after relocation in the decision-making process. and target movement area The impact of the supply-demand gap requires considering the change in the total number of people on the profitable path in the region after moving to the new area. This value indirectly reflects the change in the feasibility of the idle vehicle matching path. The agent must also weigh the mobility cost of relocation; therefore, the agent reward function will be designed from three aspects: supply-demand balance, matching path feasibility, and mobility cost. The reward function is positively correlated with the supply-demand balance reward and the matching path feasibility reward, and negatively correlated with the mobility cost reward. Taking city A as an example, the same is true for city B. The expression is as follows:

[0134] ;

[0135] in, Representing an Agent The reward function of For intelligent agents Supply and demand balance rewards; For intelligent agents Matching path feasibility reward; For intelligent agents The moving cost reward.

[0136] The supply and demand balance reward refers to: after each relocation, calculate the current area and target movement area The absolute value of the supply-demand difference is reduced to evaluate the supply-demand situation between the two areas after relocation; the supply-demand balance reward is obtained by multiplying the reduction by the unit fare. The specific calculation formula is as follows:

[0137] ;

[0138] ;

[0139] ;

[0140] in, For the current region The absolute value of the supply-demand gap decreases, Move area to target The absolute value of the reduction in the difference between supply and demand is measured in number of people; The price is for one person.

[0141] Agent When relocating idle vehicles to the target mobility area, it is necessary to consider not only whether supply and demand are balanced, but also whether the idle vehicles are conducive to matching with the carpooling path. Only when the idle vehicles can be better matched with the carpooling path can the relocation be considered an effective adjustment. The total number of people on the regional profitable path can reflect the feasibility of the region in achieving effective carpooling.

[0142] The matching path feasibility reward refers to the change in the total number of people on the regional profitable path after moving to the new area, so as to effectively guide the agent to learn the relocation strategy; and then multiply the change value by the unit fare to obtain the matching path feasibility reward, and its calculation formula is:

[0143] .

[0144] Agent When considering the mobility cost of relocating idle vehicles, there is a difference from the general mobility cost calculation method. In the scenario of intercity carpooling passenger vehicles, the vehicle eventually needs to travel to the intercity exit point. When the target moving area of ​​​​relocation is farther away from the intercity exit point relative to the current area, a detour is required, that is, more distance may need to be traveled. On the contrary, if the target moving area is closer to the intercity exit point, there is no need to take a detour. In order to comprehensively guide the agent to consider the distance and the distance from the intercity exit point when selecting the relocation target, for moving to an area farther away from the intercity exit point, the cost path needs to be multiplied by the corresponding coefficient to offset the subsequent detour route cost.

[0145] The movement cost reward refers to the route cost of the target movement area to be relocated relative to the current area to the intercity exit point, and its calculation formula is as follows:

[0146] ;

[0147] ;

[0148] in, Indicates from point To point The per-kilometer distance cost; K is the ratio of the distance from the target moving area to the intercity exit point to the distance from the current area to the intercity exit point.

[0149] Finally, the intelligent agent was constructed The reward function is positively correlated with the supply-demand balance reward and the matching path feasibility reward, and negatively correlated with the mobility cost reward. The purpose of this design is to optimize the decision-making strategy of the agent to better adapt to the actual needs of inter-city carpooling services. The positively correlated supply-demand balance reward and matching path feasibility reward encourage the agent to pay more attention to the supply-demand balance during the relocation process, while ensuring that idle vehicles are more effectively matched to carpooling paths. The negatively correlated mobility cost reward encourages the agent to choose the relocation target more carefully, reducing the cost of moving vehicles, thereby increasing the platform's profits overall.

[0150] Similarly, construct the intelligent entity of city B.

[0151] S6, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each area to capture the spatial relationship of each area and obtain an approximate optimal Q value function; based on the approximate optimal Q value function, a relocation decision for relocating idle vehicles to other areas is obtained.

[0152] In this embodiment, when constructing the agent of each area, a multi-agent graph deep Q learning network MAGDQN algorithm is adopted. The MAGDQN algorithm introduces a multi-head graph attention network. Each agent has the ability to decide whether to move idle vehicles in its area to any area. There is a mutual capture relationship between the state characteristics of the area to which it belongs and the state characteristics of other areas. An undirected graph is constructed for the area to which each agent belongs and other areas, and the multi-head graph attention network MGAT is used to learn and capture this mutual relationship.

[0153] like Figure 2 As shown in the figure, taking 4 regional nodes as an example, 1, 2, 3, and 4 represent four regional nodes respectively. Each regional node will be affected by the other 3 regional nodes and will also have an impact on the other 3 regional nodes. In this way, the new state space of each region can more accurately reflect its differences and correlations with other regions, thereby better realizing the difference learning of each region, which is used to coordinate the intelligent agents of all regions to relocate idle vehicles on both sides of the city.

[0154] The multi-agent graph deep Q learning network algorithm is based on a multi-head graph attention network, which constructs an undirected graph for each agent's region and other regions. At the same time, the algorithm introduces two identical structural networks: one is the behavior network, used to learn to predict Q values; the other is the target The network is used to calculate the target Q value; the input of the algorithm is the state space composed of all regional states, and the output is the Q value function corresponding to the state action. For example, the specific formula is as follows:

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] in, is the normalized attention weight, Indicates Head Image Attention Network, Indicates the number of heads of the multi-head graph attention network; For intelligent agents At the time of relocation scheduling The state space of For intelligent agents At the time of relocation scheduling The state space, It is a splicing operation. , , is the weight matrix to be learned, is the weight vector to be learned, is the activation function, It is an exponential operation with a natural constant as the base;

[0161] For Region No. New node features of the relationship between regions captured by the head image attention network;

[0162] For Region New node features that capture the interrelationships between regions; For splicing operation;

[0163] is the output hidden state of the first layer linear transformation, For intelligent agents The first layer linearly transforms the weight matrix to be learned, is the bias of the first layer linear transformation, is the activation function;

[0164] is the Q value function corresponding to the state action, For intelligent agents The second layer linearly transforms the weight matrix to be learned, is the bias of the second layer linear transformation.

[0165] Furthermore, when calculating the target Q value of each agent, the average of the target Q values ​​of other agents is added to the target Q value of the current agent to consider the synergistic effect of different agents, and the mean square error loss function is used to measure the gap between the predicted value and the target value of the Q value. The expression is as follows:

[0166] ;

[0167] ;

[0168] ;

[0169] in, Representing an Agent Goal The target Q value of the network, Representing an Agent Goal network, represents the discount factor for future rewards, Indicated in Moment Agent Rewards, Representing an Agent The next state, Representing an Agent Behavior The next action chosen by the network in the next state, Indicates the target The network parameters of the network, Indicates the total number of areas in city A;

[0170] Representing an Agent Behavior network, Indicates choosing the action with the largest Q value in the next state. Expressing behavior Network parameters of the network;

[0171] For intelligent agents The loss gap between the predicted Q value and the target Q value, is the mean square error function.

[0172] Furthermore, in the training process based on the MAGDQN algorithm, collaborative learning of multiple agents is involved. Figure 3 As shown, the learning and training process of the multi-agent graph deep Q learning network algorithm is as follows:

[0173] Initialize the behavior Q network and target Q network of each agent;

[0174] Each agent behaves according to the Q network and Strategy selection action;

[0175] The agent executes the selected action, calculates the reward value through the two-layer optimization model, obtains the next state after executing the action, and converts the current state of the agent into ,action , Reward Value and the next state an experience replay buffer stored in the experience memory unit;

[0176] Randomly select sampled data from the experience replay buffer to update the behavior Q network parameters of each agent ;

[0177] Iteratively update the target Q network parameters of each agent by minimizing the mean square error (MSE) loss function between the behavior Q value and the target Q value. , to improve the stability of the algorithm, when the algorithm reaches the optimal relocation scheduling strategy in the current environment, the approximate optimal Q value function is obtained;

[0178] According to the approximate optimal Q value function, each agent selects the action with the largest Q value in the action space that satisfies the constraints, thereby obtaining a relocation decision.

[0179] S7, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if reached, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and iterate until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization solution that integrates the relocation of vehicles within the city.

[0180] In this embodiment, the present invention integrates three algorithm models of inter-city multi-region demand forecasting, vehicle relocation and inter-city dynamic carpooling, and proposes an inter-city dynamic carpooling optimization process. Each carpooling scheduling moment solves a segmented dynamic carpooling sub-problem, and each relocation scheduling moment optimizes the imbalance between vehicle supply and demand over a period of time. The scheduling intervals of the carpooling scheduling moment and the relocation scheduling moment are inconsistent. The carpooling scheduling focuses on the request matching between the previous scheduling time period and the current scheduling time period, and more frequent scheduling needs need to be provided to meet dynamic carpooling requests. Relocation scheduling focuses on the future supply and demand balance, and the scheduling frequency is determined according to the time span of the predicted future supply and demand situation. This can reduce frequent vehicle movements and reduce costs while ensuring the effectiveness of relocation. The specific process is as follows: Figure 4 As shown:

[0181] A1, initialize the current scheduling time t=1, the current relocation scheduling time =1;

[0182] A2, determine whether the current scheduling time has reached the relocation scheduling interval; if so, continue to A3; otherwise jump to A8:

[0183] A3, based on deep learning prediction algorithm, predicts arrive The demand in the time period is used to obtain the forecast demand;

[0184] A4, calculate the number of idle vehicles, the supply-demand balance difference and the total number of people on the profitable path in each area;

[0185] A5, relocate the idle vehicle based on MAGDON and obtain the relocation decision;

[0186] A6, determine whether all idle vehicles have been decided, if not, jump back to step A5; if yes, continue to A7;

[0187] A7, order relocation scheduling time = +1;

[0188] A8, using multi-round bipartite graph matching MRBGM algorithm for inter-city dynamic carpooling scheduling;

[0189] A9, let the current scheduling time = +1;

[0190] A10, determine whether t reaches the number of carpooling scheduling time sets | |, if not, jump back to step A2, if yes, end the iteration.

[0191] In this embodiment, a multi-round bipartite graph matching (MRBGM) algorithm is a multi-round matching strategy to solve the maximum matching problem of a bipartite graph. A bipartite graph is an undirected graph whose vertex set can be divided into two non-intersecting subsets X and Y. Each edge in the graph connects two vertices belonging to X and Y respectively. Matching is the edge set selected in the bipartite graph so that any two edges have no common endpoints.

[0192] In another preferred embodiment, in order to verify the effectiveness of the method model and model solution proposed by the present invention, the application of the present invention will be explained with a practical example.

[0193] Among them, the round trip between Anxi County and Xiamen City in July 2023 of a certain company's intercity online car-hailing service was selected as the research object, and two identical typical days were selected as research cases. One typical day (i.e., Typical Day 1) was when the number of passenger requests in the two cities was close, while the other typical day (i.e., Typical Day 2) was when the number of passenger requests in one city was significantly greater than that in the other city. In the data sets of Anxi County and Xiamen City, two typical day data were selected for testing, and the remaining data were used for training the MAGDQN algorithm.

[0194] This embodiment adds the multi-agent graph deep Q learning network MAGDQN algorithm to the branch and price BAP (Branch and Price) algorithm and the multi-round bipartite graph matching MRBGM algorithm, and compares the scheduling performance of the two algorithms without relocation scheduling. In addition, the prediction-driven heuristic best relocation BR (Best Relocation) algorithm is also compared. The core idea of ​​this algorithm is to drive the idle vehicles in the area by predicting the demand in each area, and give priority to moving those vehicles that can cover large demand and have low movement times and mileage costs.

[0195] The evaluation index is the sum of profits at all scheduling time points. The specific calculation formula is as follows:

[0196] ;

[0197] ;

[0198] Among them, the profit function of the current scheduling period Profit function with the previous scheduling period The difference between the two measures the profit brought by each scheduling time point. Indicates the single fare. Represents the cost per kilometer of distance. The profit is calculated by subtracting the cost incurred by the vehicle's driving path from the revenue brought by the carpooling service request.

[0199] The parameter settings of the MAGDQN algorithm are shown in Table 1; Table 2 shows the profit comparison results of different algorithms on the round-trip dataset.

[0200] Table 1. Parameter settings of the MAGDQN algorithm

[0201] Parameter Description set up Anxi County and Xiamen City: Number of GDQNs 21 Initial discount factor 0.9 Initial Exploration Factor 1.00 Exploring decay rates 0.995 Minimum Exploration Factor 0.01 Learning Rate 0.001 Input size 3 Figure 2. Number of attention neurons 32 Number of neurons in nonlinear hidden layer 64 Output size 1 Graph Neural Attention Heads 2 Iterations 800

[0202] Table 2. Profit comparison of different methods on the Anxia round-trip dataset

[0203]

[0204] By comparison, it can be found that after the branch-based pricing BAP algorithm and the MRBGM algorithm are added to the repositioning algorithm (i.e., BAP+MAGDQN, MRBGM+MAGDQN), the profits of the algorithms are improved, which verifies the effectiveness of the repositioning algorithm.

[0205] Further comparison of the profit performance of the two relocation algorithms shows that the MAGDQN algorithm proposed in the present invention shows a higher improvement rate under the same scheduling algorithm compared to the heuristic BR algorithm. In addition, it was observed that under the same example, although the improvement rate of the two relocation algorithms combined with the BAP algorithm was higher than that of the MRBGM algorithm, the final profit was lower than the latter. The reason for this phenomenon is that the profit performance of the MRBGM algorithm in dynamic carpooling scheduling is better than that of the BAP algorithm. Although the introduction of the relocation algorithm helps to narrow the gap between the two scheduling algorithms, it does not completely make up for the gap in the scheduling algorithms. However, the addition of the relocation algorithm can still effectively improve the overall profit level of the system.

[0206] like Figure 5 As shown in the figure, the comparison results of the supply and demand difference on the Anxia round-trip dataset are shown. It can be observed that after adding the relocation scheduling algorithm, the absolute difference between supply and demand has been effectively reduced. It should be pointed out that when the imbalanced quantity relationship between supply and demand of vehicles and requests has been determined, the relocation scheduling algorithm can alleviate this uncoordinated relationship, but it cannot completely eliminate this imbalance. On the two round-trip datasets, it can be concluded that the proposed MAGDQN algorithm performs more significantly in reducing the absolute difference between supply and demand than the BR algorithm, providing a more effective solution to alleviate the imbalanced supply and demand problem of online-hailing vehicles.

[0207] like Figure 6 As shown in the figure, the rewards of the MAGDQN algorithm in relocation training are shown. It can be observed that: at the beginning, due to the exploration stage, the reward values ​​obtained by the multi-agent are oscillating. At this stage, the relocation behavior is often random, which helps to accumulate training experience. However, as the number of training increases, the MAGDQN algorithm gradually obtains enough training batches, and by adjusting the exploration decay rate, the relocation behavior is no longer randomly selected, but the relocation is performed by the decision output by the model. In this case, the rewards obtained by the multi-agent gradually increase, while the oscillation amplitude gradually decreases, and finally converges. This verifies the effectiveness of the MAGDQN algorithm training.

[0208] Table 3 summarizes the running time of different relocation algorithms. It can be observed that the running time of the MAGDQN algorithm is shorter than that of the BR algorithm. When the trained MAGDQN is added to the scheduling algorithm, the relocation scheduling decision can be completed in less than 0.1 seconds on average, which takes up almost no time of the original scheduling.

[0209] Table 3. Comparison of running time of relocation algorithms

[0210]

[0211] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0212] The present invention aims at the dynamic problem of inter-city carpooling that integrates vehicle relocation, and establishes a two-level planning model based on inter-city dynamic carpooling and vehicle relocation. The present invention designs the MAGDQN algorithm, which combines multi-region demand forecasts to optimize the relocation decisions of idle vehicles, while considering factors such as supply and demand balance, matching path feasibility, and mobility costs. The present invention proposes an optimization process for inter-city dynamic carpooling that integrates multi-region demand forecasts and vehicle relocation. Through experiments comparing different algorithms without relocation, the scheme proposed by the present invention can enable inter-city dynamic carpooling to obtain higher profits, and can effectively alleviate the problem of imbalance between supply and demand of online ride-hailing vehicles.

[0213] Embodiment 2

[0214] like Figure 7 As shown, the second embodiment of the present invention further provides an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation, comprising:

[0215] A data acquisition unit is used to obtain the road network data of city A and city B on a certain operation route of an intercity online car-hailing service, divide the data into several areas, and obtain the round-trip passenger carpooling travel data in each area in real time;

[0216] A passenger demand prediction unit is used to predict the passenger travel demand of each area of ​​city A and city B in a future period of time at each relocation scheduling time in the relocation scheduling time set, given a scheduling time set and a relocation scheduling time set;

[0217] A supply and demand index calculation unit is used to calculate the number of idle vehicles, the supply and demand balance difference and the total number of people on the profitable path in each area of ​​City A and City B according to the predicted passenger travel demand;

[0218] A two-layer model building unit is used to build a two-layer optimization model based on the calculated number of idle vehicles, the supply-demand gap, the total number of profitable paths and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after relocation;

[0219] A multi-agent construction unit, used to construct an independent agent for each area in city A and city B; wherein the agent includes a state space, an action space and a reward function;

[0220] A vehicle relocation unit is used to obtain a relocation decision, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each area to capture the spatial relationship of each area and obtain an approximate optimal Q value function; based on the approximate optimal Q value function, a relocation decision for an idle vehicle is obtained;

[0221] The dynamic carpooling scheduling unit is used to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if it reaches it, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and it is iterated until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city.

[0222] Embodiment 3

[0223] The third embodiment of the present invention also provides an inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation, which includes a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation as described above.

[0224] Embodiment 4

[0225] The fourth embodiment of the present invention further provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation as described above is implemented.

[0226] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0227] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0228] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0229] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0230] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0231] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0232] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0233] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation, characterized in that: include: S1, after obtaining the road network data of city A and city B on a certain operation route of an intercity online car-hailing service, divide them into several areas respectively, and obtain the round-trip passenger carpooling travel data in each area in real time; S2, given a set of scheduling times and a set of relocation scheduling times, at each relocation scheduling time in the relocation scheduling time set, predict the passenger travel demand of each area of ​​city A and city B in the future period of time through a deep learning algorithm; S3, based on the predicted passenger travel demand, calculate the number of idle vehicles, the supply-demand balance difference and the total number of people on the profitable path in each area of ​​City A and City B respectively; S4, constructing a two-layer optimization model based on the calculated number of idle vehicles, the supply-demand gap, the total number of people on the profitable path and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after relocation; The upper model includes an upper objective function, an upper decision variable and an upper constraint condition; wherein the upper objective function is: ; ; The upper-level decision variables are: Represents a 0-1 decision variable. If the value is 1, it indicates a carpooling request. By vehicle service; otherwise the value is 0; where, ; , is the set of all points in the road networks of the two cities; Represents a 0-1 decision variable. If the value is 1, it means that the vehicle From point To point ; otherwise the value is 0; where, ; is the number of passengers in the passenger data; Among them, the scheduling time set , t is each scheduling time, there are Schedule time; is the profit at the scheduling time t, which is the revenue brought by the carpooling service request minus the cost of the vehicle's driving path; is the sum of profits at all scheduling time points; is a set of requests to be matched, is a set of matched requests, is a set of vehicles currently available on the road network; For scheduling time , the set of all points; Indicates the single fare. represents the cost per kilometer of distance; Indicate point The shortest travel time between the roads Indicate point The shortest road travel distance between Will The set of pick-up and drop-off points for all carpooling requests in is set to ; The set of pick-up and drop-off points for all carpooling requests in is set to ,point Matching vehicle express; is the boarding point in the passenger data, is the alighting point in the passenger data; Order, at the scheduling time When , the set of all points is: ; in, The exit point of the departure city; The entry point to the destination city; For the current starting point, is the current endpoint; For the current location, The location of the next trip; The lower-level model includes a lower-level objective function, lower-level decision variables, and lower-level constraints; wherein the expression of the lower-level objective function is: ; in, To relocate the scheduling time set, Scheduling time for each relocation; is the regional node set of city A, For each area of ​​city A; is the set of regional nodes of city B, and b is each region of city B; Potential profit after relocation The function consists of three factors: the absolute value of the reduction in the difference between supply and demand in each area, the increase in the number of matching demands, and the cost of moving distance for relocation. The first two factors are multiplied by the single fare to obtain the potential revenue, and the moving cost of the third factor is subtracted from it to obtain the potential profit after relocation. Take city A as an example. The calculation formula of the function is as follows: ; in, The potential revenue is obtained by multiplying the absolute value of the reduction in the difference between supply and demand in area a by the single-person ticket price; The potential revenue is obtained by multiplying the increase in the number of matching demands in area a by the single-person ticket price; is the moving distance cost of relocation of area a; Similarly, the potential profit after relocation of city B is obtained: ; The potential revenue is the reduction in the absolute value of the difference between supply and demand in area b multiplied by the single-person ticket price; =The potential revenue obtained by multiplying the increase in the number of matching demands in area b by the single-person ticket price; is the moving distance cost of relocation of area b; The lower-level decision variables are: Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate and dispatch from the current starting point to the target moving area The road network centroid ; otherwise the value is 0, ; Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate the dispatch from the current starting point to the target area The road network centroid ; otherwise the value is 0, ; in, For Region The set of idle vehicles; For Region A collection of idle vehicles; The absolute value of the reduction in the supply-demand gap in each area of ​​City A The calculation formula is as follows: ; in, is the supply and demand balance difference of city A; For vehicles The number of vacancies; The growth in the number of matching demands The calculation formula is as follows: ; in, For Region Total number of profitable paths; For Region Total number of profitable paths; The moving distance cost of relocation The calculation formula is as follows: ; S5, constructing an independent intelligent agent for each area in city A and city B, wherein the intelligent agent includes a state space, an action space, and a reward function; The state space of the agent includes: the number of adjustable idle vehicles in the area, the supply-demand gap, the total number of people on the profitable path and the moving cost; the action task of the agent is to move the idle vehicles to other appropriate areas or not to move, and the numbers of all areas are used as the action space of the agent; the reward function of the agent is designed from three aspects: supply-demand balance, matching path feasibility and moving cost; wherein, the steps of constructing the agent of city A are: Step 1: In the state space, assume that city A is The agent is represented as ; Agent At the time of relocation scheduling The state space is represented as ; Then, the region The state is represented as: ; Among them, the area Adjustable number of idle vehicles , which is equal to the cardinality of the set of idle vehicles in the area ; Then, any area The state space of contains the states of all regions and can be expressed as , Indicates |The status of the region; Step 2: In terms of action space, let the number set of all areas in city A be: , in, Represents the total number of nodes in all areas of city A; Agent Action Space It is expressed as: ; The value is expressed as the value from the current area Move to the area indicated by the arrow number; if the area indicated by the arrow number is the same as the current area It is the same area, which means no movement; Step 3: In terms of reward function, the agent The reward function is positively correlated with the supply-demand balance reward and the matching path feasibility reward, and negatively correlated with the mobile cost reward. The expression is as follows: ; in, Indicated in Moment Agent Rewards For intelligent agents Supply and demand balance rewards; For intelligent agents Matching path feasibility reward; For intelligent agents The mobile cost reward; The supply and demand balance reward refers to: after each relocation, calculate the current area and target movement area The absolute value of the supply-demand difference is reduced to evaluate the supply-demand situation between the two areas after relocation; the supply-demand balance reward is obtained by multiplying the reduction by the unit fare. The specific calculation formula is as follows: ; ; ; in, For the current region The absolute value of the supply-demand gap decreases, Move area to target The absolute value of the reduction in the difference between supply and demand is measured in number of people; The price is for a single person; The matching path feasibility reward refers to the change in the total number of people on the regional profitable path after moving to the new area, so as to effectively guide the agent to learn the relocation strategy; and then multiply the change value by the unit fare to obtain the matching path feasibility reward, and its calculation formula is: ; The movement cost reward refers to the route cost of the target movement area to be relocated relative to the current area to the intercity exit point, and its calculation formula is as follows: ; ; in, Indicates from point To point The cost per kilometer of distance; K is the ratio of the distance from the target moving area to the intercity exit point to the distance from the current area to the intercity exit point; Similarly, construct the intelligent agent of city B; S6, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each region to capture the spatial relationship of each region and obtain an approximate optimal Q value function; according to the approximate optimal Q value function, a relocation decision of the idle vehicle is obtained; S7, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if reached, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and iterate until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization solution that integrates the relocation of vehicles within the city.

2. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 1 is characterized in that ,The real-time passenger carpooling travel data in each area includes passenger data and vehicle data; The passenger data includes: boarding point , Drop-off point , Order time , Boarding time , Service Hours Number of people traveling with you ; Among them, the order time refers to the time when the passenger places a ride-sharing request order; the boarding time refers to the time the passenger expects to board the car; the service time refers to the time it takes for the passenger to complete the boarding at the boarding point; The vehicle data includes: current starting point , Current destination , garage point , Current location , the location of the next trip , Passenger capacity , Number of trips and itinerary path set ;vehicle Number of passes In the itinerary path set Locate your current itinerary path ,vehicle The itinerary path , which starts from , the requested pickup point served to , Departure city exit point , Destination city entry point , the requested drop-off point being served to and end point composition; Each requested boarding time has an earliest boarding time window constraint and the latest boarding time window constraint ,vehicle Time window Arrive at the requested pick-up point within 10 days; Assume that the maximum number of round trips of a vehicle is The shortest rest time of the trip is , the maximum waiting time for the return trip is ; Vehicle Arrival point The time is , , is the set of all points in the road network of the two cities, and the vehicles At the current starting point The departure time is , before any new requests are inserted into the route to the point The time is ,in, A set of matched requests.

3. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 2 is characterized in that , according to the regional node set of city A , then each area of ​​city A Contains: Road network centroids , forecast demand , poor balance between supply and demand , Total number of profit paths , idle vehicle collection and expected passing area Passenger vehicle collection ; Among them, the forecast demand The area output by the prediction model exist arrive Demand within a time period; total number of profitable paths , refers to the total number of people in all feasible carpooling route solutions requested in area a; the balance difference between supply and demand Refers to the supply and demand relationship of area a, which is calculated by subtracting the predicted demand of the area from the number of empty and non-empty supply spaces. The calculation formula is as follows: ; in, For vehicles The number of vacancies, For vehicles The expected number of coverage areas.

4. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 3 is characterized in that , the upper constraint conditions are: Define the decision variable value as binary, then: , ; Ensure that at most one vehicle serves a request: ; Make sure each vehicle has a starting point and a destination on its route: ; The path flow conservation constraint ensures that the demand point served by the vehicle must have a point before and after it: ; To ensure that when a request is served by a vehicle, both the pickup and drop-off points must be served by the same vehicle: , ; Make sure the match between the request and the vehicle formed in the previous time period cannot be changed: 。 5. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 4 is characterized in that , the lower level constraints are: Ensure that each idle vehicle can only be repositioned once at each decision: ; ; Make sure that the idle vehicle being relocated has already started its next trip and is not resting: ; in, Indicates vehicle The first +1 drop-off service point, i.e. vehicle The end point of the travel path; Indicates vehicle Arrival at the destination time; Ensure that the number of trips of the relocated idle vehicle does not exceed the maximum number of round trips of the vehicle: ; Ensure that repositioning cannot be performed when an idle vehicle is returning empty-handed after waiting for the maximum waiting time: ; Ensure that idle vehicles can travel to the relocation area within the predicted time range: 。 6. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 5 is characterized in that The multi-agent graph deep Q learning network algorithm is based on a multi-head graph attention network, which constructs an undirected graph for each agent's region and other regions. The algorithm introduces two identical structural networks: one is the behavior network, used to learn to predict Q values; the other is the target The network is used to calculate the target Q value. The input of the algorithm is the state space composed of all regional states, and the output is the Q value function corresponding to the state action. The specific formula is as follows: ; ; ; ; ; in, is the normalized attention weight, Indicates Head Image Attention Network, Indicates the number of heads of the multi-head graph attention network; For intelligent agents At the time of relocation scheduling The state space of For intelligent agents At the time of relocation scheduling The state space, It is a splicing operation. , , is the weight matrix to be learned, is the weight vector to be learned, , is the activation function, It is an exponential operation with a natural constant as the base; For Region No. New node features of the relationship between regions captured by the head image attention network; For Region New node features that capture the interrelationships between regions; For splicing operation; is the output hidden state of the first layer linear transformation, For intelligent agents The first layer linearly transforms the weight matrix to be learned, is the bias of the first layer linear transformation, is the activation function; is the Q value function corresponding to the state action, For intelligent agents The second layer linearly transforms the weight matrix to be learned, is the bias of the second layer linear transformation.

7. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 6 is characterized in that When calculating the target Q value of each agent, the average of the target Q values ​​of other agents is added to the target Q value of the current agent to consider the synergistic influence of different agents, and the mean square error loss function is used to measure the gap between the predicted value and the target value of the Q value. The expression is as follows: ; ; ; in, Representing an Agent Goal The target Q value of the network, Representing an Agent Goal network, represents the discount factor for future rewards, Indicated in Moment Agent Rewards, Representing an Agent The next state, Representing an Agent Behavior The next action chosen by the network in the next state, Indicates the target The network parameters of the network, Represents the total number of nodes in all areas of city A; Representing an Agent Behavior network, Indicates choosing the action with the largest Q value in the next state. Expressing behavior Network parameters of the network; For intelligent agents The loss gap between the predicted Q value and the target Q value, is the mean square error function.

8. The inter-city dynamic carpooling scheduling optimization method integrating intra-city vehicle relocation according to claim 7 is characterized in that ,The learning and training process of the multi-agent graph deep Q learning network algorithm is as follows: Initialize the behavior Q network and target Q network of each agent; Each agent behaves according to the Q network and Strategy selection action; The agent executes the selected action, calculates the reward value through the two-layer optimization model, obtains the next state after executing the action, and stores the current state, action, reward value and next state of the agent in the experience replay buffer; Randomly select sampled data from the experience replay buffer to update the behavior Q network parameters of each agent; By minimizing the mean square error between the behavior Q value and the target Q value, the target Q network parameters of each agent are iteratively updated to improve the stability of the algorithm. When the algorithm reaches the optimal relocation scheduling strategy in the current environment, an approximate optimal Q value function is obtained. According to the approximate optimal Q value function, each agent selects the action with the largest Q value in the action space that satisfies the constraints, thereby obtaining a relocation decision.

9. An inter-city dynamic carpooling scheduling optimization device integrating intra-city vehicle relocation, characterized in that: include: A data acquisition unit is used to obtain the road network data of city A and city B on a certain operation route of an intercity online car-hailing service, divide the data into several areas, and obtain the round-trip passenger carpooling travel data in each area in real time; A passenger demand prediction unit is used to predict the passenger travel demand of each area of ​​city A and city B in a future period of time at each relocation scheduling time in the relocation scheduling time set, given a scheduling time set and a relocation scheduling time set; A supply and demand index calculation unit is used to calculate the number of idle vehicles, the supply and demand balance difference and the total number of people on the profitable path in each area of ​​City A and City B according to the predicted passenger travel demand; A two-layer model building unit is used to build a two-layer optimization model based on the calculated number of idle vehicles, the supply-demand gap, the total number of profitable paths and the round-trip passenger carpooling travel data; wherein the two-layer optimization model includes an upper model and a lower model; the upper model aims to maximize the sum of profits at all scheduling time points; and the lower model aims to maximize the potential profit after relocation; The upper model includes an upper objective function, an upper decision variable and an upper constraint condition; wherein the upper objective function is: ; ; The upper-level decision variables are: Represents a 0-1 decision variable. If the value is 1, it indicates a carpooling request. By vehicle service; otherwise the value is 0; where, ; , is the set of all points in the road networks of the two cities; Represents a 0-1 decision variable. If the value is 1, it means that the vehicle From point To point ; otherwise the value is 0; where, ; is the number of passengers in the passenger data; Among them, the scheduling time set , t is each scheduling time, there are Schedule time; is the profit at the scheduling time t, which is the revenue brought by the carpooling service request minus the cost of the vehicle's driving path; is the sum of profits at all scheduling time points; is a set of requests to be matched, is a set of matched requests, is a set of vehicles currently available on the road network; For scheduling time , the set of all points; Indicates the single fare. represents the cost per kilometer of distance; Indicate point The shortest travel time between the roads Indicate point The shortest road travel distance between Will The set of pick-up and drop-off points for all carpooling requests in is set to ; The set of pick-up and drop-off points for all carpooling requests in is set to ,point Matching vehicle express; is the boarding point in the passenger data, is the alighting point in the passenger data; Order, at the scheduling time When , the set of all points is: ; in, The exit point of the departure city; The entry point to the destination city; For the current starting point, is the current endpoint; For the current location, The location of the next trip; The lower-level model includes a lower-level objective function, lower-level decision variables, and lower-level constraints; wherein the expression of the lower-level objective function is: ; in, To relocate the scheduling time set, Scheduling time for each relocation; is the regional node set of city A, For each area of ​​city A; is the set of regional nodes of city B, and b is each region of city B; Potential profit after relocation The function consists of three factors: the absolute value of the reduction in the difference between supply and demand in each area, the increase in the number of matching demands, and the cost of moving distance for relocation. The first two factors are multiplied by the single fare to obtain the potential revenue, and the moving cost of the third factor is subtracted from it to obtain the potential profit after relocation. Take city A as an example. The calculation formula of the function is as follows: ; in, The potential revenue is obtained by multiplying the absolute value of the reduction in the difference between supply and demand in area a by the single-person ticket price; The potential revenue is obtained by multiplying the increase in the number of matching demands in area a by the single-person ticket price; is the moving distance cost of relocation of area a; Similarly, the potential profit after relocation of city B is obtained: ; The potential revenue is the reduction in the absolute value of the difference between supply and demand in area b multiplied by the single-person ticket price; =The potential revenue obtained by multiplying the increase in the number of matching demands in area b by the single-person ticket price; is the moving distance cost of relocation of area b; The lower-level decision variables are: Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate and dispatch from the current starting point to the target moving area The road network centroid ; otherwise the value is 0, ; Represents a 0-1 decision variable. If the value is 1, it means the area Idle vehicles Relocate the dispatch from the current starting point to the target area The road network centroid ; otherwise the value is 0, ; in, For Region The set of idle vehicles; For Region A collection of idle vehicles; The absolute value of the reduction in the supply-demand gap in each area of ​​City A The calculation formula is as follows: ; in, is the supply and demand balance difference of city A; For vehicles The number of vacancies; The growth in the number of matching demands The calculation formula is as follows: ; in, For Region Total number of profitable paths; For Region Total number of profitable paths; The moving distance cost of relocation The calculation formula is as follows: ; A multi-agent construction unit, used to construct an independent agent for each area in city A and city B; wherein the agent includes a state space, an action space and a reward function; The state space of the agent includes: the number of adjustable idle vehicles in the area, the supply-demand gap, the total number of people on the profitable path and the moving cost; the action task of the agent is to move the idle vehicles to other appropriate areas or not to move, and the numbers of all areas are used as the action space of the agent; the reward function of the agent is designed from three aspects: supply-demand balance, matching path feasibility and moving cost; wherein, the steps of constructing the agent of city A are: Step 1: In the state space, assume that city A is The agent is represented as ; Agent At the time of relocation scheduling The state space is represented as ; Then, the region The state is represented as: ; Among them, the area Adjustable number of idle vehicles , which is equal to the cardinality of the set of idle vehicles in the area ; Then, any area The state space of contains the states of all regions and can be expressed as , Indicates |The status of the region; Step 2: In terms of action space, let the number set of all areas in city A be: , in, Represents the total number of nodes in all areas of city A; Agent Action Space It is expressed as: ; The value is expressed as the value from the current area Move to the area indicated by the arrow number; if the area indicated by the arrow number is the same as the current area It is the same area, which means no movement; Step 3: In terms of reward function, the agent The reward function is positively correlated with the supply-demand balance reward and the matching path feasibility reward, and negatively correlated with the mobile cost reward. The expression is as follows: ; in, Indicated in Moment Agent Rewards For intelligent agents Supply and demand balance rewards; For intelligent agents Matching path feasibility reward; For intelligent agents The mobile cost reward; The supply and demand balance reward refers to: after each relocation, calculate the current area and target movement area The absolute value of the supply-demand difference is reduced to evaluate the supply-demand situation between the two areas after relocation; the supply-demand balance reward is obtained by multiplying the reduction by the unit fare. The specific calculation formula is as follows: ; ; ; in, For the current region The absolute value of the supply-demand gap decreases, Move area to target The absolute value of the reduction in the difference between supply and demand is measured in number of people; The price is for a single person; The matching path feasibility reward refers to the change in the total number of people on the regional profitable path after moving to the new area, so as to effectively guide the agent to learn the relocation strategy; and then multiply the change value by the unit fare to obtain the matching path feasibility reward, and its calculation formula is: ; The movement cost reward refers to the route cost of the target movement area to be relocated relative to the current area to the intercity exit point, and its calculation formula is as follows: ; ; in, Indicates from point To point The cost per kilometer of distance; K is the ratio of the distance from the target moving area to the intercity exit point to the distance from the current area to the intercity exit point; Similarly, construct the intelligent agent of city B; A vehicle relocation unit is used to obtain a relocation decision, using a multi-agent graph deep Q learning network algorithm to perform differential learning on the agents in each area to capture the spatial relationship of each area and obtain an approximate optimal Q value function; based on the approximate optimal Q value function, a relocation decision for an idle vehicle is obtained; The dynamic carpooling scheduling unit is used to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city, traverse each scheduling time of the scheduling time set, and determine whether the current time reaches the relocation scheduling interval of the relocation scheduling time set; if it reaches it, the passenger travel demand is predicted, and after the relocation decision of the idle vehicle is obtained according to the corresponding intelligent agent, a multi-round bipartite graph matching method is used to perform inter-city dynamic carpooling scheduling; otherwise, a multi-round bipartite graph matching algorithm is directly used for dynamic carpooling scheduling, and it is iterated until all scheduling times are completed to obtain an inter-city dynamic carpooling scheduling optimization plan that integrates the relocation of vehicles within the city.