A global and local collaborative double Q network car-hailing scheduling method

Abstract

The present application relates to a kind of global and local cooperative double Q network car-hailing scheduling method, belong to network car-hailing scheduling technical field.The method includes: first, extract car single mode from historical data and train an environment simulator;Subsequently, grid road network and offline training supply-demand prediction model and area value evaluation model;Second, according to the latest car single distribution of environment simulator real-time upload, execute dynamic decoupling mechanism, real-time rolling correction previous optimal scheduling strategy;Then, the competition intensity between vehicles is calculated, so as to construct bipartite graph and initialize car single decision tree;Finally, under the guidance of supply-demand prediction model and area value evaluation model, bidirectional integration multi-agent Q learning and combination optimization algorithm KM iteratively update car single decision tree, and output final optimal network car-hailing scheduling strategy.The technical problem of structure bias caused by random ordering of bipartite graph and scheduling shortsightedness caused by prediction limitation in prior art is solved.

Description

Technical Field

[0001] This invention relates to a dual-Q ride-hailing dispatching method that combines global and local coordination, belonging to the field of ride-hailing dispatching technology. Background Technology

[0002] In recent years, ride-hailing services have become one of the main modes of transportation for urban residents due to their efficiency and convenience. Designing their scheduling algorithms is a highly challenging research problem. On the one hand, it requires simultaneously considering multiple conflicting indicators such as platform revenue, driver income, and user experience; on the other hand, the temporal dependencies between scheduling decisions and the dynamic changes in the spatial distribution of vehicles and orders further complicate algorithm design. Many researchers have attempted to use combinatorial optimization algorithms (such as the KM algorithm) or methods combining combinatorial optimization algorithms with reinforcement learning to implement ride-hailing scheduling. Although these methods have achieved a certain degree of multi-objective optimization while ensuring a certain level of scheduling efficiency, they still face the following four shortcomings:

[0003] (1) Structural bias caused by random ordering in bipartite graphs: When the number of vehicle orders is large and the candidate space is complex, bipartite graphs often have multiple candidate orderings of nodes, and different orderings correspond to different optimal scheduling solutions. These solutions have little difference in immediate benefits in the current window, but have different impacts on future vehicle order distribution, and further have significant differences in their impact on long-term benefits. Existing studies have ignored this point when constructing bipartite graphs and generally use random ordering, which leads to structural bias in bipartite graphs and outputs suboptimal solutions from a long-term perspective, failing to maximize long-term benefits.

[0004] (2) Conflict between multi-objective optimization and scheduling efficiency: Ride-hailing scheduling needs to simultaneously consider multiple conflicting objectives such as platform revenue, driver income, and passenger experience. Its most commonly used reinforcement learning techniques can provide long-term value modeling, but currently commonly used exploration strategies (e.g., The computational cost is too high, and the time delay is also too large.

[0005] (3) Compatibility between local and global decision-making: The choice of agent architecture will directly determine the performance of the reinforcement learning scheduling model. Complete decentralization will lead to excessive competition among agents, resulting in convergence to a local optimum; complete centralization will face the explosion of state space and action space; semi-centralization has the advantages of both, but still requires coordination between local policies and global scheduling.

[0006] (4) The limitations of offline prediction models lead to short-sighted scheduling: Prediction models trained on historical data can provide supply and demand trends, but they lack the ability to predict random factors such as order upload time, start point, and end point, making the optimal strategy of the preceding window still short-sighted in the latest window. Therefore, an online and effective compensation mechanism is needed to dynamically correct the suboptimal decisions of the preceding window.

[0007] The dynamic game model proposed in patent "CN120975486A, a method and system for online car-hailing dispatch based on dynamic game model" completes the modeling of personalized preferences of drivers and passengers, which can balance the interests of both parties, but ignores the impact of online car-hailing dispatch order on long-term benefits.

[0008] The BiLSTM-GRU-Attention combined prediction model proposed in patent "CN121581350A, Vehicle Dispatch Optimization Method Based on Ride-Hailing Passenger Demand Location and Hotspot Areas" can predict order demand and, based on this, execute K-means clustering, hierarchical clustering, and Greedy-NSGA-II algorithms to complete ride-hailing dispatch, improving the service quality and operational efficiency of the entire ride-hailing system. However, the prediction model has difficulty predicting some random factors (such as order origin and destination), which can lead to short-term limitations in the dispatch strategy, thereby harming long-term benefits. Summary of the Invention

[0009] The purpose of this invention is to provide a dual-Q ride-hailing scheduling method that combines global and local collaboration, aiming to solve the technical problems of structural bias caused by random sorting of bipartite graphs and scheduling shortsightedness caused by prediction limitations in the prior art.

[0010] To achieve the above objectives, the technical solution of the present invention is: a global and local collaborative dual-Q ride-hailing dispatch method, comprising the following steps:

[0011] Step 1: Obtain historical vehicle order data and extract vehicle order patterns from the historical vehicle order data;

[0012] Step 2: Based on the vehicle order model, perform offline environment modeling and build an environment simulator;

[0013] Step 3: Divide the road network into several grids of equal size, assign a unique ID to each grid, and map the vehicle and the order to different grids based on the real-time location of the vehicle and the origin and destination of the order.

[0014] Step 4: After mapping vehicles and orders to different grids, use Temporal Residual Network (TRN) to train the regional value assessment model and the supply and demand prediction model offline on historical vehicle order data with grid IDs; wherein, the regional value assessment model is used to evaluate local scheduling strategies, and the supply and demand prediction model is used to evaluate global scheduling strategies.

[0015] Step 5: After obtaining the latest vehicle order data from the environment simulator in real time, a dynamic decoupling mechanism is executed to continuously correct the previous suboptimal scheduling strategy in real time. A circular search area is constructed for each idle vehicle, and the orders in the circular search area constitute the candidate order set for each vehicle. The competitive intensity of different vehicles with other vehicles in all orders is calculated.

[0016] Step 6: Sort the available vehicles in ascending order of competition intensity, and use the sorted available vehicles as nodes on one side and waiting orders as nodes on the other side to construct a bipartite graph; where the weight of the connection between the available vehicle node and the waiting order node represents the value of matching the current available vehicle with the waiting order.

[0017] Step 7: Initialize the Vehicle Order Decision Tree (VODT); wherein, the initialization is to insert idle vehicles into each level of the VODT as nodes in order of increasing competition intensity, the depth of the tree is equal to the number of idle vehicles, the branches of the nodes represent the candidate orders of the current vehicle, the number of branches is equal to the number of candidate orders, and the vehicle order decision tree is traversed downward from the root node to generate different paths, corresponding to different global scheduling strategies and global Q values, and each node and corresponding branch in the path represents a local scheduling strategy and corresponding local Q value;

[0018] Step 8: Iteratively update the VODT using the MAQL model, and output the optimal ride-hailing dispatch strategy for the current iteration round based on the bipartite graph and the KM algorithm using the updated VODT. The iteration stops when the optimal ride-hailing dispatch strategy output by the KM algorithm converges, and the final optimal ride-hailing dispatch strategy is output. The MAQL model is a dual-Q model, consisting of a local Q model and a global Q model. The MAQL model updates the VODT iteratively under the guidance of a regional value assessment model and a supply and demand forecasting model. Specifically:

[0019] The local Q-model evaluates the value of local scheduling strategies and updates the local Q-value of VODT under the guidance of the regional value assessment model, while the global Q-model evaluates the value of global scheduling strategies and updates the global Q-value of VODT under the guidance of the supply and demand forecasting model.

[0020] Optionally, the offline training of the regional value assessment model and the supply and demand prediction model using a temporal residual network (TRN) on historical vehicle order data with grid IDs specifically involves:

[0021]

[0022]

[0023] in, and These are supply and demand forecasting models and regional valuation models, respectively. and For the training function, Used to encapsulate the training and optimization process of TRN for supply and demand forecasting models. Used to encapsulate the training and optimization process of TRN for regional value assessment models. and These are real-time vehicle datasets and order datasets generated by an environment simulator, respectively. Representing the A time window, and This is the set of learnable parameters for TRN.

[0024] Optionally, the implementation of the dynamic decoupling mechanism to continuously and dynamically correct the preceding suboptimal scheduling strategy specifically involves:

[0025] First, the dynamic decoupling mechanism is defined as follows: when a new vehicle order arrives, the dispatching platform not only needs to match the vehicle orders in the current time window, but also needs to re-include the previously matched vehicle orders into the matching space, and perform rolling correction on the suboptimal decisions of the previous order to achieve continuous optimization of the continuous window.

[0026] Then, the triggering condition for the dynamic decoupling mechanism is defined as follows:

[0027] Condition 1: The vehicle is currently matched with an order and has not yet reached the starting point of the matched order;

[0028] Condition 2: The new order is located in the vehicle's candidate region;

[0029] Condition 3: The weighted sum of the immediate profit and potential future revenue of the new order is greater than that of the currently matched orders;

[0030] Condition 4: The remaining tolerance time of the matched orders is greater than or equal to the remaining tolerance time of the new orders;

[0031] If all four conditions are met, the dynamic decoupling mechanism is triggered.

[0032] Finally, for new orders that have triggered the dynamic decoupling mechanism and already matched vehicle orders, the following steps are executed sequentially:

[0033] Action 1: Decouple matched vehicle orders;

[0034] Action 2: Match vehicles with new orders;

[0035] Action 3: Update the status of the old order to "waiting" and re-enter the scheduling process in the next time window.

[0036] Optionally, the competition intensity specifically refers to:

[0037]

[0038]

[0039] in, Indicates vehicle For orders The intensity of competition, Indicates vehicle Competition intensity for all orders; Indicates vehicle The set of candidate orders Indicates order The candidate vehicle set, Representing the A time window, A function to calculate the length of a set. This indicates the total number of orders. This indicates the total number of vehicles.

[0040] Optionally, Step 8 specifically includes:

[0041] Step 8.1: Treat each idle vehicle as an independent intelligent agent A local scheduling strategy is formulated using the KM algorithm. The scheduling platform will be used as the central intelligent agent. The KM algorithm is used to aggregate all local scheduling policies into a single global scheduling policy. ;

[0042] Step 8.2: Calculate the local scheduling strategy Instant rewards The expression is:

[0043]

[0044] in, Indicates the first An independent intelligent agent Complete the order Duration required Indicates the discount factor. Indicates the profit per vehicle order;

[0045] The initial local Q-value of the VODT branch, obtained from the immediate reward, is expressed as:

[0046]

[0047] in, Branches representing VODT The initial local Q value, Indicates the grid of the order's starting point. Real-time value assessment Indicates the order endpoint grid Future value assessment at the end of the service period Representing the One time window;

[0048] Using the local Q model in the MAQL model in the regional value assessment model Evaluate local scheduling strategies under the guidance of The value of VODT is updated and its local Q-value is determined by the expression:

[0049]

[0050] in, Represents a local Q-model. and They represent the first In this iteration, the branches in VODT and The local Q value, and As weight, and , Denotes the balance factor, where, This is the attenuation coefficient, used to control the convergence rate of the balance factor. Indicates branch Number of explorations Indicates the maximum number of iterations;

[0051] Step 8.3: Applying the global Q-model in the MAQL model to the supply and demand forecasting model Evaluate global scheduling strategies under guidance The value of VODT is updated and its global Q value is determined by the expression:

[0052]

[0053] in, Represents the global Q-model. This indicates the supply-demand ratio for the current time window, where the supply-demand ratio is the ratio of available vehicles to waiting orders. This represents the supply and demand ratio for the next time step, as provided by the supply and demand forecasting model.

[0054] The beneficial effects of this invention are:

[0055] (1) This invention proposes the concept of multi-agent competition intensity, which is used to quantify vehicle scheduling priority and construct vehicle-to-decision tree (VODT), thereby providing a structured policy space, alleviating the structural bias caused by multiple optimal solutions in bipartite graphs and accelerating policy convergence.

[0056] (2) This invention integrates MAQL and the combinatorial optimization algorithm KM in both directions. MAQL continuously updates the decision tree VODT under the guidance of the offline model, while KM searches for the optimal strategy in the tree and updates MAQL in reverse, forming an iterative "update-explore-update" closed-loop collaborative process, which achieves multi-objective optimization while ensuring online scheduling efficiency.

[0057] (3) The present invention designs a dynamic decoupling mechanism (DDM) to compensate for the limitations of the prediction model from the perspective of real-time calculation and realize the rolling correction of the previous suboptimal matching. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the technical process of the present invention.

[0059] Figure 2 This is a technical roadmap for the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0061] Example 1: As Figure 1 and Figure 2 The diagram shown is a technical flowchart and roadmap for a globally and locally collaborative dual-Q ride-hailing dispatching method (GLDQL). Figure 2 In this embodiment, the offline model training phase includes acquiring historical vehicle order data, extracting vehicle order patterns, gridding the road network, and training the offline model, corresponding to Steps 1-4. The vehicle order decision tree initialization phase includes generating and uploading vehicle order data in real time using the environment simulator, executing a dynamic decoupling mechanism based on the real-time data, and calculating the competition intensity for idle vehicles, thereby constructing a bipartite graph and initializing the vehicle order decision tree (VODT), corresponding to Steps 5-7. Finally, the MAQL and KM algorithms are bidirectionally integrated to iteratively update the VODT and output the final optimal ride-hailing dispatch strategy, corresponding to Step 8. Specifically, it includes the following steps:

[0062] Step 1: Obtain historical vehicle order data and extract vehicle order patterns from the historical vehicle order data;

[0063] Optionally, in this embodiment, the extracted vehicle order mode includes order upload mode, order cancellation mode, and ride-hailing operation mode;

[0064] Step 2: Based on the vehicle order model, perform offline environment modeling and build an environment simulator;

[0065] It is understandable that this embodiment constructs an environment simulator to generate and upload the latest vehicle order data in real time, serving as the data foundation for subsequent online dispatching of ride-hailing vehicles. Compared to simply replaying ride-hailing and order data, the environment simulator additionally obtains information such as vehicle online / offline time / location and order cancellation time from the vehicle order mode.

[0066] Step 3: Divide the road network into several grids of equal size, assign a unique ID to each grid, and map the vehicle and the order to different grids based on the real-time location of the vehicle and the origin and destination of the order.

[0067] Step 4: After mapping vehicles and orders to different grids, use Temporal Residual Network (TRN) to train the regional value assessment model and the supply and demand prediction model offline on historical vehicle order data with grid IDs; wherein, the regional value assessment model is used to evaluate local scheduling strategies, and the supply and demand prediction model is used to evaluate global scheduling strategies.

[0068] Optionally, the offline training of the regional value assessment model and the supply and demand prediction model using a temporal residual network (TRN) on historical vehicle order data with grid IDs specifically involves:

[0069]

[0070]

[0071] in, and These are supply and demand forecasting models and regional valuation models, respectively. and For the training function, Used to encapsulate the training and optimization process of TRN for supply and demand forecasting models. Used to encapsulate the training and optimization process of TRN for regional value assessment models. and These are real-time vehicle datasets and order datasets generated by an environment simulator, respectively. Representing the A time window, and This is the set of learnable parameters for TRN.

[0072] It should be understood that the regional value assessment model and the supply and demand forecasting model constructed in this embodiment are both based on the TRN model obtained by the Temporal Residual Network (TRN). Due to differences in the training process, input and output, the regional value assessment model and the supply and demand forecasting model were obtained.

[0073] Step 5: After obtaining the latest vehicle order data from the environment simulator in real time, a dynamic decoupling mechanism is executed to continuously correct the previous suboptimal scheduling strategy in real time. A circular search area is constructed for each idle vehicle, and the orders in the circular search area constitute the candidate order set for each vehicle. The competitive intensity of different vehicles with other vehicles in all orders is calculated.

[0074] It is important to understand that the core idea of ​​the Dynamic Decoupling Mechanism (DDM) constructed in this embodiment is: when a new vehicle order arrives, the dispatching platform not only needs to match the vehicle orders in the current time window, but also needs to re-include the previously matched vehicle orders into the matching space, perform rolling correction on the suboptimal decisions of the previous order, and realize continuous optimization of the continuous window.

[0075] Optionally, the implementation of the dynamic decoupling mechanism to continuously and dynamically correct the preceding suboptimal scheduling strategy specifically involves:

[0076] First, the dynamic decoupling mechanism is defined as follows: when a new vehicle order arrives, the dispatching platform not only needs to match the vehicle orders in the current time window, but also needs to re-include the previously matched vehicle orders into the matching space, and perform rolling correction on the suboptimal decisions of the previous order to achieve continuous optimization of the continuous window.

[0077] Then, the triggering condition for the dynamic decoupling mechanism is defined as follows:

[0078] Condition 1: The vehicle is currently matched with an order and has not yet reached the starting point of the matched order;

[0079] Condition 2: The new order is located in the vehicle's candidate region;

[0080] Condition 3: The weighted sum of the immediate profit and potential future revenue of the new order is greater than that of the currently matched orders;

[0081] Condition 4: The remaining tolerance time of the matched orders is greater than or equal to the remaining tolerance time of the new orders;

[0082] If all four conditions are met, the dynamic decoupling mechanism is triggered.

[0083] Understandably, this embodiment avoids impacting orders that are currently being served by defining strict DDM trigger conditions, ensuring that decoupling behavior can improve the revenue of both the platform and drivers and prevent the original orders from being canceled more easily due to decoupling, thereby controlling the platform's order cancellation rate.

[0084] Finally, for new orders that have triggered the dynamic decoupling mechanism and already matched vehicle orders, the following steps are executed sequentially:

[0085] Action 1: Decouple matched vehicle orders;

[0086] Action 2: Match vehicles with new orders;

[0087] Action 3: Update the status of the old order to "waiting" and re-enter the scheduling process in the next time window.

[0088] Optionally, the competition intensity specifically refers to:

[0089]

[0090]

[0091] in, Indicates vehicle For orders The intensity of competition, Indicates vehicle Competition intensity for all orders; Indicates vehicle The set of candidate orders Indicates order The candidate vehicle set, Representing the A time window, A function to calculate the length of a set. This indicates the total number of orders. This indicates the total number of vehicles.

[0092] It is understandable that the competition intensity metric constructed in this embodiment reflects two basic principles: first, the more candidate orders there are, the less competitive pressure a vehicle faces on a single order; second, if an order can be served by multiple vehicles, then the competition intensity of that order on each vehicle is greater.

[0093] It is understandable that this embodiment constructs a dynamic decoupling mechanism (DDM) to continuously correct the preceding suboptimal scheduling strategy in real time, thereby compensating for the scheduling shortsightedness caused by the prediction range of the prediction model from the perspective of real-time computing, and breaking the inherent assumption of the traditional scheduling algorithm that "the preceding decision cannot be modified".

[0094] Step 6: Sort the available vehicles in ascending order of competition intensity, and use the sorted available vehicles as nodes on one side and waiting orders as nodes on the other side to construct a bipartite graph; where the weight of the connection between the available vehicle node and the waiting order node represents the value of matching the current available vehicle with the waiting order.

[0095] Step 7: Initialize the Vehicle Order Decision Tree (VODT); wherein, the initialization is to insert idle vehicles into each level of the VODT as nodes in order of increasing competition intensity, the depth of the tree is equal to the number of idle vehicles, the branches of the nodes represent the candidate orders of the current vehicle, the number of branches is equal to the number of candidate orders, and the vehicle order decision tree is traversed downward from the root node to generate different paths, corresponding to different global scheduling strategies and global Q values, and each node and corresponding branch in the path represents a local scheduling strategy and corresponding local Q value;

[0096] Understandably, while bipartite graphs store different local and global scheduling strategies and can record the local Q-values ​​of local scheduling strategies through the weights of connections between nodes, they are not a strictly defined data structure and therefore cannot record the global Q-values ​​of global scheduling strategies. Therefore, this embodiment introduces a VODT (Variable Target Dataset) to organize the data structure in the form of a multi-way tree, thereby enabling the recording of the global Q-values ​​of global scheduling strategies.

[0097] Step 8: Iteratively update the VODT using the MAQL model, and output the optimal ride-hailing dispatch strategy for the current iteration round based on the bipartite graph and the KM algorithm using the updated VODT. The iteration stops when the optimal ride-hailing dispatch strategy output by the KM algorithm converges, and the final optimal ride-hailing dispatch strategy is output. The MAQL model is a dual-Q model, consisting of a local Q model and a global Q model. The MAQL model updates the VODT iteratively under the guidance of a regional value assessment model and a supply and demand forecasting model. Specifically:

[0098] The local Q-model evaluates the value of local scheduling strategies and updates the local Q-value of VODT under the guidance of the regional value assessment model, while the global Q-model evaluates the value of global scheduling strategies and updates the global Q-value of VODT under the guidance of the supply and demand forecasting model.

[0099] Step 8.1: Adopt a semi-centralized architecture: Treat each idle vehicle as an independent intelligent agent. A local scheduling strategy is formulated using the KM algorithm. The scheduling platform will be used as the central intelligent agent. The KM algorithm is used to aggregate all local scheduling policies into a single global scheduling policy. ;

[0100] Step 8.2: Calculate the local scheduling strategy Instant rewards The expression is:

[0101]

[0102] in, Indicates the first An independent intelligent agent Complete the order Duration required Indicates the discount factor. Indicates profit per vehicle order; instant rewards. This not only reflects the impact of service duration on revenue in vehicle-order matching, but also provides time-sensitive feedback signals for value assessment in reinforcement learning.

[0103] The initial local Q-value of the VODT branch, obtained from the immediate reward, is expressed as:

[0104]

[0105] in, Branches representing VODT The initial local Q value, Indicates the grid of the order's starting point. Real-time value assessment Indicates the order endpoint grid Future value assessment at the end of the service period Representing the A time window; specifically, both immediate and future value assessments are conducted using a regional value assessment model. get;

[0106] It is understandable that this embodiment ensures that the local Q value not only reflects immediate benefits but also explicitly reflects future regional value by initializing the local Q value, thereby guiding vehicles to move to high-value areas.

[0107] Using the local Q model in the MAQL model in the regional value assessment model Evaluate local scheduling strategies under the guidance of The value of VODT is updated and its local Q-value is determined by the expression:

[0108]

[0109] in, Represents a local Q-model. and They represent the first In this iteration, the branches in VODT and The local Q-value reflects the competitive equilibrium mechanism. and As weight, and , Denotes the balance factor, where, This is the attenuation coefficient, used to control the convergence rate of the balance factor. Indicates branch Number of explorations Indicates the maximum number of iterations;

[0110] It is understood that this embodiment uses... The update enables the strategy to maintain stability while making full use of the strategy feedback provided by the combinatorial optimization algorithm, so that the local Q value can quickly converge to the optimal interval within a finite number of iterations.

[0111] Step 8.3: Applying the global Q-model in the MAQL model to the supply and demand forecasting model Evaluate global scheduling strategies under guidance The value of VODT is updated and its global Q value is determined by the expression:

[0112]

[0113] in, Represents the global Q-model. This indicates the supply-demand ratio for the current time window, where the supply-demand ratio is the ratio of available vehicles to waiting orders. This represents the supply and demand ratio for the next time point, as provided by the supply and demand forecasting model. This is a discount factor used to balance the importance of current supply and demand with future supply and demand.

[0114] Optionally, in this embodiment, the step of the KM algorithm outputting the optimal ride-hailing scheduling strategy converges specifically means that if the same optimal ride-hailing scheduling strategy is output in two consecutive iterations, it is determined that the strategy has converged.

[0115] It is understood that this embodiment bidirectionally integrates multi-agent Q-learning (MAQL) and combinatorial optimization algorithm KM. The MAQL model updates the VODT, and the KM algorithm outputs the current optimal ride-hailing scheduling strategy based on the current VODT. The output optimal ride-hailing scheduling strategy is used as a feedback signal to update the MAQL model, ultimately forming an iterative closed-loop process of update-exploration-update.

[0116] Furthermore, the technical solution of the present invention will be further illustrated below through a specific implementation example.

[0117] S1: Obtain historical data of 10,000 ride-hailing vehicles and 20,400,000 orders in Shenzhen, and extract the following vehicle-order patterns: (i) Order upload pattern: upload time, origin, destination, and price for each order. (ii) Order cancellation pattern: maximum tolerable waiting time for each order; orders are automatically cancelled after this time. (iii) Ride-hailing operation pattern: online time, online location, offline time, offline location, and total revenue during the operating time for each ride-hailing vehicle;

[0118] S2: Based on the vehicle order model, offline environment modeling is performed to build an environment simulator for generating and uploading the latest vehicle order data in real time: the latest online vehicles (including online time and online location) and the latest uploaded orders (including upload time, origin, destination, price and maximum tolerable waiting time).

[0119] S3: Divide the road network into 100 equally sized grids. Each grid is assigned a unique ID from 0 to 99. Based on the real-time location of the vehicle and the origin and destination of the order, map the vehicle and the order to different grids. For example... At any time, ride-hailing In the grid ;Order The starting point is the grid. The endpoint is the grid. ;

[0120] S4: Using Temporal Residual Network (TRN), offline training is conducted on regional value assessment and supply-demand prediction models based on large-scale historical vehicle order data.

[0121] S5: At that moment, there were 3 pending orders. ~ 1 available vehicle and a matching pair ( , ). At that moment, the environment simulator uploaded a new order. Two new vehicles were added. , .Discover Triggered ( , The decoupling conditions for (i) are: Matched And has not yet arrived. (ii) The starting point; lie in (iii) in the candidate region; The weighted sum of immediate profits and future potential earnings is greater than (iv) The remaining tolerance time is greater than or equal to The remaining tolerance time. Entering the decoupling phase: (i) and Decoupling; (ii) and Match; (iii) will The status is marked as waiting, and it will participate in subsequent scheduling.

[0122] With each idle vehicle , , Three circular search areas were constructed with a center of 4 kilometers and a radius of 4 kilometers. The candidate order set is , The candidate order set is , The candidate order set is .calculate and Competition intensity , , .

[0123] S6: Sort idle vehicles in ascending order of competition intensity. , , Treat it as a node on one side. Construct a bipartite graph using nodes on the other side.

[0124] S7: When initializing the Vehicle Order Decision Tree (VODT): Order the vehicles according to their competition strength from smallest to largest. , , Insert nodes sequentially into levels 0, 1, and 2 of the VODT, resulting in a tree depth of 3. Level 0 node. There is one branch ; Level 1 node There are 3 branches ; Level 2 nodes There are 2 branches There are 6 global scheduling strategies: , , , , , However, based on the principle of one-to-one vehicle order matching, there are only two effective global scheduling strategies in practice. and For each different global Q value, there are three corresponding local scheduling policies. as well as Corresponding to different local Q values.

[0125] S8: In GLDQL, MAQL iteratively updates the global and local Q-values ​​in VODT. KM outputs the current optimal ride-hailing scheduling strategy in each iteration and uses this strategy to update the MAQL model. When the optimal ride-hailing scheduling strategy output by the KM algorithm converges, the iteration stops, and the final optimal ride-hailing scheduling strategy is output. .

[0126] In summary, this invention first acquires historical vehicle order data, extracts vehicle order patterns, and constructs an environment simulator based on these patterns. Then, it grids the road network and assigns grid indices to vehicles and orders during the online scheduling phase. Next, based on historical data, it uses a Temporal Residual Network (TRN) to train a regional value assessment model and a supply-demand prediction model offline. Then, after the environment simulator generates and uploads the latest vehicle order data in real time, it executes a Dynamic Decoupling Mechanism (DDM) and calculates the competition intensity for idle vehicles, thereby constructing a bipartite graph and initializing a Vehicle Order Decision Tree (VODT). Finally, GLDQL integrates Multi-Agent Q-Learning (MAQL) and the Combinatorial Optimization (KM) algorithm to iteratively update the VODT, ultimately outputting the optimal ride-hailing scheduling strategy. Specifically, MAQL is a dual-Q model, updating the VODT through local and global Q-models under the guidance of the regional value assessment model and the supply-demand prediction model; the KM algorithm outputs the current optimal ride-hailing scheduling strategy based on the current VODT, serving as a feedback signal to update the MAQL model, forming an iterative closed-loop process of update-exploration-update. Compared to existing technologies, this invention addresses the real-time scheduling needs of large-scale ride-hailing scenarios by designing an environment simulator, an offline model, a dynamic decoupling mechanism, a vehicle-order decision tree, and a GLDQL model that integrates bidirectional multi-agent Q-learning and combinatorial optimization algorithm KM, thereby ensuring scheduling efficiency.

[0127] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A dual-Q ride-hailing dispatching method with global and local coordination, characterized in that, The method includes the following steps: Step 1: Obtain historical vehicle order data and extract vehicle order patterns from the historical vehicle order data; Step 2: Based on the vehicle order model, perform offline environment modeling and build an environment simulator; Step 3: Divide the road network into several grids of equal size, assign a unique ID to each grid, and map the vehicle and the order to different grids based on the real-time location of the vehicle and the origin and destination of the order. Step 4: After mapping vehicles and orders to different grids, use Temporal Residual Network (TRN) to train the regional value assessment model and the supply and demand prediction model offline on historical vehicle order data with grid IDs; wherein, the regional value assessment model is used to evaluate local scheduling strategies, and the supply and demand prediction model is used to evaluate global scheduling strategies. Step 5: After obtaining the latest vehicle order data from the environment simulator in real time, a dynamic decoupling mechanism is executed to continuously correct the previous suboptimal scheduling strategy in real time. A circular search area is constructed for each idle vehicle, and the orders in the circular search area constitute the candidate order set for each vehicle. The competitive intensity of different vehicles with other vehicles in all orders is calculated. Step 6: Sort the available vehicles in ascending order of competition intensity, and use the sorted available vehicles as nodes on one side and waiting orders as nodes on the other side to construct a bipartite graph; where the weight of the connection between the available vehicle node and the waiting order node represents the value of matching the current available vehicle with the waiting order. Step 7: Initialize the Vehicle Order Decision Tree (VODT); wherein, the initialization is to insert idle vehicles into each level of the VODT as nodes in order of increasing competition intensity, the depth of the tree is equal to the number of idle vehicles, the branches of the nodes represent the candidate orders of the current vehicle, the number of branches is equal to the number of candidate orders, and the vehicle order decision tree is traversed downward from the root node to generate different paths, corresponding to different global scheduling strategies and global Q values, and each node and corresponding branch in the path represents a local scheduling strategy and corresponding local Q value; Step 8: Iteratively update the VODT using the MAQL model, and output the optimal ride-hailing dispatch strategy for the current iteration round based on the bipartite graph and the KM algorithm using the updated VODT. The iteration stops when the optimal ride-hailing dispatch strategy output by the KM algorithm converges, and the final optimal ride-hailing dispatch strategy is output. The MAQL model is a dual-Q model, consisting of a local Q model and a global Q model. The MAQL model updates the VODT iteratively under the guidance of a regional value assessment model and a supply and demand forecasting model. Specifically: The local Q-model evaluates the value of local scheduling strategies and updates the local Q-value of VODT under the guidance of the regional value assessment model, while the global Q-model evaluates the value of global scheduling strategies and updates the global Q-value of VODT under the guidance of the supply and demand forecasting model.

2. The dual-Q ride-hailing dispatching method with global and local coordination according to claim 1, characterized in that, The specific steps involved in using a Temporal Residual Network (TRN) to offline train regional value assessment and supply-demand prediction models on historical vehicle order data with grid IDs are as follows: ； ； in, and These are supply and demand forecasting models and regional valuation models, respectively. and For the training function, Used to encapsulate the training and optimization process of TRN for supply and demand forecasting models. Used to encapsulate the training and optimization process of TRN for regional value assessment models. and These are real-time vehicle datasets and order datasets generated by an environment simulator, respectively. Representing the A time window, and This is the set of learnable parameters for TRN.

3. The dual-Q ride-hailing dispatching method with global and local coordination according to claim 1, characterized in that, The aforementioned dynamic decoupling mechanism, which continuously and dynamically corrects the pre-order suboptimal scheduling strategy, specifically involves: First, the dynamic decoupling mechanism is defined as follows: when a new vehicle order arrives, the dispatching platform not only needs to match the vehicle orders in the current time window, but also needs to re-include the previously matched vehicle orders into the matching space, and perform rolling correction on the suboptimal decisions of the previous order to achieve continuous optimization of the continuous window. Then, the triggering condition for the dynamic decoupling mechanism is defined as follows: Condition 1: The vehicle is currently matched with an order and has not yet reached the starting point of the matched order; Condition 2: The new order is located in the vehicle's candidate region; Condition 3: The weighted sum of the immediate profit and potential future revenue of the new order is greater than that of the currently matched orders; Condition 4: The remaining tolerance time of the matched orders is greater than or equal to the remaining tolerance time of the new orders; If all four conditions are met, the dynamic decoupling mechanism is triggered. Finally, for new orders that have triggered the dynamic decoupling mechanism and already matched vehicle orders, the following steps are executed sequentially: Action 1: Decouple matched vehicle orders; Action 2: Match vehicles with new orders; Action 3: Update the status of the old order to "waiting" and re-enter the scheduling process in the next time window.

4. The dual-Q ride-hailing dispatching method with global and local coordination according to claim 1, characterized in that, The specific competitive intensity is as follows: ； ； in, Indicates vehicle For orders The intensity of competition, Indicates vehicle Competition intensity for all orders; Indicates vehicle The set of candidate orders Indicates order The candidate vehicle set, Representing the A time window, A function to calculate the length of a set. This indicates the total number of orders. This indicates the total number of vehicles.

5. The dual-Q ride-hailing dispatching method with global and local coordination according to claim 1, characterized in that, Step 8 specifically refers to: Step 8.1: Treat each idle vehicle as an independent intelligent agent A local scheduling strategy is formulated using the KM algorithm. The scheduling platform will be used as the central intelligent agent. The KM algorithm is used to aggregate all local scheduling policies into a single global scheduling policy. ; Step 8.2: Calculate the local scheduling strategy Instant rewards The expression is: ； in, Indicates the first An independent intelligent agent Complete the order Duration required Indicates the discount factor. Indicates the profit per vehicle order; The initial local Q-value of the VODT branch, obtained from the immediate reward, is expressed as: ； in, Branches representing VODT The initial local Q value, Indicates the grid of the order's starting point. Real-time value assessment Indicates the order endpoint grid Future value assessment at the end of the service period Representing the One time window; Using the local Q model in the MAQL model in the regional value assessment model Evaluate local scheduling strategies under the guidance of The value of VODT is updated and its local Q-value is determined by the expression: ； in, Represents a local Q-model. and They represent the first In this iteration, the branches in VODT and The local Q value, and As weight, and , Denotes the balance factor, where, This is the attenuation coefficient, used to control the convergence rate of the balance factor. Indicates branch Number of explorations Indicates the maximum number of iterations; Step 8.3: Applying the global Q-model in the MAQL model to the supply and demand forecasting model Evaluate global scheduling strategies under guidance The value of VODT is updated and its global Q value is determined by the expression: ； in, Represents the global Q-model. This indicates the supply-demand ratio for the current time window, where the supply-demand ratio is the ratio of available vehicles to waiting orders. This represents the supply and demand ratio for the next time step, as provided by the supply and demand forecasting model.

Images