Reinforcement Learning Carpool Dispatching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ride-hailing platforms face inefficiencies due to limited vehicle supply and asymmetric passenger demand, which reinforcement learning technologies struggle to address effectively, especially in scaling decisions involving thousands of driver-order pairs within milliseconds, while also considering spatial-temporal effects.
Innovation Solution
A cloud-based system using a trained machine learning model, specifically a reinforcement learning agent, to optimize carpool dispatching by obtaining utility scores for candidate carpool combinations based on vehicle and order data, including static features and historical data, to match vehicles with optimal carpool combinations, thereby improving supply-demand balance and reducing wait times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reinforcement learning is used to balance supply and demand in ride-hailing systems, then supply-demand balance is improved, but the system cannot effectively handle large-scale decisions involving thousands of driver-order pairs within milliseconds
Solution Approach 1:
The patent segments the ride-hailing dispatch problem into two hierarchical levels: (1) strategic supply-demand balancing using reinforcement learning over longer time horizons, and (2) tactical real-time matching using optimized algorithms for immediate execution. This segmentation allows the RL component to focus on policy learning while the matching component handles rapid decision-making, resolving the contradiction between reliability improvement and productivity maintenance.
2Quantity of substance
If drivers are allowed to pick up multiple orders (carpooling), then supply capacity is doubled or tripled, but deciding which orders to pool becomes combinatorially complex and intractable at scale
Solution Approach 1:
The patent applies preliminary action by pre-computing compatibility matrices and order pooling feasibility assessments before real-time dispatch decisions. The system pre-identifies which order combinations are spatially and temporally feasible, storing these as lookup tables or pre-processed features. During actual dispatch, the system only needs to query these pre-computed results rather than performing full combinatorial optimization, thus enabling high supply capacity while managing complexity.
3Measurement precision
If exact combinatorial optimization is used to determine order pooling, then optimal matching is achieved, but the computation becomes intractable in large-scale online systems requiring decisions within hundreds of milliseconds
Solution Approach 1:
The patent changes the problem parameters by transforming the combinatorial optimization task into a structured prediction problem suitable for machine learning models. Instead of solving exact combinatorial optimization (NP-hard), the system reformulates it as learning from historical data with features like spatial proximity, temporal overlap, and demand patterns. This parameter transformation enables near-optimal solutions with polynomial-time complexity, resolving the contradiction between matching precision and computation speed.
Data Source
AI summary
A method includes: obtaining a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to a plurality of candidate carpool combinations each comprising one or more unassigned transportation orders, wherein: the plurality of first signals comprise a current time, a location of the vehicle at the current time, and one or more static features corresponding to the vehicle, the plurality of second signals comprise timestamps, origins, and destinations of the unassigned transportation orders, and the vehicle has an on-going transportation order at the current time; inputting the plurality of first and second signals to a trained machine learning model; and obtaining, from an output of the trained machine learning model, a utility score of each of the plurality of candidate carpool combinations.