Dynamic order dispatching radius generation system based on UCB online learning

Abstract

The invention belongs to the technical field of online car-hailing, and particularly relates to a dynamic order dispatching radius generation system based on UCB online learning, which comprises a context environment generation module, a parameterized action space and an online learning engine, and is characterized in that the context environment generation module divides space-time supply and demand states into discrete grades; each Arm of the parameterized action space corresponds to a combination of a group of ETA and EDA parameters; and the online learning engine dynamically selects an optimal Arm based on a UCB algorithm, and generates a dispatching radius. A system and a method for dynamically generating a dispatching radius through a UCB algorithm in a multi-arm Huger machine frame are used for realizing dynamic balance of supply and demand. And the context-aware MAB framework is used for realizing scenarized parameter optimization by taking the space-time supply and demand state as a context. And dual-target parameter joint optimization: combined learning of ETA and EDA gives consideration to efficiency and user experience. According to the cold start solution, historical data are migrated through context similarity, and convergence is accelerated.

Description

Technical Field

[0001] The present invention relates to the technical field of online car-hailing, and specifically to a dynamic dispatching radius generation system based on UCB online learning. Background Art

[0002] Online car-hailing, namely the short name of online reservation taxi operation service, refers to the business activity that constructs a service platform relying on Internet technology, integrates supply and demand information, uses eligible vehicles and drivers, and provides non-cruising reservation taxi services. Users can select vehicle models and drivers through mobile application software, quickly call a car and enjoy travel services. Compared with traditional taxis, online car-hailing has the advantages of transparent prices, convenient car-calling, and controllable service quality, so it has become rapidly popular around the world.

[0003] Existing online car-hailing dispatching systems usually adopt a fixed dispatching radius or dynamic adjustment based on static rules, resulting in the following problems:

[0004] 1. Spatiotemporal supply-demand imbalance: The fixed radius cannot adapt to the demand fluctuations during peak hours and in special areas.

[0005] 2. Parameter solidification: ETA (Estimated Time of Arrival) and EDA (Estimated Dispatching Accuracy) rely on manual experience for setting and lack dynamic optimization.

[0006] 3. Exploration-exploitation dilemma: Traditional methods are difficult to balance the relationship between "trying new strategies" and "adopting the optimal strategy". Summary of the Invention

[0007] The purpose of the present invention is to provide a dynamic dispatching radius generation system based on UCB online learning to solve the problems raised in the above background art.

[0008] To achieve the above purpose, the present invention provides the following technical solution: A dynamic dispatching radius generation system based on UCB online learning, including:

[0009] A context environment generation module, a parameterized action space, and an online learning engine;

[0010] Among them, the context environment generation module divides the spatiotemporal supply-demand state into discrete levels, each Arm of the parameterized action space corresponds to a combination of a set of ETA and EDA parameters, and the online learning engine dynamically selects the optimal Arm based on the UCB algorithm to generate the dispatching radius;

[0011] The specific method for generating the dynamic dispatching radius is as follows:

[0012] Step 1: Context division;

[0013] Spatiotemporal feature extraction: Divide the city into grid areas and count the supply-demand ratio according to time slices;

[0014] - Supply and demand level classification: The supply and demand status is divided into 5 levels using the threshold method;

[0015] Step 2: Action space definition;

[0016] Each context group is bound to a set of candidate Arms, each Arm contains:

[0017] -ETA, which is the maximum time the driver will arrive;

[0018] -EDA, i.e. the confidence interval of the probability of order acceptance;

[0019] -Dispatch radius calculation function: Radius = f (ETA, EDA, real-time traffic conditions)

[0020] Step 3: UCB online study;

[0021] - Revenue feedback: Instant rewards are calculated based on indicators such as order completion rate and driver response time;

[0022] -Dynamic update strategy.

[0023] Preferably, the discrete levels include, for example, high demand-low supply and low demand-high supply.

[0024] Preferably, the time slice is 15 minutes.

[0025] Preferably, the supply and demand status includes the following five levels:

[0026] Level 1: severe shortage; Level 2: supply is less than demand; Level 3: supply and demand are balanced; Level 4: supply exceeds demand; Level 5: severe surplus

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] A system and method for dynamically generating dispatch radius through the UCB (Upper Confidence Bound) algorithm in the Multi-Armed Bandit (MAB) framework to achieve dynamic balance between supply and demand. Context-aware MAB framework: Use the spatiotemporal supply and demand status as context to achieve scenario-based parameter optimization. Dual-objective parameter joint optimization: Combined learning of ETA and EDA, taking into account both efficiency and user experience. Cold start solution: Migrate historical data through context similarity to accelerate convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The system architecture diagram of the present invention;

[0030] Figure 2 : Link diagram of UCB algorithm in matchmaking. Detailed implementation mode

[0031] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative work shall fall within the protection scope of the present invention.

[0032] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0033] Embodiment 1:

[0034] Please refer to Figure 1-2 , the present invention provides a technical solution: a dynamic dispatching radius generation system based on UCB online learning:

[0035] Core solution

[0036] System architecture:

[0037] Context environment generation module: Divide the spatio-temporal supply and demand state into discrete levels (such as high demand - low supply, low demand - high supply).

[0038] - Parameterized action space (Arms): Each Arm corresponds to a combination of a set of ETA and EDA parameters. For example, Arm_k = (ETA = 3min, EDA = 85%).

[0039] - Online learning engine: Dynamically select the optimal Arm based on the UCB algorithm to generate the dispatching radius.

[0040] The UCB algorithm is a solution to the multi-armed bandit problem based on the upper confidence bound. It maximizes the cumulative reward by balancing exploration and exploitation. Its core lies in dynamically calculating the upper confidence bound value of each option, preferentially selecting options with high estimated value or insufficient information, and gradually converging to the optimal strategy.

[0041] Core mechanism

[0042] Calculation of the upper confidence bound: The evaluation value of each option consists of the historical average reward (exploitation) and the exploration term. The formula is:

[0043] UCB value = average reward + √(2 · ln(total number of attempts) / number of attempts for this option)

[0044] The second term reflects the exploration intensity, and the value of this term is larger for options with fewer attempts.

[0045] Selection - update loop:

[0046] Initialization: Assign initial number of attempts and reward estimates to each option

[0047] Selection phase: Prioritize selecting the option with the highest current UCB value

[0048] Update phase: Update the average estimate and number of attempts for this option based on the actual reward obtained

[0049] Dynamic balance: As the total number of attempts increases, options that have not been fully explored will get selected due to the increasing exploration term

[0050] Application scenarios

[0051] The UCB algorithm is widely used in fields such as online recommendation systems, advertising placement optimization, and clinical trial design. For example, in restaurant recommendations, when the user's experience with Burger King fluctuates, the algorithm will adjust its priority through the confidence interval, and then try other options such as McDonald's, and finally lock in the optimal choice.

[0052] Algorithm advantages

[0053] No prior dependence: It does not require a pre - set reward distribution and learns completely through interaction data

[0054] Theoretical guarantee: It can lock in the optimal option with high probability after a finite number of attempts and is mathematically proven to have the property of sub - linear regret growth

[0055] This algorithm belongs to the solutions of the multi - armed bandit problem, such as the epsilon - Greedy and Softmax algorithms. However, it quantifies uncertainty through the confidence interval and achieves a more efficient exploration - exploitation balance.

[0056] Technical details

[0057] Step 1: Context partitioning

[0058] - Spatiotemporal feature extraction: Divide the city into grid regions and count the supply - demand ratio by time slices (15 minutes).

[0059] - Supply - demand level classification: Use the threshold method to classify the supply - demand status into 5 levels (S1: severe shortage ~ S5: severe surplus).

[0060] Step 2: Action space definition

[0061] -Each context group is bound to a set of candidate Arms, each Arm contains:

[0062] -ETA (driver's arrival time limit)

[0063] -EDA (confidence interval of the probability of order acceptance)

[0064] -Dispatch radius calculation function: Radius = f (ETA, EDA, real-time traffic conditions)

[0065] Step 3: Study online at UCB

[0066] - Revenue feedback: Instant rewards are calculated based on indicators such as order completion rate and driver response time.

[0067] -Dynamic update strategy:

[0068] python#UCB selection formula def select_arm(context): for armin context.arms:ucb_score=arm.avg_reward+sqrt(2*ln(total_pulls) / arm.pulls) return arm_with_max(ucb_score)-Backbone mechanism: when a new context appears, use the optimal Arm initialization parameters of historical similar scenarios.

[0069] Data collection: Receive orders, driver locations, and traffic status data in real time.

[0070] Context matching: Mapping the current spatiotemporal state to a predefined level (e.g., S3-level supply-demand balance).

[0071] Arm selection and execution: Call the UCB algorithm to select the optimal ETA / EDA combination and calculate the dynamic radius.

[0072] Feedback loop: record order allocation results and update the reward mean and confidence interval of each Arm.

[0073] Example:

[0074] In the evening peak business district (context = S1), UCB may choose Arm = (ETA = 5 min, EDA = 75%), resulting in a radius of 2.5 km.

[0075] - In the early morning residential area (context = S5), select Arm = (ETA = 8min, EDA = 95%), generating a radius of 1.2km.

[0076] 2. Context-aware MAB framework: Taking the spatiotemporal supply and demand status as context to achieve scenario-based parameter optimization.

[0077] 2. Joint optimization of dual objective parameters: The combined learning of ETA and EDA takes into account both efficiency and user experience.

[0078] 3. Cold start solution: Transfer historical data through context similarity to accelerate convergence.

[0079] The foregoing has shown and described the basic principles, main features and advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention, and any reference signs in the claims should not be regarded as limiting the claims involved.

[0080] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dynamic dispatching radius generation system based on UCB online learning, characterized in that, include: Context generation module, parameterized action space and online learning engine; The context environment generation module divides the spatiotemporal supply and demand state into discrete levels, each Arm in the parameterized action space corresponds to a combination of a set of ETA and EDA parameters, and the online learning engine dynamically selects the optimal Arm based on the UCB algorithm to generate a dispatch radius; The dynamic dispatch radius generation method is as follows: Step 1: Context division; Temporal and spatial feature extraction: Divide the city into grid areas and calculate the supply-demand ratio by time slice; - Supply and demand level classification: The supply and demand status is divided into 5 levels using the threshold method; Step 2: Action space definition; Each context group is bound to a set of candidate Arms, each Arm contains: -ETA, which is the maximum time the driver will arrive; -EDA, i.e. the confidence interval of the probability of order acceptance; -Dispatch radius calculation function: Radius = f (ETA, EDA, real-time traffic conditions) Step 3: UCB online study; - Revenue feedback: Instant rewards are calculated based on indicators such as order completion rate and driver response time; -Dynamic update strategy.

2. The dynamic dispatching radius generation system based on UCB online learning according to claim 1, characterized in that: The discrete levels include, for example, high demand-low supply, and low demand-high supply.

3. The dynamic order assignment radius generation system based on UCB online learning according to claim 1, wherein: The time slice is 15 minutes.

4. The dynamic order dispatching radius generation system based on UCB online learning according to claim 1, characterized in that: The supply and demand status includes the following five levels: Level 1: severe shortage; Level 2: Supply is less than demand; Level 3: Supply and demand are balanced; Level 4: oversupply; Level 5: serious oversupply.

Images