A Deep Reinforcement Learning-Based Food Delivery Scheduling Method

Abstract

This invention provides a deep reinforcement learning-based food delivery scheduling method, comprising: constructing multiple time-chain sequences starting from the order creation time and padded with zeros according to the fulfillment status; inputting these sequences into a GRU-Transformer hybrid network to predict the cumulative time of four nodes: order acceptance, arrival at the store, food pickup, and delivery; calculating relevant times based on the predicted time; constructing an extended state space by combining an adjacency matrix and warning labels; establishing a Markov decision process; and obtaining a scheduling strategy through multi-stage progressive training using a graph attention policy network and a proximal policy optimization algorithm; generating adaptive confidence intervals during the inference stage; using Lagrange relaxation to constrain the timeout rate; and achieving joint path planning in multi-rider scenarios through weighted priority and time conflict suppression. This invention deeply couples time-series prediction with reinforcement learning scheduling, effectively reducing the timeout rate and improving delivery efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics scheduling technology, specifically to a food delivery scheduling method based on deep reinforcement learning, and particularly to a food delivery path joint optimization technology based on time chain sequence encoding for parallel prediction of order fulfillment time, construction of graph attention strategy network, and multi-stage progressive knowledge transfer training. Background Technology

[0002] Online-to-offline (O2O) food delivery is a typical high-efficiency scenario with delivery times measured in minutes. As of December 2024, the number of online food delivery users in China reached 592 million, accounting for 53.4% ​​of the total number of internet users. The entire order fulfillment process directly determines the user's ordering decision, consumer satisfaction, and platform retention rate. The entire food delivery order fulfillment process can be broken down into four core stages: rider accepting the order, rider arriving at the store, rider picking up the food, and rider delivering it. The time taken in each stage is affected by multiple dynamic factors such as the merchant's food preparation speed, rider location, traffic conditions, and peak and off-peak order periods.

[0003] In existing food delivery platforms, intelligent dispatch systems typically include two independent functional modules: an estimated delivery time prediction module and a rider route planning module. The estimated delivery time prediction module is used to display the estimated delivery time on the merchant's details page, while the route planning module is used to assign orders to riders and plan the delivery sequence.

[0004] The prior art most relevant to this invention employs the following methods: First, an estimated delivery time prediction method: Based on historical order data, static features such as the distance from the merchant to the user, the quantity of food items, and weather are extracted, and the total delivery time is predicted using gradient boosting trees or time series models. Some improved solutions predict food preparation time and delivery time separately, but all use static features after order creation as input. Second, a delivery route planning method: The food delivery is modeled as a pickup and delivery vehicle routing problem with time windows, and heuristic algorithms or metaheuristic algorithms are used to solve the delivery order.

[0005] However, the aforementioned prior art has the following technical problems:

[0006] First, the estimated delivery time forecasting system is incomplete and cannot meet the needs of all scenarios. When a user has not placed an order, the rider's core information is unknown, and the traditional average delivery time estimation has a large error, affecting users' willingness to place an order. Moreover, existing forecasts mostly focus on single-point prediction of total delivery time and cannot predict key nodes in the entire process such as order acceptance and store arrival in parallel. At the same time, the order volume fluctuates greatly during peak and off-peak periods, and the forecast accuracy is unstable, making it difficult to support refined operations.

[0007] Second, the route planning model lacks adaptability and is out of touch with dynamic fulfillment scenarios. Existing models treat travel time and waiting time as fixed values, failing to consider dynamic factors such as merchants' food preparation pressure and real-time road conditions. This results in a large deviation between the planned route and the actual execution, leading to frequent delays for riders. In addition, the tight capacity during peak and off-peak periods prevents the model from adjusting its plans in real time, further impacting fulfillment efficiency and on-time performance. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a food delivery scheduling method based on deep reinforcement learning. This method decomposes the entire order fulfillment process into four parallel time chains using a time chain simulation method, solving the prediction problem under conditions of missing information before order creation. A deep learning architecture combining gated recurrent units and Transformers is designed to achieve parallel prediction at four key moments. Prediction time is integrated as a dynamic feature into the state space of a graph attention network, and a combined reward function containing positive rewards for on-time delivery and negative penalties for late delivery is designed to achieve deep integration of prediction and path planning. Multi-stage progressive knowledge transfer training improves the robustness of peak-valley prediction and decision-making.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] This invention provides a food delivery scheduling method based on deep reinforcement learning, comprising:

[0011] For each order in the historical order data, starting from the order creation time, information on different fulfillment stages is encoded using left-padding and dynamic feature updates to construct multiple time-chain sequences;

[0012] Based on the multiple time chain sequences, with the cumulative time for order acceptance, cumulative time for arrival at the store, cumulative time for food pickup, and cumulative time for delivery as prediction targets, a time prediction network is trained to obtain a trained time prediction model.

[0013] Based on the cumulative pickup time and cumulative delivery time obtained from the time prediction model after training, they are converted into the merchant's food preparation completion time and the customer's delivery deadline, respectively. The predicted waiting time and predicted margin time are calculated and used as the core state features to construct a Markov decision process for food delivery that includes a state space, an action space, and a reward function.

[0014] Based on the Markov decision process for food delivery, a graph attention policy network is constructed, trained using a proximal policy optimization algorithm, and trained using a multi-stage progressive knowledge transfer training strategy to complete the training of the cross-scale scheduling policy, thus obtaining the trained scheduling policy.

[0015] Optionally, the steps of constructing the multiple time chain sequences and the trained time prediction model include:

[0016] For each order in the historical order data, four time chains are constructed according to the different starting states of order fulfillment. Each time chain is padded with left zeros and dynamically updated according to the corresponding fulfillment state to obtain four time chain sequences.

[0017] The four time-chain sequences are input into the encoder to extract temporal evolution features. In the decoding stage, future information is masked by a sequence mask matrix. The temporal evolution features are interacted through a multi-head cross-attention mechanism. The cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time are output in parallel through a linear layer.

[0018] Using the cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time as supervision targets, and the mean squared error as the loss function, the Adam optimizer is used to train the time prediction network to obtain the trained time prediction model.

[0019] Optionally, the specific structure and training method of the encoder and decoder of the time prediction network include:

[0020] The encoder is a combination of a gated loop unit and a Transformer. The gated loop unit performs stepwise temporal encoding on the four time chain sequences, and the output is normalized to obtain the encoded features.

[0021] The encoded features are input into the decoder, which adopts a multi-layer Transformer structure. The decoder masks future time information through a sequence masking matrix and aligns the encoded features with the mask sequence through a multi-head cross-attention mechanism. The decoder outputs the cumulative order acceptance time, the cumulative time to store arrival time, the cumulative food pickup time, and the cumulative delivery time in parallel through a linear layer.

[0022] When training the time prediction network, multiple time chain sequences corresponding to consecutive historical orders from the same merchant are used as input. The mean squared error between the cumulative order acceptance time, cumulative store arrival time, cumulative food pickup time, and cumulative delivery time output by the decoder of the encoded features and the actual cumulative order acceptance time, actual store arrival time, actual food pickup time, and actual delivery time of the latest order is used as the loss function for supervised training, thereby obtaining the trained time prediction model.

[0023] Optionally, the steps of constructing the Markov decision process for food delivery and training the graph attention policy network include:

[0024] Based on the predicted waiting time and the predicted margin time, the global normalized adjacency matrix, node features and warning labels are incorporated into the state space, wherein the warning label is set to one when the predicted margin time is lower than a preset threshold, and otherwise set to zero, to obtain the expanded state space;

[0025] Based on the extended state space, the action space is defined as a discrete choice between picking up or delivering food, and the first-pick-then-delivery constraint is enforced through a masking mechanism. The reward function is defined as a combined reward that includes a positive reward for on-time delivery and a negative penalty for late delivery, thus obtaining a fully defined Markov decision process for food delivery.

[0026] Based on the defined Markov decision process for food delivery, a graph attention policy network is constructed. The graph attention policy network encodes the node features with the global normalized adjacency matrix as the edge weights to obtain graph attention encoded features. Based on the graph attention encoded features, the action probability distribution and scalar state value estimate are output respectively to obtain the constructed graph attention policy network.

[0027] Based on the constructed graph attention policy network, a near-end policy optimization algorithm is adopted to perform multi-stage progressive training according to the order size from small to large. In each stage, the model weights completed in the previous stage are loaded as initialization parameters to complete cross-scale knowledge transfer and obtain the trained scheduling policy.

[0028] Optionally, the method further includes steps for real-time inference, visualization, and policy evaluation of the time prediction model and the scheduling policy upon completion of training, including:

[0029] The system receives rider location information and order status information in real time through a data interface. It then uses the trained time prediction model to infer the current order and obtain the real-time predicted duration value. Finally, it uses the trained scheduling strategy to make decision-making inferences for the current delivery task and obtain the real-time delivery route decision result.

[0030] Based on the real-time predicted duration and the real-time delivery route decision results, a multi-dimensional visualization display is performed through a visualization rendering engine, outputting an order full-process time prediction panel, a rider delivery route map, an overtime risk level dashboard, and a prediction and decision consistency monitoring chart, thus obtaining a visualization display interface.

[0031] Based on the visualization interface, the hyperparameters of the trained scheduling strategy can be remotely configured. Based on the real-time prediction duration and the real-time delivery path decision results, the scheduling strategy versions under different hyperparameter configurations can be compared and tested in terms of on-time delivery rate, average delivery time, and late delivery penalty cost, so as to obtain the strategy evaluation results.

[0032] Optionally, the scheduling execution phase may also include:

[0033] After each scheduling action is executed, the trained time prediction model is called again to refresh the predicted duration of orders that have not yet been fulfilled. For orders that have started to be fulfilled, the predicted duration is estimated by the sum of the actual time elapsed and the remaining predicted duration to obtain the updated predicted duration.

[0034] Based on the updated prediction duration, the difference between the actual waiting time and the predicted waiting time is calculated. When the difference exceeds a preset deviation threshold, a prediction deviation penalty term is triggered in the reward function to obtain an updated reward signal.

[0035] Based on the updated reward signal, the network parameters of the trained scheduling policy are updated online to obtain the online-updated scheduling policy.

[0036] Optionally, the prediction output of the trained time prediction model further includes the following adaptive confidence interval generation step:

[0037] Based on the time prediction model that has been trained, multiple parallel forward propagations are performed on the same order input to be predicted during the inference phase. The mean of each prediction target is calculated as the point prediction value and the standard deviation is calculated as the local uncertainty estimate.

[0038] Based on the local uncertainty estimation, a sliding calibration window is maintained, and the predicted value, actual performance time and timestamp of historical completed orders are stored. The cumulative sum statistic is used to detect the distribution change of the residual sequence in the sliding calibration window. When a change point is detected, an exponentially decaying weight is applied to the samples before the change point to obtain the change point-aware exponentially decaying weight sequence.

[0039] Based on the exponentially decaying weight sequence with the change point awareness, the non-consistency score is normalized using the local uncertainty estimate, and the weighted quantile is calculated using the exponentially decaying weight sequence. For each prediction target, an adaptive confidence interval is output with the predicted point value as the center and the product of the weighted quantile and the local uncertainty estimate as the radius.

[0040] Based on the adaptive confidence interval, the width of the adaptive confidence interval is normalized and injected into the state space. The upper bound of the confidence interval of the cumulative delivery time is used as the dynamic risk boundary to replace the fixed delivery deadline for the calculation of the reward function, so as to obtain the updated state space and reward function.

[0041] Based on the updated state space and reward function, after each order is fulfilled, the actual observation value is appended to the sliding calibration window, triggering the change point detection and weighted quantile update, and obtaining the updated adaptive confidence interval parameters.

[0042] Optionally, the adaptive confidence interval further includes the following parameterized inconsistency metric optimization steps:

[0043] Based on the adaptive confidence interval, the upper and lower bounds of the adaptive confidence interval are mapped to parameterized inconsistency scores through a learnable parameterized transformation.

[0044] Based on the parameterized inconsistency score, and using the parameterized inconsistency score as the optimization variable, under the constraint that the prediction coverage of the sliding calibration window is not lower than the preset target confidence level, the iterative optimization method is used to minimize the conformal set radius to obtain the optimized parameterized inconsistency metric.

[0045] Based on the optimized parameterized inconsistency metric, the equivalent uncertainty scalar for each prediction target is calculated. Based on the relationship between the equivalent uncertainty scalar and the prediction margin time, a green label is used when the prediction margin time is greater than the equivalent uncertainty scalar, a yellow label is used when the prediction margin time is not greater than the equivalent uncertainty scalar but is greater than zero, and a red label is used when the prediction margin time is not greater than zero, thus obtaining a three-level risk label.

[0046] Optionally, it also includes:

[0047] Using the orders marked in yellow in the three-level risk labels and the sliding calibration window as input, a subset of historical orders is retrieved in the sliding calibration window whose absolute value of the difference between the predicted grace time and the predicted grace time of the order does not exceed a preset neighborhood radius. The actual timeout probability of the historical order subset is calculated as the expected timeout cost of the order. When the number of samples in the historical order subset is less than the preset minimum number of samples, it degenerates into the global on-time rate estimation of the sliding calibration window to obtain the expected timeout cost estimate.

[0048] Based on the expected timeout cost estimate, the expected timeout cost estimate is multiplied by a preset timeout cost coefficient and added as an additional penalty to the reward function to obtain the extended reward function;

[0049] Based on the extended reward function, a total loss function is constructed in the form of a weighted sum of the policy gradient objective, state value regression loss, and timeout probability classification loss. A Lagrange relaxation constraint term is introduced into the total loss function, and the preset timeout rate upper limit is encoded as a probability constraint. The Lagrange multiplier in the Lagrange relaxation constraint term is adaptively updated through a dual optimization method to obtain the total loss function with timeout rate constraint.

[0050] Based on the total loss function with introduced timeout rate constraints, a state value estimation head and a timeout probability auxiliary output head are extended in the graph attention policy network. The graph attention policy network is then trained using the total loss function with introduced timeout rate constraints to obtain a training completion scheduling policy that satisfies the timeout rate constraints.

[0051] Optionally, the step of performing multi-rider joint path planning using the trained scheduling strategy during the inference phase includes:

[0052] Using the negative of the predicted grace time for each rider's orders as the priority weight, calculate the weighted priority index for each rider, and determine the rider decoding order according to the weighted priority index from largest to smallest to obtain the rider decoding order sequence;

[0053] According to the rider decoding sequence, for the current rider to be decoded, the initial score of each candidate node is calculated by integrating the node features and the predicted waiting time; when the current rider has included the target pickup node in the current rider's delivery route, the time conflict degree between the target pickup node and the current rider to be decoded is calculated based on the predicted waiting time, and the initial score of the target pickup node is suppressed accordingly to obtain the adjusted candidate node score distribution;

[0054] Based on the adjusted candidate node score distribution, the candidate node with the highest score is selected as the next action node for the current rider to be decoded. The next action node is then incorporated into the delivery path of the current rider to be decoded. The process is repeated for the next rider to be decoded, following the rider decoding sequence. The initial score and score suppression steps for each candidate node are calculated by fusing the node features and the predicted waiting time, until all riders have completed the allocation of all pick-up and delivery nodes in their respective delivery paths, thus obtaining the joint optimal delivery path for all riders.

[0055] The beneficial effects of this invention are as follows:

[0056] 1. This invention solves the prediction problem under conditions of missing information before order creation by using a time chain simulation method. The entire order fulfillment process is decomposed into four progressive time chains. Through left-padding and state evolution design, the model can complete predictions even without rider information or dynamic features.

[0057] 2. This invention employs a combined architecture of gated recurrent units and Transformers for time series forecasting. The gated recurrent unit module captures the peak and trough trends of food delivery orders, while the Transformer module achieves parallel forecasting and long-distance dependency modeling through multi-head attention; the two complement each other.

[0058] 3. This invention incorporates prediction time as a dynamic feature into the reinforcement learning state space, achieving a deep fusion of prediction and path planning; the decision model can predict the expected time for each order in each stage of order acceptance, arrival at the store, food pickup, and delivery, thereby making a better delivery order decision.

[0059] 4. This invention employs a multi-stage progressive knowledge transfer training method, making full use of the scale-independent nature of graph attention networks;

[0060] 5. This invention uses Huber loss as the loss function of the Critic network, which effectively solves the gradient explosion problem caused by time window cliffs in the food delivery scenario. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating the food delivery scheduling method based on deep reinforcement learning of the present invention.

[0063] Figure 2 This is a schematic diagram of the encoding and decoding structure of the time prediction network of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment provides a food delivery scheduling method based on deep reinforcement learning, including:

[0067] Step S1: For each order in the historical order data, starting from the order creation time, encode the information of different fulfillment stages by padding with zeros and dynamic feature updates to construct multiple time chain sequences;

[0068] Step S2: Based on the multiple time chain sequences, with the cumulative time for order acceptance, cumulative time for arrival at the store, cumulative time for food pickup, and cumulative time for delivery as prediction targets, train the time prediction network to obtain the trained time prediction model.

[0069] Step S3: Based on the cumulative pickup time and cumulative delivery time obtained from the time prediction model after training, convert them into the merchant's food preparation completion time and the customer's delivery deadline, respectively. Estimate the predicted waiting time and the predicted margin time, and use the predicted waiting time and the predicted margin time as core state features to construct a Markov decision process for food delivery that includes a state space, an action space, and a reward function.

[0070] Step S4: Based on the Markov decision process for food delivery, construct a graph attention policy network, train it using a near-end policy optimization algorithm, and complete the training of the cross-scale scheduling policy using a multi-stage progressive knowledge transfer training strategy to obtain the trained scheduling policy.

[0071] Specifically, firstly, the fulfillment trajectory of each order is extracted, including order creation time, rider acceptance time, rider arrival time at the store, rider pickup time, and rider delivery time. Starting from the order creation time, the fulfillment process is decomposed into four state processes: S0 (order created, rider not accepted), S1 (rider accepted, not arrived at the store), S2 (rider arrived at the store, not picked up the food), and S3 (rider picked up the food, not delivered). A corresponding time chain is constructed for each state process. Each time chain starts from the order creation time and has a length of 4 time steps. The starting point is aligned by padding with zeros on the left. At each time step, dynamic features such as the rider's distance from the merchant and the rider's distance from the user are dynamically updated according to the current state. The features corresponding to other state stages that have not yet occurred are set to zero, thus completely encoding the dynamic evolution process of order fulfillment without relying on real-time rider information.

[0072] Secondly, multiple constructed time-chain sequences are input into the time prediction network. The prediction targets are the cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time. The mean squared error (MSE) is used as the loss function, and the Adam optimizer is employed for supervised training of the time prediction network. The time prediction network uses a combined encoder-decoder architecture of gated recurrent units and Transformers. The encoder uses gated recurrent units to progressively extract the temporal evolution features from the time-chain sequences. The decoder masks future information using a sequence masking matrix and then outputs four predicted times in parallel through a multi-head cross-attention mechanism. After training convergence, a trained time prediction model is obtained. This model achieves a root mean square error (RMSE) of less than 8.5 minutes for delivery time prediction during core operating periods such as lunch and evening peak hours.

[0073] Next, the trained time prediction model is used to infer the current order, obtaining predicted values ​​for the cumulative pickup time and cumulative delivery time. The predicted cumulative pickup time is added to the order creation time to convert it into the merchant's food preparation completion time; the predicted cumulative delivery time is added to the order creation time to convert it into the customer's delivery deadline. Based on this, the predicted waiting time (i.e., the estimated waiting time for the rider to arrive at the merchant, taken as max(0, merchant's food preparation completion time - (current cumulative time + rider's travel time to the merchant)), normalized to [0,1]) and the predicted grace time (i.e., the remaining time for the rider to complete the delivery relative to the delivery deadline, taken as max(0, customer's delivery deadline - (current cumulative time + rider's travel time to the user)), normalized to [0,1]) are calculated based on the rider's current location and current cumulative time. By incorporating the predicted waiting time and the predicted allowable time as core state features into the state space, and by constructing a discrete action space for selecting either the pickup or delivery node, a multi-level hybrid reward function is designed that includes physical time cost, single order delivery reward and penalty, and a global zero-timeout reward, thus obtaining a complete Markov decision process for food delivery.

[0074] Finally, using the Markov decision process for food delivery as the training environment, a graph attention policy network is constructed as the policy network. The graph attention policy network uses a globally normalized adjacency matrix as edge weights, performs two layers of graph attention encoding on the node feature matrix, outputs graph attention-encoded features, and outputs the probability distribution of legal actions through the Actor decoder and the scalar state value estimate through the Critic decoder. A proximal policy optimization algorithm is used for training, the advantage function is calculated using generalized advantage estimation, Huber loss is used as the loss function for the Critic network to prevent gradient explosion, and the update magnitude is limited by a policy pruning coefficient ε=0.2. In terms of training strategy, a multi-stage progressive knowledge transfer training is adopted according to the order size from small to large: the first stage trains the baseline model in a fixed small-scale order scenario (N=3); the second stage performs generalization training in a randomly generated scenario of the same size, so that the zero timeout rate is stabilized above 99%; the third stage transfers the model weights to a larger-scale order scenario (N=4, N=5, N=6), and takes advantage of the characteristic that the parameters of the graph attention network are independent of the number of nodes, and continues to train until convergence at a lower learning rate in the new scale, thus completing the cross-scale knowledge transfer and obtaining the training completed scheduling strategy.

[0075] Example 2

[0076] In this embodiment, the steps of constructing the multiple time chain sequences and obtaining the trained time prediction model include:

[0077] For each order in the historical order data, four time chains are constructed according to the different starting states of order fulfillment. Each time chain is padded with left zeros and dynamically updated according to the corresponding fulfillment state to obtain four time chain sequences.

[0078] The four time-chain sequences are input into the encoder to extract temporal evolution features. In the decoding stage, future information is masked by a sequence mask matrix. The temporal evolution features are interacted through a multi-head cross-attention mechanism. The cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time are output in parallel through a linear layer.

[0079] Using the cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time as supervision targets, and the mean squared error as the loss function, the Adam optimizer is used to train the time prediction network to obtain the trained time prediction model.

[0080] Specifically, in the context of food delivery, the complete fulfillment process of an order can be broken down into four sequential fulfillment states: S0 (order created, rider not yet accepted), S1 (rider accepted, not yet arrived at the merchant), S2 (rider arrived at the merchant, not yet picked up the food), and S3 (rider picked up the food, not yet delivered to the user). For each order in the historical order data, its complete fulfillment trajectory is extracted, including the order creation time, rider acceptance time, rider arrival time, rider pickup time, and rider delivery time. Based on this, the cumulative time for accepting the order, the cumulative time for arriving at the merchant, the cumulative time for picking up the food, and the cumulative time for delivery are calculated as prediction target labels.

[0081] This embodiment introduces the concept of a "time chain" to explicitly serialize and model the order fulfillment process. A time chain is a feature sequence formed by encoding the characteristic information of an order at different fulfillment progress states, starting from the order creation time and following a fixed length (each chain in this embodiment contains four time steps, corresponding to four fulfillment stages). The core design idea of ​​the time chain is that for any prediction target (such as predicting the cumulative order duration), at the order creation time, riders have not yet been assigned, and the dynamic characteristics related to riders (such as the distance between the rider and the merchant) are objectively unknown. Therefore, the dynamic characteristics of the time steps in the time chain where rider information is unknown are filled with zero values ​​(i.e., left-padding with zeros), and the true feature values ​​are only filled in when rider information is known. This "left-padding with zeros" design allows the model to form an effective temporal input even when features are missing, fundamentally solving the problem of unpredictability caused by missing information before order creation.

[0082] Specifically, for each historical order, four time-chain sequences are constructed as follows: Time Chain 1 (corresponding to the cumulative order acceptance time prediction target): This chain has 4 time steps. All dynamic features in the first 3 time steps are filled with zero values. Only the 4th time step is filled with features in state S0 (at this point, the rider has not yet accepted the order, rider-related features are set to 0, and only static features such as merchant and user are retained). Time Chain 2 (corresponding to the cumulative in-store arrival time prediction target): The first 2 time steps are filled with zero values. The 3rd time step is filled with features in state S0, and the 4th time step is filled with features in state S1 (the rider has accepted the order, updating dynamic features such as the rider's real-time distance to the merchant). Time Chain 3 (corresponding to the cumulative pickup time prediction target): The 1st time step is filled with zero values. The 2nd time step is filled with features in state S0, the 3rd time step is filled with features in state S1, and the 4th time step is filled with features in state S2 (the rider has arrived at the merchant, and the rider-to-merchant distance feature is reset to 0). Time chain 4 (corresponding to the cumulative delivery time prediction target): The first to fourth time steps are filled with the state features S0, S1, S2 and S3 respectively. The fourth time step updates features such as the distance between the merchant and the user. At this time, the rider has completed picking up the food and is on the way to delivery.

[0083] The feature vector at each time step consists of both static and dynamic features. Static features include information that can be determined at the time of order creation, such as the merchant identifier code, the straight-line distance between the user and the merchant, and the quantity of food items, and remain unchanged throughout all time steps of all time chains for the same order. Dynamic features include information that changes dynamically as the fulfillment progresses, such as the distance between the rider and the merchant, the distance between the rider and the user, and the peak and valley indicators for the current time period. Real values ​​are only entered in the corresponding states, and zeroed out in other stages.

[0084] In the actual training data preparation phase, historical orders were grouped according to merchant identifiers. A sliding time window strategy was adopted for the same merchant, using 12 time chain sequences of 3 consecutive historical orders (3 orders × 4 time chains) as model input, and the four cumulative duration values ​​of the 4th most recent order as supervision labels. There are strong time dependencies between consecutive orders from the same merchant (for example, slow preparation of previous orders will affect the merchant's overall food preparation rhythm and thus affect subsequent orders). This grouping strategy enables the encoder to capture the fulfillment patterns and peak-valley trends at the merchant level.

[0085] Secondly, the four time-chain sequences mentioned above are input into the encoder of the time prediction network to extract temporal evolution features. The encoder performs stepwise temporal encoding on the input time-chain sequences to capture the time dependence of historical orders and the trend changes of peak and trough periods (such as the pattern of continuous increase in the time for merchants to prepare meals during the lunch peak), and outputs the encoded feature representation.

[0086] During the decoding phase, a sequence masking matrix is ​​introduced to mask future information. The sequence masking matrix is ​​a 0-1 binary matrix. Its function is to mask the time steps corresponding to rider dynamic information that belong to the "future" (i.e., are not yet known at the current prediction time) to negative infinity during the attention calculation process of the decoder. After Softmax normalization, the attention weights at these positions approach zero, ensuring that the model can only use information available at the time of order creation during the inference phase, thus preventing information leakage problems during the training phase.

[0087] The decoder uses a multi-head cross-attention mechanism to perform cross-sequence alignment and feature interaction between the temporal evolution features output by the encoder and the masked target sequence. In this embodiment, the specific role of the multi-head cross-attention mechanism is to allow the decoder to selectively focus on different historical temporal positions of the encoder output when generating each predicted target, thereby discriminatively extracting the historical fulfillment patterns most relevant to the current predicted target. Finally, the feature representation after cross-attention interaction outputs four predicted values—cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time—in parallel through a linear layer, achieving one-time parallel prediction of four key fulfillment nodes. This has a significant advantage in inference efficiency compared to sequential prediction.

[0088] Finally, the time prediction network was trained under end-to-end supervision, using the cumulative order acceptance time, cumulative in-store arrival time, cumulative pickup time, and cumulative delivery time as the supervision targets, and the mean squared error (MSE) between the four predicted values ​​and their corresponding true values ​​as the loss function. The MSE loss penalizes the squared prediction bias, allowing the model to prioritize correcting larger prediction errors during training, making it suitable for continuous value regression tasks such as time prediction. The Adam optimizer (learning rate set to 1e-4, weight decay set to 1e-5) was used for gradient updates. The Adam optimizer adaptively adjusts the learning rate of each parameter, exhibiting stable convergence performance when handling takeout order data with peak and trough distribution differences. An early stopping strategy (20 rounds) was employed during training; training was terminated early when the validation set loss no longer decreased after 20 consecutive rounds to prevent overfitting. After the complete training process, the trained time prediction model was obtained. The model has a root mean square error (RMSE) of less than 8.5 minutes in delivery time prediction during core operating periods such as lunch and evening peak hours, and has stable cross-time period generalization ability.

[0089] Example 3

[0090] like Figure 2 As shown, in this embodiment, the specific structure and training method of the encoder and decoder of the time prediction network include:

[0091] The encoder is a combination of a gated loop unit and a Transformer. The gated loop unit performs stepwise temporal encoding on the four time chain sequences, and the output is normalized to obtain the encoded features.

[0092] The encoded features are input into the decoder, which adopts a multi-layer Transformer structure. The decoder masks future time information through a sequence masking matrix and aligns the encoded features with the mask sequence through a multi-head cross-attention mechanism. The decoder outputs the cumulative order acceptance time, the cumulative time to store arrival time, the cumulative food pickup time, and the cumulative delivery time in parallel through a linear layer.

[0093] When training the time prediction network, multiple time chain sequences corresponding to consecutive historical orders from the same merchant are used as input. The mean squared error between the cumulative order acceptance time, cumulative store arrival time, cumulative food pickup time, and cumulative delivery time output by the decoder of the encoded features and the actual cumulative order acceptance time, actual store arrival time, actual food pickup time, and actual delivery time of the latest order is used as the loss function for supervised training, thereby obtaining the trained time prediction model.

[0094] Specifically, firstly, the encoder of the time prediction network adopts a hybrid structure combining a gated recurrent unit (GRU) and a Transformer, with the two complementing each other functionally. A GRU is a variant of a recurrent neural network whose core mechanism involves selectively remembering and forgetting historical information through update and reset gates: the update gate controls how much information to retain from the historical hidden state at the current time step, while the reset gate controls the degree of influence of historical information on the current time step. In the food delivery scenario, order fulfillment time is affected by factors such as the pressure on merchants to prepare food and the saturation of rider capacity, showing a continuous upward trend during the lunch and evening peak hours. This local temporal trend is precisely the type of signal that GRU excels at capturing. Compared to LSTM, GRU has fewer parameters and higher computational efficiency, exhibiting superior training convergence speed with large datasets of food delivery orders.

[0095] The encoder uses a 3-layer stacked GRU network, with each layer having 128 hidden units and a dropout rate of 0.2 to prevent overfitting. The four time-chain sequences (3 historical orders × 4 time chains, totaling 12 sequences) constructed in Example 2 are sequentially input into the GRU encoder. The GRU fuses the input features at each time step with the hidden state of the previous time step, outputting the hidden state representation of the current time step. After layer-by-layer encoding by the 3-layer GRU, an intermediate representation containing historical fulfillment trend information is obtained. This is then processed by layer normalization to obtain the final encoded features. Layer normalization normalizes the feature dimensions of each sample, eliminating numerical instability introduced by differences in units between different time-chain sequences and improving the training stability of the subsequent Transformer decoder.

[0096] Second, the decoder adopts a multi-layer Transformer structure (in this embodiment, there are two Transformer decoder layers, each containing a four-head multi-attention mechanism, and the feedforward network has a dimension of 256). The Transformer decoder undertakes two core functions in this task: first, it masks future time-information through a sequence masking matrix to prevent data leakage; second, it aligns the encoded features with the decoder's input sequence across sequences through a multi-head cross-attention mechanism to extract discriminative information about historical performance patterns.

[0097] The sequence mask matrix is ​​constructed as follows: For a target sequence of length L, an L×L upper triangular matrix is ​​constructed, with the diagonal and lower positions set to 1 (allowing attention), and the upper triangular positions set to 0 (disallowing attention; these positions are set to negative infinity when calculating attention scores). In the scenario of predicting the time of takeout orders, the physical meaning of the sequence mask is: when predicting the cumulative time of order acceptance, the model must not use information such as the distance to the store and the order pickup status, which can only be obtained after the rider accepts the order; when predicting the cumulative time to store arrival, it must not use information such as the current length of the restaurant's food preparation queue, which can only be known after the rider arrives at the store, thereby ensuring that the prediction process strictly simulates the availability of real information at the time of order creation.

[0098] The decoder input is the masked target sequence. A multi-head cross-attention mechanism is used to perform cross-attention calculations between the decoder's current feature representation (as the Query) and the encoder's output encoded features (as the Key and Value). The multi-head design (4 attention heads) allows the model to independently learn the correspondence between encoded and decoded features from multiple subspaces. For example, one attention head might focus on capturing the correspondence between peak and trough trends, while another head might focus on the historical patterns of the merchant's food preparation speed. The outputs of each head are concatenated and linearly transformed to obtain the fused features. Finally, the fused features are output in parallel through a linear output layer (containing ReLU activation) to produce four predicted values: cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time, with an output dimension of 4.

[0099] The complementary functions of the GRU encoder and the Transformer decoder are reflected in the fact that the GRU is responsible for capturing the local trends of food delivery orders on the timeline (such as whether the preparation time of multiple consecutive orders continues to increase), while the Transformer is responsible for establishing long-distance dependencies across sequences in the global field of vision through the attention mechanism (such as the historical correlation between the rider's order acceptance speed of the first order and the arrival time of the third order). The combination of the two enables the time prediction network to have both the ability to perceive local trends and model global patterns.

[0100] Third, when training the time prediction network, multiple time-chain sequences corresponding to consecutive historical orders from the same merchant are used as input. Specifically, the 12 time-chain sequences of the first 3 historical orders from the same merchant are used as encoder input, and the time-chain sequence of the 4th most recent order, after being masked, is used as decoder input. The mean squared error between the predicted values ​​of cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time output by the decoder and the actual cumulative order acceptance time, actual arrival time, actual pickup time, and actual delivery time of the 4th most recent order is used as the loss function for supervised training.

[0101] The design of using multiple consecutive orders from the same merchant as input batches is based on the following: there is a strong temporal correlation between the fulfillment times of consecutive orders from the same merchant—if the merchant's food preparation is slow in the previous order (resulting in longer waiting times for riders), it usually means that the merchant is currently under high pressure in food preparation, and the food preparation time for subsequent orders is also likely to be longer. The GRU encoder can encode this cross-order merchant-dimensional temporal dependency into the hidden state by progressively encoding these 12 time chain sequences, thereby providing the decoder with more predictive conditional information and significantly improving the prediction accuracy of the fulfillment time of the current new order.

[0102] The training hyperparameters are configured as follows: batch size is 64, total training epochs are 300, early stopping epochs are 20; the Adam optimizer learning rate is set to 1e-4, and weight decay is set to 1e-5. After the above supervised training process, a trained time prediction model is obtained. This model can predict the cumulative duration of four key fulfillment nodes of a new order in parallel, based solely on the time chain sequence features of the three most recent historical orders from the same merchant, at the time of new order creation.

[0103] Example 4

[0104] In this embodiment, the steps for constructing the Markov decision process for food delivery and training the graph attention strategy network include:

[0105] Based on the predicted waiting time and the predicted margin time, the global normalized adjacency matrix, node features and warning labels are incorporated into the state space, wherein the warning label is set to one when the predicted margin time is lower than a preset threshold, and otherwise set to zero, to obtain the expanded state space;

[0106] Based on the extended state space, the action space is defined as a discrete choice between picking up or delivering food, and the first-pick-then-delivery constraint is enforced through a masking mechanism. The reward function is defined as a combined reward that includes a positive reward for on-time delivery and a negative penalty for late delivery, thus obtaining a fully defined Markov decision process for food delivery.

[0107] Based on the defined Markov decision process for food delivery, a graph attention policy network is constructed. The graph attention policy network encodes the node features with the global normalized adjacency matrix as the edge weights to obtain graph attention encoded features. Based on the graph attention encoded features, the action probability distribution and scalar state value estimate are output respectively to obtain the constructed graph attention policy network.

[0108] Based on the constructed graph attention policy network, a near-end policy optimization algorithm is adopted to perform multi-stage progressive training according to the order size from small to large. In each stage, the model weights completed in the previous stage are loaded as initialization parameters to complete cross-scale knowledge transfer and obtain the trained scheduling policy.

[0109] Specifically, firstly, a Markov Decision Process (MDP) is a framework for mathematically modeling sequential decision problems. It consists of four elements: a state space, an action space, state transition probabilities, and a reward function. At each time step, the agent observes the current state, selects an action, receives a reward, and transitions to the next state. The goal is to learn a decision strategy that maximizes long-term cumulative rewards. In this embodiment, the food delivery scheduling problem is modeled as an MDP, and the rider's pickup and delivery decision-making process is the interaction between the agent and the environment.

[0110] The core components of the state space are the globally normalized adjacency matrix and the node feature matrix. The globally normalized adjacency matrix describes the temporal distance relationships between nodes in the delivery network. For the actual travel time from node i to node j, the normalized adjacency matrix element values ​​are obtained by dividing it by the global maximum travel time (60 minutes in this embodiment) and constraining it to the [0,1] interval using the clip function. The purpose of the normalization operation is to ensure the numerical stability of the attention coefficient calculation in the subsequent graph attention network and to prevent gradient explosion caused by excessively large travel time dimensions.

[0111] The node feature matrix has the following dimensions: total number of nodes × feature dimension (in this embodiment, total number of nodes = 2N + 1, where N is the current number of orders, including 1 rider origin node, N pickup nodes, and N delivery nodes; feature dimension is 10). Each node's features include the following elements: unique hot encoding of node type (distinguishing between origin, pickup, and delivery nodes); unique hot encoding of order status (distinguishing between unpicked, picked-up but not delivered, and delivered); predicted waiting time and predicted allowance time (inference results from the time prediction model in Embodiment 1, after normalization); warning label; and spatial anchor point (identifying the rider's current node).

[0112] The predicted waiting time and predicted allowance time are two core custom state features introduced in this embodiment, defined as follows: The predicted waiting time refers to the estimated time a rider will need to wait for the merchant to finish preparing the food when arriving at a merchant's pickup point. Specifically, it is calculated by estimating the merchant's estimated completion time from the cumulative pickup time output by the time prediction model, then calculating the difference between this time and the rider's arrival time at the merchant based on the current time plus the estimated travel time, taking a non-negative value and normalizing it to [0,1]. The predicted waiting time reflects the waiting pressure a rider may face upon arrival at the store and is a key variable affecting the scheduling order. The predicted allowance time refers to the remaining time margin for a rider to complete the delivery of an order. Specifically, it is calculated by estimating the customer's delivery deadline (i.e., the latest allowed delivery time for the order) from the cumulative delivery time output by the time prediction model, then calculating the difference between this deadline and the rider's completion time based on the current time plus the estimated travel time, taking a non-negative value and normalizing it to [0,1]. The estimated grace time reflects the urgency of the current order's timeout; the smaller the grace time, the more urgent the order.

[0113] The warning label is set to 1 when the predicted grace time is less than a preset threshold (in this embodiment, the threshold corresponds to 5 minutes before normalization), and otherwise set to 0, providing the agent with an explicit warning signal of timeout risk. After incorporating the globally normalized adjacency matrix, the node feature matrix containing all the above features, and the warning label into the state space, an expanded state space is obtained. This state space deeply integrates the prediction results of the time prediction model with the topological information of the delivery network, enabling the agent to perceive the temporal constraints of the real environment when making decisions.

[0114] Secondly, the action space is defined as a one-dimensional discrete space, where each action corresponds to a legally accessible target node (pickup node or delivery node) in the delivery network. To enforce the business constraints of food delivery (riders must pick up the food from the merchant before delivering it to the user, i.e., the "pickup first, delivery later" constraint), a dynamic masking mechanism is introduced. The dynamic masking vector is updated synchronously with the state. A legal action must simultaneously satisfy three conditions: the target node is not the rider's current node; if the target node is a pickup node, the corresponding order must be in an unpicked state; if the target node is a delivery node, the corresponding order must be in a picked-up but not yet delivered state. The position corresponding to illegal actions in the masking vector is set to 0. Before the subsequent Softmax calculation of the policy network, the score corresponding to illegal actions is set to negative infinity, fundamentally preventing the generation of illegal actions without relying on penalty terms for soft constraints.

[0115] The reward function is defined as a combined reward that includes a positive reward for on-time delivery and a negative penalty for late delivery. The reward function consists of three parts: first, physical time cost, which deducts the sum of travel time and waiting time after each scheduling action, encouraging the agent to choose more time-efficient delivery routes; second, single-order delivery reward and penalty, where a rider receives a positive reward (20 points for base reward plus 30 points for on-time reward, totaling 50 points) if the actual delivery time does not exceed the delivery deadline predicted by the time prediction model, and a negative penalty (60 points for base penalty plus an additional penalty that increases linearly with the late delivery duration, with a coefficient of 2.0 points / minute); and third, a global zero-late delivery bonus, where an additional 50 points are awarded when all orders are delivered on time, encouraging the agent to pursue overall optimization rather than local suboptimal. The waiting time, delivery deadline, and other time parameters in the reward function are directly derived from the output of the time prediction model, ensuring that the reward signal accurately reflects the decision-making quality under time constraints.

[0116] Next, based on the completed food delivery MDP defined above, a graph attention policy network (GAT) is constructed as a joint policy and value estimation model for reinforcement learning. A graph attention network (GAT) is a neural network capable of extracting features from graph-structured data. Its core mechanism is to learn an attention coefficient for each node's edges with its neighbors, updating the node's feature representation by weighted aggregation of neighbor features. The attention coefficient is dynamically calculated from the feature similarity of node pairs, without needing to pre-specify fixed aggregation weights. In this embodiment, the special feature of GAT is that it incorporates the element values ​​of the globally normalized adjacency matrix as edge weight factors into the calculation of the attention coefficient, making the attention weight higher between node pairs that are spatially closer (shorter travel time), thereby implicitly encoding the spatiotemporal topology of the delivery network into the node feature representation.

[0117] The policy network adopts an Actor-Critic dual-head architecture, with both sharing the same GAT encoder. The GAT encoder consists of two layers of graph attention convolutional layers: the first layer maps node features from the input dimension to 64-dimensional intermediate features; the second layer further maps the 64-dimensional features to 128-dimensional high-level features, and finally outputs a node feature matrix containing global topological information and temporal constraint information.

[0118] The Actor decoder is responsible for outputting the probability distribution of legal actions. It concatenates the 128-dimensional features of the rider's current node with the average pooling result of the 128-dimensional features of all nodes in the entire graph to obtain a 256-dimensional query vector. This vector contains both local information about the rider's current position and macroscopic information about the global state of the delivery network. The features of each candidate target node are used as key vectors, and a scaled inner product operation is performed with the query vector to obtain the raw score for each action. After dynamic masking filtering, the probability distribution of legal actions is obtained through the Softmax function. During training, actions are sampled from this distribution to explore the environment; during inference, the action with the highest probability is selected to implement a greedy decision. The Critic decoder shares the GAT encoder weights with the Actor. It takes the 256-dimensional query vector as input and outputs a scalar state value estimate through a three-layer fully connected network (with hidden layer dimensions of 128 and 64 respectively), representing the expected cumulative reward obtainable from the current state according to the current policy. The shared encoder design between the Actor and Critic allows the error signal of the value network to directly guide the feature learning of the GAT layer, achieving co-evolution of representation capabilities.

[0119] Finally, based on the constructed graph attention policy network, the Proximal Policy Optimization (PPO) algorithm is used for training. PPO introduces an importance sampling ratio and prunes it to limit the policy update magnitude within a safe range (in this embodiment, the pruning threshold ε=0.2, i.e., the ratio is constrained in the range [0.8, 1.2]), preventing policy degradation or training collapse caused by excessively large single update steps. The value network uses Huber loss instead of mean squared error loss. Huber loss exhibits squared loss when the error is small, and degenerates into linear loss when the error is large, effectively suppressing the impact of abnormal gradients on training stability caused by the "time window cliff" (i.e., the reward function changes drastically when the rider is close to the delivery deadline) in the food delivery scenario. During training, the generalized advantage estimation (discount factor γ=0.99, GAE smoothing parameter λ=0.95) is used to calculate the advantage function, and an entropy regularization term is introduced to encourage the policy to remain exploratory.

[0120] The design of the multi-stage progressive knowledge transfer training strategy is based on the scale-independent property of GAT: the parameters of GAT (attention weight matrix) depend only on the node feature dimension and are independent of the number of nodes. Therefore, the model weights trained in a small-scale order scenario can be directly transferred to a larger-scale scenario without re-initialization. Training was conducted in three phases, progressing from small to large order sizes: The first phase involved training the model to convergence at a high learning rate (5e-4) in a fixed small-scale scenario (N=3), enabling the model to learn basic pickup and delivery timing constraints. The second phase loaded the weights from the first phase, reduced the learning rate to 1e-4, and performed generalization training on randomly generated delivery instances of the same size. To prevent infeasible "dead ends" in random instances, a reverse-engineered data generator (first constructing feasible delivery paths, then reverse-engineering time window parameters) was used to ensure that each training instance had at least one optimal feasible solution with zero timeout. The third phase seamlessly loaded the model weights trained in the second phase onto a larger-scale order scenario (N=4 to N=6), continuing training at a lower learning rate (5e-5) until convergence, completing cross-scale knowledge transfer. Experimental results show that compared to training from scratch, the three-stage progressive transfer training improved the model's convergence speed by 2.5 times in the N=6 scale scenario, increased the zero-timeout rate to 87%, and yielded a successfully trained scheduling strategy.

[0121] Example 5

[0122] In this embodiment, the steps of performing real-time inference, visualization, and policy evaluation on the time prediction model and the scheduling policy after training are included:

[0123] The system receives rider location information and order status information in real time through a data interface. It then uses the trained time prediction model to infer the current order and obtain the real-time predicted duration value. Finally, it uses the trained scheduling strategy to make decision-making inferences for the current delivery task and obtain the real-time delivery route decision result.

[0124] Based on the real-time predicted duration and the real-time delivery route decision results, a multi-dimensional visualization display is performed through a visualization rendering engine, outputting an order full-process time prediction panel, a rider delivery route map, an overtime risk level dashboard, and a prediction and decision consistency monitoring chart, thus obtaining a visualization display interface.

[0125] Based on the visualization interface, the hyperparameters of the trained scheduling strategy can be remotely configured. Based on the real-time prediction duration and the real-time delivery path decision results, the scheduling strategy versions under different hyperparameter configurations can be compared and tested in terms of on-time delivery rate, average delivery time, and late delivery penalty cost, so as to obtain the strategy evaluation results.

[0126] Specifically, firstly, the system continuously receives GPS location information and order status information reported by rider apps, as well as historical order feature data from the dispatch center, through an open data interface. The GPS location information includes the rider's real-time latitude and longitude coordinates and corresponding timestamps. The order status information includes the current fulfillment stage of each order (not accepted, accepted but not yet at the store, arrived at the store but not picked up, picked up but not delivered, delivered) and the actual time of each stage. The data interface uses an incremental push mechanism, triggering data reporting only when the rider's location or order status changes, thus reducing communication bandwidth consumption.

[0127] Upon receiving new data, the trained time prediction model is first invoked to infer the current active orders. During inference, 12 time-chain sequences from the three most recent historical orders of the current merchant are used as input. These are combined with the real-time status features of the current order (dynamic features such as the rider's current distance from the merchant and peak / valley indicators for the current time period) to generate four time-chain sequences for the current order according to the time-chain construction rules described in Example 2. These sequences are then input into the trained time prediction model for forward propagation, and four real-time predicted duration values ​​are output in parallel: cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time. For prediction requests initiated by users browsing the merchant's details page, if the current merchant has fewer than three historical orders, the missing portion is filled with global default features (such as the average feature of the merchant category) to ensure the robustness of the inference process. The inference latency for a single prediction is controlled within 28 milliseconds, meeting the response time requirements of real-time scheduling.

[0128] After obtaining the real-time predicted duration, the prediction results are transformed into the merchant's food preparation completion time and the customer's delivery deadline. These, along with the rider's current location and the real-time status of each order, are used to construct the current MDP state representation (i.e., the globally normalized adjacency matrix and node feature matrix defined in Example 4). This representation is then input into the trained graph attention policy network for decision-making and reasoning. The policy network outputs the rider's next access node in a greedy manner (selecting the legal action with the highest probability in the Actor's output probability distribution). This process iteratively continues until all pickup and delivery nodes for all pending orders are assigned, resulting in a complete real-time delivery path decision, including the rider's node access sequence and the corresponding expected arrival time sequence.

[0129] Secondly, based on real-time predicted duration and real-time delivery route decision results, a multi-dimensional visual display interface for operations personnel is generated through a visualization rendering engine. The visual display interface consists of four functional panels.

[0130] The first panel is the order fulfillment time prediction panel, which displays the predicted times for the four key fulfillment stages (order acceptance, arrival at the store, pickup, and delivery) of currently active orders in a timeline format. It compares the predicted times with the order creation time and the current time, intuitively presenting the progress status and estimated remaining time for each fulfillment stage. Operations personnel can use this panel to monitor the full-process time distribution of each order in real time and identify abnormal situations such as order delays or rider issues.

[0131] The second panel is a rider delivery route map, which uses a geographic map as the base map and displays the rider's current location (marked with a rider icon), visited nodes (marked in gray), currently heading to a node (marked in highlight), nodes to be visited (marked in color according to the type of food pickup / delivery), and the complete delivery route planned by the strategy network (connected by broken lines). This allows operations personnel to intuitively monitor the deviation between the rider's actual movement trajectory and the planned route.

[0132] The third panel is the timeout risk level dashboard, which classifies all active orders into timeout risk levels based on the estimated grace time. The dashboard displays the percentage of orders at each risk level in green, yellow, and red, and clearly marks yellow (estimated grace time is less than the preset threshold) and red (estimated time has exceeded) orders, allowing operations staff to quickly locate high-risk orders and intervene manually.

[0133] The fourth panel is a monitoring chart for the consistency of prediction and decision-making. It displays the deviation trend between the predicted waiting time and the actual waiting time for each historical completed order in the form of a line graph. This helps operations personnel assess the accuracy and stability of the time prediction model during the current operating period and identify the time periods when prediction deviations occur in concentrated areas (such as the lunch peak switching time), providing a basis for model update strategies.

[0134] Finally, based on the visual display interface, operators can remotely configure the hyperparameters of the trained scheduling strategy. Configurable items include the on-time delivery positive reward weight in the reward function, the timeout penalty coefficient, the prediction bias penalty threshold, as well as PPO training hyperparameters (pruning coefficient, discount factor, entropy regularization coefficient) and the refresh frequency of the time prediction model. Hyperparameter configurations are distributed to the scheduling server through parameterized configuration files and take effect without redeploying the service.

[0135] Supports A / B testing frameworks, allowing multiple scheduling strategy versions with different hyperparameter configurations to run simultaneously. For delivery tasks entering the system, they are routed to different strategy versions according to preset traffic allocation ratios for decision-making inference. Based on real-time predicted delivery time and real-time delivery route decision results, each strategy version is statistically compared in real time on three core metrics: on-time delivery rate (the percentage of orders whose actual delivery time does not exceed the delivery deadline), average delivery time (the average total time from order creation to rider delivery), and overtime penalty cost (the total penalty calculated based on the accumulated overtime time). The strategy evaluation results are then presented in a visual display interface in the form of comparison tables and trend line charts, providing data support for operations personnel to select the optimal hyperparameter configuration.

[0136] Example 6

[0137] In this embodiment, the online update step during the scheduling execution phase includes:

[0138] After each scheduling action is executed, the trained time prediction model is called again to refresh the predicted duration of orders that have not yet been fulfilled. For orders that have started to be fulfilled, the predicted duration is estimated by the sum of the actual time elapsed and the remaining predicted duration to obtain the updated predicted duration.

[0139] Based on the updated prediction duration, the difference between the actual waiting time and the predicted waiting time is calculated. When the difference exceeds a preset deviation threshold, a prediction deviation penalty term is triggered in the reward function to obtain an updated reward signal.

[0140] Based on the updated reward signal, the network parameters of the trained scheduling policy are updated online to obtain the online-updated scheduling policy.

[0141] Specifically, firstly, during the scheduling execution phase, each time a rider completes a scheduling action (i.e., arrives at a pickup or delivery node), the predicted duration refresh process is immediately triggered to ensure that the time parameters on which subsequent decisions are based always reflect the latest fulfillment progress.

[0142] The predicted duration is updated incrementally to reduce system computational overhead. For orders that have not yet started any fulfillment process (i.e., orders where the rider has not yet traveled to the corresponding merchant), the trained time prediction model is re-invoked. The dynamic features of the time chain sequence are updated with the latest historical order features of the merchant and the rider's current location information. Forward propagation is then performed again to obtain the updated predicted values ​​for the cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time. This allows for the recalculation of the merchant's food preparation completion time and the customer's delivery deadline. For orders that have already started fulfillment (the rider has already set off for the merchant or has picked up the food and is en route), the prediction model is not re-invoked for full prediction. Instead, an incremental estimation method is used: the difference between the current time and the order's start time is taken as the actual time elapsed, and the original total predicted time minus the actual time elapsed is taken as the remaining predicted time. The sum of these two values ​​yields the updated predicted duration estimate for the order. This incremental estimation method, while ensuring the reasonableness of the estimation, compresses the amount of update calculation for orders that have already begun to be fulfilled to a constant level, so that the overall prediction refresh delay after a single action is controlled within an acceptable range, ensuring the real-time nature of decision response.

[0143] After completing the above refresh, the updated predicted duration of all orders is resubmitted into the node feature matrix defined in Example 4 to update the predicted waiting time, predicted leeway time, warning labels, and other state features, forming the latest MDP state representation at the current moment for use in subsequent decision-making steps.

[0144] Second, based on the updated prediction duration, the prediction deviation is calculated for orders that have already completed the waiting process (i.e., orders where the rider has finished waiting for the food at the merchant and picked it up before leaving). The prediction deviation is defined as the absolute value of the difference between the actual waiting time and the predicted waiting time, where the actual waiting time is the actual time elapsed from the moment the rider arrives at the merchant to the moment they finish picking up the food, and the predicted waiting time is the estimated waiting time calculated by the time prediction model before the scheduling action is executed.

[0145] The prediction deviation penalty is a custom reward adjustment mechanism introduced in this embodiment. Its design aims to cultivate a perceptual preference for prediction reliability during the learning process, prioritizing orders from merchants with low historical prediction errors and stable preparation times. This proactively avoids high-risk decisions in scenarios with high prediction uncertainty. When the absolute value of the calculated prediction deviation exceeds a preset deviation threshold (set to 5 minutes in this embodiment), the prediction deviation penalty is triggered in the reward function. The penalty value is proportional to the deviation amount (penalty coefficient is 0.1, maximum penalty value is 1.0). When the prediction deviation does not exceed the threshold, no additional penalty is triggered, and the reward signal maintains the original reward function structure defined in Embodiment 4. The updated reward signal is obtained by superimposing the original reward signal with the prediction deviation penalty. The updated reward signal simultaneously includes direct incentives for delivery timeliness (on-time delivery reward and late delivery penalty) and implicit incentives for prediction consistency (deviation penalty), enabling the scheduling strategy to optimize delivery efficiency while maintaining the coupling stability of prediction and decision-making.

[0146] Third, based on the updated reward signal, the network parameters of the training-completed scheduling policy are updated online. The trigger condition for online update is: after accumulating a sufficient number of new interaction samples (including state, action, updated reward signal and complete trajectory fragment of the next state), the network parameters are updated once using the PPO algorithm and total loss function (a weighted sum of policy loss, Huber value loss and entropy regularization term) defined in Example 4.

[0147] The learning rate during the online update phase is significantly lower than that during the offline training phase (set to one-tenth of the offline training learning rate) to prevent excessive perturbation of the converged network weights by a small number of new samples in the online phase. Simultaneously, it allows the strategy to adapt appropriately to new distribution characteristics in the current operating environment (such as drift in travel time distribution due to sudden weather changes, or sudden surges in order volume due to marketing campaigns). The online update does not replace the base model weights obtained from offline training; instead, it fine-tunes them to preserve the generalization ability of the basic strategy. After the online update, a scheduling strategy is obtained that can continuously respond to dynamic changes in the real operating environment and maintain a high on-time delivery rate.

[0148] Example 7

[0149] In this embodiment, the adaptive confidence interval generation step for the prediction output of the trained time prediction model includes:

[0150] Based on the time prediction model that has been trained, multiple parallel forward propagations are performed on the same order input to be predicted during the inference phase. The mean of each prediction target is calculated as the point prediction value and the standard deviation is calculated as the local uncertainty estimate.

[0151] Based on the local uncertainty estimation, a sliding calibration window is maintained, and the predicted value, actual performance time and timestamp of historical completed orders are stored. The cumulative sum statistic is used to detect the distribution change of the residual sequence in the sliding calibration window. When a change point is detected, an exponentially decaying weight is applied to the samples before the change point to obtain the change point-aware exponentially decaying weight sequence.

[0152] Based on the exponentially decaying weight sequence with the change point awareness, the non-consistency score is normalized using the local uncertainty estimate, and the weighted quantile is calculated using the exponentially decaying weight sequence. For each prediction target, an adaptive confidence interval is output with the predicted point value as the center and the product of the weighted quantile and the local uncertainty estimate as the radius.

[0153] Based on the adaptive confidence interval, the width of the adaptive confidence interval is normalized and injected into the state space. The upper bound of the confidence interval of the cumulative delivery time is used as the dynamic risk boundary to replace the fixed delivery deadline for the calculation of the reward function, so as to obtain the updated state space and reward function.

[0154] Based on the updated state space and reward function, after each order is fulfilled, the actual observation value is appended to the sliding calibration window, triggering the change point detection and weighted quantile update, and obtaining the updated adaptive confidence interval parameters.

[0155] Specifically, firstly, this step introduces a local uncertainty estimation method based on test-time Dropout. The core idea of ​​this method is that the Dropout mechanism configured during the neural network training phase (in this embodiment, the Dropout rate of the GRU encoder is 0.2) is typically activated only during training and deactivated during inference. However, if Dropout is kept active during inference, the combination of neurons randomly dropped in the network differs each time the network propagates forward, resulting in slightly different outputs for the same input in multiple forward propagations. This output difference statistically reflects the model's uncertainty about the current input: when the input features are close to the training distribution, the difference between multiple outputs is small; when the input features deviate from the training distribution (such as extreme feature values ​​during peak periods), the difference increases significantly, naturally capturing the signal of "whether the model is confident about the current input," without needing to modify the model structure or retrain.

[0156] In practice, the Dropout layer of the GRU-Transformer model is kept active during the inference phase. T=10 parallel forward propagations are performed on the time-chain sequence input of the same order to be predicted, resulting in 10 sets of four-dimensional prediction vectors. The mean of each prediction target (cumulative order acceptance time, cumulative arrival time, cumulative pickup time, and cumulative delivery time) over the 10 propagation results is calculated as the point prediction value for that target. The standard deviation of the 10 results for each prediction target is calculated as an estimate of local uncertainty. Thanks to batch parallel computing, the additional inference overhead of 10 parallel forward propagations is approximately 2 to 3 times that of a single inference, while the total latency of a single decision still meets the real-time constraint of less than 28 milliseconds.

[0157] Second, this step involves two custom core mechanisms: sliding calibration window and variable point perception exponential decay weighting, which need to be explained in conjunction with the non-stationary characteristics of the food delivery scenario.

[0158] A sliding calibration window is a dynamic data structure that stores historical fulfillment order information with a fixed capacity (M=500 in this embodiment). Each record contains the order's point prediction vector, actual fulfillment time vector, and order completion timestamp. The 500 most recent records are retained in chronological order, and the oldest records are discarded in a first-in, first-out manner when the capacity is exceeded. The purpose of the sliding calibration window is to provide a historical reference distribution for subsequent inconsistency score calibration, enabling the generation of confidence intervals to be based on the recent true prediction error distribution, rather than relying on a fixed assumed distribution.

[0159] A turning point is the moment when the statistical characteristics (mean, variance, or distribution pattern) of time series data undergo a sudden change. In the food delivery scenario, the transition between lunch and dinner peak hours is a typical turning point: at the beginning of the lunch peak, the distribution of restaurant food preparation delays rapidly changes from a narrow distribution during off-peak hours to a wide-tailed distribution. If confidence intervals are calculated using a calibration window that includes off-peak data, the interval width will systematically underestimate the prediction uncertainty of the peak period, rendering the confidence intervals invalid.

[0160] This embodiment uses the Cumulative Sum (CUSUM) statistic to detect abrupt changes in the distribution of the residual sequence within the calibration window in real time. The CUSUM statistic accumulates the residuals (the absolute value of the difference between the actual delivery time and the predicted point value) up to the current time. When the accumulated amount exceeds a preset threshold h (h=5 in this embodiment), a change point is determined to have occurred. Specifically, the CUSUM statistic is continuously calculated for the residual sequence within the calibration window, parameterized by the mean μ0 of the absolute value of the residuals during the off-peak period and the allowable fluctuation κ (set as 0.5 times the standard deviation of the off-peak period residuals). When the statistic exceeds the threshold, a change point marker is triggered.

[0161] Upon detecting a change point, an exponentially decaying weight is applied to historical samples prior to the change point within the calibration window: the weight of each sample is calculated using the difference between the current time and the timestamp of each historical sample as the exponent and the decay factor γ as the base. During periods of rapid switching between peak and off-peak times (such as the midday peak starting at 11:00), γ is set to 0.9, allowing recent data to quickly dominate the calibration process; during periods of stable operation, γ is set to 0.99, fully utilizing historically accumulated data to improve calibration stability. This process yields a change-point-aware exponentially decaying weight sequence, which enables the confidence interval generation process to adaptively respond to the intraday non-stationarity of the food delivery scenario without manual intervention or periodic reset.

[0162] Third, the inconsistency score is a core metric in the conformal prediction framework for measuring the degree of difference between new samples and the historical calibration set. In this embodiment, for each historical record within the calibration window, the normalized inconsistency score is obtained by dividing the absolute value of the difference between the predicted point value and the actual delivery time of each prediction target by the corresponding local uncertainty estimate. The purpose of normalization using the local uncertainty estimate is to make the inconsistency scores under different uncertainty levels comparable—when the model uncertainty is high during peak periods, the normalized inconsistency score corresponding to the same prediction error will naturally decrease, thereby automatically tightening the judgment criteria for "abnormal deviations" and preventing normal prediction errors during peak periods from being misjudged as abnormal.

[0163] Based on a change-point-aware exponential decay weighted sequence, weighted quantiles are calculated for the normalized inconsistency scores of each historical sample within the calibration window (the confidence level is dynamically adjusted according to the current time period: a wider confidence level is used during peak periods to ensure more robust coverage, and a narrower confidence level is used during off-peak periods to reduce interference with decision-making). The weighted quantiles are calculated as follows: the inconsistency scores of each historical sample are arranged in ascending order, normalized with the corresponding exponential decay weights, and used as the cumulative probability. The score at which the cumulative probability first does not fall below the target quantile is taken as the weighted quantile result.

[0164] For each prediction target, an adaptive confidence interval is constructed with the predicted point value as the center and the product of the weighted quantile and the local uncertainty estimate as the radius. The width of this confidence interval adaptively shrinks or expands with changes in prediction uncertainty (local uncertainty estimate) and historical calibration error distribution (weighted quantile), satisfying the marginal coverage guarantee under finite sample conditions, without specifying any parameterized distribution assumptions.

[0165] Fourth, the adaptive confidence interval widths (i.e., twice the product of the weighted quantile and the local uncertainty estimate) of each of the four prediction targets are normalized to [0,1] and then injected as new node feature dimensions into the state space node feature matrix defined in Example 4. This expands the original 10-dimensional features by four uncertainty width feature dimensions, correspondingly expanding the node feature dimensions. Only the input projection matrix dimension of the first layer of the graph attention strategy network needs to be adjusted, without affecting the rest of the network structure. The uncertainty width feature enables the graph attention strategy network to perceive the credibility of each order time prediction during decision-making, guiding the strategy to proactively adopt more conservative scheduling decisions when prediction uncertainty is high.

[0166] Simultaneously, the fixed delivery deadline in the reward function (directly calculated from the point prediction value) is replaced with a dynamic risk boundary based on the upper bound of the confidence interval for cumulative delivery duration. The upper bound of the confidence interval for cumulative delivery duration (i.e., the latest reasonable delivery time under a pessimistic prediction scenario) is used as the time boundary for determining whether a timeout has occurred in the reward function. The risk coefficient is adaptively adjusted according to the current timeout rate target: a larger risk coefficient (corresponding to a more pessimistic delivery time estimate) is used during peak periods to proactively reserve a safety margin; a smaller risk coefficient is used during off-peak periods to avoid overly conservative decision-making. This results in an updated state space and reward function, enabling the reinforcement learning agent to manage timeout risk in a mathematically controllable manner during both training and inference.

[0167] Fifth, after each order's full-process fulfillment (the rider completes the delivery action and reports the actual delivery time), the order's point prediction vector, actual fulfillment time vector, and delivery timestamp are appended as new records to the sliding calibration window. Simultaneously, the CUSUM change point detection process and the weighted quantile update process are triggered. Change point detection is performed incrementally immediately after the new record is appended, requiring only an update to the CUSUM statistics for the latest residual record; the computational overhead is negligible. Weighted quantile updates are triggered for recalculation when the number of records in the calibration window changes. If the window is full, the oldest record is deleted simultaneously, maintaining the window capacity. The entire online update process requires no retraining of the GRU-Transformer main model, achieving continuous online adaptation of confidence interval parameters with zero additional training overhead. This results in updated adaptive confidence interval parameters, ensuring effective coverage performance of the confidence interval across different operational periods.

[0168] Example 8

[0169] In this embodiment, the parameterized inconsistency metric optimization step for adaptive confidence intervals includes:

[0170] Based on the adaptive confidence interval, the upper and lower bounds of the adaptive confidence interval are mapped to parameterized inconsistency scores through a learnable parameterized transformation.

[0171] Based on the parameterized inconsistency score, and using the parameterized inconsistency score as the optimization variable, under the constraint that the prediction coverage of the sliding calibration window is not lower than the preset target confidence level, the iterative optimization method is used to minimize the conformal set radius to obtain the optimized parameterized inconsistency metric.

[0172] Based on the optimized parameterized inconsistency metric, the equivalent uncertainty scalar for each prediction target is calculated. Based on the relationship between the equivalent uncertainty scalar and the prediction margin time, a green label is used when the prediction margin time is greater than the equivalent uncertainty scalar, a yellow label is used when the prediction margin time is not greater than the equivalent uncertainty scalar but is greater than zero, and a red label is used when the prediction margin time is not greater than zero, thus obtaining a three-level risk label.

[0173] Specifically, this step first introduces a custom metric, the parameterized inconsistency score. Its design motivation stems from the inherent conservatism of the adaptive confidence interval constructed in Example 7. In the conformal prediction framework of Example 7, the inconsistency score is obtained by dividing the absolute value of the prediction error by the local uncertainty estimate. This linear normalization method tends to generate overly wide confidence intervals when the prediction error distribution is heavy-tailed (e.g., extreme delays during peak periods), leading to an overly conservative scheduling strategy. Therefore, this example introduces a learnable parameterized transformation to perform a more flexible nonlinear mapping of the confidence interval boundaries, compressing the redundancy width of the confidence interval while maintaining coverage.

[0174] The parameterized inconsistency score is constructed as follows. Using the adaptive confidence interval obtained in Example 7 as input, its upper and lower bounds are linearly scaled and offset through an affine transformation containing learnable parameters, resulting in mapped upper and lower bound pseudo-values. The absolute value of the difference between the mapped upper bound pseudo-value and the corresponding actual performance time is then used as the parameterized inconsistency score for that historical sample. The learnable parameters consist of two scalars: a scaling parameter controls the overall contraction or expansion of the confidence interval, and an offset parameter controls the systematic translation of the confidence interval. Both are initialized to values ​​that degenerate the transformation to an identity mapping (scaling parameter initialized to 1, offset parameter initialized to 0), ensuring that the parameterized inconsistency score degenerates to the original inconsistency score in Example 7 when the parameters are not optimized, thus ensuring the rationality of the optimization starting point. By introducing the above-mentioned learnable parameterized transformation, the model gains the ability to actively adjust the boundaries of the confidence interval, providing optimizable degrees of freedom for subsequent constraint optimization steps.

[0175] Secondly, the conformal set radius refers to the quantile value obtained by taking the quantile of the inconsistency score of historical samples within the calibration window at the target confidence level within the conformal prediction framework. The smaller the conformal set radius, the tighter the confidence interval constructed with that quantile as the threshold, corresponding to more accurate uncertainty quantification. This step uses the parameterized inconsistency score as the optimization variable. Under the constraint that the prediction coverage rate of historical samples within the sliding calibration window (i.e., the proportion of actual performance time falling into the confidence interval) is not lower than the preset target confidence level (set to 0.9 in this embodiment, i.e., 90% of historical samples should be covered by the confidence interval), the conformal set radius is minimized, thereby obtaining the most compact parameterized inconsistency measure under the premise of effective coverage guarantee.

[0176] The optimization process employs an iterative optimization method. In each iteration, the parameterized inconsistency score of all historical samples within the calibration window is calculated using the current learnable parameters. The weighted quantile corresponding to the target confidence level (the weights are the change-point perception exponential decay weight sequence obtained in Example 7) is taken as the current conformal set radius. Simultaneously, the proportion of historical samples whose actual fulfillment time falls within the confidence interval constructed with the current parameters is calculated as the coverage estimate. If the coverage meets the constraint (not less than 0.9), the gradient of the conformal set radius with respect to the learnable parameters is calculated, and the parameters are updated along the gradient descent direction to reduce the radius. If the coverage does not meet the constraint, the parameters are updated along the gradient ascent direction to expand the confidence interval until the coverage constraint is met again. The iteration step size is set to 0.01, and the maximum number of iterations is 100. The process terminates early when the change in the conformal set radius between two adjacent iterations is less than 1e-4. After iterative optimization, the optimized parameterized inconsistency metric is obtained, corresponding to a set of learnable parameter values ​​that can compress the confidence interval width to the greatest extent while ensuring coverage. Compared to the original scheme in Example 7, the optimized parameterized inconsistency metric reduces the average width of the confidence interval by about 18% during the midday and evening peak hours, while maintaining a measured coverage of no less than 90%.

[0177] Finally, the equivalent uncertainty scalar is a custom concept introduced in this embodiment. It is defined as a single scalar value expressed as the product of the optimized parameterized inconsistency measure (i.e., the optimized conformal set radius) and the local uncertainty estimate. Physically, it represents the time length (in minutes) corresponding to the confidence interval radius at the current level of prediction uncertainty. The equivalent uncertainty scalar integrates the model's local uncertainty (reflected by the standard deviation inferred from Dropout) and the global calibration error distribution (reflected by the optimized conformal set radius), compressing the confidence interval width information into a single comparable time-dimensional scalar. This facilitates direct comparison with the predicted margin of error, enabling risk grading.

[0178] Specifically, the equivalent uncertainty scalar is calculated in the following way:

[0179] Let the optimized parameterized inconsistency metric be q' (i.e., the conformal set radius after iterative optimization, which is a dimensionless scalar), and the local uncertainty estimate of the current prediction target be σ. local (i.e., the standard deviation of the predicted values ​​obtained from multiple parallel forward propagations of the same order during the inference phase, in minutes), then the equivalent uncertainty scalar σ eq The calculation formula is:

[0180] σ eq =q'×σ local

[0181] Where, σ eq The unit is minutes, and its physical meaning is: the length of time corresponding to the radius of the confidence interval at the current level of prediction uncertainty. This scalar integrates the global calibration error distribution (represented by q') and the local model uncertainty (represented by σ). local This approach compresses the confidence interval width information into a single time-dimensional scalar, allowing for direct numerical comparison with the predicted margin of time.

[0182] Based on the equivalent uncertainty scalar σ eq With the anticipated grace time T margin Based on the size relationship, the risk classification rules are as follows:

[0183] When T margin >σ eq When this occurs, it is marked as green (low risk).

[0184] When 0 <T margin ≤σ eq When this occurs, it is marked as yellow (medium risk).

[0185] When T margin When the value is ≤0, it is marked in red (high risk).

[0186] The three-tiered risk label classification is based on the following criteria: When the predicted time margin is greater than the equivalent uncertainty scalar, it indicates that even with a deviation in the current level of uncertainty in the prediction, the rider still has sufficient time margin to ensure on-time delivery, and the order is marked as green (low risk). When the predicted time margin is not greater than the equivalent uncertainty scalar but is still greater than zero, it indicates that the current predicted time margin has been completely covered by the predicted uncertainty. In a pessimistic prediction scenario, there is a risk of timeout, but the current prediction still shows on-time delivery, so the order is marked as yellow (medium risk), prompting the dispatch system to pay close attention to and prioritize the dispatch of this order. When the predicted time margin is not greater than zero, it indicates that the current prediction has determined that the delivery time has exceeded the delivery deadline, so the order is marked as red (high risk), triggering emergency dispatch intervention. The three risk labels combine information from both the accurate prediction value and the uncertainty estimate, significantly reducing the misjudgment rate during peak periods with high uncertainty compared to a single threshold rule that only relies on the point prediction value.

[0187] Example 9

[0188] In this embodiment, the steps of introducing expected timeout cost estimation and timeout rate constraints include:

[0189] Using the orders marked in yellow in the three-level risk labels and the sliding calibration window as input, a subset of historical orders is retrieved in the sliding calibration window whose absolute value of the difference between the predicted grace time and the predicted grace time of the order does not exceed a preset neighborhood radius. The actual timeout probability of the historical order subset is calculated as the expected timeout cost of the order. When the number of samples in the historical order subset is less than the preset minimum number of samples, it degenerates into the global on-time rate estimation of the sliding calibration window to obtain the expected timeout cost estimate.

[0190] Based on the expected timeout cost estimate, the expected timeout cost estimate is multiplied by a preset timeout cost coefficient and added as an additional penalty to the reward function to obtain the extended reward function;

[0191] Based on the extended reward function, a total loss function is constructed in the form of a weighted sum of the policy gradient objective, state value regression loss, and timeout probability classification loss. A Lagrange relaxation constraint term is introduced into the total loss function, and the preset timeout rate upper limit is encoded as a probability constraint. The Lagrange multiplier in the Lagrange relaxation constraint term is adaptively updated through a dual optimization method to obtain the total loss function with timeout rate constraint.

[0192] Based on the total loss function with introduced timeout rate constraints, a state value estimation head and a timeout probability auxiliary output head are extended in the graph attention policy network. The graph attention policy network is then trained using the total loss function with introduced timeout rate constraints to obtain a training completion scheduling policy that satisfies the timeout rate constraints.

[0193] Specifically, firstly, the core objective of this step is to perform fine-grained timeout probability quantification on the orders marked as yellow (medium risk) in Example 8, thereby providing a differentiable continuous probability signal for the subsequent reward function design, replacing the binarized timeout determination logic, and enabling the scheduling strategy to perceive the timeout risk differences of orders within the yellow interval using gradient signals.

[0194] The estimation of expected timeout costs relies on a nonparametric probability estimation method based on neighborhood retrieval. The core idea of ​​this method is to predict that historical orders with similar grace times have high similarity in delivery conditions (rider driving time, merchant food preparation time distribution, etc.). Therefore, the actual timeout rate of these historical neighboring orders is used as the timeout probability estimate of the current order, realizing probabilistic inference based on historical experience without the need to build a parameterized probability model.

[0195] The specific implementation steps are as follows: For each current order marked with a yellow risk, use its predicted grace time as the query key. Traverse all historical samples within a sliding calibration window (capacity M=500) to filter out a subset of historical orders whose absolute value of the difference between the predicted grace time and the current order's predicted grace time does not exceed a preset neighborhood radius (in this embodiment, the neighborhood radius is set to 2 minutes, corresponding to the original time unit before normalization). For each record in this subset of historical orders, query its actual fulfillment result (whether the actual delivery time exceeds the delivery deadline). Calculate the proportion of actually overdue orders within the subset to the total number of samples, using this as the estimated expected overdue cost for the current order.

[0196] When the sample size of the historical order subset is less than the preset minimum sample size (set to 10 in this embodiment), the samples in the neighborhood are insufficient to support a reliable probability estimate. In this case, the estimate is degraded to the global historical timeout rate of the sliding calibration window (i.e., the proportion of actually timed-out orders among all historical samples in the calibration window) as a conservative estimate. This adaptive degradation mechanism ensures the robustness of the estimate during the system cold start phase (when historical sample accumulation is insufficient) or when current operating conditions are extremely rare (when the predicted grace time is in an extreme range and historical neighbors are very few). Finally, the expected timeout cost estimate for each yellow-risk order is obtained, which ranges from [0,1]. A higher value indicates a greater probability of timeout for the current order under similar historical circumstances.

[0197] Specifically, the actual timeout probability of the aforementioned subset of historical orders is calculated in the following manner:

[0198] Let the subset of historical orders contain N subset For each of the historical order records, extract its actual delivery time t. actual With the corresponding delivery deadline t deadline (Calculated by adding the order creation time to the predicted cumulative delivery time output by the time prediction model) Compare this to the total number of orders, N, whose actual delivery time exceeds the delivery deadline. overtime That is, satisfying t actual >t deadline The actual timeout probability P is the number of records. overtime The calculation formula is:

[0199] P overtime =N subset / N overtime

[0200] When the sample size N of the historical order subset subset Less than the preset minimum sample size N min (N in this embodiment) min When =10), it degenerates into a global on-time rate estimate using all historical samples within the sliding calibration window, i.e.:

[0201] P overtime =1−R global

[0202] Among them, R global R is the ratio of the number of orders actually delivered on time to the total number of orders in the sliding calibration window. global =N on-time / N total N on-time N represents the number of orders whose actual delivery time within the window does not exceed the delivery deadline. total This represents the total number of orders within the window.

[0203] Second, based on the estimated expected timeout cost, it is multiplied by a preset timeout cost coefficient (in this embodiment, the timeout cost coefficient is set to 15, with the same dimensions as the reward / penalty score in the reward function of Embodiment 4) to obtain an additional penalty for yellow-risk orders. For green-risk orders, the estimated expected timeout cost is zero (the historical timeout rate is close to zero under ample safety margin), and the additional penalty is not activated; for red-risk orders, the original timeout penalty in Embodiment 4 already covers this situation, and the additional penalty is not superimposed, only effective for medium-risk orders in the yellow range.

[0204] Adding the aforementioned additional penalty terms to the original reward function defined in Example 4 (which includes physical time cost, single order delivery reward / penalty, and global zero timeout reward) yields the extended reward function. Based on the original binary reward / penalty structure, the extended reward function introduces a continuously changing risk penalty gradient for the yellow zone: the closer the predicted grace time is to zero (the higher the historical nearest neighbor timeout rate), the greater the additional penalty, driving the scheduling strategy to prioritize yellow orders with high expected timeout costs, thereby fundamentally reducing the probability of yellow orders turning into red orders.

[0205] Third, this step constructs a total loss function in the form of a weighted sum of the policy gradient objective, state value regression loss, and timeout probability classification loss, and introduces a Lagrange relaxation constraint term to encode the timeout rate upper limit as a hard constraint.

[0206] The timeout probability classification loss is a newly added auxiliary loss term in this step. Its function is to help the timeout probability auxiliary output head of the supervised graph attention policy network accurately predict the timeout probability of each order, providing a differentiable constraint violation metric for the Lagrange constraint term. The timeout probability classification loss adopts a binary cross-entropy form, using the expected timeout cost estimate as the soft label and the output value of the timeout probability auxiliary output head as the predicted probability, and calculates the cross-entropy loss between the two.

[0207] Lagrange relaxation is a classic method for transforming constrained optimization problems into unconstrained optimization problems. The basic idea is to multiply the constraints by a non-negative Lagrange multiplier and add it to the objective function to form a Lagrange function. The original constrained optimization problem is solved by minimizing the Lagrange function with respect to the original variables and maximizing it with respect to the Lagrange multiplier (i.e., finding the saddle point). In this embodiment, a preset timeout rate cap (set to 10% in this embodiment, meaning the actual timeout rate of the scheduling strategy on the validation set is required not to exceed 10%) is encoded as a probabilistic constraint. The difference between the mean of the timeout probabilities of all orders in the current batch and the timeout rate cap is used as the constraint violation, multiplied by the Lagrange multiplier, and added to the total loss function as a Lagrange relaxation constraint term.

[0208] The adaptive update of the Lagrange multiplier employs a dual ascent method: after each training batch, the constraint violation amount (the difference between the average actual timeout probability and the upper limit of the timeout rate) of the current batch is calculated. The Lagrange multiplier is updated by multiplying the constraint violation amount by a preset step size (0.001 in this embodiment), and the multiplier value is truncated to a non-negative range (negative Lagrange multipliers are not allowed). When the actual timeout rate exceeds the upper limit, the multiplier increases, the constraint penalty term in the total loss function is strengthened, and the policy optimization direction shifts towards satisfying the constraints; when the actual timeout rate is lower than the upper limit, the multiplier decreases, the constraint penalty term is weakened, and the policy optimization direction shifts towards improving delivery efficiency. Through this dual optimization mechanism, the Lagrange multiplier is adaptively adjusted to a level that matches the constraint tightness, so that the actual timeout rate of the scheduling policy on the validation set after training convergence naturally stabilizes near the preset upper limit, without the need for manual search of reward and penalty weight parameters.

[0209] The total loss function consists of three weighted terms: policy gradient target loss (weight 1.0), Huber loss for state value regression (weight 0.5), and binary cross-entropy loss for timeout probability classification (weight 0.3). Adding the Lagrange relaxation constraint term (weight dynamically determined by the Lagrange multiplier) yields the total loss function with timeout rate constraint.

[0210] Fourth, based on the total loss function with introduced timeout rate constraints, a timeout probability auxiliary output head is added to the Actor-Critic dual-head architecture of the graph attention policy network constructed in Example 4. The timeout probability auxiliary output head shares the 128-dimensional node features output by the GAT encoder with the Critic. Taking the delivery node features of each order as input, it outputs a scalar prediction value (range [0,1]) of the timeout probability of the order through a two-layer fully connected network (64 hidden layer dimensions, Sigmoid activation function of the output layer), representing the predicted probability of the order timeout under the current state and scheduling policy. The state value estimation head structure remains unchanged.

[0211] The extended graph attention strategy network described above was trained end-to-end using a total loss function with introduced timeout rate constraints. Training employed the PPO algorithm framework defined in Example 4, with hyperparameter settings consistent with Example 4 (pruning threshold ε = 0.2, discount factor γ = 0.99, GAE smoothing parameter λ = 0.95). A dual update step using Lagrange multipliers was then added on top of this. During training, after each training epoch, the actual timeout rate was calculated on the independent validation set. If the actual timeout rate was below a preset upper limit for five consecutive epochs, early stopping was triggered to save the current model weights. After the above training process, a training completion scheduling strategy that satisfies the timeout rate constraint was obtained. This strategy maintained a stable actual timeout rate of 9.2% on the test set, meeting the preset 10% timeout rate upper limit constraint. Simultaneously, it improved the on-time delivery reward score by 3.1% compared to the unconstrained baseline strategy, achieving an effective balance between delivery timeliness and the hard constraint of the timeout rate.

[0212] Example 10

[0213] In this embodiment, the step of performing multi-rider joint path planning using the trained scheduling strategy during the inference phase includes:

[0214] Using the negative of the predicted grace time for each rider's orders as the priority weight, calculate the weighted priority index for each rider, and determine the rider decoding order according to the weighted priority index from largest to smallest to obtain the rider decoding order sequence;

[0215] According to the rider decoding sequence, for the current rider to be decoded, the initial score of each candidate node is calculated by integrating the node features and the predicted waiting time; when the current rider has included the target pickup node in the current rider's delivery route, the time conflict degree between the target pickup node and the current rider to be decoded is calculated based on the predicted waiting time, and the initial score of the target pickup node is suppressed accordingly to obtain the adjusted candidate node score distribution;

[0216] Based on the adjusted candidate node score distribution, the candidate node with the highest score is selected as the next action node for the current rider to be decoded. The next action node is then incorporated into the delivery path of the current rider to be decoded. The process is repeated for the next rider to be decoded, following the rider decoding sequence. The initial score and score suppression steps for each candidate node are calculated by fusing the node features and the predicted waiting time, until all riders have completed the allocation of all pick-up and delivery nodes in their respective delivery paths, thus obtaining the joint optimal delivery path for all riders.

[0217] Specifically, in the scenario of multi-rider joint route planning, the delivery system needs to simultaneously assign pickup and delivery node sequences to multiple riders (K=3 to 6 in this embodiment) to form a joint delivery route scheme. This problem belongs to the multi-agent combinatorial optimization problem. A naive parallel greedy method will produce suboptimal or even infeasible route schemes because it ignores the resource competition among riders (multiple riders may compete for the same merchant's pickup node at the same time, resulting in accumulated waiting time). This embodiment uses a sequential decoding mechanism to solve the above problem. Its core idea is to determine the priority of decoding (route allocation) according to the rider's urgency. The rider with the highest urgency completes the route allocation first, and subsequent riders make decisions under the constraints of the path of the preceding rider, thereby avoiding scheduling failures caused by resource competition.

[0218] The weighted priority index is a custom scalar used to quantify the overall urgency of each rider's current situation. Its calculation method is as follows: For rider k, firstly, the predicted time leeway for all pending orders under his / her jurisdiction (including all accepted but not yet delivered orders) is collected. The negative of the predicted time leeway for each order (i.e., a negative predicted time leeway value) is used as the priority weight for each order. A negative predicted time leeway indicates that the order has timed out; in this case, its negative value is positive, and the priority weight is the highest. A smaller positive predicted time leeway indicates that the order is about to time out; the absolute value of the negative value is larger, and the priority weight is higher. Sufficient predicted time leeway results in a smaller absolute value of the negative value and a lower priority weight. The priority weights of all orders under rider k's jurisdiction are weighted and summed (weights normalized to [0,1]) to obtain rider k's weighted priority index. Intuitively, the larger the weighted priority index, the higher the overall risk of timeout for the orders currently under the rider's jurisdiction, and route allocation should be prioritized to secure key pickup nodes.

[0219] The riders are sorted in descending order of their weighted priority indices to obtain the rider decoding order sequence. When multiple riders have the same weighted priority index, the rider number is used as the secondary sorting key to ensure the determinism of the decoding order. The rider decoding order sequence is recalculated based on the latest status after each scheduling action (i.e., after completing the route allocation step for one rider) to respond to the dynamic impact of real-time fulfillment progress on the urgency of each rider.

[0220] Secondly, following the rider decoding sequence, the candidate node score calculation and suppression process is performed sequentially for each rider to be decoded. For the current rider to be decoded, the Actor decoder of the trained graph attention policy network is invoked. The concatenation result of the 128-dimensional features of the current rider's node and the mean of the features of all nodes in the graph is used as the query vector. Scaled inner product attention is then calculated for all legal candidate nodes (the set of nodes after dynamic masking) to obtain the basic attention score of each candidate node.

[0221] Based on the basic attention score, the initial score is adjusted by incorporating the predicted waiting time. Specifically, for each candidate pickup node, its predicted waiting time is normalized to [0,1], and then the negative of the predicted waiting time (i.e., the negative predicted waiting time) is added to the basic attention score to obtain the initial score. The shorter the predicted waiting time of the pickup node, the greater the positive correction, and the higher the corresponding initial score. The guidance strategy prioritizes merchants with shorter waiting times to avoid riders spending long periods waiting for food. For candidate delivery nodes, since they do not involve the waiting process, the initial score is directly equal to the basic attention score, and no additional correction is made.

[0222] The degree of time conflict is a custom metric introduced in this embodiment to characterize the risk of overlapping waiting times at the target pickup node between the rider currently waiting to be decoded and the rider who has already completed route allocation. Its definition and calculation method are as follows: If the rider in the previous order (i.e., the rider whose decoding order precedes the current rider and whose route allocation has been completed) has included a target pickup node in their delivery route, it means that the previous rider is expected to arrive at the merchant at a certain time to pick up the food and occupy the merchant's food preparation resources. If the rider currently waiting to be decoded also plans to go to the same merchant to pick up the food, the overlapping arrival times of the two riders may cause the current rider to wait additionally. The degree of time conflict is measured by the reciprocal of the absolute value of the difference between the expected arrival time of the previous rider and the expected arrival time of the current rider (the smaller the absolute value of the difference, the closer the times, and the higher the degree of conflict), multiplied by the merchant's current estimated waiting time (the longer the waiting time, the greater the actual impact of the conflict) as an amplification factor, thus obtaining the degree of time conflict between the target pickup node and the current rider waiting to be decoded.

[0223] For target food pickup nodes whose time conflict exceeds a preset conflict threshold (in this embodiment, the preset conflict threshold is set to 0.6, with the same dimensions as the normalized predicted waiting time), their initial scores are suppressed: the initial score is subtracted from the time conflict level multiplied by a suppression coefficient (in this embodiment, the suppression coefficient is set to 2.0), which significantly reduces the score of food pickup nodes with serious time conflicts with the preceding rider, guiding the current rider to actively avoid merchants where waiting times may overlap. After the above initial score calculation and score suppression processing, the adjusted scores of all legal candidate nodes are normalized using the Softmax function to obtain the adjusted candidate node score distribution.

[0224] Next, based on the adjusted candidate node score distribution, the candidate node with the highest score is selected as the next action node for the current rider to be decoded. This action node is included in the current rider's delivery route sequence and marked as "assigned" from the global set of legal candidate nodes (but this is only done for the current rider's delivery node; in special cases, multiple riders may be allowed to go to the pickup node, in which case it is not removed from the global candidate set, but only conflict suppression is applied to the score).

[0225] After assigning an action node to the current rider, the system switches to the next rider to be decoded according to the rider decoding sequence. The initial score calculation and time conflict suppression process is repeated to determine the next action node for the next rider. When switching to the next rider, the path node information and corresponding estimated arrival time of the previous rider are updated to the global state to ensure that the conflict detection of subsequent riders uses the latest path information.

[0226] The aforementioned sequential decoding and score suppression process is executed cyclically among all riders. Each iteration assigns one action node to each rider until all pickup and delivery nodes in all riders' delivery paths are assigned (i.e., all pickup and delivery actions for all orders have been assigned to their respective rider paths), at which point the iteration terminates. The final output includes the complete sequence of nodes visited by each rider (all permutations of pickup and delivery nodes) and the corresponding joint delivery path scheme with the expected arrival time, i.e., the joint optimal delivery path for all riders. The generation latency of the joint optimal delivery path (including the sequential decoding iterations for all riders) is controlled within 50 milliseconds, meeting the response requirements of real-time scheduling. Experimental results show that, compared to the parallel greedy baseline that ignores conflicts between riders, the sequential decoding and conflict suppression mechanism in this embodiment improves the on-time delivery rate of multi-rider joint delivery by 5.3 percentage points and reduces the average delivery time by 2.1 minutes in a K=4 scale scenario.

[0227] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0228] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0229] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments of this application are sometimes grouped together into a single embodiment, figure, or description thereof.

[0230] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying abstract and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying abstract and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0231] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can design alternative embodiments without departing from the scope. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

[0232] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A take-out delivery scheduling method based on deep reinforcement learning, characterized in that, include: For each order in the historical order data, starting from the order creation time, information on different fulfillment stages is encoded using left-padding and dynamic feature updates to construct multiple time-chain sequences; Based on the multiple time chain sequences, with the cumulative time for order acceptance, cumulative time for arrival at the store, cumulative time for food pickup, and cumulative time for delivery as prediction targets, a time prediction network is trained to obtain a trained time prediction model. Based on the cumulative pickup time and cumulative delivery time obtained from the time prediction model after training, they are converted into the merchant's food preparation completion time and the customer's delivery deadline, respectively. The predicted waiting time and predicted margin time are calculated and used as the core state features to construct a Markov decision process for food delivery that includes a state space, an action space, and a reward function. Based on the Markov decision process for food delivery, a graph attention policy network is constructed, trained using a proximal policy optimization algorithm, and trained using a multi-stage progressive knowledge transfer training strategy to complete the training of the cross-scale scheduling policy, thus obtaining the trained scheduling policy.

2. The method of claim 1, wherein, The steps of constructing the multiple time chain sequences and the obtained trained time prediction model include: For each order in the historical order data, four time chains are constructed according to the different starting states of order fulfillment. Each time chain is padded with left zeros and dynamically updated according to the corresponding fulfillment state to obtain four time chain sequences. The four time-chain sequences are input into the encoder to extract temporal evolution features. In the decoding stage, future information is masked by a sequence mask matrix. The temporal evolution features are interacted through a multi-head cross-attention mechanism. The cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time are output in parallel through a linear layer. Using the cumulative order acceptance time, cumulative in-store time, cumulative food pickup time, and cumulative delivery time as supervision targets, and the mean squared error as the loss function, the Adam optimizer is used to train the time prediction network to obtain the trained time prediction model.

3. The method according to claim 2, characterized in that, The specific structure and training method of the encoder and decoder of the time prediction network include: The encoder is a combination of a gated loop unit and a Transformer. The gated loop unit performs stepwise temporal encoding on the four time chain sequences, and the output is normalized to obtain the encoded features. The encoded features are input into the decoder, which adopts a multi-layer Transformer structure. The decoder masks future time information through a sequence masking matrix and aligns the encoded features with the mask sequence through a multi-head cross-attention mechanism. The decoder outputs the cumulative order acceptance time, the cumulative time to store arrival time, the cumulative food pickup time, and the cumulative delivery time in parallel through a linear layer. When training the time prediction network, multiple time chain sequences corresponding to consecutive historical orders from the same merchant are used as input. The mean squared error between the cumulative order acceptance time, cumulative store arrival time, cumulative food pickup time, and cumulative delivery time output by the decoder of the encoded features and the actual cumulative order acceptance time, actual store arrival time, actual food pickup time, and actual delivery time of the latest order is used as the loss function for supervised training, thereby obtaining the trained time prediction model.

4. The method according to claim 3, characterized in that, The steps for constructing the Markov decision process for food delivery and training the graph attention policy network include: Based on the predicted waiting time and the predicted margin time, the global normalized adjacency matrix, node features and warning labels are incorporated into the state space, wherein the warning label is set to one when the predicted margin time is lower than a preset threshold, and otherwise set to zero, to obtain the expanded state space; Based on the extended state space, the action space is defined as a discrete choice between picking up or delivering food, and the first-pick-then-delivery constraint is enforced through a masking mechanism. The reward function is defined as a combined reward that includes a positive reward for on-time delivery and a negative penalty for late delivery, thus obtaining a fully defined Markov decision process for food delivery. Based on the defined Markov decision process for food delivery, a graph attention policy network is constructed. The graph attention policy network encodes the node features with the global normalized adjacency matrix as the edge weights to obtain graph attention encoded features. Based on the graph attention encoded features, the action probability distribution and scalar state value estimate are output respectively to obtain the constructed graph attention policy network. Based on the constructed graph attention policy network, a near-end policy optimization algorithm is adopted to perform multi-stage progressive training according to the order size from small to large. In each stage, the model weights completed in the previous stage are loaded as initialization parameters to complete cross-scale knowledge transfer and obtain the trained scheduling policy.

5. The method according to claim 1, characterized in that, It also includes steps for real-time inference, visualization, and policy evaluation of the training completion time prediction model and the training completion scheduling policy, including: The system receives rider location information and order status information in real time through a data interface. It then uses the trained time prediction model to infer the current order and obtain the real-time predicted duration value. Finally, it uses the trained scheduling strategy to make decision-making inferences for the current delivery task and obtain the real-time delivery route decision result. Based on the real-time predicted duration and the real-time delivery route decision results, a multi-dimensional visualization display is performed through a visualization rendering engine, outputting an order full-process time prediction panel, a rider delivery route map, an overtime risk level dashboard, and a prediction and decision consistency monitoring chart, thus obtaining a visualization display interface. Based on the visualization interface, the hyperparameters of the trained scheduling strategy can be remotely configured. Based on the real-time prediction duration and the real-time delivery path decision results, the scheduling strategy versions under different hyperparameter configurations can be compared and tested in terms of on-time delivery rate, average delivery time, and late delivery penalty cost, so as to obtain the strategy evaluation results.

6. The method according to claim 5, characterized in that, The scheduling and execution phase also includes: After each scheduling action is executed, the trained time prediction model is called again to refresh the predicted duration of orders that have not yet been fulfilled. For orders that have started to be fulfilled, the predicted duration is estimated by the sum of the actual time elapsed and the remaining predicted duration to obtain the updated predicted duration. Based on the updated prediction duration, the difference between the actual waiting time and the predicted waiting time is calculated. When the difference exceeds a preset deviation threshold, a prediction deviation penalty term is triggered in the reward function to obtain an updated reward signal. Based on the updated reward signal, the network parameters of the trained scheduling policy are updated online to obtain the online-updated scheduling policy.

7. The method according to claim 1, characterized in that, The prediction output of the trained time prediction model further includes the following adaptive confidence interval generation step, including: Based on the time prediction model that has been trained, multiple parallel forward propagations are performed on the same order input to be predicted during the inference phase. The mean of each prediction target is calculated as the point prediction value and the standard deviation is calculated as the local uncertainty estimate. Based on the local uncertainty estimation, a sliding calibration window is maintained, and the predicted value, actual performance time and timestamp of historical completed orders are stored. The cumulative sum statistic is used to detect the distribution change of the residual sequence in the sliding calibration window. When a change point is detected, an exponentially decaying weight is applied to the samples before the change point to obtain the change point-aware exponentially decaying weight sequence. Based on the exponentially decaying weight sequence with the change point awareness, the non-consistency score is normalized using the local uncertainty estimate, and the weighted quantile is calculated using the exponentially decaying weight sequence. For each prediction target, an adaptive confidence interval is output with the predicted point value as the center and the product of the weighted quantile and the local uncertainty estimate as the radius. Based on the adaptive confidence interval, the width of the adaptive confidence interval is normalized and injected into the state space. The upper bound of the confidence interval of the cumulative delivery time is used as the dynamic risk boundary to replace the fixed delivery deadline for the calculation of the reward function, so as to obtain the updated state space and reward function. Based on the updated state space and reward function, after each order is fulfilled, the actual observation value is appended to the sliding calibration window, triggering the change point detection and weighted quantile update, and obtaining the updated adaptive confidence interval parameters.

8. The method according to claim 7, characterized in that, The adaptive confidence interval further includes the following parameterized inconsistency metric optimization steps: Based on the adaptive confidence interval, the upper and lower bounds of the adaptive confidence interval are mapped to parameterized inconsistency scores through a learnable parameterized transformation. Based on the parameterized inconsistency score, and using the parameterized inconsistency score as the optimization variable, under the constraint that the prediction coverage of the sliding calibration window is not lower than the preset target confidence level, the iterative optimization method is used to minimize the conformal set radius to obtain the optimized parameterized inconsistency metric. Based on the optimized parameterized inconsistency metric, the equivalent uncertainty scalar for each prediction target is calculated. Based on the relationship between the equivalent uncertainty scalar and the prediction margin time, a green label is used when the prediction margin time is greater than the equivalent uncertainty scalar, a yellow label is used when the prediction margin time is not greater than the equivalent uncertainty scalar but is greater than zero, and a red label is used when the prediction margin time is not greater than zero, thus obtaining a three-level risk label.

9. The method according to claim 8, characterized in that, Also includes: Using the orders marked in yellow in the three-level risk labels and the sliding calibration window as input, a subset of historical orders is retrieved in the sliding calibration window whose absolute value of the difference between the predicted grace time and the predicted grace time of the order does not exceed a preset neighborhood radius. The actual timeout probability of the historical order subset is calculated as the expected timeout cost of the order. When the number of samples in the historical order subset is less than the preset minimum number of samples, it degenerates into the global on-time rate estimation of the sliding calibration window to obtain the expected timeout cost estimate. Based on the expected timeout cost estimate, the expected timeout cost estimate is multiplied by a preset timeout cost coefficient and added as an additional penalty to the reward function to obtain the extended reward function; Based on the extended reward function, a total loss function is constructed in the form of a weighted sum of the policy gradient objective, state value regression loss, and timeout probability classification loss. A Lagrange relaxation constraint term is introduced into the total loss function, and the preset timeout rate upper limit is encoded as a probability constraint. The Lagrange multiplier in the Lagrange relaxation constraint term is adaptively updated through a dual optimization method to obtain the total loss function with timeout rate constraint. Based on the total loss function with introduced timeout rate constraints, a state value estimation head and a timeout probability auxiliary output head are extended in the graph attention policy network. The graph attention policy network is then trained using the total loss function with introduced timeout rate constraints to obtain a training completion scheduling policy that satisfies the timeout rate constraints.

10. The method according to claim 4, characterized in that, The steps for multi-rider joint path planning using the trained scheduling strategy during the inference phase include: Using the negative of the predicted grace time for each rider's orders as the priority weight, calculate the weighted priority index for each rider, and determine the rider decoding order according to the weighted priority index from largest to smallest to obtain the rider decoding order sequence; According to the rider decoding sequence, for the current rider to be decoded, the initial score of each candidate node is calculated by integrating the node features and the predicted waiting time; when the current rider has included the target pickup node in the current rider's delivery route, the time conflict degree between the target pickup node and the current rider to be decoded is calculated based on the predicted waiting time, and the initial score of the target pickup node is suppressed accordingly to obtain the adjusted candidate node score distribution; Based on the adjusted candidate node score distribution, the candidate node with the highest score is selected as the next action node for the current rider to be decoded. The next action node is then incorporated into the delivery path of the current rider to be decoded. The process is repeated for the next rider to be decoded, following the rider decoding sequence. The initial score and score suppression steps for each candidate node are calculated by fusing the node features and the predicted waiting time, until all riders have completed the allocation of all pick-up and delivery nodes in their respective delivery paths, thus obtaining the joint optimal delivery path for all riders.

Images