Accurate prediction of estimated time of arrival (eta)

Abstract

The application discloses a precise ETA prediction method, and relates to the technical field of data processing methods. The method comprises the following steps: generating a pre-training decoupling distribution task autoencoder DDMAP, performing self-supervised pre-training on unlabelled takeout data, learning general representation from massive unlabelled trajectories through a multi-task target, performing dynamic fine-tuning on labelled data, utilizing a frozen encoder in a self-adaptive hybrid expert MoE Transformer architecture to construct a final ETA prediction model, and predicting the precise ETA by using the final ETA prediction model. The method has the advantages of high accuracy and strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of data processing methods, and in particular to an accurate ETA (Estimated Time of Arrival) prediction method. Background Technology

[0002] The booming development of on-demand logistics, encompassing services from food delivery to e-commerce fulfillment, has fundamentally reshaped the infrastructure of modern cities. At the heart of these platforms, providing accurate Estimated Time of Arrival (ETA) forecasts is a critical challenge—it's the cornerstone of ensuring customer satisfaction, optimizing fleet efficiency, and maintaining platform reliability. An accurate ETA system is more than just a customer-facing metric; it's a key input for a range of complex operational decisions, including dynamic order dispatch and real-time route optimization.

[0003] However, the pursuit of accurate last-mile ETA prediction faces two significant obstacles: the inherent complexity of delivery systems and the widespread scarcity of high-quality labeled data. On the one hand, the task is inherently complex, requiring models to capture dynamic spatiotemporal contexts (e.g., fluctuating traffic patterns, diverse location types, and adverse weather conditions), unique rider behaviors (e.g., experience, route familiarity), and tight sequential dependencies between delivery tasks. On the other hand, such modeling is often hampered by data limitations. For example, in newly developed residential areas or for novice riders, the lack of sufficient historical data can lead to the classic "cold start" problem, rendering supervised models without relevant training samples unreliable. To address these serious challenges, the research community has developed advanced deep learning models, particularly utilizing advanced graph neural networks (GNNs) to capture fine-grained spatial relationships and deep reinforcement learning (DRL) frameworks to optimize decision-making processes.

[0004] However, these cutting-edge models all suffer from a fundamental flaw: an over-reliance on traditional end-to-end supervised learning paradigms. This strong dependence on labeled data makes the models extremely vulnerable to significant shifts in the underlying data distribution, a common occurrence in real-time logistics scenarios. For example, large-scale urban events or sudden, prolonged road closures can render years of accumulated historical traffic and demand data invalid within a short period. In such scenarios, the model's inherent preference for historical data distribution becomes a performance bottleneck, leading to prediction failures during critical periods when accuracy is paramount. This core limitation significantly reduces the model's robustness and hinders its deployment in dynamic real-world scenarios. Besides the data dependency issue, the second key limitation lies in the static nature of their architecture. Their monolithic designs struggle to adapt to the highly heterogeneous and dynamically changing characteristics of real-world delivery scenarios. A rider's operating environment can change dramatically throughout the day, from traversing dense central business districts during the morning rush hour to serving sparsely populated suburban homes in the afternoon. Static models apply the same fixed set of parameters to these vastly different scenarios, lacking the flexibility to adjust their inference logic according to changing context. Therefore, these models often fail to learn representations that are both generalizable and truly adaptive, making it difficult to maintain high performance as the operating environment evolves. Summary of the Invention

[0005] The technical problem to be solved by this invention is how to provide a precise ETA prediction method with high accuracy and strong generalization ability.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: an accurate ETA prediction method, comprising the following steps:

[0007] S1: Generate a decoupled delivery task autoencoder DDMAP for pre-training, perform self-supervised pre-training on unlabeled food delivery data, and learn general representations from massive unlabeled trajectories through multi-task objectives;

[0008] S2: Dynamically fine-tune the labeled data, utilize the frozen encoder in an adaptive hybrid expert MoE Transformer architecture to build the final ETA prediction model, and use the final ETA prediction model to predict the accurate estimated time of arrival (ETA).

[0009] The beneficial effects of adopting the above technical solution are as follows: the method includes two stages: a self-supervised pre-training stage (DDMAP) for building a robust knowledge base from unlabeled data; followed by an adaptive fine-tuning stage with a dynamic expert hybrid (MoE) Transformer to handle diverse real-time scenarios.

[0010] The DDMAP framework is a decoupled pre-training method that uses a three-part context and a masked autoencoder to learn robust semantics from incomplete trajectory data.

[0011] Pretext Tasks Aligned with ETA: Specially designed cooperative tasks (especially ORP and MSDP) are intended to give the model basic reasoning capabilities for sequence logic and duration estimation.

[0012] Dynamic MoE Transformer: This is a context-aware architecture that uses macro-level task profiling and dynamic expert routing to enable parameters to adapt to real-time scene shifts.

[0013] Extensive experiments on large real-world datasets demonstrate that the proposed method significantly outperforms all benchmark models, showcasing its superior accuracy and generalization ability. Attached Figure Description

[0014] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0015] Figure 1 This is the main flowchart of the method described in the embodiments of the present invention;

[0016] Figure 2a This is a flowchart of the first stage of the method described in the embodiments of the present invention;

[0017] Figure 2b This is a flowchart of the second stage of the method described in the embodiments of the present invention;

[0018] Figure 3a These are the activation probabilities of different experts in each Transformer layer in the embodiments of this invention;

[0019] Figure 3b This refers to the selection frequency of each expert in the embodiments of the present invention;

[0020] Figure 4 This is a total loss curve diagram in an embodiment of the present invention;

[0021] Figure 5 These are the loss curves of each training subtask in the embodiments of the present invention;

[0022] Figure 6 This is a time distribution chart of order volume;

[0023] Figure 7a This is a distribution map of the original delivery times;

[0024] Figure 7b This is a distribution chart of logarithmic transformation delivery times; Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0027] like Figure 1 As shown in the figure, this invention discloses an accurate ETA (Estimated Time of Arrival) prediction method, which includes the following steps:

[0028] S1: Generate a decoupled delivery task autoencoder DDMAP for pre-training, perform self-supervised pre-training on unlabeled food delivery data, and learn general representations from massive unlabeled trajectories through multi-task objectives;

[0029] S2: Dynamically fine-tune the labeled data, utilize the frozen encoder in an adaptive hybrid expert MoE Transformer architecture to build the final ETA prediction model, and use the final ETA prediction model to predict the accurate estimated time of arrival (ETA).

[0030] like Figures 2a-2b As shown, the two core stages of the proposed method are illustrated. The first stage involves self-supervised pre-training on unlabeled data to generate a basic knowledge encoder. The second stage involves dynamic fine-tuning on labeled data, utilizing the frozen encoder within an adaptive hybrid expert (MoE) architecture to construct the final ETA prediction model. These two stages work together to achieve robust and accurate predictions.

[0031] The above methods will be explained in detail below with specific examples:

[0032] Task definition:

[0033] Following the ETA forecasting task formula established in the LaDe benchmark, the problem is defined as: predicting the arrival time of a rider's set of orders to be delivered at a specific time.

[0034] Given in time The query contains a set One unfinished package The goal of the model is to learn a function. This function should map the query to a prediction duration vector. Each package corresponds to a specific duration. Each package The actual delivery time (as the prediction target) is calculated as the actual delivery time. With query time The difference. This can be formally expressed as:

[0035] ;

[0036] Pre-training framework: DDMAP

[0037] The core goal of DDMAP (Decoupled Delivery Task Autoencoder for Pre-training) is to learn a powerful encoder function. It can map any given delivery order into a rich, decoupled representation vector. The framework consists of three key parts: the construction of decoupled context units, the masked autoencoder architecture, and a multi-task self-supervised learning objective.

[0038] Decoupled context unit:

[0039] For each delivery order Defined as a tuple ,in It is the destination. It's about allocating time. For riders, this application constructs a comprehensive and decoupled context unit. This unit consists of three information components:

[0040] Self-characteristics ( This vector describes the inherent static properties of the order itself.

[0041]

[0042] in This represents a trainable embedding layer that embeds the point of interest (AOI) ID, AOI type, rider ID, and day of the week. timestamp Periodic encoding is used to capture periodic patterns (such as specific times of day). This represents the normalized latitude and longitude coordinates.

[0043] Local spatial context ( The matrix captures the destination. The dynamic characteristics of the surrounding geographical environment. This application identifies distance. Recent Let _ ... . It is a matrix where each row vector Representing a neighbor Defined as:

[0044]

[0045] in In the recent time window Internal occurrence The collection of orders at the location. These characteristics represent the dominant business type, activity intensity (order volume), and average activity duration in the surrounding area.

[0046] Rider's recent time and space dynamics ( This matrix summarizes the riders'... Recent actions prior to performing the current task. Tracing the rider's most recent... One historical order, . It is a matrix where each row Representing the historical orders :

[0047]

[0048] in It is the normalized constant of the time interval. It is the sequence interval. It is an indicator function, if and If they belong to the same AOI, the value is 1; otherwise, it is 0. These features capture historical task types, relative spatiotemporal and sequence distances, and spatial locality.

[0049] Mask autoencoder architecture:

[0050] DDMAP employs an asymmetric Transformer-based encoder-decoder architecture to learn efficiently from context units.

[0051] Tokenization: Contextual Unit First, convert it into a token sequence. Through linear layer Mapping Formation Master . and Each row vector in the layer is also passed through layers. and Mapping to form a token sequence and The final input sequence is These tokens will also have learnable type embeddings and standard position embeddings added.

[0052] Masking: For the input token sequence High application ratio ( A random mask, such as 40%. Let... For the set of indices of the masked token, This is the set of indices for visible tokens.

[0053] Encoder: Depth Transformer Encoder Only process the visible token subset Generate their latent representations This localized processing method makes the pre-training phase highly efficient.

[0054] Decoder: Lightweight Transformer decoder With the encoded visible token As input. At the masked position. Insert a shared, learnable [MASK] token. The decoder's task is to reconstruct the original, complete token sequence: .

[0055] Multi-task self-supervised objectives:

[0056] Pre-training loss It is a weighted sum of losses from three self-supervised tasks, designed to capture different aspects of the delivery process.

[0057] Mask autoencoding loss ( ): This is the primary reconstruction target, used to calculate the reconstructed token. With the original token At the mask location The difference lies in the loss method. Cross-entropy loss is used for discrete features (such as ID), while mean squared error (MSE) is used for continuous features.

[0058]

[0059] Order reordering prediction loss ( To learn time logic, a length of [length missing] is provided to the model. Real delivery sequence random shuffling By adding a [CLS] token before the sequence, the model must predict whether the sequence is in the original order.

[0060]

[0061] in The encoded [CLS] token is sorted by header. If the obtained predicted values ​​are in the original order, then Otherwise, it is 0.

[0062] Mask span duration prediction loss ( This task is highly aligned with ETA predictions. This application samples an initial order from the trajectory. and an ended order Intermediate interval Each order. The models are only informed of their context units. and And must predict the time elapsed. .

[0063]

[0064] The target duration undergoes a logarithmic transformation to handle its long-tail distribution.

[0065] The ultimate goal of pre-training is:

[0066]

[0067] in It is a hyperparameter that balances various losses.

[0068] Downstream fine-tuning using dynamic MoE:

[0069] During the fine-tuning phase, the pre-trained DDMAP encoder is frozen. And use it as a powerful feature extractor. Given query time Collection of orders to be delivered by riders The objective of this application is to predict each order. ETA.

[0070] Feature extraction and fusion:

[0071] For each order in the sequence First, the DDMAP encoder is used to extract its deep decoupling context representation. This application also retains the original shallow features of the order. (e.g., coordinates, AOI type). These two sets of features are fused to form the final order representation. :

[0072]

[0073] in and It is a projection layer. It's a fusion layer. In addition, a special [CLS] token has been prepared to represent the rider's time... Status (such as speed, current load).

[0074] 3.3.2 Task Profiling-Driven Dynamic MoE Transformer

[0075] The downstream model of this application is an innovative Transformer architecture that integrates a Dynamic Expert Hybrid (MoE) mechanism.

[0076] Task profile generation: Compute the entire set of tasks to be processed before inputting the sequence into the Transformer. The macroscopic statistical representation of this task profile vector. Defined as:

[0077]

[0078] It includes the geographic center and spatial variance of the task, current load, query time, and a histogram of AOI type.

[0079] Dynamic Gating: Task Profile Passed to a lightweight gating network The network is An expert network generates route weights:

[0080]

[0081] in It is a learnable weight matrix. Operation by only keeping the previous An expert (such as) The weights are used to enhance sparsity.

[0082] MoE Transformer Layers: This application replaces the standard feedforward network (FFN) in each Transformer layer with a MoE layer. For input to the... Layer representation Output .

[0083] Prediction and training objectives:

[0084] go through After applying the MoE Transformer layer, this application obtains the final representation of each order. Shared return head Then predict the ETA for each order:

[0085]

[0086] The training objective during the fine-tuning phase is to minimize the L1 loss (mean absolute error) of all valid predictions, supplemented by a load-balanced auxiliary loss. This encourages gating networks to make balanced use of all experts.

[0087]

[0088] in It's a hyperparameter. The aim is to minimize the coefficient of variation of the task load assigned to each expert.

[0089] Experimental test:

[0090] 1) Dataset:

[0091] The experiments in this application were conducted on the LaDe benchmark dataset, which provides large-scale real-world data from last-mile delivery operations. This application utilizes this dataset in two different ways, corresponding to two phases of the DDEP framework.

[0092] In the self-supervised pre-training phase, this application constructs a large-scale unlabeled dataset using all delivery trajectories and their associated contexts in LaDe. Crucially, this application ignores the final delivery time information, treating it as unlabeled data, aiming to learn fundamental logistics knowledge from massive amounts of operational data without requiring expensive ground-truth labels.

[0093] In the downstream fine-tuning and evaluation phase of the ETA prediction task, this application strictly adheres to the benchmark protocol and uses the official LaDe-D subset dataset containing the necessary ground truth labels. The data is split chronologically in a 6:2:2 ratio into training, validation, and test sets. This temporal splitting strategy is crucial because it ensures that the model is evaluated on its ability to predict future, unseen data, realistically simulating real-world deployment scenarios.

[0094] 2) Baseline Model:

[0095] For a comprehensive evaluation, this application compares the proposed DDEP framework with a diverse range of benchmark models, from simple heuristics to state-of-the-art deep learning architectures.

[0096] 3) Evaluation indicators:

[0097] Following the LaDe benchmark [wu2024lade], this application uses three standard metrics to evaluate performance:

[0098] Mean Absolute Error (MAE): The average absolute error between the predicted and actual times, measuring overall accuracy. A lower value is better.

[0099] Root Mean Square Error (RMSE): The square root of the average of the squared errors. It penalizes larger errors more severely and measures robustness. A lower value is better.

[0100] 20-Minute Accuracy (ACC@20): The percentage of prediction errors within 20 minutes of actual time, reflecting actual user-centric performance. A higher value is better.

[0101] 4) Hyperparameters:

[0102] The model in this application is implemented using PyTorch and trained on an NVIDIA A100 GPU. All experiments used a fixed random seed to ensure reproducibility. Detailed hyperparameter settings for the DDMAP pre-training and downstream fine-tuning stages are provided in the appendix.

[0103] Test results:

[0104] Table 1 compares the different methods in the ETA prediction task.

[0105]

[0106] 1) Main results:

[0107] The comprehensive experimental results for the ETA prediction task are summarized in the table [tab:main_results_booktabs]. The DDEP framework proposed in this application consistently outperforms all benchmark methods on three different city datasets (Shanghai, Chongqing, and Yantai), achieving new state-of-the-art results on all evaluation metrics.

[0108] Overall advantages:

[0109] On the massive Shanghai dataset, DDEP achieved a MAE of 15.19 and an RMSE of 18.24, significantly lower than the strongest baseline model, RANKETPA, by 11.6% and 9.6%, respectively. This advantage continued in the mountainous city of Chongqing with its complex terrain: DDEP reduced the MAE from RANKETPA's 24.00 to 22.94. Particularly noteworthy is the most significant lead achieved on the data-scarce Yantai dataset, where the model outperformed RANKETPA by 13.1% with an MAE of 20.51 compared to RANKETPA's 23.62. This demonstrates that DDEP not only performs exceptionally well in data-rich environments but also exhibits outstanding performance in scenarios with limited historical data.

[0110] (2) Actual reliability:

[0111] Besides the mean error, the user-centric evaluation metric ACC@20 (the accuracy of predicted times falling within 20 minutes before or after the actual arrival time) is particularly crucial for ensuring customer satisfaction. DDEP improved Shanghai's ACC@20 to 0.74, achieving a relative improvement of 7.1% compared to the best baseline model. Similar improvements were also validated in Chongqing (0.54) and Yantai (0.58). These results demonstrate that DDEP can provide significantly more reliable prediction assurance under extreme conditions, thereby reducing the frequency of extreme delays that severely impact platform reputation.

[0112] Table 2 shows the comprehensive ablation experiment on data efficiency for three different target cities.

[0113]

[0114] 2) Data efficiency and cold start analysis

[0115] One of DDEP's core contributions is its ability to mitigate the cold start problem through pre-training. Table 2 shows a rigorous stress test, in which this application fine-tuned the data using only a small fraction (1% to 50%) of the target city's training data.

[0116] Resilience under Extreme Data Scarcity: The results demonstrate a stark contrast between supervised learning from scratch and the pre-trained paradigm of this application. When training data was limited to only 1% (simulating the first day of deployment in a city), the model trained from scratch collapsed, producing unacceptable errors (e.g., a MAE of 50.49 in Jilin). In contrast, leveraging knowledge transferred from Shanghai, DDEP maintained high accuracy (MAE of 19.92 in Jilin), achieving a remarkable 60.5% improvement. Even on the complex Chongqing dataset, using only 1% of the data, DDEP outperformed the baseline trained from scratch by 43.2%. Remarkably, DDEP fine-tuned with only 10% of the data often matched or surpassed supervised baselines trained with 100% of the data (e.g., in Yantai, DDEP with -10% had an MAE of 20.62, better than the 20.64 of fully supervised training with -100%). This demonstrates DDEP's superior data efficiency, making it a viable solution for rapid deployment in new regions.

[0117] Table 3. Cross-city migration performance matrix

[0118]

[0119] 3) Intercity migration

[0120] To evaluate the generality of the learned representations, this application conducted cross-city transfer experiments. The general representations are as follows: The results show that the pre-trained encoder learns general logistics logic, rather than simply memorizing city-specific maps. Transferring a model pre-trained in a small city (e.g., source city: Jilin) ​​to a large, complex city (target city: Chongqing) yielded a MAE of 22.75, surprisingly outperforming the baseline model trained from scratch in Chongqing (23.04). This indicates that the pre-training task captures robust temporal dynamics common across geographical regions. Furthermore, the general model trained on all city combination datasets consistently achieved the best performance (e.g., a 15.4% improvement in Yantai), validating that expanding the diversity of pre-training data directly translates into downstream benefits.

[0121] 4) Dynamic MoE Mechanism Analysis

[0122] To further reveal how DDEP addresses scenario heterogeneity through dynamic architecture, this application... Figure 3a The internal control decision-making process is visualized.

[0123] Specialization of Experts and Role Allocation: Figure 3aThe activation probabilities of experts at different Transformer layers are presented, revealing a clear hierarchical division of labor. In shallower layers (such as layer 0), expert 5 has an activation probability exceeding 0.8, dominating the field and indicating its specialized role in encoding spatial priors and static AOI properties. As the network deepens, the activation patterns become more diverse. In layer 3, computation primarily routes to experts 0, 1, and 6 (probabilities between 0.24 and 0.5), who may be responsible for higher-order reasoning, such as predicting delivery speeds in a specific time context or handling complex sequential dependencies. This significant contrast between "hot" and "cold" areas confirms that gating networks can accurately route multidimensional logistics knowledge to specific subnetworks based on task profiles, rather than applying a uniform average.

[0124] Load balancing and parameter utilization: Figure 3b The selection frequency for each expert in the test set is shown. While specialization is evident, the distribution remains robust. The most active expert (Expert 5) was selected approximately 568,000 times, while the least selected expert (Expert 4) still processed approximately 135,000 instances. Driven by the auxiliary load balancing loss, the selection frequency fluctuates around the "ideal balanced load" line. This prevents pattern collapse (i.e., only one expert dominates) and ensures that the full parameter capacity of MoE is utilized, thereby maximizing efficiency and robustness across various long-tail scenarios.

[0125] Why MoE is suitable for logistics tasks: MoE's effectiveness stems from its ability to handle high contextual dependencies in logistics. For example, the parameter requirements for "high-load peak hours" and "low-load late-night" deliveries are fundamentally different. Static models are forced to learn globally averaged parameters that perform poorly under extreme conditions, while MoE acts as a conditional computation engine. It switches "driving modes" based on the real-time task profile, achieving accurate adaptation to heterogeneous and non-stationary logistics data while maintaining computational efficiency.

[0126] 5) Ablation experiment:

[0127] Table 4: Contributions of Each Component

[0128]

[0129] Finally, Table 4 validates the contributions of each component. Removing the MoE module (“Transformer only”) resulted in a performance degradation (MAE increased from 22.05 to 22.36), confirming the difficulty of adapting a static architecture to dynamic contexts. However, the most significant performance degradation occurred when pre-training was removed (training the full DDEP from scratch resulted in higher errors), further highlighting the crucial role of decoupling pre-training from dynamic adaptation in achieving state-of-the-art performance for DDEP.

[0130] 6) Hyperparameters:

[0131] This application details the key hyperparameters of the DDEP framework, covering both the pre-training phase (DDMAP) and the downstream fine-tuning phase. All experiments were performed on an NVIDIA A100 GPU using the PyTorch framework, with a fixed random seed of 42 to ensure reproducibility.

[0132] Pre-training: DDMAP

[0133] The DDMAP framework is configured with a powerful and efficient asymmetric Transformer architecture.

[0134] Input representation: For decoupled context units, this application sets the number of nearest neighbor spatial neighbors of the LSC matrix to... For the RTDC matrix, this application traces the rider's most recent... A historical order. The embedding dimensions of key entities are set as follows: AOI ID is 64, rider ID is 64, and day of the week is 64.

[0135] Masked autoencoder architecture: The main Transformer encoder is a 12-layer network with hidden layers of dimension 1. It has 12 attention heads. The lightweight decoder is designed with 10 layers, and the hidden layer dimension is reduced to... It has 8 attention heads. Following the MAE methodology, this application adopts... The proportion of random masking.

[0136] Multi-task objective: Final pre-training loss It is a weighted sum of three objectives. For the novel proxy task proposed in this application, the loss weight of the Order Reordering Prediction (ORP) task is set to... The weights for the Mask Span Duration Prediction (MSDP) task are set to... .

[0137] Training details: The DDMAP model was pre-trained for 10 epochs with a batch size of 2048. This application used the AdamW optimizer with a learning rate of... A linear learning rate warmup strategy was used in the first 10% of the training steps.

[0138] Fine-tuning: Dynamic MoE Transformer

[0139] During the fine-tuning phase, the pre-trained DDMAP encoder is frozen and used as a feature extractor. The downstream architecture and training configuration are as follows:

[0140] MoE Transformer Architecture: The downstream model is a Transformer consisting of 4 MoE enhancement layers. Each hidden layer has a dimension of... It has 8 attention heads and a feedforward network (FFN) dimension of 1024. The input sequence representing the rider's orders to be delivered is padded or truncated to a maximum length of 25.

[0141] Dynamic Experts: Within each MoE layer, this application configures $M=8$ expert networks. The dynamic gating network is driven by a 9-dimensional task profile vector, selecting the first $k=2$ experts for each forward propagation. A dropout rate of 0.1 is applied for regularization.

[0142] Training details: The model was fine-tuned for 15 epochs with a batch size of 64. This application used the AdamW optimizer with a learning rate of [missing information - likely a specific value]. Weight decay is Auxiliary load balancing losses hyperparameters Set to 0.01.

[0143] Pre-training dynamic analysis:

[0144] Figure 4 visualizes the loss dynamics during the DDEP pre-training phase, providing an in-depth demonstration of the model's learning process under the multi-task self-supervised objective.

[0145] Overall convergence:

[0146] Figure 4The panel displays the total training loss, which is a weighted sum of the losses from the three proxy tasks. The graph shows a clear and stable convergence pattern. The original per-step loss (light red) exhibits high-frequency fluctuations, which is expected in stochastic gradient-based optimization. Crucially, the smoothed average (solid red line) shows an initial rapid decline, followed by a steady decrease, and finally flattens out as the model converges. This smooth convergence validates the stability of the multi-task learning framework and the effectiveness of the learning rate scheduling strategy in this application, confirming that the model can successfully learn from unlabeled data without instability.

[0147] Learning phases for specific tasks:

[0148] Figure 5 A more nuanced perspective is provided by plotting the smoothed losses for the three agent tasks separately. The unique trajectories of the masked autoencoder (MAE), order reordering prediction (ORP), and masked span duration prediction (MSDP) losses reveal different learning characteristics and complexities:

[0149] Masked Autoencoder (MAE) Loss: The MAE loss (blue dashed line) starts high and shows a continuous, gradual decrease. As a fundamental reconstruction task, MAE requires the model to learn a comprehensive representation of the entire context unit. Its steady convergence indicates that the model is gradually improving its ability to capture the underlying semantics and structure of logistics data.

[0150] Order Reordering Prediction (ORP) Loss: The ORP loss (green dashed line) starts at a relatively low value and converges very quickly to a stable low error level. This indicates that learning the temporal logic of delivery sequences is a relatively simple task for the Transformer architecture. The model can quickly grasp the inherent sequence patterns in rider trajectories, which is a key skill for downstream ETA prediction.

[0151] Masked Span Duration Prediction (MSDP) Loss: The MSDP loss (purple dashed line) shows the most significant initial decrease, indicating rapid early progress in the model's core skill of duration estimation. This task is highly aligned with the ultimate goal of ETA prediction. The significant decrease in MSDP loss strongly demonstrates that the surrogate task design of this application effectively endows the model with basic temporal reasoning capabilities before the model encounters any explicit ETA labels.

[0152] In summary, the loss curves collectively demonstrate the effectiveness of the collaborative multi-task objective of this application. The model not only exhibits stable learning processes but also demonstrates unique learning phases across different skills—Robust Representation Learning (MAE), Temporal Logical Learning (ORP), and Duration Estimation (MSDP)—which together provide a robust and generalizable foundation for accurate downstream ETA predictions.

[0153] Time-based order pattern analysis:

[0154] To understand the inherent dynamics of the logistics environment, this application analyzes the temporal distribution of orders in the dataset. Figure 6 The chart displays a heatmap of order volume aggregated by hour and day of the week. This visualization reveals strong, periodic time patterns, which are crucial for capturing in any accurate ETA forecasting model.

[0155] The most notable feature is the concentration of high order volumes during specific peak periods. Throughout the week, this application observed a significant surge in activity during the morning hours, specifically between 8:00 and 11:00 AM. Around 10:00 AM consistently represents the absolute peak, indicated by the darkest squares on the heatmap. This corresponds to the primary order dispatch window for daily deliveries in many logistics operations. Following the morning peak, order volumes decline sharply and remain at lower levels throughout the afternoon and evening.

[0156] Furthermore, the heatmap exhibits a consistent pattern across weekdays (Monday to Friday) and weekends (Saturday to Sunday), indicating a stable weekly operational rhythm. These significant peaks and troughs highlight a core challenge in ETA forecasting: the highly non-uniform operational context. Traffic conditions, rider workload, and station processing times can fluctuate dramatically between peak and off-peak hours.

[0157] This analysis validates the core motivation behind the proposed DDEP framework. A static model that applies the same parameters regardless of time will struggle to adapt to these drastic changes in operational rhythm. The clear time dependencies highlighted in Figure 6 necessitate a context-aware model architecture. The approach in this application, particularly the Dynamic Expert Hybrid (MoE) mechanism during the fine-tuning phase, is designed to address this heterogeneity. By learning to route computation to specialized "expert" subnetworks based on the current scenario profile (including time of day), the model in this application can adjust its inference logic, thereby achieving more accurate predictions during both peak and off-peak periods.

[0158] Target variable distribution analysis:

[0159] A key step in developing a robust regression model is understanding the distribution of the target variable. In the case of this application, the target variable is delivery time. Figure 7a The distribution of delivery times in the dataset is shown, including the original scale and the scale after logarithmic transformation.

[0160] Figure 7a As shown, the original distribution of delivery times is severely right-skewed, exhibiting a classic long-tail pattern. The vast majority of deliveries are concentrated in relatively short time ranges (e.g., within 100 minutes), while a very small but significant number of outliers extend to the far end of the long tail, lasting for hours or even days. This high skewness poses a challenge to standard regression models because extreme values ​​disproportionately affect the loss function, potentially causing the model to favor predicting longer durations while performing poorly in most common cases.

[0161] To address this issue, this application applies a logarithmic transformation ($\log(1+x)$) to the duration value. The effect of this transformation is as follows: Figure 7b As shown, the resulting distribution is more symmetrical and very close to a normal (Gaussian) distribution.

[0162] This transformation brings two core advantages to the modeling method of this application:

[0163] Variance stabilization: It compresses the range of the target variable, reduces variance, and makes the learning problem more stable for gradient-based optimization methods.

[0164] Mitigating the impact of outliers: By reducing the magnitude of extreme values ​​in the long tail, the logarithmic transformation ensures that the model is not overly penalized by these rare but large errors, thus enabling it to better learn more frequent short-duration delivery patterns.

[0165] Based on these advantages, this application uses logarithmically transformed duration as the prediction target during the training phase. During inference, the model's output is converted back to the original minute scale through inverse operations (i.e., exponential operations) to provide the final ETA prediction. This methodological choice is fundamental to achieving robust and accurate performance.

[0166] Detailed description of the benchmark model:

[0167] Hereinafter, this application provides a detailed description of each benchmark model used for comparison in the experiment.

[0168] SPEED: This is a simple, non-learning heuristic that estimates arrival time by dividing the remaining trip distance by the rider's historical average speed. While intuitive, it serves as a baseline benchmark for measuring the benefits of learning-based methods. Its main limitation is its inability to capture any complex, dynamic, and non-linear contexts (such as traffic, weather, and time of day) that significantly influence actual delivery times.

[0169] KNN (K-Nearest Neighbors): A non-parametric machine learning method that predicts the ETA of a given task by averaging the ETAs of the $k$ most similar historical task neighbors in the feature space. Its performance is highly dependent on the quality of feature engineering and the chosen distance metric, and it typically struggles with handling high-dimensional data and capturing complex time dependencies.

[0170] LightGBM [ke2017lightgbm]: An efficient and powerful gradient boosting decision tree (GBDT) framework. It serves as a strong benchmark representing classic feature-based machine learning. While capable of modeling complex nonlinear interactions between input features, it lacks the intrinsic architectural ability to understand sequential or spatial dependencies, treating each prediction task as an independent event.

[0171] MLP (Multilayer Perceptron): A basic feedforward neural network consisting of multiple fully connected layers. It serves as a foundational deep learning benchmark for evaluating performance improvements achieved by more complex, sequence-aware, and adaptive architectures. Its monolithic structure is inherently unsuitable for modeling the sequential nature of delivery tasks.

[0172] FDNET [gao2021deep]: An advanced deep learning model designed specifically for route and time prediction in delivery services. It typically employs a Bi-LSTM encoder and Pointer Net decoder architecture to learn representations from rider trajectory data. It represents an important class of models trained end-to-end on sequence data in a supervised manner, but its static architecture limits its adaptability to heterogeneous delivery scenarios.

[0173] RANKETPA [wen2023enough]: A state-of-the-art two-step deep learning model that first explicitly predicts future delivery routes and then estimates the arrival time of each task in that predicted sequence. As one of the strongest existing benchmarks, it excels at modeling sequence dependencies. However, like other supervised models, its performance is fundamentally limited by its static architecture and dependence on large amounts of labeled data, making it vulnerable to data-scarce and cold-start scenarios, which is precisely where the pre-training method in this application excels.

[0174] The method described in this application firstly addresses the data sparsity problem by proposing DDMAP—a self-supervised pre-training framework—to learn general representations from massive amounts of unlabeled trajectories through multi-task objectives. This endows the model with a robust intrinsic understanding of spatial semantics and temporal dynamics. Secondly, to handle scene heterogeneity, we design a context-aware Dynamic Expert Hybrid (MoE) Transformer. This mechanism dynamically routes computation to specialized expert networks based on macro-level task profiles, thereby achieving agile parameter reconstruction.

[0175] Extensive experiments on large real-world datasets demonstrate that DDEP consistently outperforms state-of-the-art benchmark models, exhibiting superior generalization ability and strong robustness, especially in cold-start and sparse data scenarios.

Claims

1. A method for accurately predicting the estimated time of arrival (ETA), characterized in that... Includes the following steps: S1: Generate a decoupled delivery task autoencoder DDMAP for pre-training, perform self-supervised pre-training on unlabeled food delivery data, and learn general representations from massive unlabeled trajectories through multi-task objectives; S2: Dynamically fine-tune the labeled data, utilize the frozen encoder in an adaptive hybrid expert MoE Transformer architecture to build the final ETA prediction model, and use the final ETA prediction model to predict the accurate estimated time of arrival (ETA).

2. The accurate ETA prediction method as described in claim 1, characterized in that, The Predicted Time of Arrival (ETA) is defined as: predicting the arrival time of a rider's set of orders to be delivered at a specific time. Given in time The query contains a set One unfinished package The goal of the model is to learn a function. ; This function should map the query to a prediction duration vector. Each package corresponds to a specific duration; each package The actual duration is calculated as the actual delivery time. With query time The difference, formally expressed as: 。 3. The accurate ETA prediction method as described in claim 2, characterized in that, The DDMAP processing method in step S1 includes the following steps: Decouple the construction of context units; The masked autoencoder DDMAP is constructed by learning from context cells; Multi-task self-supervised learning objectives.

4. The accurate ETA prediction method as described in claim 3, characterized in that, The method for constructing the decoupling context unit includes: For each delivery order Defined as a tuple ,in It is the destination. It's about allocating time. For riders, building a comprehensive and decoupled context unit. This unit comprises three information components: Self-characteristics This vector is used to describe the inherent static properties of the order itself; in This indicates a trainable embedding layer; this application embeds the ID, type, rider ID, and day of the week of the point of interest (AOI). timestamp Periodic encoding is used to capture periodic patterns. Represents normalized latitude and longitude coordinates; Local spatial context The matrix captures the destination. Dynamic characteristics of the surrounding geographical environment; identification of distance Recent Let _ ... ; It is a matrix where each row vector Representing a neighbor Defined as: In the recent time window Internal occurrence The order set at the location; these characteristics represent the dominant business type, activity intensity, and average activity duration in the surrounding area; Rider's Recent Moments This matrix is ​​used to summarize riders. Recent behavior prior to performing the current task; tracing the rider's most recent actions. One historical order: ； It is a matrix where each row Representing the historical orders : in It is the normalized constant of the time interval. It is the sequence interval. It is an indicator function, if and If they belong to the same AOI, the value is 1; otherwise, it is 0.

5. The accurate ETA prediction method as described in claim 4, characterized in that, The mask autoencoder DDMAP includes: Word segmentation: context unit First, convert it into a token sequence; Through linear layer Mapping Formation Master ; and Each row vector in the layer is also passed through layers. and Mapping to form a token sequence and The final input sequence is ; Mask: For the input token sequence Apply a high proportion of random masking; let For the set of indices of the masked token, A set of indices for visible tokens; Encoder: Depth Transformer Encoder Processing the visible token subset Generate their latent representations ; Decoder: Lightweight Transformer Decoder With the encoded visible token As input; at the masked position Insert a shared, learnable [MASK] token; the decoder's task is to reconstruct the original, complete token sequence. .

6. The accurate ETA prediction method as described in claim 4, characterized in that, The specific methods for the multi-task self-supervised learning objective include: Pre-training loss It is a weighted sum of the losses from three self-supervised tasks, used to capture different aspects of the delivery process; Mask autoencoding loss The primary reconstruction target is used to calculate the reconstructed token. With the original token At the mask location The difference lies in the location; cross-entropy loss is used for discrete features, and mean squared error (MSE) is used for continuous features: Order reordering prediction loss To learn the temporal logic, the model is provided with a length of... Real delivery sequence random shuffling Adding a [CLS] token before the sequence allows the model to predict whether the sequence is in its original order. in The encoded [CLS] token is sorted by header. If the obtained predicted values ​​are in the original order, then Otherwise, it is 0; Mask span duration prediction loss This task is highly aligned with ETA predictions; it samples an initial order from the trajectory. and an ended order Intermediate interval Each order; the model is informed of its context unit. and And must predict the time elapsed. ; The target duration undergoes a logarithmic transformation to handle its long-tail distribution; The ultimate goal of pre-training is: 。 7. The accurate ETA prediction method as described in claim 1, characterized in that, During the dynamic fine-tuning phase, the pre-trained DDMAP encoder is frozen. And use it as a powerful feature extractor, given query time Collection of orders to be delivered by riders Predict each order The precise estimated arrival time (ETA).

8. The accurate ETA prediction method as described in claim 7, characterized in that, Feature extraction and fusion are performed during the dynamic fine-tuning stage: For each order in the sequence First, the DDMAP encoder is used to extract its deep decoupling context representation. Preserve the original shallow features of the order. These two sets of features are merged to form the final order representation. : in and It is a projection layer. It is a fusion layer; and prepares a [CLS] token to represent the rider's time. The state.

9. The accurate ETA prediction method as described in claim 8, characterized in that, The dynamic MoE Transformer construction driven by task profiling during the dynamic fine-tuning phase includes the following steps: Task profile generation: Compute the entire set of tasks to be processed before inputting the sequence into the Transformer. Macro-statistical representation, task profile vector Defined as: This includes the geographic center and spatial variance of the task, current load, query time, and histograms of AOI type; Dynamic Gating: Task Profile Passed to a lightweight gating network The network is An expert network generates route weights: in It is a learnable weight matrix. Operation by only keeping the previous The weights of individual experts are used to enhance sparsity; MoE Transformer Layer: Replace the standard feedforward network FFN in each Transformer layer with a MoE layer for the input to the... Layer representation Output .

10. The accurate ETA prediction method as described in claim 9, characterized in that, Target prediction and training are performed during the dynamic fine-tuning phase: go through After the MoE Transformer layer, the final representation of each order is obtained. Shared return head Then predict the ETA for each order: The training objective during the fine-tuning phase is to minimize the L1 loss of all valid predictions, supplemented by a load-balanced auxiliary loss. This is to encourage gating networks to make balanced use of all experts; in It's a hyperparameter. The aim is to minimize the coefficient of variation of the task load assigned to each expert.

Images