Arrival time prediction optimization system and method based on pre-training learning

Abstract

The invention discloses an arrival time prediction optimization system and method based on pre-training learning, and belongs to the technical field of logistics distribution, the arrival time prediction optimization system comprises a server, a database module, a cloud deployment platform and an express terminal system, the server generates a basic model and stores the basic model in a database, the cloud deployment platform comprises a plurality of sub-platforms, and the sub-platforms are connected with the cloud deployment platform. The basic model is deployed on a plurality of cloud deployment sub-platforms to form a plurality of basic model sub-files, each sub-platform is connected with one express terminal system, and the sub-platforms transmit the basic model sub-files to the express terminal systems to realize time and route prediction. By adopting the system and the method, efficient order delivery and time prediction are realized through cooperation, the overall service level of the logistics industry is improved, the basic model can be efficiently utilized, the logistics delivery efficiency is improved, accurate time prediction service is provided, and finally the customer experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics distribution, and in particular to an arrival time prediction optimization system and method based on pre-training learning. Background Art

[0002] The existing courier pickup method is essentially to sort the orders. One of the core tasks of the service platform is to accurately predict the service route and time. With the widespread use of personal digital assistants (PDAs), a large amount of historical behavior data of couriers can be collected in daily operations, including GPS location information, task acceptance time, and task completion time. These data provide an important basis for data mining. However, due to the limited coverage of such data, research in related fields is still insufficient. The current solution is to conduct deep learning on big data, conduct in-depth analysis of historical data, and then predict the courier's estimated arrival time. However, this training time is too long and data training needs to be performed once in different regions, which is a waste of computing resources. In addition, this training also lacks the targeted consideration of nonlinear influencing factors (such as weather, road conditions, personal characteristics of takeaways, etc.) for each courier. Summary of the invention

[0003] The purpose of the present invention is to provide an arrival time prediction optimization system and method based on pre-training learning. First, the learning cost is reduced by pre-learning of data. Secondly, the nonlinear factors affecting the courier, such as weather, road conditions, deliveryman's preferences and speed, are fully considered, and the characteristics of the deliveryman are accurately learned to bring a good experience to users.

[0004] To achieve the above-mentioned purpose, the present invention provides an arrival time prediction optimization system based on pre-training learning, including a server, a database module, a cloud deployment platform and a courier-end system, the server is connected to the database module, the database module is connected to the cloud deployment platform, the cloud deployment platform includes multiple sub-platforms, each of the cloud deployment sub-platforms is connected to one of the courier-end systems, and the courier-end system is connected to a customer terminal and a courier terminal.

[0005] The present invention also provides an arrival time prediction optimization method based on pre-training learning, the steps comprising:

[0006] S1. Obtain the historical courier's completed order data and extract important parameters from the historical order data;

[0007] S2. The server first uses the Transformer model to pre-train the important parameters in the historical order information, and then uses the DRL4Route network to generate the corresponding basic model file and store it in the database;

[0008] S3. Deploy the basic model file to the cloud deployment sub-platform to form multiple basic model sub-files, and transmit the basic model sub-files to the express delivery end system;

[0009] S4, the user places an order, and the user terminal transmits the order information to the courier system;

[0010] S5. The courier end system receives the transmitted order information, processes the order information by loading the basic model sub-file, and predicts the route and arrival time;

[0011] S6, the courier system transmits the predicted route and arrival time to the customer terminal and the courier terminal;

[0012] S7. After the courier completes the order, the courier will feedback the actual arrival time and route to the courier end system, and the completed order will be temporarily stored in the cloud deployment sub-platform by the courier end system;

[0013] S8. The courier system learns and optimizes the courier feedback information, continuously adjusts the courier system parameters, and transmits the adjusted system parameters to the cloud deployment sub-platform. The cloud deployment sub-platform optimizes and updates the basic model sub-file. After the basic model sub-file is optimized and updated, the order information temporarily stored in the cloud deployment sub-platform is regularly transmitted to the database for storage;

[0014] S9. When the basic model sub-files of multiple cloud deployment sub-platforms undergo consistent changes, update the parameters in the basic model.

[0015] Preferably, the important parameters in the historical order data in step S1 include package ID, start time, end time, latitude, city information, courier ID, time of accepting the task, GPS point time closest to the acceptance time, information on when the courier accepts the task, courier pickup time, GPS point time closest to the pickup time, coordinates of the courier when receiving the task, and information on accepting the package.

[0016] Preferably, in step S2, the Transformer model is used to pre-train and learn important parameters in the historical order information, including: inputting the important parameter information in the historical order information into the Transformer model, the Transformer model predicts the pickup time, and compares the pickup time with other information to extract hidden information related to the pickup time.

[0017] Preferably, in step S2, the DRL4Route network is used to learn the route prediction agent based on the state, action, reward and state transition probability. * , the formula is:

[0018]

[0019] Where γ represents the discount factor, which is a trade-off between the importance of current rewards and future rewards, E represents the embedding matrix, and r t represents the reward for a specific task; θ represents the parameters of the current strategy; π θ Indicates the current strategy;

[0020] The route prediction agent includes:

[0021] The route prediction agent is Γ C Modeling,Following the encoder-decoder architecture, the agent selects a task from the unfinished tasks step by step, and its encoder calculates the comprehensive representation of the unfinished tasks, which is formulated as:

[0022] E=Encoder(O w );

[0023] The decoder is based on the embedding matrix E = (e 1, e 2, ,...,e n, ) Accurately calculate the predicted route where e i is the embedding of the i-th unfinished task, e i Learn the output probability distribution of all unfinished tasks, the formula is:

[0024] o t ,h t+1 =DecoderCell(E,C,h t ,Y 1:t-1 );

[0025] In the formula, O w represents the unfinished task set of courier w at query time t, Y 1:t-1 represents the output path until the t-th step decoding, h t represents the decoder hidden state, o t represents the output probability distribution of the t-th step decoding, and C represents the route constraint;

[0026] The states include:

[0027] Status t ∈S represents the state of the environment when the agent decodes at step t, and the formula is:

[0028]

[0029] In the formula, is the output route until the t-th step decoding;

[0030] The actions include:

[0031] Action a∈A t To select a task Given the current task candidate and state, the action set (a1, a2, ..., a n )∈A=A1×A2×…×A n Generate a predicted route, action space A t The task candidate selected by the agent at step t, under the route constraint, the action space changes as the decoding process changes;

[0032] The rewards include:

[0033] It is defined using the position squared deviation, and its formula is:

[0034] LSD(p,q)=(p,q) 2 ;

[0035] Where p,q≥0 represents the order of tasks in the route, LSD measures the deviation between the prediction and the label, and the smaller the LSD, the better the performance;

[0036] There are four types of rewards, and the formula is:

[0037]

[0038] In the formula, m represents the number of tasks in the label, represents the output task, Y represents the label value, Indicates the number of tasks in the prediction. Case 1 means that if the output task is not in the label and is output within the first m tasks, it will produce a deviation between the label and the prediction; case 2 means that if the output task is not in the label and is output after the first m tasks, it will not be considered in the evaluation; case 3 means that if the output task is in the label and is not output in the correct order, the deviation is calculated directly; case 4 means that if the output task is in the label and the output order is correct, a reward is obtained.

[0039] The state transition probability includes:

[0040] If in state a t Take Actions t , then from state s t to t+1 The transition probability is:

[0041]

[0042] In the formula, represents a set of non-negative real numbers, S represents the state, and A represents the action.

[0043] Preferably, the learning route predicts the optimal parameters of the agent and maximizes the expected cumulative reward, including:

[0044] Convert rewards to losses and define the negative expected cumulative reward for the complete sequence as:

[0045]

[0046] In the formula, From strategy The sampling is obtained, Representing and predicting routes Related rewards;

[0047] Use network evaluation to estimate the state value function and learn the state-action value function, so that the route prediction agent updates the strategy distribution in the direction suggested by the network evaluation. In the definition of state-action pair Q(s t ,a t ) value and V(s t ) value, the formula is:

[0048]

[0049] In the formula, Q(s t ,a t ) represents the query function, V(s t ) represents the value function;

[0050] The state-action value function is calculated recursively using dynamic programming, and the formula is:

[0051]

[0052] With trainable parameters θ c The network evaluation is used to approximate the value function The network evaluation and route prediction agents share parameters, and then use the predicted value function to estimate the advantage function The formula is:

[0053]

[0054] Secondly, we sample a set of unfinished tasks and calculate the value function in state s t+1 The expectation under , and the approximate advantage function accelerates the calculation, the formula is:

[0055]

[0056] Finally, a sampling calculation is performed on multiple unfinished task sets, and the network evaluation returns the value estimate to the route prediction agent. The route prediction agent uses the value estimate to calculate the advantage approximation function and update the loss. The formula is:

[0057]

[0058] In the formula, represents the agent strategy function of the evaluation network modeling, s i,t represents the state of the i-th sample, a i,t represents the action of the i-th sample, represents the negative expected cumulative reward of the complete sequence;

[0059] The above value function is trained using robust loss, the formula is:

[0060]

[0061] In the formula, represents the network evaluation estimate, which is called the “baseline”, smoothL1(x) represents the smoothing function, and N represents the number of samples.

[0062] Preferably, in step S7, the actual arrival time of the courier is divided into on-time arrival time and late arrival time. When the courier arrives late, the courier will feed back nonlinear factors affecting the arrival time to the courier-end system. The nonlinear factors include road conditions, weather, and personal characteristics of the deliveryman.

[0063] Therefore, the present invention adopts the above-mentioned arrival time prediction optimization system and method based on pre-training learning, which has the following beneficial effects:

[0064] (1) Generate a basic model file through Transformer pre-training learning and DRL4Route network, and form multiple basic model sub-files during deployment. The basic model sub-files are continuously optimized to reduce training costs;

[0065] (2) Taking into account the nonlinear factors that affect the courier’s arrival time, such as road conditions, weather, and the deliveryman’s personal characteristics, the model subfile learns the nonlinear factors to generate more accurate route planning and time prediction;

[0066] (3) The customer terminal can check the order status and estimated arrival time in real time, improving customer satisfaction.

[0067] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is an overall framework diagram of an embodiment of the present invention;

[0069] Figure 2 This is a schematic diagram of Transformer pre-training learning in an embodiment of the present invention;

[0070] Figure 3 Schematic diagram of encoding and decoding in Transformer according to an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram of Transformer encoding according to an embodiment of the present invention;

[0072] Figure 5 This is a graph of Transformer pre-training learning results in an embodiment of the present invention;

[0073] Figure 6 This is a diagram of the DRL4Route network architecture of an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] Example

[0076] like Figure 1 As shown, the present invention provides an arrival time prediction optimization system based on pre-training learning, including a server, a database module, a cloud deployment platform and a courier system, the server is connected to the database module, the database module is connected to the cloud deployment platform, the cloud deployment platform includes multiple sub-platforms, each cloud deployment sub-platform is connected to an courier system, each cloud deployment sub-platform can represent a region or a city, and the courier system is connected to the customer terminal and the courier terminal.

[0077] The server generates the corresponding basic model file M through Transformer pre-training learning and DRL4Route network.

[0078] The database module is used to store completed order information and basic model parameters, and also stores the basic model file M generated by deep learning.

[0079] The cloud deployment platform includes multiple sub-platforms (P1, P2, ..., Pn), each of which is responsible for processing and generating different model parameters. For example, the P2 sub-platform obtains the basic model file M through the database and stores the model parameters generated by the express delivery system. The P1, P3 to Pn sub-platforms are also responsible for storing their own model parameters, and continuously update the basic model parameters and optimize the system based on the comparison of the basic model sub-file parameters of each sub-platform.

[0080] The courier system processes the customer's order information in real time and transmits it to the customer terminal and the courier terminal. In addition to considering the constraints given by the customer, the courier system will also adjust parameters based on the courier's feedback, such as whether the courier's workload is too full, etc. It will also consider external factors such as bad weather such as rain, snow and typhoons, and traffic information during rush hours.

[0081] There are multiple customer terminals, customer 1 to customer q represent terminals of multiple customers. The terminals can display order information and the estimated arrival time of the order to ensure that the customer is aware of the progress of the order at any time.

[0082] There are multiple courier terminals. Courier 1 to courier r receive the delivery route and estimated arrival time through their own terminals. The courier end system can provide personalized route planning for each courier to ensure that their orders are delivered on time. After completing the order, the courier will feedback the actual arrival time and route to the system to further optimize the route planning and time prediction of subsequent orders.

[0083] The present invention also provides an arrival time prediction optimization method based on pre-training learning, the steps comprising:

[0084] S1. Obtain the historical courier's completed order data and extract important parameters from the historical order data.

[0085] Important parameters in historical order data include package ID, start time, end time, longitude, city information, courier ID, time of accepting the task, GPS point time closest to the acceptance time, information on when the courier accepts the task, courier pickup time, GPS point time closest to the pickup time, coordinates of the courier when receiving the task, and information on accepting the package.

[0086] S2. The server first uses the Transformer model to pre-train the important parameters in the historical order information, and then uses the DRL4Route network to generate the corresponding basic model file and store it in the database.

[0087] Transformer model pre-training learning Figure 2As shown in the figure, ENCODER means encoding and ENCODER means decoding, which converts information into a language that can be understood by computers. The steps include: inputting important parameter information in historical order information into the Transformer model, the Transformer model predicts the pickup time, and compares the pickup time with other information to extract hidden information related to the pickup time.

[0088] The Transformer model includes an encoder and a decoder. The schematic diagram of the encoder and decoder is as follows Figure 3 As shown in , the encoder contains a self-attention layer and a feed-forward layer. The decoder contains a self-attention layer, an encoder-decoder attention layer and a feed-forward layer. The encoder can learn two vectors to obtain the information correlation between the two vectors. Figure 4 As shown, the Transformer model pre-training learning results are as follows Figure 5 shown.

[0089] After Transformer pre-training, the DRL4Route network is used to sort the orders, and the optimal parameters θ of the agent are predicted based on the state, action, reward and state transition probability learning route. * , the formula is:

[0090]

[0091] Where γ represents the discount factor, which is a trade-off between the importance of current rewards and future rewards, E represents the embedding matrix, and r t represents the reward for a specific task; θ represents the parameters of the current strategy; π θ Represents the current strategy. The following introduces the route prediction agent, state, action, reward and state transition probability.

[0092] Route Prediction Agent

[0093] The route prediction agent is Γ C Modeling,Following the encoder-decoder architecture, the agent selects a task from the unfinished tasks step by step, and its encoder calculates the comprehensive representation of the unfinished tasks, which is formulated as:

[0094] E=Encoder(O w );

[0095] The decoder is based on the embedding matrix E = (e 1, e 2, ,...,e n, ) Accurately calculate the predicted route where e i is the embedding of the i-th unfinished task, e iLearn the output probability distribution of all unfinished tasks, the formula is:

[0096] o t ,h t+1 =DecoderCell(E,C,h t ,Y 1:t-1 );

[0097] In the formula, O w represents the unfinished task set of courier w at query time t, Y 1:t-1 represents the output path until the t-th step decoding, h t represents the decoder hidden state, o t represents the output probability distribution of the t-th step decoding, and C represents the route constraint;

[0098] state

[0099] Status t ∈S represents the state of the environment when the agent decodes at step t, and the formula is:

[0100]

[0101] In the formula, is the output route until the t-th step decoding;

[0102] action

[0103] Action a∈A t To select a task Given the current task candidate and state, the action set (a1, a2, ..., a n )∈A=A1×A2×…×A n Generate a predicted route, action space A t The task candidate selected by the agent at step t, under the route constraint, the action space changes as the decoding process changes;

[0104] award

[0105] It is defined using the position squared deviation, and its formula is:

[0106] LSD(p,q)=(p,q) 2 ;

[0107] Where p,q≥0 represents the order of tasks in the route, LSD measures the deviation between the prediction and the label, and the smaller the LSD, the better the performance;

[0108] There are four types of rewards, and the formula is:

[0109]

[0110] In the formula, m represents the number of tasks in the label, represents the output task, Y represents the label value, Indicates the number of tasks in the prediction. Case 1 means that if the output task is not in the label and is output within the first m tasks, it will produce a deviation between the label and the prediction; case 2 means that if the output task is not in the label and is output after the first m tasks, it will not be considered in the evaluation; case 3 means that if the output task is in the label and is not output in the correct order, the deviation is calculated directly; case 4 means that if the output task is in the label and the output order is correct, a reward is obtained.

[0111] State transition probability

[0112] If in state a t Take Actions t , then from state s t to t+1 The transition probability is:

[0113]

[0114] In the formula, represents a set of non-negative real numbers, S represents the state, and A represents the action.

[0115] The optimal parameters of the agent are predicted by learning the route through the DRL4Route network to maximize the expected cumulative reward. The DRL4Route network trained and learned by Transformer is as follows: Figure 6 As shown, specifically including:

[0116] Convert rewards to losses and define the negative expected cumulative reward for the complete sequence as:

[0117]

[0118] In the formula, From strategy The sampling is obtained, Representing and predicting routes Since the above formula can only observe the reward after the complete action sequence has been sampled, in this case the model is forced to wait until the end of the sequence to observe its performance, resulting in error accumulation and high variability in policy gradient estimates.

[0119] Therefore, it is necessary to provide reward feedback for a given action at each time step to reduce the variance of the policy gradient estimate. Specifically, the state value function (i.e., V function) is estimated using the "network evaluation", which evaluates how good the agent is at a certain state, and a state-action value function (i.e., Q value) is learned to evaluate the benefits of taking a specific action in a specific state. The network evaluation is a function estimator that tries to evaluate the model at each decoding step t. The route prediction agent updates the strategy distribution according to the direction recommended by the network evaluation. In the definition of state-action pair Q(s t ,a t ) value and V(s t ) value, the formula is:

[0120]

[0121] In the formula, Q(s t ,a t ) represents the query function, V(s t ) represents the value function;

[0122] The state-action value function is calculated recursively using dynamic programming, and the formula is:

[0123]

[0124] With trainable parameters θ c The network evaluation is used to approximate the value function The network evaluation and route prediction agents share parameters, and then use the predicted value function to estimate the advantage function The formula is:

[0125]

[0126] Secondly, we sample a set of unfinished tasks and calculate the value function in state s t+1 The expectation under , and the approximate advantage function accelerates the calculation, the formula is:

[0127]

[0128] The V function is in a specific state s t It can produce a good strategy. The Q value measures the value of choosing a specific behavior when in this state. Based on these two functions, the advantage function The relative superiority of each action is expressed by subtracting the value function V from the value function Q.

[0129] Finally, multiple unfinished task sets are sampled and calculated to reduce the variance of the gradient estimate. The network evaluation returns the value estimate to the route prediction agent, which uses the value estimate to calculate the advantage approximation function and update the loss. The formula is:

[0130]

[0131] In the formula, represents the agent strategy function of the evaluation network modeling, s i,t represents the state of the i-th sample, a i,t represents the action of the i-th sample, represents the negative expected cumulative reward for the complete sequence.

[0132] Specifically, the above value function is trained using robust loss, and the formula is:

[0133]

[0134] In the formula, represents the network evaluation estimate, which is called the “baseline”, smoothL1(x) represents the smoothing function, and N represents the number of samples.

[0135] In summary, if the tasks output by the route prediction agent match their actual order in the route, the reward measured by the test criterion will be high, which will encourage action, and vice versa. At prediction time, the agent takes all outstanding tasks as input and progressively predicts the entire service route, with the task selected at each decoding step t given by:

[0136]

[0137] in, is the output probability of task k at the tth step.

[0138] S3. Deploy the basic model file to the cloud deployment sub-platform to form multiple basic model sub-files, and transfer the basic model sub-files to the express delivery system.

[0139] S4. The user places an order, and the user terminal transmits the order information to the courier system.

[0140] S5. The courier system receives the transmitted order information, processes the order information by loading the basic model sub-file, and predicts the route and arrival time.

[0141] S6. The courier system transmits the predicted route and arrival time to the customer terminal and the courier terminal.

[0142] S7. After the courier completes the order, the courier will feedback the actual arrival time and route to the courier system. The completed order will be temporarily stored in the cloud deployment sub-platform through the courier system.

[0143] The actual arrival time is divided into on-time arrival time and late arrival time. When the arrival is late, the courier will feedback the non-linear factors affecting the arrival time to the courier system. The non-linear factors include road conditions, weather, the deliveryman’s personal characteristics and other reasons.

[0144] S8. The courier system learns and optimizes the courier’s feedback information. For example, rain will delay the courier’s delivery time. After learning, the system will delay the estimated arrival time under extreme weather conditions, and continuously adjust the courier system parameters based on the feedback information. The adjusted system parameters are transmitted to the cloud deployment sub-platform, and the basic model sub-file is optimized and updated. After the basic model sub-file is optimized and updated, the order information temporarily stored in the cloud deployment sub-platform is regularly transmitted to the database for storage;

[0145] S9. When the basic model sub-files of multiple cloud deployment sub-platforms undergo consistent changes, the system automatically or manually updates the parameters in the basic model.

[0146] Therefore, the present invention adopts the above-mentioned arrival time prediction optimization system and method based on pre-training learning, and realizes efficient order delivery and time prediction through the collaborative cooperation of cloud deployment platform, database, server and express delivery system, and improves the overall service level of the logistics industry. This not only enables the basic model to be efficiently utilized, but also improves the efficiency of logistics distribution, and provides accurate time prediction services, ultimately improving customer experience.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An arrival time prediction optimization system based on pre-training learning, characterized by: It includes a server, a database module, a cloud deployment platform and a courier system. The server is connected to the database module, the database module is connected to the cloud deployment platform, the cloud deployment platform includes multiple sub-platforms, each of the cloud deployment sub-platforms is connected to one of the courier systems, and the courier system is connected to the customer terminal and the courier terminal.

2. A method for optimizing arrival time prediction based on pre-training learning, using the system for optimizing arrival time prediction based on pre-training learning described in claim 1, characterized in that the steps include: S1. Obtain the historical courier's completed order data and extract important parameters from the historical order data; S2. The server first uses the Transformer model to pre-train the important parameters in the historical order information, and then uses the DRL4Route network to generate the corresponding basic model file and store it in the database; S3. Deploy the basic model file to the cloud deployment sub-platform to form multiple basic model sub-files, and transmit the basic model sub-files to the express delivery end system; S4, the user places an order, and the user terminal transmits the order information to the courier system; S5. The courier end system receives the transmitted order information, processes the order information by loading the basic model sub-file, and predicts the route and arrival time; S6, the courier system transmits the predicted route and arrival time to the customer terminal and the courier terminal; S7. After the courier completes the order, the courier will feedback the actual arrival time and route to the courier end system, and the completed order will be temporarily stored in the cloud deployment sub-platform by the courier end system; S8. The courier system learns and optimizes the courier feedback information, continuously adjusts the courier system parameters, and transmits the adjusted system parameters to the cloud deployment sub-platform. The cloud deployment sub-platform optimizes and updates the basic model sub-file. After the basic model sub-file is optimized and updated, the order information temporarily stored in the cloud deployment sub-platform is regularly transmitted to the database for storage; S9. When the basic model sub-files of multiple cloud deployment sub-platforms undergo consistent changes, update the parameters in the basic model.

3. The method for optimizing arrival time prediction based on pre-training learning according to claim 2, characterized in that: The important parameters in the historical order data in step S1 include package ID, start time, end time, longitude, city information, courier ID, time of accepting the task, GPS point time closest to the acceptance time, information on when the courier accepts the task, courier pickup time, GPS point time closest to the pickup time, coordinates of the courier when receiving the task, and information on accepting the package.

4. The arrival time prediction optimization method based on pre-training learning according to claim 3 is characterized in that: In step S2, the Transformer model is used to pre-train and learn important parameters in the historical order information, including: inputting the important parameter information in the historical order information into the Transformer model, the Transformer model predicts the pickup time, and compares the pickup time with other information to extract hidden information related to the pickup time.

5. The arrival time prediction optimization method based on pre-training learning according to claim 4 is characterized in that: In step S2, the DRL4Route network is used to learn the optimal parameters θ of the route prediction agent based on state, action, reward and state transition probability. * , the formula is: Where γ represents the discount factor, which is a trade-off between the importance of current rewards and future rewards, E represents the embedding matrix, and r t Indicates a specific task reward; θ represents the parameters of the current strategy; Indicates the current strategy; The route prediction agent includes: the route prediction agent is Γ C Modeling,Following the encoder-decoder architecture, the agent selects a task from the unfinished tasks step by step, and its encoder calculates the comprehensive representation of the unfinished tasks, which is formulated as: E=Encoder(The w ); The decoder is based on the embedding matrix E = (e 1, e 2, ,...,e n, ) Accurately calculate the predicted route where e i is the embedding of the i-th unfinished task, e i Learn the output probability distribution of all unfinished tasks, the formula is: o t ,h t+1 =DecoderCell(E,C,h t ,Y 1:t-1 ); In the formula, O w represents the unfinished task set of courier w at query time t, Y 1:t-1 represents the output path until the t-th step decoding, h t represents the decoder hidden state, o t represents the output probability distribution of the t-th step decoding, and C represents the route constraint; The states include: Status t ∈S represents the state of the environment when the agent decodes at step t, and the formula is: In the formula, is the output route until the t-th step decoding; The actions include: Action a∈A t To select a task Given the current task candidate and state, the action set (a1, a2, ..., a n )∈A=A1×A2×…×A n Generate a predicted route, action space A t The task candidate selected by the agent at step t, under the route constraint, the action space changes as the decoding process changes; The rewards include: It is defined using the position squared deviation, and its formula is: LSD(p,q)=(p,q) 2 ; Where p,q≥0 represents the order of tasks in the route, LSD measures the deviation between the prediction and the label, and the smaller the LSD, the better the performance; There are four types of rewards, and the formula is: Where m represents the number of tasks in the label, represents the output task, Y represents the label value, Indicates the number of tasks in the prediction. Case 1 means that if the output task is not in the label and is output within the first m tasks, it will produce a deviation between the label and the prediction; case 2 means that if the output task is not in the label and is output after the first m tasks, it will not be considered in the evaluation; case 3 means that if the output task is in the label and is not output in the correct order, the deviation is calculated directly; case 4 means that if the output task is in the label and the output order is correct, a reward is obtained. The state transition probability includes: If in state a t Take Actions t , then from state s t to t+1 The transition probability is: In the formula, represents a set of non-negative real numbers, S represents the state, and A represents the action.

6. The method for optimizing arrival time prediction based on pre-training learning according to claim 5, characterized in that: The learning path predicts the optimal parameters of the agent to maximize the expected cumulative reward, including: Convert rewards to losses and define the negative expected cumulative reward for the complete sequence as: In the formula, From strategy The sampling is obtained, Representing and predicting routes Related rewards; Use network evaluation to estimate the state value function and learn the state-action value function, so that the route prediction agent updates the strategy distribution in the direction suggested by the network evaluation. In the definition of state-action pair Q(s t ,a t ) value and V(s t ) value, the formula is: In the formula, Q(s t ,a t ) represents the query function, V(s t ) represents the value function; The state-action value function is calculated recursively using dynamic programming, and the formula is: With trainable parameters θ c The network evaluation is used to approximate the value function The network evaluation and route prediction agents share parameters, and then use the predicted value function to estimate the advantage function The formula is: Secondly, we sample a set of unfinished tasks and calculate the value function in state s t+1 The expectation under , and the approximate advantage function accelerates the calculation, the formula is: Finally, a sampling calculation is performed on multiple unfinished task sets, and the network evaluation returns the value estimate to the route prediction agent. The route prediction agent uses the value estimate to calculate the advantage approximation function and update the loss. The formula is: In the formula, represents the agent strategy function of the evaluation network modeling, s i,t represents the state of the i-th sample, a i,t represents the action of the i-th sample, represents the negative expected cumulative reward of the complete sequence; The value function is trained using robust loss, the formula is: In the formula, represents the network evaluation estimate, which is called the "baseline", smoothL1(x) represents the smoothing function, and N represents the number of samples.

7. The method for optimizing arrival time prediction based on pre-training learning according to claim 6, characterized in that: In step S7, the actual arrival time of the courier is divided into on-time arrival time and late arrival time. When the courier arrives late, the courier will feed back the nonlinear factors affecting the arrival time to the courier system. The nonlinear factors include road conditions, weather and personal characteristics of the deliveryman.

Images