Delivery route planning methods, devices, computer equipment, and storage media

By acquiring current status information and predicting the reward for the next optional task location, the problem of unreasonable delivery route planning in existing technologies is solved, enabling more reasonable route planning decisions and meeting delivery time requirements.

CN115700677BActive Publication Date: 2026-11-17SF TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110794070.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-14
Publication Date
2026-11-17
Estimated Expiration
2041-07-14

AI Technical Summary

Technical Problem

Existing delivery route planning algorithms are unable to make reasonable decisions in a short period of time and fail to consider possible future changes, resulting in inadequate delivery route planning.

Method used

By acquiring current status information, the reward for the next available task location is predicted, and the next task location for the target delivery person is planned based on current and future rewards, taking into account possible future changes.

Benefits of technology

It improves the rationality of delivery route planning, better meets delivery time requirements, and reduces additional total route time and late payment penalties.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115700677B_ABST
    Figure CN115700677B_ABST
Patent Text Reader

Abstract

This application relates to a delivery route planning method, apparatus, computer equipment, and storage medium. The method includes: when it is detected that a target delivery person has completed a task at the current task location, acquiring current status information, including: the current time, the current subsequent delivery routes of all delivery persons, and newly generated task locations; determining, based on the current subsequent delivery routes and newly generated task locations, the original subsequent delivery routes of all delivery persons and a set of next-optional task locations for the target delivery person; predicting, based on the current time, the original subsequent delivery routes, and the set of next-optional task locations, the corresponding reward after each next-optional task location in the set is selected as the next task location for the target delivery person; and planning the next task location for the target delivery person based on the corresponding reward after each next-optional task location is selected as the next task location for the target delivery person. This method can improve the rationality of delivery route planning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of logistics technology, and in particular to a delivery route planning method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the rise of mobile internet, more and more people are using O2O (Online To Offline, combining offline business opportunities with the internet) platforms for local shopping, including ordering food delivery and errands such as flowers and cakes. After receiving an order from a user, the platform needs to assign the delivery task to a suitable delivery person within a short period of time, and the delivery person needs to complete the delivery within a specified time limit.

[0003] As order volume increases, platforms need to allocate a large number of order delivery tasks to multiple delivery personnel within a short period of time, placing higher demands on delivery routes to complete more orders within the required timeframe. However, current scheduling and planning algorithms often fail to make reasonable decisions in a short time and do not consider the dynamic nature of the decision-making process, meaning they do not take future changes into account, resulting in inadequate delivery route planning. Summary of the Invention

[0004] Therefore, it is necessary to provide a delivery route planning method, apparatus, computer equipment, and storage medium that can improve the rationality of delivery route planning in response to the above-mentioned technical problems.

[0005] A delivery route planning method, the method comprising:

[0006] When it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0007] Based on the current subsequent delivery routes and the newly generated task locations, determine the original subsequent delivery routes for all delivery personnel and the set of next optional task locations for the target delivery personnel;

[0008] Based on the current time, the original subsequent delivery route, and the set of next optional task locations, predict the reward corresponding to each next optional task location in the set of next optional task locations after it is selected as the next task location for the target delivery person;

[0009] The next task location for the target delivery person is planned based on the reward corresponding to the next optional task location being selected as the next task location for the target delivery person.

[0010] A delivery route planning device, the device comprising:

[0011] The acquisition module is used to acquire current status information when it detects that the target delivery person has completed the task at the current task location. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0012] The determination module is used to determine the original subsequent delivery routes of all delivery personnel and the set of next optional task locations for the target delivery personnel based on the current subsequent delivery routes and the newly generated task locations;

[0013] The prediction module is used to predict the reward corresponding to each of the next optional task locations in the next optional task location set after it is selected as the next task location of the target delivery person, based on the current time, the original subsequent delivery route and the next optional task location set;

[0014] The planning module is used to plan the next task location for the target delivery person based on the reward corresponding to each of the next optional task locations being selected as the next task location for the target delivery person.

[0015] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0016] When it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0017] Based on the current subsequent delivery routes and the newly generated task locations, determine the original subsequent delivery routes for all delivery personnel and the set of next optional task locations for the target delivery personnel;

[0018] Based on the current time, the original subsequent delivery route, and the set of next optional task locations, predict the reward corresponding to each next optional task location in the set of next optional task locations after it is selected as the next task location for the target delivery person;

[0019] The next task location for the target delivery person is planned based on the reward corresponding to the next optional task location being selected as the next task location for the target delivery person.

[0020] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0021] When it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0022] Based on the current subsequent delivery routes and the newly generated task locations, determine the original subsequent delivery routes for all delivery personnel and the set of next optional task locations for the target delivery personnel;

[0023] Based on the current time, the original subsequent delivery route, and the set of next optional task locations, predict the reward corresponding to each next optional task location in the set of next optional task locations after it is selected as the next task location for the target delivery person;

[0024] The next task location for the target delivery person is planned based on the reward corresponding to the next optional task location being selected as the next task location for the target delivery person.

[0025] The aforementioned delivery route planning method, apparatus, computer equipment, and storage medium, when detecting that the target delivery person has completed a task at the current task location, acquire current status information, including: the current time, the current subsequent delivery routes of all delivery persons, and newly generated task locations; based on the current subsequent delivery routes and newly generated task locations, determine the original subsequent delivery routes of all delivery persons and the set of next optional task locations for the target delivery person; based on the current time, original subsequent delivery routes, and the set of next optional task locations, predict the corresponding reward after each next optional task location in the set is selected as the next task location for the target delivery person; based on the corresponding reward after each next optional task location is selected as the next task location for the target delivery person, plan the next task location for the target delivery person. The reward corresponding to each next optional task location obtained in this way considers both current and future rewards. The planning decision for the next task location is made based on both current and future rewards, thereby taking into account possible future changes in the planning decision and making the delivery route planning more reasonable. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a delivery route planning method in one embodiment;

[0027] Figure 2 This is a flowchart illustrating step S106 in one embodiment;

[0028] Figure 3 This is a flowchart illustrating step S202 in one embodiment;

[0029] Figure 4This is a flowchart illustrating a method for training value function parameters in one embodiment;

[0030] Figure 5 This is a structural block diagram of a delivery route planning device in one embodiment;

[0031] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0032] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] In one embodiment, such as Figure 1 As shown, a delivery route planning method is provided. This embodiment illustrates the method by applying it to a server. It is understood that the method can also be applied to a terminal, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps S102 to S108.

[0035] S102, when it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0036] A target delivery person refers to any one of the delivery persons recorded in the delivery system. The current task location can be the pickup location or delivery location of the currently processed order, and the corresponding completed task is either a pickup task or a delivery task. For example, for a food delivery order, the delivery task includes a pickup task and a delivery task, which respectively involve a pickup location (corresponding to the merchant's address) and a delivery location (corresponding to the customer's address). When the delivery person responsible for the delivery task of this food delivery order arrives at the pickup location, they are considered to have completed the pickup task; when the delivery person responsible for the delivery task of this food delivery order arrives at the delivery location, they are considered to have completed the delivery task.

[0037] Current status information refers to the information considered when making delivery route planning decisions. Current time refers to the time when the target delivery person completes the task at the current task location; it can also be understood as the decision time when the target delivery person replans their subsequent delivery route. Current subsequent delivery route refers to the delivery route before the decision is made at the current time. Each delivery person's current subsequent delivery route includes the task location the delivery person was originally supposed to go to after the current time; the number of such task locations can be one or more. Newly generated task locations include those newly generated between the previous time and the current time. The previous time can be understood as the time when a delivery person was last detected completing a task. If the current time is the time when a delivery person was first detected completing a task, then the previous time can be understood as the time when the task was initially assigned to the delivery person.

[0038] For example, suppose all delivery personnel include delivery personnel a, b, and c, and the initial orders include order 1, order 2, order 3, and order 4. At 10:00, order 1 is assigned to delivery personnel a, order 2 to delivery personnel b, and orders 3 and 4 to delivery personnel c. If delivery personnel a arrives at the pickup location of order 1 at 10:10 to complete the pickup task, and a new order 5 is generated between 10:00 and 10:10, then the current status information includes: the time when delivery personnel a completes the pickup task, the current subsequent delivery routes of delivery personnel a, b, and c, and the pickup and delivery locations of order 5. Among them, delivery person A completed the pickup task at 10:10; delivery person A's current subsequent delivery route is: delivery location of order 1; delivery person B's current subsequent delivery route is: pickup location of order 2 - delivery location of order 2; delivery person C's current subsequent delivery route is: pickup location of order 3 - delivery location of order 3 - pickup location of order 4 - delivery location of order 4.

[0039] S104, based on the current subsequent delivery routes and newly generated task locations, determine the original subsequent delivery routes for all delivery personnel and the set of next optional task locations for the target delivery personnel.

[0040] The original follow-up delivery route is the delivery route obtained by adding newly generated task locations to the current follow-up delivery route according to predetermined rules. In one embodiment, newly generated task locations are added to the current follow-up delivery route in a greedy manner. For example, without violating constraints such as pickup and delivery time, time window length, and delivery personnel's working hours, the newly generated task locations are added to the corresponding delivery person's current follow-up delivery route, which becomes the corresponding delivery person's original follow-up delivery route. It is understood that if no newly generated task locations are added to the delivery person's current follow-up delivery route, then the delivery person's original follow-up delivery route becomes the current follow-up delivery route.

[0041] For example, in the example above, assuming the newly generated task locations (the pickup and delivery locations of order 5) are added to the current subsequent delivery path of delivery person a, then the original subsequent delivery path of delivery person a is: the delivery location of order 1 - the pickup location of order 5 - the delivery location of order 5, the original subsequent delivery path of delivery person b is: the pickup location of order 2 - the delivery location of order 2, and the original subsequent delivery path of delivery person c is: the pickup location of order 3 - the delivery location of order 3 - the pickup location of order 4 - the delivery location of order 4.

[0042] The set of possible next task locations for a target delivery person refers to the set consisting of all possible next task locations for that target delivery person. Each element in this set represents a possible next task location for that target delivery person. It should be noted that the possible next task locations for a target delivery person do not include the next task locations for other delivery people. If the next task location for another delivery person is a pickup location, then the possible next task locations for the target delivery person also do not include the corresponding delivery locations for the next task locations of other delivery people.

[0043] For example, in the example above, the next set of possible task locations for the target delivery person (i.e., delivery person a) includes: the delivery location of order 1, the pickup location of order 4, the delivery location of order 4, the pickup location of order 5, and the delivery location of order 5.

[0044] S106, based on the current time, the original subsequent delivery route, and the set of next optional task locations, predict the corresponding reward after each next optional task location in the set of next optional task locations is selected as the next task location for the target delivery person.

[0045] For each of the next possible task locations in the set of next possible task locations for the target delivery person, once it is selected as the next task location for the target delivery person, the original subsequent delivery routes of the target delivery person and other delivery persons may change accordingly, resulting in new subsequent delivery routes. That is, every time a delivery route planning decision is made, the subsequent delivery routes of all delivery persons may change before and after the decision.

[0046] Specifically, assuming that the next optional task location A is chosen as the next task location for the target delivery person at the current time, the current reward corresponding to the selection of this next optional task location as the target delivery person's next task location can be obtained by calculating the objective function values ​​corresponding to all original and all new subsequent delivery paths, and then calculating the difference between the two objective function values. The objective function value is defined as the sum of the total path time and the total timeout penalty. Furthermore, relevant state features can be extracted based on the current state information to design a value function. This value function can then be used to predict the future reward corresponding to the selection of this next optional task location as the target delivery person's next task location.

[0047] S108, based on the reward corresponding to each next optional task location being selected as the next task location for the target delivery person, plan the next task location for the target delivery person.

[0048] After obtaining the corresponding rewards for each next optional task location being selected as the next task location for the target delivery person, one next optional task location can be determined from all the next optional task locations based on the size of each reward, and then designated as the next task location for the target delivery person.

[0049] In one embodiment, the step of planning the next task location for the target delivery person based on the reward corresponding to each next optional task location being selected as the target delivery person's next task location can specifically involve planning the next optional task location corresponding to the minimum reward as the target delivery person's next task location. Accordingly, the additional total route time and total timeout penalty resulting from the planning decision are relatively minimized, better meeting delivery timeliness requirements and improving the rationality of the planning.

[0050] In the aforementioned delivery route planning method, when it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. This current status information includes: the current time, the original subsequent delivery routes of all delivery persons, and the newly generated task location. Based on the original subsequent delivery routes and the newly generated task location, a set of next-option task locations for the target delivery person is determined. Based on the current time, the original subsequent delivery routes, and the set of next-option task locations, the reward corresponding to each next-option task location selected as the target delivery person's next task location is predicted. Based on the reward corresponding to each next-option task location selected as the target delivery person's next task location, the next task location for the target delivery person is planned. The reward corresponding to each next-option task location obtained in this way considers both current and future rewards. The planning decision for the next task location is made based on both current and future rewards, thus taking into account potential future changes in the planning decision and making the delivery route planning more reasonable.

[0051] In one embodiment, such as Figure 2 As shown, step S106 may specifically include steps S202 to S204.

[0052] S202, based on the original subsequent delivery route and the new subsequent delivery routes of all delivery personnel after each next optional task location in the set of next optional task locations is selected as the next task location of the target delivery person, predict the first reward corresponding to each next optional task location.

[0053] For each next optional task location in the set of next optional task locations for the target delivery person, when that next optional task location is selected as the target delivery person's next task location, the subsequent delivery routes for that target delivery person and other delivery people may change accordingly, thus obtaining new subsequent delivery routes for all delivery people. The first reward corresponding to the next optional task location can be understood as the current reward corresponding to the next optional task location being selected as the target delivery person's next task location.

[0054] For example, in the example above, the set of possible next task locations for the target delivery person (i.e., delivery person a) includes: the delivery location of order 1, the pickup location of order 4, the delivery location of order 4, the pickup location of order 5, and the delivery location of order 5. If the pickup location of order 4 is selected as the next task location for delivery person a, then the new subsequent delivery path for delivery person a is: the pickup location of order 4 - the delivery location of order 4 - the delivery location of order 1 - the pickup location of order 5 - the delivery location of order 5. The new subsequent delivery path for delivery person b is: the pickup location of order 2 - the delivery location of order 2. The new subsequent delivery path for delivery person c is: the pickup location of order 3 - the delivery location of order 3.

[0055] In one embodiment, such as Figure 3 As shown, step S202 may specifically include steps S302 to S306.

[0056] S302, for any selected next optional task location, obtain a first objective function value based on the travel time between adjacent task locations in the original subsequent delivery path, the arrival time of each task location and its corresponding latest arrival time. The first objective function value is the sum of the total travel time of the original subsequent delivery path and the total timeout penalty.

[0057] The travel time between adjacent task locations can be obtained by comparing the distance between them to the delivery speed. The distance between adjacent task locations can be obtained from a map, and the delivery speed can be a preset speed experience value. The arrival time of each task location can be calculated based on the current time and the travel time. The latest arrival time of each task location can be understood as the promised latest arrival time.

[0058] The first objective function value Obj1 can be calculated using the following formula:

[0059]

[0060] Where C1 represents the set of all original subsequent delivery routes for all delivery personnel; c1 represents the original subsequent delivery route for a single delivery personnel; i represents the task location number in the original subsequent delivery route c1; |c1| represents the number of task locations in the original subsequent delivery route c1. This represents the i-th task location in the original subsequent delivery path c1; This represents the (i+1)th task location in the original subsequent delivery path c1; express arrive The journey time; β represents the timeout penalty factor; express Arrival time; express The corresponding latest arrival time; if but like but

[0061] S304. Based on the travel time between adjacent task locations in the new subsequent delivery route, the arrival time of each task location and its corresponding latest arrival time, the second objective function value is obtained. The second objective function value is the sum of the total travel time and the total timeout penalty of the new subsequent delivery route.

[0062] The second objective function value Obj2 can be calculated using the following formula:

[0063]

[0064] Where C2 represents the set of new follow-up delivery routes for all delivery personnel; c2 represents the new follow-up delivery route for a single delivery personnel; j represents the task location number in the new follow-up delivery route c2; |c2| represents the number of task locations in the new follow-up delivery route c2; This represents the j-th task location in the new subsequent delivery route c2; This represents the (j+1)th task location in the new subsequent delivery route c2; express arrive The journey time; β represents the timeout penalty factor; express Arrival time; express The corresponding latest arrival time; if but like but

[0065] Based on the distance between adjacent task locations in each delivery person's original subsequent delivery route, delivery speed, and the latest arrival time corresponding to each task location, the total time spent and total overtime penalty for all delivery persons' original subsequent delivery routes can be obtained.

[0066] S306, the difference between the second objective function value and the first objective function value is used to obtain the first reward corresponding to the next optional task location.

[0067] The delivery route planning problem is modeled as a Markov decision process, where the decision point k is defined as the time point at which the target delivery person completes the task at any location p, and the state S of the model is... k Defined as the time t of the current decision point k k The set of original and subsequent delivery routes for all delivery personnel, C k The newly generated set of mission locations P k The set of these three pieces of information {t k ,S k ,P k The set of next possible task locations for the target delivery person is denoted by A(S). k ) indicates that state S k Optional actions for the next target delivery person (a) k Defined as the next optional mission location, a k ∈A(S k First reward R(S) k ,a k ) is defined as being in state S k The following action a was performed. k The difference between the objective function value of the new set of subsequent delivery routes and the objective function value of the original set of subsequent delivery routes, i.e., R(S) k ,a k ) = Obj2 - Obj1.

[0068] S204. Based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, and the number of optional task locations, predict the second reward corresponding to each next optional task location. The number of optional task locations represents the number of next optional task locations for the target delivery personnel after each next optional task location in the set of next optional task locations is selected as the next task location for the target delivery personnel.

[0069] Each delivery rider's maximum delivery time can be predetermined. A rider's remaining delivery time refers to the remaining delivery time available up to the current time, which can be obtained by subtracting the delivered time up to the current time from the maximum delivery time. The remaining number of delivery riders refers to the number of riders who can continue delivering as of the current time. If a rider's remaining delivery time is insufficient to continue performing delivery tasks, that rider is not considered a rider who can continue delivering. The second reward corresponding to the next optional task location can be understood as the future reward corresponding to the next optional task location being selected as the target rider's next task location.

[0070] In one embodiment, step S204 may specifically include: predicting the second reward corresponding to each next optional task location based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, the number of optional task locations, and the trained value function parameters; the value function parameters include coefficients corresponding to the current time, remaining delivery time, number of remaining delivery personnel, and number of optional task locations, respectively, and the value function is constructed by multiplying the current time, remaining delivery time, number of remaining delivery personnel, and number of optional task locations by their respective coefficients and summing them.

[0071] Based on the model's current state and the selected action, the following information is extracted as features: Current time (using...). (Indicated by) the number of possible next task locations after the target delivery person determines the next task location (in numerals). (indicated by) the remaining delivery time for all delivery personnel (in terms of) The number of remaining delivery personnel who can continue deliveries (indicated by...). (Representation). The second reward can be represented by a value function, V(S). k ,a k It can be designed as a quaternary equation as follows:

[0072]

[0073] in, θ represents the f-th feature; f express The corresponding coefficients, i.e., the parameters of the value function that need to be trained; V(S k ,a k This can be understood as being in state S k The following action a was performed. k The corresponding future rewards.

[0074] In one embodiment, such as Figure 4 As shown, the training method for the value function parameters includes the following steps S402 to S408.

[0075] S402, set simulator data. The simulator data includes: task location distribution, task generation time distribution, task time window length, number of tasks, timeout service penalty factor, as well as the number of delivery personnel, initial location and maximum delivery time.

[0076] The simulator data can be set based on experience. For example, orders are randomly and evenly generated within a 100*100 unit range, the order generation follows a Poisson process, the task time window length is 90 units of time, the total number of orders generated in one simulation is 100, the number of deliverymen is 3, the maximum delivery time for each deliveryman is 360, and the timeout penalty factor is 5.

[0077] S404, Initialize the simulation environment. When the simulation detects that the target deliveryman has completed the current task, select the next task location for the target deliveryman based on the current state information, the parameters of the function to be trained, and the preset strategy. The preset strategy is used to select the next optional task location with the smallest sum of the first reward and the second reward from the set of next optional task locations as the next optional task location.

[0078] The simulation environment can be initialized at a certain point in time, with all delivery personnel at their initial locations and not yet assigned orders. At this point, a series of orders are generated according to the set simulator data, and these orders are assigned to each delivery person. During each simulation decision-making process, for each next available task location of the target delivery person, the corresponding first and second rewards can be calculated. The specific calculation method can be found in the previous embodiment and will not be repeated here. Preset strategy π(S) k This can be represented as:

[0079] π(S k ) = arg min ak∈A(Sk) (R(S k ,a k )+V(S k ,a k ))

[0080] S406, when the simulation termination condition is not met, return to the step of selecting the next task location for the target delivery person based on the current state information, the value function parameters to be trained, and the preset strategy when the simulation detects that the target delivery person has completed the current task, until the simulation termination condition is met, and update the value function parameters to be trained using the value function sampled value corresponding to this simulation.

[0081] The simulation termination condition can be set according to actual needs. In one embodiment, the simulation termination condition includes: no new tasks are generated in this simulation. This can also be understood as the simulation termination condition being met when the simulator no longer generates new orders. The value function sample value is based on the sum of subsequent first rewards obtained after obtaining the first first reward in this simulation. In one embodiment, the value function sample value v corresponding to this simulation... k The formula for calculating can be as follows:

[0082]

[0083] Where k+1 represents the (k+1)th simulated decision point; K represents the total number of simulated decision points; S k+1 Let a represent the model state corresponding to the (k+1)th simulated decision point. k+1 Indicates that in state S k+1 The next task location is selected. The method of updating the parameters of the value function to be trained using sampled values ​​of the value function can be achieved using recursive least squares.

[0084] S408, if the training termination condition is not met, returns to the initialization simulation environment step until the training termination condition is met and the trained value function parameters are obtained.

[0085] The training termination condition can be set according to actual needs. In one embodiment, the training termination condition can be that the number of times the value function parameters are updated reaches a preset number (e.g., 50,000 times). If the training termination condition is not met, the process returns to steps S404 to S406 until the training termination condition is met, and the trained value function parameters are obtained for use in actual delivery route planning decisions.

[0086] In a real-world environment, each time a decision needs to be made, based on the current state information and the trained value function parameters, an action is output according to a preset strategy. This output is the next task location planned for the target delivery person, thus completing the decision. Therefore, a dynamic scheduling and planning algorithm can automatically allocate orders and plan routes for delivery people without manual scheduling and planning. It can provide better scheduling and planning solutions, thereby saving costs. The offline training followed by online use better ensures the algorithm's effectiveness and the real-time nature of the decisions.

[0087] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0088] In one embodiment, such as Figure 5 As shown, a delivery route planning device 500 is provided, including: an acquisition module 510, a determination module 520, a prediction module 530, and a planning module 540, wherein:

[0089] The acquisition module 510 is used to acquire current status information when it is detected that the target delivery person has completed the task at the current task location. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location.

[0090] The determination module 520 is used to determine the original subsequent delivery routes of all delivery personnel and the set of next optional task locations for the target delivery personnel based on the current subsequent delivery routes and newly generated task locations.

[0091] The prediction module 530 is used to predict the reward corresponding to each next optional task location in the next optional task location set after it is selected as the next task location of the target delivery person, based on the current time, the original subsequent delivery route and the next optional task location set.

[0092] The planning module 540 is used to plan the next task location for the target delivery person based on the reward corresponding to each next optional task location being selected as the next task location for the target delivery person.

[0093] In one embodiment, the prediction module 530 includes a first prediction unit and a second prediction unit; the first prediction unit is used to: predict a first reward corresponding to each next optional task location based on the original subsequent delivery route and the new subsequent delivery routes of all delivery personnel after each next optional task location in the next optional task location set is selected as the next task location of the target delivery person; the second prediction unit is used to: predict a second reward corresponding to each next optional task location based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, and the number of optional task locations, where the number of optional task locations represents the number of next optional task locations of the target delivery person after each next optional task location in the next optional task location set is selected as the next task location of the target delivery person; the reward includes the first reward and the second reward.

[0094] In one embodiment, the first prediction unit is specifically configured to: for any selected next optional task location, obtain a first objective function value based on the travel time between adjacent task locations in the original subsequent delivery path, the arrival time of each task location and its corresponding latest arrival time, wherein the first objective function value is the sum of the total travel time and the total timeout penalty of the original subsequent delivery path; obtain a second objective function value based on the travel time between adjacent task locations in the new subsequent delivery path, the arrival time of each task location and its corresponding latest arrival time, wherein the second objective function value is the sum of the total travel time and the total timeout penalty of the new subsequent delivery path; and obtain a first reward corresponding to the next optional task location by the difference between the second objective function value and the first objective function value.

[0095] In one embodiment, the second prediction unit is specifically used to: predict the second reward corresponding to each next optional task location based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, the number of optional task locations, and the trained value function parameters; the value function parameters include coefficients corresponding to the current time, the remaining delivery time, the number of remaining delivery personnel, and the number of optional task locations, respectively, and the value function is constructed by multiplying the current time, the remaining delivery time, the number of remaining delivery personnel, and the number of optional task locations by their respective coefficients and summing them.

[0096] In one embodiment, the device further includes a training module for training and obtaining value function parameters. Specifically, the training module is used to: set simulator data, including: task location distribution, task generation time distribution, task time window length, number of tasks, timeout service penalty factor, and the number of delivery personnel, initial location, and maximum deliverable time; initialize the simulation environment; when the simulation detects that a target delivery person has completed the current task, select the next task location for the target delivery person based on the current state information, the value function parameters to be trained, and a preset strategy. The preset strategy is used to select the next optional task location with the smallest sum of the first and second rewards from the set of next optional task locations; if the simulation termination condition is not met, return to the step of selecting the next task location for the target delivery person based on the current state information, the value function parameters to be trained, and the preset strategy when the simulation detects that the target delivery person has completed the current task, until the simulation termination condition is met, and update the value function parameters to be trained using the value function sampled value corresponding to this simulation; if the training termination condition is not met, return to the step of initializing the simulation environment, until the training termination condition is met, and obtain the trained value function parameters.

[0097] In one embodiment, the simulation termination condition includes: no new tasks are generated in this simulation; the value function sample value is determined based on the sum of subsequent first rewards obtained after obtaining the first first reward in this simulation.

[0098] In one embodiment, the planning module 540 is specifically used to: plan the next optional task location corresponding to the minimum reward as the next task location for the target delivery person.

[0099] Specific limitations regarding the delivery route planning device can be found in the limitations of the delivery route planning method described above, and will not be repeated here. Each module in the aforementioned delivery route planning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0100] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a delivery route planning method.

[0101] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a delivery route planning method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0102] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the various method embodiments described above.

[0104] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0105] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in the various method embodiments described above.

[0106] It should be understood that the terms "first," "second," etc., used in the above embodiments are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A delivery route planning method, characterized in that, The method includes: When it is detected that the target delivery person has completed the task at the current task location, the current status information is obtained. The current status information includes: the current time, the current subsequent delivery routes of all delivery persons, and the newly generated task location. Based on the current subsequent delivery routes and the newly generated task locations, determine the original subsequent delivery routes for all delivery personnel and the set of next optional task locations for the target delivery personnel; the set of next optional task locations for the target delivery personnel refers to the set consisting of all the next optional task locations that the target delivery personnel can choose. For any of the selected next optional task locations, calculate the difference between the objective function value corresponding to the original subsequent delivery path and the objective function value corresponding to the new subsequent delivery path to obtain the current reward corresponding to the next optional task location being selected as the next task location for the target delivery person; the original subsequent delivery path is the delivery path obtained after adding the newly generated task location to the current subsequent delivery path according to a predetermined rule; the new subsequent delivery path is the delivery path obtained after the next optional task location is selected as the next task location for the target delivery person; the objective function value is the total travel time and timeout of the corresponding delivery path. The sum of total penalties; based on the current state information and the action of selecting the next optional task location as the next task location for the target delivery person, relevant state features are extracted; the state features include the current time, the number of next task locations that the target delivery person can choose after the next optional task location is selected as the next task location, the remaining delivery time for all delivery persons, and the number of remaining delivery persons who can continue delivery; a value function is designed based on the state features; using the value function, the future reward corresponding to the next optional task location being selected as the next task location for the target delivery person is predicted; The next task location for the target delivery person is planned based on the current reward and the future reward corresponding to each of the next optional task locations being selected as the next task location for the target delivery person.

2. The method according to claim 1, characterized in that, The method further includes: Based on the original subsequent delivery route and the new subsequent delivery routes of all delivery personnel after each of the next optional task locations in the set of next optional task locations is selected as the next task location of the target delivery person, the first reward corresponding to each of the next optional task locations is predicted; the first reward corresponding to the next optional task location refers to the current reward corresponding to the next optional task location after it is selected as the next task location of the target delivery person. Based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, and the number of optional task locations, a second reward corresponding to each of the next optional task locations is predicted. The number of optional task locations represents the number of next optional task locations for the target delivery personnel after each of the next optional task locations in the set of next optional task locations is selected as the next task location for the target delivery personnel. The second reward corresponding to the next optional task location refers to the future reward corresponding to the next optional task location after it is selected as the next task location for the target delivery personnel.

3. The method according to claim 2, characterized in that, Based on the original subsequent delivery route and the new subsequent delivery routes of all delivery personnel after each of the next optional task locations in the set of next optional task locations is selected as the next task location for the target delivery person, predict the first reward corresponding to each of the next optional task locations, including: For any of the selected next optional task locations, a first objective function value is obtained based on the travel time between adjacent task locations in the original subsequent delivery route, the arrival time of each task location and its corresponding latest arrival time. The first objective function value is the sum of the total travel time of the original subsequent delivery route and the total timeout penalty. The second objective function value is obtained based on the travel time between adjacent task locations in the new subsequent delivery route, the arrival time of each task location and its corresponding latest arrival time. The second objective function value is the sum of the total travel time and the total timeout penalty of the new subsequent delivery route. The difference between the second objective function value and the first objective function value is used to obtain the first reward corresponding to the next optional task location.

4. The method according to claim 2, characterized in that, Based on the current time, the remaining delivery time for all delivery personnel, the number of remaining delivery personnel who can continue delivery, and the number of optional task locations, predict the second reward corresponding to each of the next optional task locations, including: Based on the current time, the remaining delivery time of all delivery personnel, the number of remaining delivery personnel who can continue delivery, the number of optional task locations, and the trained value function parameters, predict the second reward corresponding to each of the next optional task locations; The value function parameters include coefficients corresponding to the current time, the remaining delivery time, the remaining number of delivery personnel, and the number of optional task locations. The value function is constructed by multiplying the current time, the remaining delivery time, the remaining number of delivery personnel, and the number of optional task locations by their respective coefficients and then summing them.

5. The method according to claim 4, characterized in that, The training methods for the value function parameters include: The simulator data includes: task location distribution, task generation time distribution, task time window length, number of tasks, timeout service penalty factor, and the number of delivery personnel, initial location, and maximum delivery time. The simulation environment is initialized. When the simulation detects that the target deliveryman has completed the current task, the next task location is selected for the target deliveryman based on the current status information, the parameters of the value function to be trained, and the preset strategy. The preset strategy is used to select the next optional task location with the smallest sum of the first reward and the second reward from the set of next optional task locations as the next optional task location. If the simulation termination condition is not met, return to the step of selecting the next task location for the target delivery person based on the current status information, the value function parameters to be trained, and the preset strategy when the simulation detects that the target delivery person has completed the current task, until the simulation termination condition is met, and update the value function parameters to be trained using the value function sampled value corresponding to this simulation. If the training termination condition is not met, return to the initialization simulation environment step until the training termination condition is met and the trained value function parameters are obtained.

6. The method according to claim 5, characterized in that, The simulation termination conditions include: no new tasks will be generated in this simulation; the value function sample value is determined based on the sum of subsequent first rewards obtained after obtaining the first first reward in this simulation.

7. The method according to any one of claims 1 to 6, characterized in that, Based on the current reward and the future reward corresponding to each of the next optional task locations being selected as the next task location for the target delivery person, the next task location for the target delivery person is planned, including: The next optional task location corresponding to the minimum reward will be designated as the next task location for the target delivery person.

8. A delivery route planning device, characterized in that, The device includes: The acquisition module is used to acquire current status information when it is detected that the target delivery person has completed the task at the current task location. The current status information includes: the current time, the original subsequent delivery routes of all delivery persons, and the newly generated task location. The determining module is used to determine the set of next optional task locations for the target delivery person based on the original subsequent delivery route and the newly generated task location; the set of next optional task locations for the target delivery person refers to the set consisting of all the next optional task locations that the target delivery person can choose; The prediction module is used to calculate, for any selected next optional task location, the difference between the objective function value corresponding to the original subsequent delivery path and the objective function value corresponding to the new subsequent delivery path, to obtain the current reward corresponding to the next optional task location being selected as the next task location for the target delivery person; the original subsequent delivery path is the delivery path obtained after adding the newly generated task location to the current subsequent delivery path according to a predetermined rule; the new subsequent delivery path is the delivery path obtained after the next optional task location is selected as the next task location for the target delivery person; the objective function value is the total travel time of the corresponding delivery path. The sum of the total penalties for exceeding the time limit; based on the current state information and the action of selecting the next optional task location as the next task location for the target delivery person, relevant state features are extracted; the state features include the current time, the number of next task locations that the target delivery person can choose after the next optional task location is selected as the next task location for the target delivery person, the remaining delivery time for all delivery persons, and the number of remaining delivery persons who can continue delivery; a value function is designed based on the state features; using the value function, the future reward corresponding to the next optional task location being selected as the next task location for the target delivery person is predicted; The planning module is used to plan the next task location of the target delivery person based on the current reward and the future reward corresponding to each of the next optional task locations being selected as the next task location of the target delivery person.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Distribution route planning method and distribution route planning device

    CN109242214A

  • Control method and device applied to automatic driving, electronic equipment and medium

    CN112256037A