An adaptive logistics scheduling method and system
Through the adaptive logistics scheduling method, using combined feature vectors to divide sliding windows and combined with deep reinforcement learning, the problem of difficulty in improving the accuracy of express scheduling and platform benefits in the existing technology is solved, and more efficient express matching and platform benefits are achieved.
Patent Information
- Application Number
- CN202111227271.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-21
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2041-10-21
AI Technical Summary
The existing urban express scheduling methods are difficult to improve the receipt efficiency and platform benefits at the same time, especially the manual batch size adjustment is not effective in real-time.
Adaptive logistics scheduling method is adopted to adaptively divide the sliding windows according to the combined feature vectors related to express delivery at each time step, and use a deep reinforcement learning network to determine the size of the sliding window, so as to accurately match the courier and the receiving request task, improving the matching quality.
It achieves more accurate matching results, improves the accuracy of express scheduling and receipt efficiency, and improves the overall benefits of the platform.
Smart Images
Figure CN113888012B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an adaptive logistics scheduling method and system. Background Art
[0002] Existing urban express dispatching methods can be divided into two types: one is to allocate packages immediately when they appear; the other is to accumulate a batch of packages through a sliding window and then allocate them in batches. First, the queue processing strategy is easy to implement, but it usually produces poor matching results. Although the latter can improve the quality of the results to a certain extent, manual adjustment of the batch size is not effective for solving real-time situations, and it may be difficult to specify a reasonable batch size. In addition, the optimization of platform revenue is also one of the important considerations for commercial platforms. However, existing technical means mainly focus on improving the efficiency of receiving parcels rather than increasing platform revenue, and cannot consider both the efficiency of receiving parcels and platform revenue at the same time. Summary of the invention
[0003] In view of this, the present invention provides an adaptive logistics scheduling method and system.
[0004] The present invention adopts the following technical solutions:
[0005] The beneficial effects of the present invention are as follows: the sliding window is adaptively divided according to the combined feature vectors related to express delivery at each time step, the size of the sliding window can be accurately obtained, and compared with the method of manually adjusting the batch size, a more accurate matching result can be obtained, the accuracy of subsequent express delivery scheduling is improved, and the efficiency of receiving parcels is improved. Moreover, according to the maximum task number threshold of the receiving request determined and the preset rules, the receiving request tasks in the sliding window are packaged into task groups, and according to the greedy strategy, the corresponding task groups are assigned to each courier, so that high-quality matching can be performed for the couriers and the receiving request tasks in the sliding window, and higher overall benefits can be obtained for the platform. Therefore, the adaptive logistics scheduling method provided by the present invention can not only improve the efficiency of receiving parcels, but also improve the platform benefits. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following briefly introduces the drawings required for use in the embodiment:
[0007] Figure 1 is an overall flow chart of the adaptive logistics scheduling method provided in an embodiment of the present application;
[0008] Figure 2 It is a schematic diagram of the characteristic value of the free capacity of the courier’s backpack;
[0009] Figure 3 It is a schematic diagram of the location distribution eigenvalues;
[0010] Figure 4is a schematic diagram of time eigenvalues;
[0011] Figure 5 It is a flowchart of using deep reinforcement learning to adaptively divide the sliding window;
[0012] Figure 6 It is a schematic diagram of the overall architecture of the adaptive logistics scheduling system provided in the embodiment of the present application. DETAILED DESCRIPTION
[0013] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0014] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0015] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0016] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0017] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0018] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0019] In order to illustrate the technical solution described in this application, a specific implementation method is provided below.
[0020] In this embodiment, the express delivery platform can be modeled as an intelligent agent to intelligently execute the adaptive logistics scheduling method.
[0021] See also Figure 1 , is a flow chart of the adaptive logistics scheduling method provided in an embodiment of the present application. For ease of explanation, only the part related to the embodiment of the present application is shown.
[0022] Step 1: Get the combined feature vector related to express delivery at each time step:
[0023] A time step represents a time period. The length of a time step is set according to actual needs. Accordingly, the number of time steps divided into one day is set according to actual conditions. Each time step corresponds to a combined feature vector, which is a feature vector related to express delivery. The feature sources of the combined feature vector are the courier status information (such as current location, quantity, backpack capacity, and return station time) and the collection request task status information (such as the collection request location, quantity, and expiration time).
[0024] In this embodiment, the combined feature vector includes the courier quantity feature value The number of receiving request tasks Characteristic value of free capacity of courier backpack Location distribution characteristic value and time characteristic values It should be understood that each eigenvalue in the combined eigenvector is the eigenvalue of the corresponding time step. In this embodiment, the specific form of the combined eigenvector is h is the hth time step, and h will be reset to 0 each time a sliding window is divided.
[0025] is the number of all active and valid couriers in time step h. Active and valid couriers can be understood as couriers who are in the normal working process. This eigenvalue can be obtained by statistics. Used to evaluate the number of couriers in the current sliding window.
[0026] is the number of all the receiving request tasks in the time step h. In this embodiment, the receiving request task also specifically refers to the active and valid receiving request task, where the active and valid receiving request task can be understood as the receiving request task that is being carried out normally, and the characteristic value can be obtained by statistics. Used to evaluate the number of receiving request tasks in the current sliding window.
[0027] Characteristic value of free capacity of courier backpack The calculation formula is as follows:
[0028]
[0029] Where C is the set of couriers (i.e., the set of all active and valid couriers in time step h), k c is the total capacity of courier c’s backpack, c∈C, |Γ| c is the number of parcels to be delivered in the backpack of courier c, |Λ| c is the number of parcels received in the backpack of courier C. In this embodiment, the courier backpack is an object used to place express delivery, such as a backpack or a courier vehicle. Used to evaluate the free capacity characteristics of the courier's backpack in the current sliding window.
[0030] Figure 2 is the characteristic value of the free capacity of the courier's backpack In the feature diagram, in S201, it can be seen that each courier’s backpack is sufficient, so they can continue to wait for more collection request tasks, and it is not suitable to divide the sliding window at this time; in S202, the courier’s backpack’s free capacity can accommodate more collection request tasks, and it is more suitable to divide the sliding window at this time, so as to match immediately.
[0031] Location distribution characteristic value The calculation formula is as follows:
[0032]
[0033] Where Λ is the collection of receiving request tasks (i.e., the collection of all active and valid receiving request tasks within time step h), D M (l λ ,D C ) is l λ and D C The Mahalanobis distance, lλ is the location of the receiving request task λ, λ∈Λ, D C is the location distribution of the courier, that is, the location distribution of the courier within time step h. It is used to evaluate the spatial distribution characteristics of couriers and pickup request tasks in the current sliding window.
[0034] Figure 3 is the location distribution characteristic value Feature diagram, in S301, the distribution of couriers and collection request tasks is relatively scattered, and the matching effect may be very poor at this time. In contrast, the distribution of couriers and collection request tasks in S302 is very compact, which is more suitable for dividing the sliding window.
[0035] Time characteristic value The calculation formula is as follows:
[0036]
[0037] Among them, τ is the current time, t c is the deadline for courier c to return to the site, t λ The expiration deadline of the receiving request task λ. The calculation formula of is: c -τ} and {t λ -τ} to find the minimum value x in two sets. For example, there are currently two couriers, t c They are 8, 9, and three request tasks, t λ are 6, 8, and 12 respectively. If the current time τ is 5, then {t c -τ} set is {3, 4}, {t λ -τ} set is {1, 3, 7}, and x is 1 at this time.
[0038] Used to evaluate the time distribution characteristics of couriers and request tasks. The smaller it is, the closer the deadline for the courier or collection request task is, and it is appropriate to divide the sliding window at this time. Figure 4 is the time characteristic value Characteristic diagram, in S401, the deadlines for the courier and the collection request task are still sufficient, and you can continue to wait in order to match a better courier or collection request task. It is not suitable to divide the sliding window at this time; however, in S402, the deadlines for the courier and the collection request task are approaching, and the sliding window should be divided as soon as possible to match in time.
[0039] Step 2: Based on the combined feature vector, adaptively divide the sliding window:
[0040] The sliding window is adaptively divided according to the combined feature vector, and the sliding window is specifically a time window. In this embodiment, a deep reinforcement learning network is used to adaptively divide the sliding window. That is, the sliding window decision process is modeled using a Markov decision process (MDP), and a deep reinforcement learning method is used to adaptively determine the size of the sliding window according to multiple combined feature vectors.
[0041] In a deep reinforcement learning network, the action space A = [h min ,h max ], where h min ,h max ∈N is the minimum and maximum time step for partitioning the sliding window. A sliding window is composed of a group of consecutive time steps, which is often called a batch, which includes new collection request tasks that appear in this sliding window (that is, during this period of time) and collection request tasks left over from the previous sliding window, as well as couriers who are valid in this sliding window (for example, they have sufficient backpack capacity and sufficient time).
[0042] The reward function of the deep reinforcement learning network is as follows: Reward is the key to motivating learning in the right direction. This embodiment uses E(M) to represent the sum of the rewards generated by all couriers in a matching process, that is, E(M) = ∑ c∈C,λ∈Λ R(c,λ). R(c,λ) is the reward a single courier receives for completing a single delivery request.
[0043] It should be understood that a deep reinforcement learning network can be trained by combining feature vectors corresponding to multiple time steps.
[0044] Input the combined feature vector of the current time step into the deep reinforcement learning network and output the corresponding action value a h and the reward value r h , in order to perform sliding window division.
[0045] Action value a h h min and h max A discrete value between (i.e., the number of time steps), indicating that the division is most appropriate at this time step. h = h, then the time window is divided at time step h, and the reward value is r h is the total revenue generated by matching in this sliding window. Otherwise, no division is performed at time step h, r h =0.
[0046] like Figure 5As shown in the figure, the DRL-based adaptive sliding window partitioning scheme consists of two modules: learning and planning. For planning, in time step h, the state feature vector Encode the current environment. Then, h Enter the target network Make a decision, that is, in [h min ,h max ] Select an action a h . Note that if with action value a h If the corresponding time step is consistent with the current time step, the sliding window is divided to execute the subsequent TBM matching algorithm; otherwise, it will continue to wait for the next decision. Stored in ReplayMemory (experience pool). For learning, first randomly extract a batch of experience from Replay Memory to update Q-network Q, then Synchronize with the parameters of Q-network Q.
[0047] It should be understood that the number of sliding windows obtained by division is determined by actual conditions.
[0048] In this embodiment, steps 1 and 2 correspond to Figure 6 S101 in, steps 3-5 correspond to Figure 6 In S102, steps 3-5 constitute a time-aware batch matching (TBM) algorithm, which performs matching according to the sliding window obtained by the division. Specifically, the input of the TBM algorithm is all the collection request tasks and couriers in a sliding window, and the output is a feasible allocation plan for the current sliding window.
[0049] Step 3: Based on the divided sliding window, determine the maximum task quantity threshold of the receiving request corresponding to the sliding window:
[0050] Since the data processing process corresponding to each sliding window is the same, any sliding window is used as an example for explanation below. Based on the sliding window obtained by division, the maximum task number threshold m of the receiving request corresponding to the sliding window is determined. The maximum task number threshold m of the receiving request represents the maximum number of receiving request tasks in a task group. The maximum task number threshold m of the receiving request is set according to actual needs. In this embodiment, the maximum task number threshold m of the receiving request can be understood as the maximum size of a task package in the packing operation. This threshold can be determined according to the experience of experienced couriers. This threshold generally does not exceed the maximum capacity of a courier's backpack, because in real scenarios, the number of tasks that can form a task group is generally small, and limiting the maximum size of the task group can greatly reduce the time complexity of the algorithm, and it may even be very small, because the temporal and spatial distribution of more receiving request tasks is too complex, and it is generally difficult to pack them into a task package.
[0051] Step 4: Based on the maximum task quantity threshold of the receiving request, the receiving request tasks in the sliding window are packaged into task groups according to preset rules to obtain at least two task groups:
[0052] The task packaging rules are set according to actual needs. In this embodiment, the preset rules include time constraints and distance constraints. Accordingly, the receiving request tasks that meet the time constraints and distance constraints in the sliding window are packaged into a task group. Among them, the time constraint condition is that the receiving request tasks in the same task group will not expire if they are completed in sequence. The specific judgment conditions for expiration are set according to actual needs. For example, starting from the time when the receiving request task is generated, it is judged to be expired if it exceeds 2 hours. The distance constraint condition is that the distance between each receiving request task in the same task group does not exceed the preset distance threshold. The distance threshold is set according to actual needs. Moreover, the number of receiving request tasks in each task group is less than the maximum number of receiving request tasks threshold m.
[0053] The above-mentioned task packaging operation means combining a group of collection request tasks with similar appearance time and distance into a task group, which will form a feasible collection path inside to ensure that all tasks can be collected by the courier before expiration. For example, the collection request tasks that appear between 9:00 and 9:30 in a community can be packaged into a task group, that is, regarded as one task for the subsequent allocation operation, which can greatly save the allocation time.
[0054] It should be understood that at least two task groups can be generated after task packaging, and the specific number of task groups is determined by the actual grouping situation.
[0055] Step 5: According to the greedy strategy, assign corresponding task groups to each courier:
[0056] In this embodiment, the income obtained by each task group is calculated and processed for each courier in turn, and the task group with the largest income is assigned to the courier. Specifically: a sequence can be set for all couriers, and the allocation operation is performed on each courier in sequence according to the sequence. The allocation operation is specifically: for a certain courier, the income obtained by each task group is calculated and processed for the courier, the largest income is found, and the task group with the largest income is assigned to the courier, and then the allocation operation is performed on the next courier. It should be understood that the task group assigned this time will no longer participate in the subsequent calculations, and the income obtained by the remaining task groups is calculated and processed for the courier, and the largest income is found, and the task group with the largest income is assigned to the courier, and then the allocation operation is performed on the next courier. It should be understood that the task group assigned this time will no longer participate in the subsequent calculations, and the income obtained by the remaining task groups is calculated and processed for the courier, and the largest income is found, and the task group with the largest income is assigned to the courier, and so on, until all task groups are assigned or the courier cannot accept the task group, and the matching between the task group and the courier is completed. Finally, the matching results are output for the express delivery platform to dispatch couriers.
[0057] It should be understood that the task group that is not assigned in this sliding window can be decomposed into a single receiving request task, and enter the next sliding window, and continue to perform the operations of the above steps 3-5 for assignment.
[0058] In this embodiment, after step 5, the adaptive logistics scheduling method further includes the following steps:
[0059] Step 6: For the successfully matched couriers and task groups, notify the couriers to go to the requested location to perform the pickup operation, and notify the dispatched courier of the customer who sent the pickup request task:
[0060] In this embodiment, step 6 corresponds to Figure 6 S103 in the process. For the couriers and task groups that have been successfully matched, the courier platform can notify the corresponding courier to go to the requested location to perform the pickup operation, and at the same time notify the customer who issued the pickup request task of the relevant information of the dispatched courier, such as the courier's name and phone number.
[0061] In this embodiment, after step 5, the adaptive logistics scheduling method further includes the following steps:
[0062] Step 7: For the couriers and task groups that have not been successfully matched, return the couriers and task groups that have not been successfully matched to the courier flow and task flow to wait for the next matching.
[0063] In this embodiment, step 7 corresponds to Figure 6S104 in. For the couriers and task groups that have not been successfully matched, the couriers and task groups that have not been successfully matched are returned to the courier flow and task flow to wait for the next matching.
[0064] It should be understood that there is no strict order between step 6 and step 7.
[0065] This embodiment also provides an adaptive logistics scheduling system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the adaptive logistics scheduling method described above are implemented.
[0066] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. An adaptive logistics scheduling method, characterized in that: The steps include: Get the combined feature vector related to express delivery at each time step; Based on the combined feature vector, adaptively divide the sliding window; Based on the sliding window obtained by the division, determining a maximum task quantity threshold of the receiving request corresponding to the sliding window; Based on the maximum task quantity threshold of the receiving request, the receiving request tasks in the sliding window are packaged into task groups according to a preset rule to obtain at least two task groups; According to the greedy strategy, each courier is assigned a corresponding task group; The combined feature vector includes a courier quantity feature value, a collection request task quantity feature value, a courier backpack spare capacity feature value, a location distribution feature value, and a time feature value; The calculation formula of the courier backpack free capacity characteristic value is as follows: Among them, C is the set of couriers, k c is the total capacity of courier c’s backpack, c∈C, |Γ| c is the number of parcels to be delivered in the backpack of courier c, |Λ| c is the number of parcels received in the backpack of courier c; The calculation formula of the position distribution characteristic value is as follows: Among them, Λ is the collection of receiving request tasks, D M (l λ ,D C ) is the Mahalanobis distance, l λ is the location of the receiving request task λ, λ∈Λ, D C The location distribution of couriers; The calculation formula of the time characteristic value is as follows: Among them, τ is the current time, t c is the deadline for courier c to return to the site, t λ The expiration deadline of the receiving request task λ; The calculation formula is as follows: c -τ} and {t λ -τ} to find the minimum value x in two sets.
2. The adaptive logistics scheduling method according to claim 1, characterized in that: The adaptively dividing the sliding window based on the combined feature vector comprises: The combined feature vector is input into a preset deep reinforcement learning network, and the corresponding action value and reward value are output to perform sliding window division.
3. The adaptive logistics scheduling method according to claim 1, characterized in that: Based on the maximum task quantity threshold of the receiving request, the receiving request tasks in the sliding window are packaged into task groups according to a preset rule to obtain at least two task groups, including: Packing the receiving request tasks that meet the time constraint and the distance constraint in the sliding window into a task group; The time constraint condition is that all the receiving request tasks in the same task group must be completed in sequence and will not expire; the distance constraint condition is that the distance between each receiving request task in the same task group does not exceed the preset distance threshold; the number of receiving request tasks in each task group is less than the maximum number of receiving request tasks threshold.
4. The adaptive logistics scheduling method according to claim 1, characterized in that: According to the greedy strategy, the corresponding task group is assigned to each courier, including: The profit obtained by processing each task group is calculated for each courier in turn, and the task group with the largest profit is assigned to the courier.
5. The adaptive logistics scheduling method according to claim 1, characterized in that: After assigning a corresponding task group to each courier according to the greedy strategy, the adaptive logistics scheduling method further includes the following steps: For the couriers and task groups that have been successfully matched, the couriers are notified to go to the requested location to perform the pickup operation, and the customer who issued the pickup request task is notified of the dispatched courier.
6. The adaptive logistics scheduling method according to claim 1, characterized in that: After assigning a corresponding task group to each courier according to the greedy strategy, the adaptive logistics scheduling method further includes the following steps: For couriers and task groups that are not successfully matched, the couriers and task groups that are not successfully matched are returned to the courier flow and task flow to wait for the next matching.
7. An adaptive logistics scheduling system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the adaptive logistics scheduling method according to any one of claims 1 to 6 are implemented.