Dynamic order delivery method and system based on space crowdsourcing

Abstract

The invention discloses a dynamic order delivery method and system based on spatial crowdsourcing. The method comprises the following steps: updating order information successfully distributed before a current decision point and position information of a rider in a delivery state; reading newly acquired order information and rider information; allocating orders and planning a rider delivery route, specifically, allocating all unallocated orders through a heuristic greedy method according to order information and rider information, and inserting starting points and ending points of the orders into the rider delivery route by using a minimum distance insertion method to obtain an initial solution # imgabs0 # of order allocation; adjusting the initial solution by using a hyper-heuristic framework, specifically, calculating the state of the initial solution as the input of the DQN network, and adjusting the solution according to an output operator to obtain a new solution; re-planning a delivery route for the rider adjusted for the order, and starting delivery by the rider; and calculating the meeting state of the rider and adjusting the order, and optimizing the distribution and path planning of the rider, thereby improving the total profit of the order distribution system.

Description

Technical Field

[0001] The present invention belongs to the technical field of task allocation, and in particular relates to a dynamic order distribution method and system based on spatial crowdsourcing. Background Art

[0002] With the popularization of the Internet and smart devices, many people prefer to buy various goods or services online for convenience. In particular, online delivery services (OD) have become increasingly popular in recent years. This service first requires people (i.e. customers) to place an order, then the supplier prepares it, and finally the rider completes the delivery.

[0003] At present, a large number of crowdsourcing online delivery systems have emerged, providing various services. A typical crowdsourcing online delivery (COD) system has four stakeholders, namely the platform, crowdsourcing riders, suppliers (such as food and medicine suppliers), and customers.

[0004] The framework of the COD system is described as follows: Due to the dynamic nature of orders and riders, the platform requires an instant allocation mechanism after the order arrives. As the core of the COD system, the platform faces two major challenges in the order allocation process. First, the platform should encourage more crowdsourced riders to participate in delivery, because many crowdsourced riders have no additional income besides participating in delivery. Crowdsourced riders are willing to deliver more orders at a time to get more profits. To solve this problem, the platform usually assigns one rider to be responsible for multiple orders with similar destinations. Another challenge is the huge dynamics and uncertainty in the system environment. In the COD system, customers' online orders arrive at the platform dynamically, and crowdsourced riders are randomly distributed around suppliers. The dynamics and uncertainty bring great challenges to order allocation, especially during busy hours (such as commuting time). Finding the global optimal solution for large-scale instances is NP-hard and usually requires exponential resources. To address these challenges, heuristic methods are used to find high-quality solutions in a reasonable time. In addition to heuristic methods, various strategies are also adopted to optimize the allocation of orders. For example, some studies have adopted a delay strategy to assign orders that are not yet ready to riders, allowing riders to wait for the order to be completed on the way to the order destination. In addition, in order to cope with the dynamics and uncertainty of the future environment, a variety of prediction-based methods have been proposed. These prediction-based strategies use a large amount of historical data to train models, predict future features, and apply these predictions to order allocation methods. Although experimental results show the effectiveness of these strategies, they may not always be successful due to the lack of historical data and the limitation of computing resources.

[0005] Overall, the design and optimization of the COD system is a complex issue that requires comprehensive consideration of multiple aspects such as order allocation, rider incentives, dynamic environment adaptability, and prediction methods. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a dynamic order delivery method based on spatial crowdsourcing, uses a heuristic method to realize the instant allocation mechanism of crowdsourcing online delivery services, proposes a two-stage order adjustment strategy, and optimizes rider allocation and path planning, thereby improving the total profit of the order delivery system and building an efficient order delivery service.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: a dynamic order delivery method based on spatial crowdsourcing, comprising the following steps: Update the order information that was successfully assigned before the current decision point and the rider information that is in the delivery status; read the order information that is newly entered into the system and rider information ; Assign orders and plan delivery routes for riders, specifically, based on order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; Use the hyper-heuristic framework to adjust the initial solution. Specifically, calculate the initial solution Status , as the input of the DQN network, the solution is adjusted according to the output operator to obtain a new solution ; The delivery routes are re-planned for the riders whose orders have been adjusted, and the riders start delivering; Calculate the rider encounter status and adjust the order, including: based on the rider's encounter radius ,calculate The encounter status of all riders in the system at all times, randomly selects the rider who meets, and tries to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed, and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver; Repeat the above steps.

[0008] Furthermore, update the order information that was successfully assigned before the current decision point and the location information of the riders in the delivery state; read the newly acquired order information : Order information and rider information definition, orders are submitted by customers through the online platform, and an order is represented by its ID , Start time ,expiration date , Start position , Target location and reward budget Composition, expressed as ; The rider receives the order from the platform, picks up the goods from the supplier, and delivers the items to the customer. The rider is identified by his / her ID , Current Location , total capacity , Current remaining capacity , assign orders Composition, expressed as ; Update the order information previously assigned by the system and update the status of unassigned orders: check whether the order has timed out, if so, delete it from the system, otherwise reallocate it at the next decision point; update the orders in the delivery state, check whether the order has been picked up / delivered, and update it to the corresponding status; Read the order information obtained between the current decision point and the previous decision point And the order information of the failed allocation at the previous decision point , together constitute the order list to be allocated at this decision point ; Get the online crowdsourced rider list , as an available rider for order assignment.

[0009] Furthermore, when the deep reinforcement learning hyper-heuristic framework is constructed and trained: a deep neural network is constructed , used to learn the mapping of order and rider features to order adjustment strategies from the dataset, evaluate the environment state to select actions; use the experience replay mechanism and strategies, respectively used to store and reuse historical interaction experience and build the target network ,After the construction is completed, the order training set and the rider training set are used for training.

[0010] Furthermore, in the deep reinforcement learning hyper-heuristic framework, the agent considers additional information in the search process, such as rider information, order information, and system information as the state of the current environment. , as follows: ① Rider status information Rider status information including the rider's total remaining capacity , Number of active riders , the weighted sum of the two is used as the rider's status information. The specific calculation formula is as follows:

[0011] ②Order status information Order status information includes the number of orders allocated in the system at the current moment 、Number of completed orders , the calculation formula is as follows:

[0012] ③System status information The system status information is the profit difference between the current solution and the previous solution, expressed as:

[0013] in, are all weight parameters, and the final environment state is expressed as: .

[0014] Furthermore, the construction and training of the deep reinforcement learning hyper-heuristic framework specifically includes the following steps: S1, build training network , defines the input of the network , the dimension is state The dimension is , and the DQN network is constructed using a fully connected layer and a ReLU activation function to learn the mapping relationship between states and actions. The output vector represents the estimated value of performing the corresponding action in the current state, and the dimension is the number of optional actions; S2, initialize the training network , target network And the experience replay pool R, randomly initialize the training network Parameters , copy the parameters of the trained network Initialize the target network , initialize the experience replay pool R, set the experience replay pool capacity, and store the experience tuples of the interaction between the agent and the environment ; S3, execute S4-S9 for a set number of times, and use the order training set and the rider training set for training; S4, get the initial state of the environment , according to the training network The output is used Strategy selection action ; S5, execute action , get reward , the environment state becomes ; S6, the experience tuple Put it into the experience replay pool R; S7: If the amount of data in the experience replay pool R is sufficient, randomly sample N data from it. ; For each sampled data, use the target network to calculate the target value ,in is the discount factor, ranging from [0,1]; S8, minimize the target loss , to update the training network The parameters, that is ; S9, regularly update the parameters of the target network ,Right now .

[0015] Furthermore, the heuristic greedy method dynamically allocates the arriving delivery orders including: S11, get the order list that needs to be allocated and a list of available riders ; S12, determine the delivery rider for the order, traverse the order list, and greedily find the rider with the largest profit increment from the list of available riders and return it. S13, planning the delivery route for the rider. If the allocation is successful, the delivery route for the rider is planned and the rider starts to deliver the order; otherwise, the order is delayed to the next decision point for allocation.

[0016] Furthermore, when calculating the rider encounter status and adjusting the order, based on the collaboration between crowdsourcing riders, the following are specifically included: Calculate the rider encounter status and set the rider encounter radius , calculate all the riders in the system The encounter status at the moment (i.e. the current moment) is obtained to obtain the list of encountered riders; From two encounter riders Randomly select an order from the list of delivery orders ; Use order exchange and order transfer strategies to adjust riders and Delivery orders and corresponding delivery routes; implement Order exchange attempts: from the rider and List of orders being shipped Randomly select orders If in exchange order and If the time and capacity constraints are still met and the profit increases after the exchange, the order exchange operation is performed and the new delivery routes are adjusted for the two riders. and ;After the operation is completed, the rider starts delivering; implement Order transfer attempts: Each time an order is randomly selected from the order list carried by the two riders If the order If the constraints are still met after the order is transferred to another rider and the profit increases after the transfer, the order transfer operation is performed, and then the delivery routes of the two riders are adjusted. and , the riders deliver according to the new route.

[0017] The present invention can also provide a dynamic order distribution system based on spatial crowdsourcing, including an information cycle update module, an initial solution acquisition module, an allocation module, and an adjustment exchange module; The information cycle update module includes an information acquisition unit and a cycle unit. The information acquisition unit is used to repeatedly update the order information successfully allocated before the current decision point and the location information of the riders in the delivery state; read the newly acquired order information and rider information ; The initial solution acquisition module is used to allocate orders and plan the delivery routes of riders. Specifically, according to the order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; The allocation module uses a hyper-heuristic framework to adjust the initial solution. Specifically, it calculates the initial solution Status , as the input of the DQN network, according to the network output use Strategy selection operator, adjust the solution to get a new solution ; The delivery routes are re-planned for the riders whose orders have been adjusted, and the riders start delivering; The adjustment exchange module is used to calculate the rider encounter status and adjust the order, including: according to the rider's encounter radius ,calculate The encounter status of all riders in the system at all times, randomly selects the rider who meets, and tries to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed, and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver; The loop unit is used to execute the above process cyclically.

[0018] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the dynamic order delivery method based on spatial crowdsourcing described in the present invention.

[0019] At the same time, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the dynamic order delivery method based on spatial crowdsourcing described in the present invention can be implemented.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects: This paper proposes a two-stage order adjustment strategy to optimize rider allocation and path planning, thereby improving the total profit of the order delivery system. In the first stage, deep reinforcement learning is used as a hyper-heuristic framework for adaptive large neighborhood search to adjust the initial solution generated by the heuristic method; in the second stage, two order adjustment strategies, order transfer and order exchange, are used to optimize rider allocation and path planning to generate the final delivery plan; and extensive experiments are conducted using synthetic and real-world datasets. The results show that the proposed strategies and methods are effective compared with the baseline. For the problem of crowdsourced online order delivery, this paper uses a heuristic method to implement an instant allocation mechanism for crowdsourced online delivery services and build an efficient order delivery service system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of an embodiment of the present invention.

[0022] Figure 2 Using a flowchart for the hyperheuristic framework. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the following embodiments.

[0024] Example 1 exist Figure 1 In the present embodiment, the dynamic order delivery method based on spatial crowdsourcing includes the following steps: S1, build and train a deep reinforcement learning hyper-heuristic framework; including: building a deep neural network , used to learn the mapping from order and rider features to order adjustment strategies from the dataset, thereby evaluating the environment state to select appropriate actions; using the experience replay mechanism and Strategies are used to store and reuse historical interaction experience, improve sample efficiency and learning stability, and balance exploration and utilization. Finally, in order to stabilize the training network learning process, it is also necessary to build a target network ,After the construction is completed, the order training set and the rider training set are used for training.

[0025] The environment state contains additional information during the search process, such as rider information, order information, and system information, which is calculated as the state of the current environment. , leading to better decision making. Figure 2 , the status information considered is as follows: (1) Rider status information Rider status information including the rider's total remaining capacity , Number of active riders , the weighted sum of the two is used as the rider's status information. The specific calculation formula is as follows:

[0026] (2) Order status information Order status information includes the number of orders allocated in the system at the current moment 、Number of completed orders , the calculation formula is as follows:

[0027] (3) System status information The system status information is the profit difference between the current solution and the previous solution, expressed as:

[0028] in are all weight parameters, and the final environment state is expressed as:

[0029] The construction and training of the deep neural network to select the order adjustment operator specifically includes the following steps: (1) Building a training network . Define the input of the network , the dimension is the state The DQN network is constructed using a fully connected layer and ReLU activation function to learn the mapping between states and actions. The output is a vector with a length equal to the total number of actions, representing the estimated value of executing the corresponding action in the current state.

[0030] (2) Initialize the training network, target network, and experience replay pool. Randomly initialize the training network Parameters , copy the parameters of the trained network Initialize the target network Initialize the experience replay pool R with a capacity of 1000 to store the experience tuples of the agent’s interaction with the environment ; (3) Perform steps (4) to (9) E times, where E is the number of model training times. , take an integer, and use the order training set and the rider training set for training; (4) Obtaining the initial state of the environment , according to the training network The output is used Strategy selection action ; (5) Execute action , get reward , the environment state becomes ; (6) The experience tuple Put it into the experience replay pool R; (7) If the amount of data in the experience replay pool R is sufficient, randomly sample N data from it. ; For each sampled data, use the target network to calculate the target value ,in is the discount factor, ranging from [0,1]; (8) Minimize target loss , to update the training network The parameters, that is ; (9) Regularly update the parameters of the target network ,Right now .

[0031] S2, update order information and rider information . Including: updating the order information successfully assigned before the current decision point and the location information of the riders in the delivery state; reading the order information and rider information newly entered into the system; S3, assign orders and plan delivery routes for riders. Including: Based on the order information in the system and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; S4, using the hyper-heuristic framework to adjust the initial solution. This includes: first calculating the current solution Status , as the input of the DQN network, according to the network output use Strategy selection operator, adjust the solution to get a new solution ; Then the delivery route is re-planned for the rider whose order has been adjusted, and the rider starts delivering; S5: Calculate the rider encounter status and adjust the order, including: based on the rider encounter radius ,calculate The encounter status of all riders in the system at all times is collected, and the encountered rider is randomly selected to try to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver and the system continues to receive new orders.

[0032] Further, the updating of order information and rider information in step S2 includes: (1) Definition of order information and rider information: An order is submitted by a customer through an online platform. An order is identified by its ID. , Start time ,expiration date , Start position , Target location and reward budget Composition, expressed as ; The rider receives the order from the platform, picks up the goods from the supplier, and delivers the items to the customer. The rider is identified by his / her ID , Current Location , total capacity , Current remaining capacity , assign orders Composition, expressed as .

[0033] (2) Update the order information previously assigned by the system. First, update the status of unassigned orders: check whether the order has timed out. If so, delete it from the system. Otherwise, reallocate it at the next decision point. Next, update the orders in the delivery state. Check whether the order has been picked up / delivered and update it to the corresponding state. (3) Read the order information that enters the system between the current decision point and the previous decision point , Order information for which the previous decision point allocation failed , together constitute the order list to be allocated at this decision point ; Read the list of online crowdsourcing riders in the system , as an available rider for order assignment.

[0034] The dynamic order delivery method based on spatial crowdsourcing described in step S3, i.e., the heuristic greedy method, includes the following steps: (1) Get the list of orders that need to be allocated and a list of available riders .

[0035] (2) Determine the delivery rider for the order, traverse the order list to find a suitable rider, and greedily find the rider with the largest profit increment from the list of available riders and return it.

[0036] (3) Plan the rider’s delivery route. If the order is successfully returned to the rider, the rider’s delivery route is planned and the rider starts delivering the order. Otherwise, the order is delayed until the next decision point for allocation.

[0037] The collaboration among crowdsourcing riders in the dynamic order delivery method based on spatial crowdsourcing described in step S4 specifically includes the following steps: (1) Calculate the rider encounter status. Set the rider encounter radius , calculate all the riders in the system The encounter status at the moment, and get the list of encountered riders.

[0038] (2) Determine the adjusted order list. Randomly select one order from the delivery order list .

[0039] (3) Adjust orders and use order exchange and order transfer strategies to adjust riders and Delivery orders and corresponding delivery routes.

[0040] 1) Execution Order exchange attempts: from the rider and List of orders being shipped Randomly select orders If in exchange order and If the time and capacity constraints are still met and the profit increases after the exchange, the order exchange operation is performed and the new delivery routes are adjusted for the two riders. and ;After the operation is completed, the rider starts delivering.

[0041] 2) Execution Order transfer attempts: Each time an order is randomly selected from the order lists of the two riders If the order If the constraints are still met after the order is transferred to another rider and the profit increases after the transfer, the order transfer operation is performed, and then the delivery routes of the two riders are adjusted. and , the riders deliver according to the new route.

[0042] Example 2, in order to verify the effectiveness of the method of the present invention, the method of Example 1 was compared with FR, BR, FA, and BA. Among them, FR assigns the order to the first available rider who meets the constraints; BR assigns the order to the rider who makes the most profit among the available riders. These two methods do not adjust the order allocation results. Both FA and BA belong to adaptive large neighborhood search methods, that is, after the initial solution is generated, the destruction operator and the repair operator are used to adjust the initial solution. The difference between FA and BA is that FR and BR are used to initially allocate orders, respectively. The destruction operator and repair operator used in the solution adjustment process are introduced as follows: (1) Destruction operator 1) Single route removal: randomly select a rider and delete the new orders assigned to the rider in this allocation; 2) Worst removal: remove the two orders with the smallest profit.

[0043] (2) Repair Operator 1) Sequence insertion: directly insert the starting point and end point of the order to be inserted into the end of the current rider's path; 2) Optimal insertion: insert the starting point and end point of the order into the current rider's delivery path to minimize the distance increase.

[0044] In the FA and BA method flow, a roulette wheel is first used to select the destruction operator and the repair operator. After the destruction and repair operations are completed, the weights of the operators are updated according to the scores of the new solutions until the iteration is completed.

[0045] The experimental results of each method are analyzed based on five indicators: total profit, total delivery distance, average waiting time for orders, number of overdue orders, and number of benefited riders.

[0046] (1) Total profit, which is the sum of the profits of all riders. When the order data is the same, the greater the total profit, the better the efficiency of the method; (2) Total delivery distance: the total distance traveled by all riders to complete deliveries. The total distance is the main indicator that determines the delivery cost. The smaller the total distance, the greater the profit that can be obtained. (3) The average waiting time for orders, that is, the average time it takes for all orders to be delivered from the time the rider starts delivering them to the time they are successfully delivered, which affects user satisfaction; (4) The number of benefited riders refers to the number of riders involved in order distribution, reflecting the efficiency of order delivery.

[0047] Compare the experimental results, refer to Table 1 and Table 2, where the bold ones are the optimal results under the indicator. Table 1 compares the experimental results of different indicators of Example 1 and comparative methods FR, BR, FA, and BA on 6 synthetic data set instances (i.e., S100, S200, S300, S400, S500, and S600). S100-S600 represents the number of orders. It can be seen from the table that with the increase in the number of orders, the various indicators of all methods are improved, but the effect of the present invention is the best. Table 2 compares the experimental results of different indicators of Example 1 and comparative methods FR, BR, FA, and BA on 4 real data set instances (i.e., R100, R200, R300, and R400). It can be seen that the experimental results on the real instance are similar to those on the synthetic instance: the present invention performs best in terms of total profit, total distance, average waiting time, and number of canceled orders.

[0048] Table 1

[0049] Table 2

[0050] In summary, the present invention provides a dynamic order delivery method based on spatial crowdsourcing, which includes the following steps: updating the order information successfully allocated before the current decision point and the location information of the riders in the delivery state; reading the newly acquired order information; and rider information ; Assign orders and plan delivery routes for riders, specifically, based on order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; Use the hyper-heuristic framework to adjust the initial solution. Specifically, calculate the initial solution Status , as the input of the DQN network, according to the network output use Strategy selection operator, adjust the solution to get a new solution ; Re-plan the delivery route for the rider whose order is adjusted, and the rider starts delivering; Calculate the rider's encounter status and adjust the order, including: according to the rider's encounter radius ,calculate The encounter status of all riders in the system at the moment, randomly select the encounter rider, try to adjust the order, if the profit increases after adjustment, execute the order adjustment operation, and re-plan the delivery route for the rider; otherwise, do not execute the order adjustment operation; after completing the above operations, the rider continues to deliver; repeat the above steps. Using the heuristic method, an efficient order delivery service system is constructed, and a two-stage order adjustment strategy is proposed to optimize the rider allocation and path planning, thereby improving the total profit of the order delivery system, including: first, modeling the crowdsourcing online order delivery problem, using the heuristic method to realize the instant allocation mechanism of the crowdsourcing online delivery service; in the first stage, using deep reinforcement learning as a hyper-heuristic framework for adaptive large neighborhood search, adjust the initial solution generated by the heuristic method; in the second stage, using the collaboration between riders, that is, the two order adjustment strategies of order transfer and order exchange optimize the rider allocation and path planning, and generate the final delivery plan.

[0051] Embodiment 2, based on the concept of the method described in the present application, also provides a dynamic order distribution system based on spatial crowdsourcing, including an information cycle update module, an initial solution acquisition module, an allocation module and an adjustment exchange module; The information cycle update module includes an information acquisition unit and a cycle unit. The information acquisition unit is used to repeatedly update the order information successfully allocated before the current decision point and the location information of the riders in the delivery state; read the newly acquired order information and rider information ; The initial solution acquisition module is used to allocate orders and plan the delivery routes of riders. Specifically, according to the order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; The allocation module uses a hyper-heuristic framework to adjust the initial solution. Specifically, it calculates the initial solution Status , as the input of the DQN network, according to the network output use Strategy selection operator, adjust the solution to get a new solution ; The delivery routes are re-planned for the riders whose orders have been adjusted, and the riders start delivering; The adjustment exchange module is used to calculate the rider encounter status and adjust the order, including: according to the rider's encounter radius ,calculate The encounter status of all riders in the system at all times, randomly selects the rider who meets, and tries to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed, and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver; The loop unit is used to execute the above process cyclically.

[0052] On the other hand, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the dynamic order delivery method based on spatial crowdsourcing described in the present invention can be implemented.

[0053] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the dynamic order delivery method based on spatial crowdsourcing described in the present invention.

[0054] The computer device may be a laptop computer, a desktop computer or a workstation.

[0055] The processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).

[0056] The memory of the present invention may be an internal storage unit of a laptop computer, a desktop computer or a workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0057] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drive (SSD) or optical disk, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).

[0058] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A dynamic order delivery method based on spatial crowdsourcing, characterized in that: The following steps are involved: Update the order information that was successfully assigned before the current decision point and the rider information that is in the delivery status; read the order information that is newly entered into the system and rider information ; Assign orders and plan delivery routes for riders, specifically, based on order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; Use the hyper-heuristic framework to adjust the initial solution. Specifically, calculate the initial solution Status , as the input of the DQN network, the solution is adjusted according to the output operator to obtain a new solution ; The delivery routes are re-planned for the riders whose orders have been adjusted, and the riders start delivering; Calculate the rider encounter status and adjust the order, including: based on the rider's encounter radius ,calculate The encounter status of all riders in the system at all times, randomly selects the rider who meets, and tries to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed, and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver; Repeat the above steps.

2. The dynamic order distribution method based on spatial crowdsourcing according to claim 1 is characterized in that: Update the order information that was successfully assigned before the current decision point and the rider information that is in the delivery status; read the order information that is newly entered into the system and rider information include: Order information and rider information definition, orders are submitted by customers through the online platform, and an order is represented by its ID , Start time ,expiration date , Start position , Target location and reward budget Composition, expressed as ; The rider receives the order from the platform, picks up the goods from the supplier, and delivers the items to the customer. The rider is identified by his / her ID , Current Location , total capacity , Current remaining capacity , assigned orders Composition, expressed as ; Update the order information previously assigned by the system and update the status of unassigned orders: check whether the order has timed out, if so, delete it from the system, otherwise reallocate it at the next decision point; update the orders in the delivery state, check whether the order has been picked up / delivered, and update it to the corresponding status; Read the order information that entered the system between the current decision point and the previous decision point And the order information of the failed allocation at the previous decision point , together constitute the order list to be allocated at this decision point ; Get the online crowdsourcing rider list , as an available rider for order assignment.

3. The dynamic order distribution method based on spatial crowdsourcing according to claim 1 is characterized in that: The deep reinforcement learning hyper-heuristic framework is constructed and trained: Building a deep neural network , used to learn the mapping of order and rider features to order adjustment strategies from the dataset, evaluate the environment state to select actions; use the experience replay mechanism and Strategies, used to store and reuse historical interaction experience, and to balance exploration and exploitation; building a target network ,After the construction is completed, the order training set and the rider training set are used for training.

4. The dynamic order distribution method based on spatial crowdsourcing according to claim 3 is characterized in that: In the deep reinforcement learning hyper-heuristic framework, the agent considers additional information in the search process, such as rider information, order information, and system information as the state of the current environment , as follows: ① Rider status information Rider status information including the rider's total remaining capacity , Number of active riders , the weighted sum of the two is used as the rider's status information. The specific calculation formula is as follows: ②Order status information Order status information includes the number of orders allocated in the system at the current moment 、Number of completed orders , the calculation formula is as follows: ③System status information The system status information is the profit difference between the current solution and the previous solution, expressed as: in, are all weight parameters with a value range of [0,1]. The final environment state is expressed as: 。 5. The dynamic order distribution method based on spatial crowdsourcing according to claim 3 is characterized in that: The construction and training of the deep reinforcement learning hyper-heuristic framework specifically includes the following steps: S1, build training network , defines the input of the network , the dimension is state The dimension is , and the DQN network is constructed using a fully connected layer and a ReLU activation function to learn the mapping relationship between states and actions. The output vector represents the estimated value of performing the corresponding action in the current state, and the dimension is the number of optional actions; S2, initialize the training network , target network And the experience replay pool R, randomly initialize the training network Parameters , copy the parameters of the trained network Initialize the target network , initialize the experience replay pool R, set the experience replay pool capacity, and store the experience tuples of the interaction between the agent and the environment ; S3, execute S4-S9 for a set number of times, and use the order training set and the rider training set for training; S4, get the initial state of the environment , according to the training network The output is used Strategy selection action ; S5, execute action , get reward , the environment state becomes ; S6, the experience tuple Put it into the experience replay pool R; S7: If the amount of data in the experience replay pool R is sufficient, randomly sample N data from it. ; For each sampled data, use the target network to calculate the target value ,in is the discount factor, ranging from [0,1]; S8, minimize the target loss , to update the training network The parameters, that is ; S9, regularly update the parameters of the target network ,Right now .

6. The dynamic order distribution method based on spatial crowdsourcing according to claim 1 is characterized in that: The heuristic greedy method dynamically allocates the arriving delivery orders including: S11, get the order list that needs to be allocated and a list of available riders ; S12, determine the delivery rider for the order, traverse the order list, and greedily find the rider with the largest profit increment from the list of available riders and return it; S13, planning the delivery route for the rider. If the allocation is successful, the delivery route for the rider is planned and the rider starts to deliver the order; otherwise, the order is delayed to the next decision point for allocation.

7. The dynamic order distribution method based on spatial crowdsourcing according to claim 1 is characterized in that: When calculating the rider encounter status and adjusting the order, it is based on the collaboration between crowdsourcing riders, specifically including: Calculate the rider encounter status and set the rider encounter radius , calculate all the riders in the system The encounter status at the moment, and get the list of encountered riders; From two encounter riders Randomly select an order from the list of delivery orders ; Use order exchange and order transfer strategies to adjust riders and Delivery orders and corresponding delivery routes; implement Order exchange attempts: from the rider and List of orders being shipped Randomly select orders , if in exchange order and If the time and capacity constraints are still met and the profit increases after the exchange, the order exchange operation is performed and the new delivery routes are adjusted for the two riders. and ;After the operation is completed, the rider starts delivering; implement Order transfer attempts: Each time an order is randomly selected from the order lists of the two riders If the order If the constraints are still met after the order is transferred to another rider and the profit increases after the transfer, the order transfer operation is performed, and then the delivery routes of the two riders are adjusted. and , the riders deliver according to the new route.

8. A dynamic order delivery system based on spatial crowdsourcing, characterized in that: It includes an information cycle update module, an initial solution acquisition module, an allocation module, and an adjustment and exchange module; The information cycle update module includes an information acquisition unit and a cycle unit. The information acquisition unit is used to repeatedly update the order information successfully allocated before the current decision point and the location information of the riders in the delivery state; read the newly acquired order information and rider information ; The initial solution acquisition module is used to allocate orders and plan the delivery routes of riders. Specifically, according to the order information and rider information , all unassigned orders are assigned through a heuristic greedy method, and the starting and ending points of the orders are inserted into the delivery routes of the riders using the minimum distance insertion method to obtain the initial solution for order assignment. ; The allocation module uses a hyper-heuristic framework to adjust the initial solution. Specifically, it calculates the initial solution Status , as the input of the DQN network, according to the network output use Strategy selection operator, adjust the solution to get a new solution ; The delivery routes are re-planned for the riders whose orders have been adjusted, and the riders start delivering; The adjustment exchange module is used to calculate the rider encounter status and adjust the order, including: according to the rider's encounter radius ,calculate The encounter status of all riders in the system at all times, randomly selects the rider who meets, and tries to adjust the order. If the profit increases after the adjustment, the order adjustment operation is performed, and the delivery route is re-planned for the rider; otherwise, the order adjustment operation is not performed; after completing the above operations, the rider continues to deliver; The loop unit is used to execute the above process cyclically.

9. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store computer executable programs, the processor reads part or all of the computer executable programs from the memory and executes them, and when the processor executes part or all of the computer executable programs, it can implement the dynamic order delivery method based on spatial crowdsourcing as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the dynamic order delivery method based on spatial crowdsourcing as described in any one of claims 1 to 7.

Images