Order distribution and delivery method for instant delivery

Abstract

The invention discloses an order distribution and delivery method for instant delivery, and relates to the technical field of intelligent logistics optimization, and the method comprises the steps: S1, carrying out the grouping of riders according to historical operation characteristics, employing a grouping actor near-end strategy optimization algorithm, learning a rider reservation strategy in an order matching process and a rider delivery strategy in a delivery process, and carrying out the learning of the rider reservation strategy and the rider delivery strategy; obtaining a policy network corresponding to each group; s2, according to the strategy network corresponding to each group, a rider reservation instruction of a matching stage and a rider distribution instruction of a distribution stage are given to riders of each group; s3, according to the rider reservation instruction, a delayed matching algorithm is adopted to distribute the order to the rider; and S4, based on the order distribution result and the rider distribution instruction obtained in the S3, a greedy adjustment sorting algorithm is adopted to optimize the distribution path and speed of the rider, and the state of the rider is updated after execution so as to optimize the strategy network. According to the invention, the order completion rate and the delivery efficiency can be improved, the order timeout rate is reduced, and the workload of a rider is balanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent logistics optimization, and particularly relates to an order allocation and delivery method for instant delivery. BACKGROUND

[0002] With the popularity of smart phones and the acceleration of consumer life pace, instant delivery services such as take-out, errand and flash delivery have developed vigorously. Instant delivery platforms need to efficiently match between a large number of orders and riders, and plan delivery routes to ensure service efficiency and timeliness. However, the existing instant delivery order allocation and delivery method mainly faces the following challenges: (1) dynamic and complexity: the delivery process involves real-time new orders, rider position changes, traffic condition changes and other dynamic factors, making order allocation and delivery a complex multi-objective optimization problem that needs to balance the interests of multiple parties. (2) balance between efficiency and fairness: the platform needs to consider both delivery efficiency (such as order completion rate, delivery time) and rider work fairness (such as workload balance, reasonable income). (3) large-scale real-time decision-making: the platform needs to process a large number of orders and riders in a short time, with high requirements for the calculation efficiency and real-time performance of the algorithm. (4) long-term optimization: most methods focus on optimization within the decision-making point, and extending the optimization target to a longer period is a major challenge.

[0003] Traditional optimization method-based solutions, such as dynamic programming and heuristic algorithms, are difficult to cope with the above challenges simultaneously. This is because these methods can usually only handle static or simplified problems, and are difficult to adapt to highly dynamic and complex instant delivery environments. At the same time, they often only focus on short-term optimization, making it difficult to achieve long-term efficiency and fairness balance. SUMMARY

[0004] Therefore, the present application provides an order allocation and delivery method for instant delivery to solve the technical problems in the related art.

[0005] The present application provides an order allocation and delivery method for instant delivery, comprising: S1, grouping riders according to historical operation characteristics, using a grouped actor proximal policy optimization algorithm to learn rider reservation strategies in the order matching process and rider delivery strategies in the delivery process, and obtaining a strategy network corresponding to each group; wherein each group corresponds to a strategy network, and all riders in each group share the same strategy network; S2, according to the strategy network corresponding to each group, giving each group of riders rider reservation instructions in the matching stage and rider delivery instructions in the delivery stage; S3, according to the rider reservation instructions, using a delayed matching algorithm to allocate orders to riders; S4, based on the rider delivery instruction and the order distribution result obtained in S3, a greedy adjustment sorting algorithm is used to optimize the delivery path and speed of the rider, and after execution, the state of the rider is updated to optimize the strategy network; the state of the rider includes: the number of orders to be completed, the current position and current speed of the rider.

[0006] In an optional implementation, S1 includes: S11, obtaining historical delivery data of riders in the delivery platform; the historical delivery data includes order information, matching of orders and riders, and rider information; S12, grouping riders according to historical operation characteristics according to the historical delivery data; S13, using a grouping actor proximal policy optimization algorithm to learn rider reservation strategies and rider delivery strategies within each group, and obtaining a strategy network corresponding to each group; S14, using an actor-critic architecture, each group is regarded as an actor, each rider within each group is regarded as an agent, and each agent is used to make a decentralized action according to local observation; a critic is constructed to evaluate and optimize the strategy network of each group according to the action and reward of the agents within the actor group.

[0007] In an optional implementation, each agent is used to make a decentralized action according to local observation, which includes: Each rider independently makes a decision according to the local environmental information observed by itself, and the decision includes: whether to participate in the next matching, current route selection and speed adjustment; After making a decision, the trajectory of the interaction between each rider and the local environmental information is recorded, and the trajectory is stored in a buffer shared by each group; the trajectory is a sequence number including state, action and reward.

[0008] In an optional implementation, S2 includes: S21, obtaining historical order information in a past preset time period, and inputting the historical order information into a pre-trained time-space demand prediction model to output order demand of different regions in a future preset time period; S22, obtaining real-time rider information and order information at the current time, and inputting the order demand of different regions in the future preset time period, the rider information and the order information at the current time into the corresponding strategy network to generate rider reservation instructions in the matching stage and rider delivery instructions in the delivery stage for each group of riders; the rider delivery instruction carries future order demand distribution information; the future order demand distribution information is determined based on the order demand of different regions in the future preset time period.

[0009] In an alternative embodiment, the execution process of the delay matching algorithm is as follows: temporarily storing all to-be-matched order information at the current time in a matching pool; According to the rider reservation instruction obtained in S2, the matching time window of part of the riders is delayed, the pickup locations and delivery locations in all to-be-matched order information are clustered, and the batch is distributed to the riders by using bipartite graph matching, so as to realize globally optimal order distribution.

[0010] In an alternative embodiment, the execution process of the greedy adjustment sorting algorithm is as follows: obtaining the current position of the target rider; According to the order distribution result obtained in S3, the target order corresponding to the target rider is obtained; and the pickup location and delivery location in the target order are obtained; According to the current position of the target rider, the pickup location and delivery location in the target order, an initial path minimizing the current delivery distance is calculated; checking the initial path by using a double-checking mechanism and performing local adjustment to generate a checked delivery path; the double-checking mechanism includes that the average speed of delivery is less than or equal to the road speed limit, and the arrival time is earlier than the promised time; According to the rider delivery instruction, the checked delivery path is adjusted in real time to obtain a final delivery path and speed.

[0011] The present application has the following beneficial effects: (1) Improving order delivery efficiency: the framework can improve the overall delivery efficiency, such as increasing the total order completion rate, improving the delivery efficiency, reducing the order overtime rate, reducing the pickup time, reducing the delay time, etc. (2) Improving system fairness: the instant delivery order distribution and scheduling hybrid decision framework based on reinforcement learning, namely (Adaptive Delivery Matching with Agent-based Courier Reinforcement Optimization, referred to as AD-MACRO), helps to more evenly distribute the workload of riders, so that they can get more rest time, less riding time and less riding distance, and the system overspeed rate is also reduced; (3) Scalability and adaptability: the framework is designed to adapt to dynamic and large-scale delivery environments and maintain its effectiveness under different operation scales and external disturbances, such as when adding rider order rejection behavior and expanding the experimental area, the framework is better than the existing results, which reflects the robustness of the framework. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the drawings required to be used in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0013] Figure 1 is a flowchart of an order allocation and delivery method for instant delivery according to an embodiment of the present application; Figure 2 is a structural diagram of a hybrid decision framework for instant delivery order allocation and scheduling based on reinforcement learning according to an embodiment of the present application; Figure 3 is a framework structure diagram of a grouped actor proximal policy optimization algorithm according to an embodiment of the present application; Figure 4 is a framework structure diagram of a lower adaptive matching and delivery execution layer according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0015] As shown in Figure 1 , the present application provides an order allocation and delivery method for instant delivery, comprising: S1, grouping the riders according to historical operation characteristics, using a grouped actor proximal policy optimization (GAPPO) algorithm to learn the rider reservation strategy in the order matching process and the rider delivery strategy in the delivery process, and obtaining a strategy network corresponding to each group; wherein each group corresponds to a strategy network, and all riders in each group share the same strategy network.

[0016] In step S1, the grouped actor proximal policy optimization (GAPPO) algorithm is an improved proximal policy optimization (PPO) algorithm designed for multi-agent reinforcement learning scenarios. The core innovation lies in improving the collaboration efficiency and training stability of the multi-agent system through grouped policy optimization and parameter sharing mechanism.

[0017] Based on this, the present application regards each rider as an agent, and through grouping the agents according to historical operation characteristics (agent grouping mechanism), the agents in each group will follow similar strategy logic (parameter sharing mechanism) when making decisions, while retaining individual differences, and each rider can still make the final decision according to his own observation.

[0018] The present application adopts the GAPPO algorithm, which can solve the problem of computational complexity caused by a large number of riders, so that the reinforcement learning model can be efficiently applied to the actual distribution scene. Through grouping and sharing strategies, the GAPPO algorithm can still retain the individual differences of the riders while reducing the complexity of the model, and realize the effective update of the strategy.

[0019] Specifically, the similar strategy logic followed by all riders in each group can be regarded as a strategy network, and all riders in each group share the same strategy network. The strategy network is obtained after learning the rider reservation strategy in the order matching process and the rider distribution strategy in the distribution process. The historical operation characteristics are obtained by obtaining the historical distribution data of the riders in the distribution platform, and are extracted from the historical distribution data. The historical operation characteristics can include employment types, distribution modes, etc. For example, full-time riders and part-time riders can belong to different groups, and employed riders and crowd-sourced riders can belong to different groups.

[0021] In an optional embodiment, step S1 comprises: S11, obtaining historical distribution data of riders in the distribution platform; the historical distribution data comprises order information, order and rider matching conditions, and rider information.

[0022] In step S11, the distribution platform continuously collects and stores detailed historical distribution data of the riders. By obtaining the historical distribution data, the historical rules can be learned using the reinforcement learning algorithm, i.e. learning the rider reservation strategy in the order matching process and the rider distribution strategy in the distribution process, and then customizing an effective strategy network.

[0023] The historical distribution data includes order information, order and rider matching conditions, and rider information. The order information includes order location, matching and distribution time, etc. For example, the order location includes merchant location and customer location, which is used to analyze order distribution and distribution distance. The matching and distribution time represents the time information from the generation of the order to the matching of the rider, and then to the final delivery to the customer, which is used to analyze the distribution efficiency. The rider information includes rider ID, rider location, rider historical distribution record, and number of uncompleted orders when matching, etc., which is used to analyze the rider ability and behavior mode.

[0024] S12, grouping the riders according to historical operation characteristics based on historical delivery data.

[0025] S13, learning the rider reservation strategy and the rider delivery strategy in each group by using a group actor proximal policy optimization algorithm, to obtain a strategy network corresponding to each group.

[0026] The rider reservation strategy refers to reserving some important riders to the next matching window to balance the cost, efficiency and quality. The rider delivery strategy refers to guiding the riders to go to high-demand areas to maximize the delivery efficiency, reduce the empty running rate and improve the rider income.

[0027] S14, using an actor-critic architecture, taking each group as an actor and each rider in the group as an agent, and each agent is used to make a decentralized action according to local observation; a critic is constructed to evaluate and optimize the strategy network of each group according to the action and reward of the agents in the actor group.

[0028] In an optional implementation, each agent is used to make a decentralized action according to local observation, including: Each rider makes a decision independently according to the local environmental information observed by itself, and the decision includes whether to participate in the next matching, current route selection and speed adjustment. After the decision is made, the trajectory of the interaction between each actor and the local environmental information is recorded, and the trajectory is stored in the buffer shared in each group; the trajectory is a sequence number including state, action and reward.

[0029] Specifically, in the scene of order allocation strategy and rider delivery strategy, the actor is applicable to local decision (individual difference) while sharing the strategy network, each actor selects action according to observation, and the trajectory is stored in the buffer shared in the group.

[0030] The action refers to the decision made by each rider (actor) according to its own observation, such as whether to participate in the next matching, delivery route selection and speed adjustment. The local environmental information refers to the local information observed by itself, such as the current position, nearby order information (destination, distance, potential demand), self-state (the number of uncompleted orders, related information of uncompleted orders) and the like. The rider can only make a decision according to these local observations, but cannot see all the global information.

[0031] Specifically, the local observation (Observation): the rider can only see the orders, traffic conditions and their own state (such as the number of uncompleted orders, the relevant information of uncompleted orders) in their vicinity, and cannot see all the global information. Independent strategy (Policy): each rider has its own decision logic (which may be a rule-based system or a machine learning model), and the input only depends on the local observation.

[0032] Secondly, after the rider completes the order delivery, the decision-making process and results (such as order matching selection, delivery path selection, delivery time) will be recorded and stored in a shared buffer. These decision-making processes and results are stored in the form of trajectories in the buffer for subsequent analysis, optimization of scheduling strategies or training of machine learning models.

[0033] The trajectory includes a sequence of states, actions and rewards. For example: (past state, rider reservation instruction, rider delivery instruction, order matching decision, rider delivery decision, reward: + delivery fee, new state), (past state, rider reservation instruction, rider delivery instruction, order matching decision, rider delivery decision, reward: - time cost, new state) and so on.

[0034] Examples: Rider A observes that there is an order nearby (destination is a mall, distance is 2 kilometers, and the estimated delivery fee is 10 yuan), but in the next time window, there may be a new high-value order (destination is a school, distance is 1 kilometer, and the estimated delivery fee is 15 yuan). Rider B observes that there are two orders nearby (one is a hospital, distance is 1 kilometer, and delivery fee is 8 yuan; the other is an office building, distance is 3 kilometers, and delivery fee is 15 yuan).

[0035] Upper guidance: From the global information, the platform gives the following reservation and delivery guidance to riders A and B: in the current time window, rider A does not participate in matching, and rider B participates in matching. Rider A can approach the school during the delivery process to obtain a new order, and rider B can approach the office building during the delivery process to obtain a new order.

[0036] Local decision: each rider independently selects actions (accepts an order or refuses) according to its own observation (order information, traffic conditions, and its own state) and upper guidance (reservation instruction, delivery instruction).

[0037] Trajectory storage: Rider A's trajectory: [state: carrying an order, distance 2 kilometers from the mall] → reservation guidance: do not participate in matching → delivery guidance: approach the school during the delivery process → order matching decision: do not participate → rider delivery decision: adjust the delivery sequence and approach the school → reward: - time cost (assuming 5 minutes).

[0038] Rider B's trajectory: [state: carrying two orders, 1 km away from the hospital] → reserved guidance: participate in matching → delivery guidance: approaching the office building during the delivery process → order matching decision: matched with the hospital order → rider delivery decision: adjust the delivery sequence, approach the office building → reward: +8 yuan (delivery fee) - time cost (assuming 5 minutes).

[0039] The trajectories of all riders are stored in a shared buffer.

[0040] S2, according to the strategy network corresponding to each group, give the rider reservation instructions in the matching stage and the rider delivery instructions in the delivery stage to the riders of each group.

[0041] In an optional implementation, step S2 includes: S21, obtain historical order information in a past preset time period, and input the historical order information into a pre-trained spatiotemporal demand prediction model to output order demand quantities of different regions in a future preset time period; S22, obtain rider information and order information at the current time in real time, and input the order demand quantities of different regions in the future preset time period, the rider information and the order information at the current time into the corresponding strategy network to generate the rider reservation instructions in the matching stage and the rider delivery instructions in the delivery stage for the riders of each group; the rider delivery instructions carry future order demand distribution information; the future order demand distribution information is determined based on the order demand quantities of different regions in the future preset time period.

[0042] S3, according to the rider reservation instructions, use a delayed matching algorithm (Delayed Matching with Forecasting, DM&F) to assign orders to riders.

[0043] In an optional implementation, the execution process of the delayed matching algorithm is as follows: temporarily store all orders to be matched at the current time in a matching pool; delay the matching time window of part of the riders according to the rider reservation instructions obtained in S2, cluster the pickup locations and delivery locations in all orders to be matched, and use bipartite graph matching to assign the orders to the riders in batches to achieve globally optimal order assignment.

[0044] S4, based on the order assignment result obtained in S3 and the rider delivery instructions, use a greedy adjustment sequence algorithm (Greedy Adjustment Sequence, GAS) to optimize the delivery path and speed of the riders, and update the state of the riders after execution to optimize the strategy network; the state of the riders includes: the number of orders to be completed, the current position and current speed of the riders.

[0045] In an alternative embodiment, the execution process of the greedy adjustment sorting algorithm is as follows: According to the order allocation result obtained in S3, a target order corresponding to the target courier is obtained; and a pickup location and a delivery location in the target order are obtained; According to the current position of the target courier, the pickup location and the delivery location in the target order, an initial path minimizing the current delivery distance is calculated; The initial path is checked and locally adjusted using a double-checking mechanism to generate a checked delivery path; the double-checking mechanism includes that the average speed of delivery is less than or equal to the road speed limit, and the arrival time is earlier than the promised time; According to the courier delivery instruction, the checked delivery path is adjusted in real time to obtain a final delivery path and speed.

[0046] As shown in Figure 2 The method of the present application adopts an instant delivery order allocation and scheduling hybrid decision framework based on reinforcement learning, namely (Adaptive Delivery Matching with Agent-based Courier Reinforcement Optimization, referred to as AD-MACRO). The framework aims to solve the challenges faced by instant delivery platforms in order allocation and delivery optimization. AD-MACRO framework is a hybrid decision framework that combines reinforcement learning with optimization methods to achieve optimal control of instant delivery process. The framework contains two main levels: Upper layer: policy optimization layer. The platform as a high-level agent, evaluates and guides the decision; the courier as a low-level agent, makes a decentralized decision according to its own state. In the upper layer reinforcement learning part, the group actor proximal policy optimization algorithm is adopted, which reduces the decision dimension and improves the algorithm efficiency by grouping the couriers according to similar operating characteristics.

[0047] Lower layer: adaptive matching and delivery execution layer. This layer executes order allocation and delivery tasks according to the strategy provided by the upper layer. In the order allocation stage, a delayed matching algorithm is used in combination with the future demand predicted by the space-time demand forecasting (STDFM) to achieve more reasonable order allocation. The courier adjusts the delivery route and speed in real time according to the delivery guidance provided by the upper layer and the feasible delivery route generated by the greedy adjustment sorting (GAS) algorithm.

[0048] The upper and lower layers interact iteratively to continuously optimize the order allocation and scheduling strategy to adapt to the dynamically changing delivery environment.

[0049] Specifically, the upper layer is responsible for long-term decision-making of the delivery platform and riders. It receives system information from the lower layer, such as new orders, available riders, and future demand forecasts. Based on global complete information, the upper layer formulates rider reservation strategies for the platform, reserving riders for high-value or urgent orders that may arise in the future, and matching riders with more suitable orders to improve long-term order matching quality. At the same time, the upper layer formulates delivery strategies for riders based on their local observation information, guiding them to complete delivery faster and maximize potential order acquisition.

[0050] The lower layer, guided by the upper layer, uses optimization methods to execute the implementation decisions of the platform and riders. The platform uses the DM&F algorithm in combination with rider reservation guidance to match orders with riders. Then, riders go to the corresponding locations to pick up and deliver orders based on the newly assigned orders. Based on local observation information, combined with upper-layer delivery guidance, riders adjust their routes and speeds in real time to seek more potential orders, rather than strictly following the pre-set delivery sequence.

[0051] A continuous feedback loop is formed between the upper and lower layers: real-time state updates of the lower layer are used to improve the decisions of the upper layer, and the guidance of the upper layer guides the execution of the lower layer.

[0052] As shown in Figure 3 , the architecture diagram of the Grouped Actor Proximal Policy Optimization algorithm (GAPPO) is shown, which clearly shows the core idea of the algorithm through the three core components of actors, critics, and buffers, and their interaction logic. The following explains the algorithm in detail in combination with the components, connection relationships, and examples in the figure: 1. Actor Role: The figure shows 2 actors, each corresponding to multiple agents (such as agent 1 to agent n), and the actor is responsible for interacting with the environment.

[0053] Grouping mechanism: Grouping basis: Agents are divided into different actor groups according to operating characteristics (such as employment type, delivery mode). For example, in the logistics scenario, full-time delivery personnel and part-time delivery personnel may belong to different actor groups.

[0054] Strategy sharing: Agents in the same group share the same actor's strategy network (such as agent 1 to agent k share strategy network 1, and agent k+1 to agent n share strategy network 2).

[0055] Local decision-making: Each agent selects an action based on its own observation (observation), and the trajectory is stored in the buffer shared within the group.

[0056] 2. Critic Role: Located above the individual components, responsible for evaluating the quality of actor strategies. Figure 3

[0057] Evaluation Method: Receive trajectory information such as the state (stat e ) and action (action) of agents in the same actor group.

[0058] Calculate advantage function (measure the degree of action better than average level) or state value function (predict long-term returns).

[0059] Provide feedback to update actor strategies, ensuring that strategies are optimized towards high returns.

[0060] 3. Buffers Role: Figure 3 Two buffers (buffer 1 and buffer 2) are displayed, each corresponding to an actor group.

[0061] Function: Store experience: Record trajectory data such as the state, action, and reward of agents in the same actor group.

[0062] Data reuse: Train policy networks by sampling experiences in buffers to improve data utilization.

[0063] In-group sharing: Agents in the same actor group share a buffer, promoting experience dissemination and collaborative optimization of strategies.

[0064] II. Connection between components and data flow 1. Actor and environment interaction Action execution: Agents in the actor group select actions (such as agent 1 performing action a1) according to the actor's policy network, and interact with the environment.

[0065] State feedback: The environment returns the new state (stat e ) and reward (reward), forming an interactive closed loop.

[0066] 2. Actor and critic interaction Strategy update: The actor is connected to the critic through the arrow, indicating that the necessary trajectory information and other feedback is fed back to the critic.

[0067] Feedback delivery: The critic is connected to the actor through the arrow, delivering advantage function or value function to guide strategy optimization.

[0068] 3. Actor and experience buffer interaction ​Experience storage: actors store their trajectory data into a group buffer (e.g., agent 1 to agent k store into buffer 1).

[0069] 4. Update process Iterative optimization: Figure 3 The "update" arrow represents the update process between actors and critics.

[0070] Optimization goal: actors update their policy network parameters based on critic feedback and buffer experience, maximizing long-term rewards through gradient descent.

[0071] The GAPPO architecture diagram realizes efficient collaboration of multi-agent systems through actor grouping, policy sharing, experience buffer, and critic feedback. Its core lies in balancing sharing and heterogeneity: sharing policies within the same group reduces complexity, and independent decision-making between groups retains flexibility. This design has broad application prospects in logistics, transportation, game AI, and other fields, providing a new paradigm for agent collaboration in complex systems.

[0072] The benefits of using the GAPPO algorithm include: (1) Reducing model complexity: compared to each agent having an independent policy network, sharing a policy network greatly reduces the number of parameters that need to be learned.

[0073] (2) Preserving agent heterogeneity: although policies are shared, each agent can still interact with the environment based on its own observations and experience, and store its trajectory in the buffer shared by its respective group.

[0074] (3) Improving computational efficiency and personalized decision-making ability: sharing policies reduces computational cost, while the grouping mechanism allows more refined policy learning within the group, thereby improving personalized decision-making ability.

[0075] As shown in Figure 4 , the structure of the adaptive matching and delivery execution layer is displayed, including three core executors: the platform and two types of riders. This layer is the core part of the delivery system, responsible for intelligently matching orders with riders and efficiently executing delivery tasks.

[0076] Specifically: 1. Delivery platform (core management unit, high-level agent) The platform is the brain of the entire adaptive matching and delivery execution layer, playing the role of a command center. Its main responsibilities are: Receive new orders: the platform receives new delivery orders from users or merchants, and the order information includes order content, destination, delivery time requirements, etc.

[0077] Coordinating Delivery Process: The platform is responsible for coordinating the matching and delivery process of orders, ensuring that orders can be efficiently and accurately allocated to the appropriate riders, and monitoring the delivery process to ensure that orders are delivered on time.

[0078] 2. Two types of riders (executive body, low-level agent) There are two types of riders in the system, who work together to complete the delivery task: (1) Company-employed riders: Characteristics: Have a fixed employment relationship with the platform, and are the stable force of the platform's delivery network.

[0079] Advantages: Strong reliability, rich delivery experience, suitable for handling regular orders and ensuring basic delivery capacity, and less order rejection behavior.

[0080] (2) Crowd-sourced riders: Characteristics: Participate in delivery through the crowd-sourcing mode, with high flexibility.

[0081] Advantages: Can choose to accept orders according to their own time and location, providing additional delivery resources for the platform, especially during peak order periods or in specific areas, effectively supplementing delivery capacity, and urgently needing to deliver orders quickly.

[0082] 3. Order matching process (intelligent allocation) Order matching is one of the core functions of the platform, aiming to intelligently allocate orders to the most suitable riders. Figure 4 The following shows the key technologies in this process: Bipartite graph matching algorithm: The platform uses a bipartite graph matching algorithm for order matching. This algorithm will consider the following factors to find the best order- rider matching: Candidate rider information: Rider's location, delivery capacity, current order volume, historical delivery efficiency, etc.

[0083] Order information: Order destination, delivery time requirement, order priority, etc.

[0084] Rider reservation instructions: Reserve certain riders for the next time window for matching to optimize order allocation and reserve delivery capacity.

[0085] 4. Delivery execution process (task execution) After successful order matching, the delivery execution phase begins, where riders are responsible for completing the delivery task of orders. This process includes: Rider delivery instructions: The platform will provide delivery guidance to riders to help them efficiently complete delivery, and move to areas with high order volume.

[0086] Pick-up: The rider will go to the merchant to pick up the goods according to the order information.

[0087] Delivery: Riders autonomously choose delivery routes based on their current state and deliver orders to customers.

[0088] Order Delivery: Customers receive and sign for orders, completing the delivery process.

[0089] 5. Strategic Guidance (Optimized Decision Making) To improve delivery efficiency and optimize decision making, the system provides two strategic guidelines: Rider Reservation Guidance: Purpose: Predict future order demand and reserve rider resources in advance.

[0090] Effect: Ensure sufficient delivery capacity during order peaks or in specific areas, avoiding order backlog due to insufficient riders.

[0091] Rider Delivery Guidance: Purpose: Provide strategic guidance for riders during the delivery process.

[0092] Effect: Includes optimal delivery routes, potential high-order areas, etc., ensuring that riders can efficiently and safely complete delivery tasks and maximize profits.

[0093] 6. System State Transition Phase (Dynamic Adjustment) The system state transition phase reflects the system's adaptive ability. The system will dynamically adjust based on the following information to achieve state transition: Order Status: Addition of new orders, delivery progress of existing orders, and whether there are abnormalities.

[0094] Observation Information: Real-time traffic conditions, rider locations, weather conditions, etc.

[0095] Estimated Demand: Predict future order volume, etc.

[0096] Through dynamic adjustment, the system can adapt to changing environments and maintain efficient operation.

[0097] 7. Closed-loop Mechanism (Continuous Optimization) The closed-loop mechanism is the key to the system's continuous optimization. Through closed-loop feedback, the system can: Collect Data: Collect real-time data during delivery, such as order delivery time, rider efficiency, customer satisfaction, etc.

[0098] Analyze Data: Analyze collected data to evaluate the effectiveness of current strategies and identify existing problems.

[0099] Optimize Strategies: Based on data analysis results, continuously optimize order matching algorithms, delivery guidance strategies, etc., to improve overall delivery efficiency and service quality.

[0100] The application is based on a hybrid decision framework combining reinforcement learning and optimization methods to achieve optimal control of the instant delivery process. By optimizing order allocation and rider scheduling, the order completion rate and delivery efficiency are improved, the order overtime rate is reduced, the workload of the riders is balanced, and their rest time is increased, thereby achieving a win-win situation for the platform, riders and customers.

[0101] In summary, the application has the following advantages: (1) Improve order delivery efficiency: the framework can improve the overall delivery efficiency, such as increasing the total order completion rate, improving the delivery efficiency, reducing the order overtime rate, reducing the pickup time, reducing the delay time, etc. (2) Improve system fairness: AD-MACRO helps to more evenly distribute the workload of riders, so that they can get more rest time, less riding time and less riding distance, and the system overspeed rate is also reduced; (3) Scalability and adaptability: the framework is designed to adapt to dynamic and large-scale delivery environments and maintain its effectiveness under different operation scales and external disturbances, such as when adding rider refusal behavior and expanding the experimental area, the framework is better than existing results, which reflects the robustness of the framework.

[0102] Although the embodiments of the application are described in conjunction with the drawings, various modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the application, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for order allocation and delivery for instant delivery, characterized in that, include: S1. Group riders according to their historical operational characteristics, and use the group actor near-end strategy optimization algorithm to learn the rider reservation strategy in the order matching process and the rider delivery strategy in the delivery process to obtain the strategy network corresponding to each group; where each group corresponds to a strategy network, and all riders in each group share the same strategy network. S2. Based on the strategy network corresponding to each group, give rider reservation instructions for the matching stage and rider delivery instructions for the delivery stage to riders in each group. S3. Based on the rider's pre-reserved instructions, the order is assigned to the rider using a delayed matching algorithm; S4. Based on the rider delivery instructions and the order allocation results obtained in S3, a greedy adjustment sorting algorithm is used to optimize the rider's delivery path and speed. After execution, the rider's status is updated to optimize the strategy network. The rider's status includes: the number of orders to be completed, the rider's current location, and the current speed.

2. The method according to claim 1, characterized in that, S1 includes: S11. Obtain historical delivery data of riders from the delivery platform; the historical delivery data includes order information, order-rider matching information, and rider information; S12. Based on the historical delivery data, group the riders according to their historical operational characteristics; S13. The group actor near-end strategy optimization algorithm is adopted to learn the rider reservation strategy and the rider delivery strategy in each group, so as to obtain the strategy network corresponding to each group. S14. Adopting an actor-critic architecture, each group is treated as an actor, and each rider within each group is treated as an agent. Each agent is used to make decentralized actions based on local observations. A critic is constructed to evaluate and optimize the policy network of each group based on the actions and rewards of the agents within the actor group.

3. The method according to claim 2, characterized in that, Each agent is used to perform decentralized actions based on local observations, including: Each rider makes independent decisions based on the local environmental information they observe, including: whether to participate in the next match, current route selection, and speed adjustment. After a decision is made, the trajectory of each rider's interaction with their local environment is recorded and stored in a shared buffer within each group; the trajectory is a sequence number including status, action, and reward.

4. The method according to claim 1, characterized in that, S2 includes: S21. Obtain historical order information within a preset time period, and input the historical order information into a pre-trained spatiotemporal demand prediction model to output the order demand in different regions within a preset time period in the future. S22. Obtain rider and order information in real time, and input the order demand in different regions within a future preset time period, the rider and order information at the current moment into the corresponding strategy network, and generate rider reservation instructions for the matching stage and rider delivery instructions for the delivery stage for each group of riders; the rider delivery instructions carry future order demand distribution information; the future order demand distribution information is determined based on the order demand in different regions within a future preset time period.

5. The method according to claim 3, characterized in that, The execution process of the delayed matching algorithm is as follows: Temporarily store all pending order information in the matching pool at the current moment; Based on the rider reservation instructions obtained in S2, the matching time window for some riders is delayed, the pick-up and delivery locations in all order information to be matched are clustered, and bipartite graph matching is used to batch assign them to riders in order to achieve globally optimal order allocation.

6. The method according to claim 5, characterized in that, The execution process of the greedy adjustment sorting algorithm is as follows: Get the current location of the target rider; Based on the order allocation results obtained in S3, obtain the target order corresponding to the target rider; and obtain the pick-up location and delivery location in the target order; Based on the target rider's current location and the pickup and delivery locations in the target order, calculate an initial path that minimizes the current delivery distance; A dual verification mechanism is used to check the initial path and make local adjustments to generate a checked delivery path. The dual verification mechanism includes: the average delivery speed is less than or equal to the road speed limit, and the delivery time is earlier than the promised time; Based on the rider's delivery instructions, the inspected delivery route is adjusted in real time to obtain the final delivery route and speed.

Images