Method and system for real-time planning of autonomous flow vehicle cluster collaborative distribution path based on multi-agent reinforcement learning

Abstract

This invention relates to the field of unmanned logistics delivery, and more particularly to a real-time planning method and system for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning. The method includes: acquiring real-time status information of each vehicle in the cluster to construct collaborative delivery data; then, responding to preset trigger events and generating a global collaborative scheme based on the collaborative delivery data; next, acquiring local environmental perception information of each vehicle and independently generating a local execution strategy based on the global collaborative scheme; determining whether neighboring vehicles meet conflict triggering conditions based on the local execution strategy, and if so, generating a collaborative interaction strategy to resolve the conflict; finally, outputting the real-time delivery path planning results for each vehicle based on the execution results of the collaborative interaction strategy. This invention effectively improves the delivery efficiency and safety of unmanned logistics vehicle clusters in dynamic environments by combining global centralized planning with distributed execution and introducing an active conflict resolution mechanism.

Description

Technical Field

[0001] This application relates to the field of unmanned logistics delivery technology, and in particular to a method and system for real-time planning of collaborative delivery paths for unmanned logistics vehicle clusters based on multi-agent reinforcement learning. Background Technology

[0002] Currently, path planning for unmanned logistics vehicle swarms mainly relies on centralized scheduling or decentralized decision-making. Centralized methods struggle to cope with real-time dynamic changes on the road, such as temporary congestion or new orders, while decentralized decision-making easily leads to a lack of coordination among vehicles and path conflicts. More importantly, most methods separate global path planning from local execution control, meaning that once the delivery plan is set, it is rarely adjusted based on real-time information, resulting in poor adaptability.

[0003] Secondly, in actual logistics and delivery, multiple vehicles often need to relay to complete an order. For example, one vehicle delivers to a transfer point, and another picks up the goods and continues the delivery. Existing scheduling systems typically treat each segment of the task as an independent entity, rarely recording the dependencies between tasks. If a vehicle performing an upstream task experiences an anomaly, such as insufficient battery power, malfunction, or severe delay, current technologies often only reschedule that vehicle's own task, while downstream vehicles continue to drive as originally planned, leading to wasted effort, missed time windows, or even the failure of the entire task chain. Therefore, existing methods have not yet provided an effective solution for how to quickly identify the affected task chain and coordinate the adjustment of upstream and downstream vehicles when anomalies occur. Summary of the Invention

[0004] To address one or more problems in the existing technology, the main objective of this application is to provide a real-time planning method and system for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning.

[0005] To achieve the aforementioned objectives, this application proposes a real-time planning method for collaborative delivery routes of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, the method comprising:

[0006] Real-time status information of each unmanned logistics vehicle in the cluster;

[0007] Based on the real-time status information, construct collaborative delivery data for global planning;

[0008] In response to a preset trigger event, a global collaborative solution is generated by combining the collaborative delivery data.

[0009] Acquire local environmental perception information of each vehicle, and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently.

[0010] Based on the local execution strategy of each vehicle, determine whether the conflict triggering conditions are met between neighboring vehicles;

[0011] If the conflict triggering conditions are met between adjacent vehicles, a collaborative interaction strategy for resolving the conflict is generated based on the determination result.

[0012] Based on the execution results of the aforementioned collaborative interaction strategy, the real-time delivery route planning results for each vehicle are output.

[0013] This application also provides a real-time path planning system for collaborative delivery of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, including:

[0014] The first acquisition module is used to acquire the real-time status information of each unmanned logistics vehicle in the cluster.

[0015] A construction module is used to construct collaborative delivery data for global planning based on the real-time status information;

[0016] The first generation module is used to generate a global collaborative solution in response to a preset trigger event and in combination with the collaborative delivery data.

[0017] The second acquisition module is used to acquire the local environmental perception information of each vehicle, and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently.

[0018] The judgment module is used to determine whether the conflict triggering conditions are met between neighboring vehicles based on the local execution strategy of each vehicle.

[0019] The second generation module is used to generate a collaborative interaction strategy to resolve the conflict based on the determination result if the conflict triggering conditions are met between adjacent vehicles.

[0020] The output module is used to output the real-time delivery route planning results of each vehicle based on the execution results of the collaborative interaction strategy.

[0021] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0022] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0023] This application presents a real-time planning method and system for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning. By acquiring the status information of each vehicle in the cluster in real time and dynamically updating the collaborative delivery data, the global planning no longer relies on static offline data but can truly reflect the current vehicle distribution, resource availability, and task progress, thereby effectively reducing decision-making bias caused by information lag. Secondly, it adopts an event-driven global collaborative solution generation mechanism, triggering replanning only when new tasks, vehicle anomalies, or road condition changes are required, reducing the computational burden of continuous recalculation while ensuring the solution can quickly adapt to dynamic changes. The centralized training and distributed execution framework of multi-agent reinforcement learning allows the global planning layer to output task allocation and path guidance that are both efficient and feasible, while each vehicle independently fine-tunes its local strategy based on local environmental information collected by its own sensors, achieving a balance between order and micro-flexibility. Furthermore, by introducing conflict prediction and distributed collaborative interaction mechanisms during local execution, vehicles can anticipate the risk of spatiotemporal overlap with neighboring vehicles and proactively negotiate yielding or speed matching, transforming passive risk avoidance into proactive collaboration. This reduces the probability of collisions and the frequency of emergency braking, while also improving traffic efficiency in dense traffic scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a real-time collaborative delivery path planning method for unmanned logistics vehicle clusters based on multi-agent reinforcement learning, according to an embodiment of this application.

[0025] Figure 2 This is a flowchart illustrating a real-time collaborative delivery path planning method for unmanned logistics vehicle clusters based on multi-agent reinforcement learning, according to an embodiment of this application.

[0026] Figure 3 This is a schematic block diagram of a real-time collaborative delivery path planning system for unmanned logistics vehicle clusters based on multi-agent reinforcement learning, according to an embodiment of this application.

[0027] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of a task-dependent directed graph and a chain-recursive traversal according to an embodiment of this application.

[0029] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] Reference Figure 1 This application provides a real-time planning method for collaborative delivery routes of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, the method comprising:

[0032] S1. Obtain real-time status information of each unmanned logistics vehicle in the cluster;

[0033] S2. Based on the real-time status information, construct collaborative delivery data for global planning;

[0034] S3. In response to a preset trigger event, and in conjunction with the collaborative delivery data, generate a global collaborative solution;

[0035] S4. Obtain the local environmental perception information of each vehicle, and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently.

[0036] S5. Based on the local execution strategy of each vehicle, determine whether the conflict triggering conditions are met between neighboring vehicles;

[0037] S6. If the conflict triggering conditions are met between adjacent vehicles, a collaborative interaction strategy for resolving the conflict is generated based on the determination result.

[0038] S7. Based on the execution result of the collaborative interaction strategy, output the real-time delivery route planning results for each vehicle.

[0039] As described in steps S1-S4 above, in step one, during the actual delivery process, the status of each vehicle changes continuously over time, including its current location, current speed, remaining battery power, quantity of loaded goods, and unfinished task list. This information forms the basis for all subsequent planning and decision-making. This solution maintains a continuous communication connection with each vehicle and periodically collects this dynamic data at set time intervals. The collected information is uniformly formatted for easy subsequent processing. The principle behind this step is that only by understanding the true status of each vehicle in the cluster can the system make effective collaborative decisions. If battery power information is lacking, vehicles may run out of energy during delivery; if task information is lacking, the global solution cannot reasonably allocate new orders. By acquiring this status information in real time, this solution provides reliable input for the multi-agent reinforcement learning system. Step two: Simple status information is fragmented and insufficient to directly guide global collaboration. Therefore, after acquiring the status of each vehicle, this solution needs to integrate this information into a structured collaborative delivery data that is easy for the global planning algorithm to use. For example, the locations of all vehicles are mapped onto an electronic map to form a vehicle distribution density map; the remaining battery power and cargo capacity of each vehicle are organized into resource vectors; and unfinished tasks are sorted according to priority and time window. This data is encapsulated into a unified collaborative data model, which is equivalent to preparing a global situation snapshot for the global scheduling module. Transforming the raw, scattered state information into planning inputs with clear semantics reduces the complexity of subsequent reinforcement learning algorithms in processing data, while ensuring the spatiotemporal consistency of information from different sources. Step three: The delivery process is not static; new orders are constantly being added, vehicles may malfunction, and roads may become congested. To cope with these changes, this solution introduces an event-driven mechanism. Preset trigger events can include: the arrival of a new delivery task, a vehicle reporting that its remaining battery power is below the safety line, a vehicle experiencing mechanical failure, or the system receiving a real-time report of abnormal road conditions. Once any of these events is detected, the system will call a multi-agent reinforcement learning algorithm, using the collaborative delivery data constructed in the previous step as input, to generate a new global collaborative scheme for the entire cluster. This solution comprises at least two layers: first, task allocation decisions, determining which vehicle(s) will handle newly arriving orders; and second, global path guidance information, defining a general driving corridor for each vehicle. The principle behind this step is that through centralized training in reinforcement learning, the system learns how to efficiently allocate tasks and plan routes under various circumstances. When a triggering event occurs, the system doesn't search from scratch but quickly provides a solution based on the learned strategies, achieving a response time in seconds. The centralized training and distributed execution characteristics of multi-agent reinforcement learning are key features. During training, all vehicles share experience, and during execution, a global module rapidly outputs a solution based on a unified strategy.Step four, the global coordination scheme, provides a broad spatiotemporal corridor rather than a fine-grained path precise to every point in time. This is because the global scheduling module possesses global information but cannot perceive the micro-dynamics around each vehicle in real time, such as pedestrians suddenly entering the lane, temporarily stopped vehicles, or sudden braking by vehicles ahead. Therefore, this scheme requires each unmanned logistics vehicle to utilize its onboard sensors, such as LiDAR, cameras, and millimeter-wave radar, to acquire real-time environmental information, including the position of nearby obstacles, the speed and heading of nearby vehicles, and the position of lane lines. Then, the onboard intelligent system combines this local perception information with the global coordination scheme issued by the global scheduling module to independently plan a local execution strategy. This strategy can be a specific speed adjustment command, lane change timing selection, or temporary stopping decision. Combining global macro-guidance with local micro-autonomy ensures the overall orderliness of the cluster while giving each vehicle sufficient autonomy to cope with rapidly changing nearby road conditions. This avoids sluggish vehicle response due to communication delays or excessive global processing load.

[0040] As described in steps S5-S7 above, even if each vehicle follows its own local execution strategy in steps five and six, mutual interference may still occur at congested road sections or intersections. For example, two vehicles may simultaneously approach a merging point, or one vehicle may attempt to change lanes and cut into the safe interval of another vehicle. To address this issue, this solution adds a distributed conflict detection and resolution layer to the single-vehicle execution strategy. While executing its own strategy, each vehicle continuously monitors the driving intentions of surrounding vehicles, such as obtaining the predicted trajectories of neighboring vehicles through inter-vehicle communication. The system calculates the degree of spatiotemporal overlap of the predicted trajectories of multiple vehicles in real time. When this overlap exceeds a preset safety threshold, a conflict condition is considered triggered. Once a conflict is detected, the relevant vehicles need to generate a cooperative interaction strategy. This strategy can take the form of negotiating yielding, adjusting speeds to stagger passage times, or temporarily changing lanes. The principle of this step is to introduce a localized cooperation mechanism based on distributed decision-making, enabling vehicles to achieve implicit coordination through algorithms, much like human drivers reaching an understanding through eye contact and gestures at unsignalized intersections. Its effect is to quickly and locally resolve most potential conflicts without relying on a global scheduling module, thereby ensuring driving safety while maintaining high traffic efficiency. Step seven is the final output of all the above steps, which is to generate one or a series of real-time executable path planning results for each unmanned logistics vehicle. These results are not output once and then remain unchanged, but are continuously updated as the above steps iterate. That is, when the vehicle completes a journey, detects new road condition changes, or receives a new global solution, the system will re-execute the aforementioned process and output the updated path. This dynamic output mechanism ensures that the vehicle always travels along the latest optimal path. Its effect is to keep the entire delivery process sensitive and adaptable to the environment, realizing a leap from static open-loop control to dynamic closed-loop control.

[0041] As mentioned above, by acquiring the status information of each vehicle in the cluster in real time and dynamically updating the collaborative delivery data, the global planning no longer relies on static offline data, but can truly reflect the current vehicle distribution, resource availability, and task progress, thereby effectively reducing decision-making bias caused by information lag. Secondly, an event-driven global collaborative solution generation mechanism is adopted, triggering replanning only when new tasks, vehicle anomalies, or road condition changes are required, reducing the computational burden of continuous recalculation while ensuring that the solution can quickly adapt to dynamic changes. The centralized training and distributed execution framework of multi-agent reinforcement learning enables the global planning layer to output task allocation and path guidance that are both efficient and feasible, while each vehicle independently fine-tunes its local strategy based on local environmental information collected by its own sensors, achieving a balance between order and micro-flexibility. Furthermore, by introducing conflict prediction and distributed collaborative interaction mechanisms during local execution, vehicles can predict the risk of spatiotemporal overlap with neighboring vehicles in advance and actively negotiate yielding or speed matching, transforming passive risk avoidance into active collaboration, reducing the probability of collisions and the frequency of emergency braking, while improving traffic efficiency in dense traffic scenarios.

[0042] Reference Figure 2 In one embodiment, the step of generating a global collaborative solution in response to a preset triggering event, combined with the collaborative delivery data, includes:

[0043] S31. Obtain the preset trigger event, and determine the set of vehicles or task sets to be planned according to the type of the trigger event;

[0044] S32. Extract the remaining battery power, current cargo load, and list of unfinished tasks for each vehicle from the collaborative delivery data;

[0045] S33. Based on the extracted results, construct a joint resource constraint vector;

[0046] S34. Call the graph attention network. Through the graph attention network, use the joint resource constraint vector as node features and the distance between vehicles and the task overlap as edge weights to generate a resource dependency graph between vehicles.

[0047] S35. Invoke the multi-agent reinforcement learning algorithm, and use the resource dependency graph as the basis for collaborative decision-making to generate the global collaborative scheme for the set of vehicles or task sets to be planned.

[0048] As described above, in actual operation, there may be various types of events that trigger global replanning. For example, when the system receives a brand new delivery order, it only needs to select suitable vehicles from those that are idle or about to complete their current tasks, without involving all vehicles in the replanning. Conversely, if a vehicle experiences a serious malfunction while on the road, not only does the vehicle's own task need to be reassigned, but vehicles around it that may be affected by congestion may also need to adjust their plans. Therefore, this solution first analyzes the type of triggering event and dynamically determines a smaller, more targeted set of vehicles or tasks based on the nature of the event. The benefits of doing this are obvious: it avoids scanning the entire fleet for each replanning, significantly reducing computational overhead and speeding up response time. After determining the scope to be planned in Step Two, the system needs to extract the key attributes of these vehicles from the previously constructed collaborative delivery data. Remaining battery power directly determines how far a vehicle can travel and whether it can accept new long-distance orders; current cargo load determines how much free capacity the vehicle has to load new goods; and the incomplete task list records the delivery tasks that each vehicle has committed to and their time window constraints. These three pieces of information are indispensable; together they constitute a vehicle's resource profile at the current moment. Without battery level information, a vehicle about to run out of energy might be assigned to a long-distance order, ultimately leading to a breakdown. Without cargo capacity information, overloading or wasted capacity might occur. Without a task list, it's impossible to determine whether a new order conflicts with an existing task in terms of time. Step three integrates the extracted battery level, cargo capacity, and key information from the task list into a compact numerical vector called the joint resource constraint vector. This vector assigns each vehicle a set of multi-dimensional resource labels, which can be simply understood as a vehicle's capability profile. For example, one vehicle might be in a state of high battery, low cargo, and few tasks, while another vehicle might be in a state of low battery, full load, and intensive tasks. After unifying and summarizing these vectors, the system obtains a snapshot describing the resource distribution of the entire cluster. The technical significance of this step lies in standardizing the original data, which was originally of varying formats and dimensions, making it directly processable by graph neural networks and reinforcement learning algorithms, avoiding tedious manual feature engineering. Step four, after obtaining the joint resource constraint vector, introduces a graph attention network to construct a resource dependency graph. In this graph, each node represents an unmanned logistics vehicle, and the initial feature of the node is the joint resource constraint vector of that vehicle. Whether nodes are connected and the strength of the connection are determined by two factors: the spatial distance between vehicles and the degree of task overlap.The smaller the spatial distance, the greater the likelihood of two vehicles meeting or influencing each other on the road; therefore, the graph needs to reflect strong connections. Task overlap refers to the degree to which the unfinished tasks of two vehicles intersect at pickup points, delivery points, or time windows. Higher overlap indicates a stronger dependency between the two vehicles in terms of resource competition or task collaboration. The graph attention network dynamically learns the importance weight of each edge, enabling the final resource dependency graph to highlight the key interactions that truly affect collaborative efficiency. The principle is that the collaborative decision-making of an unmanned logistics vehicle cluster is essentially a graph-based reasoning problem. Each vehicle's decision depends not only on its own state but also on the states of its neighboring vehicles. Through the graph attention network, the system can automatically capture this influence without manually pre-setting rules. Compared to simple fully connected or distance-based graphs, this method can more precisely reflect the actual degree of dependency between vehicles. For example, a fully loaded vehicle and an idle vehicle that is very close may have a high task transfer dependency, while the dependency between two vehicles that are very close but traveling in opposite directions and have no overlapping tasks can be ignored. Step 5: After obtaining the resource dependency graph, this scheme calls a multi-agent reinforcement learning algorithm to finally generate a global collaborative solution. This multi-agent reinforcement learning algorithm adopts a centralized training and distributed execution paradigm. During the training phase, the algorithm learns how to output reasonable actions for each node (i.e., each vehicle) in the current resource dependency graph. These actions include accepting new tasks, relinquishing some tasks to other vehicles, and adjusting speed to avoid obstacles. After training, during actual execution, the system only needs to input the current resource dependency graph into the trained policy network, and the network can quickly output a collaborative decision-making scheme, including how to allocate newly arrived tasks and delineate approximate spatiotemporal path corridors for each vehicle.

[0049] In this embodiment, the core problem addressed is how to reasonably handle the complex and dynamic resource dependencies between vehicles when generating a global collaborative solution. Traditional methods often treat each vehicle as an independent decision-making unit, allocating tasks solely based on its own state and simple rules, ignoring the mutual constraints between vehicles in multiple dimensions such as battery level, cargo capacity, and task time windows. This leads to seemingly reasonable allocation results locally, but cause resource mismatch, path conflicts, and efficiency reduction globally. For example, a vehicle with low battery might be assigned a long-distance order, while a nearby vehicle with high battery might be handling a short-distance task; or two vehicles with highly overlapping tasks might be guided to the same narrow road segment, causing unnecessary time waste. This solution constructs a joint resource constraint vector, unifying the multi-dimensional resource states of each vehicle into a computable mathematical space. Then, using a graph attention network, it automatically learns the spatial distance associations and task overlap dependencies between vehicles, generating a dynamically updated resource dependency graph. This graph is equivalent to a resource collaboration map within the cluster, clearly indicating which vehicles have complementary relationships and which have competitive relationships. Finally, based on this graph, the multi-agent reinforcement learning algorithm can output the globally optimal collaborative solution. The significant advancement of this design lies in integrating joint modeling of resource constraints with relational reasoning on graph structures, making global planning no longer blind.

[0050] In one embodiment, after the step of determining whether the conflict triggering conditions are met between adjacent vehicles, the method includes:

[0051] Obtain real-time status information of nearby vehicles and local road topology information, wherein the local road topology information includes road segment width constraints and intersection location identifiers;

[0052] The position and speed of each vehicle are determined based on the real-time status information;

[0053] Combining the position, speed, and road width constraints of each vehicle, it is determined whether there is a spatial competition relationship between adjacent vehicles. Specifically, when the spatiotemporal overlap of the predicted driving trajectories of multiple vehicles at the same road segment or intersection exceeds a preset threshold, it is determined that there is a spatial competition relationship.

[0054] When a spatial competition relationship is determined to exist, the conflict triggering condition is determined to be met.

[0055] Based on the determination that the conflict triggering conditions are met, the passage urgency of each vehicle is calculated according to the remaining battery power and current cargo load of each vehicle. The passage urgency is negatively correlated with the remaining battery power and positively correlated with the current cargo load.

[0056] Based on the urgency of passage and the right-of-way allocation order determined by the intersection location identifier, a collaborative interaction strategy including speed adjustment instructions or yielding actions is generated.

[0057] As mentioned above, in step one, when two or more vehicles are close together, simply knowing their current positions is insufficient. To accurately determine the risk of collision, the system first needs to collect real-time status information of nearby vehicles, mainly including their current position, speed, and direction of travel. Simultaneously, it needs to obtain local road topology information from a high-precision map, with two crucial data points being road segment width constraints and intersection location markers. Road segment width constraints reflect the space allowable for vehicles to pass side-by-side; for example, the passable space differs greatly between a narrow two-lane road and a wide four-lane road. Intersection location markers indicate whether there are crossroads, T-junctions, or intersections without traffic lights ahead, which are often high-risk areas for collisions. By acquiring these two types of information, the system can use the actual physical characteristics of the road as the basis for judgment, rather than arbitrarily estimating the risk. Step two involves parsing and calibrating the raw data received from nearby vehicles to accurately calculate each vehicle's precise coordinates, speed, and acceleration direction on the map at the current moment. This step, seemingly simple, actually requires handling errors caused by sensor latency and communication jitter. Kalman filtering or similar prediction algorithms are typically used to smooth the data, making subsequent trajectory predictions more reliable. Step three, after obtaining accurate position, speed, and road width information, involves the system performing the core judgment of spatial competition. Its logic is as follows: First, based on each vehicle's current position and speed, the predicted trajectory for a short period in the future is extrapolated, such as the path three to five seconds later. Then, these predicted trajectories are projected onto the same road segment or intersection area, calculating their degree of spatiotemporal overlap. This overlap depends not only on whether vehicles appear at the same coordinate point simultaneously but also on the actual road width. For example, if two vehicles are traveling side-by-side on a sufficiently wide road, even if their predicted trajectories overlap in position, the vehicles can pass each other without interference because there is sufficient lateral space. Therefore, the system compares the vehicle widths, preset safe lateral distances, and road widths of the two vehicles. Only when the predicted trajectories highly overlap in time and space, and the road width is insufficient for the two vehicles to safely travel side-by-side, is a spatial competition relationship determined. This judgment method is far more accurate than simple geofencing or distance alarms. Step four confirms the existence of spatial competition, determining that the conflict triggering conditions have been met. Step five addresses the unavoidable conflict requiring a decision on priority passage, introducing a passage urgency level. This urgency level is calculated based on two key parameters: the vehicle's remaining battery power and its current cargo load. The design principle is that vehicles with lower remaining battery power have a greater urgent need to pass through congested areas to reduce energy consumption and unnecessary detours, avoiding being stranded due to depleted battery power. Therefore, urgency is negatively correlated with remaining battery power; that is, the lower the battery power, the higher the urgency.Meanwhile, vehicles with higher current loads carry more delivery mission value. If they are forced to wait for a long time, it will not only affect their own mission completion time but may also delay a series of subsequent delivery plans. Therefore, urgency is positively correlated with current load; the higher the load, the higher the urgency. By combining these two factors with certain weights, an urgency score between 0 and 1 can be calculated for each vehicle involved in the conflict. This score reflects a principle based on resource allocation and task priority, rather than a simple first-come, first-served or random yielding. Step six: After determining the urgency, the actual structure of the intersection needs to be considered to determine the final right-of-way allocation order. For example, in a typical crossroads, vehicles going straight usually have higher right-of-way than vehicles turning left, but this default rule can be overridden by urgency. If a vehicle going straight with extremely low remaining battery power and a vehicle turning left with moderate load power approach the intersection simultaneously, the system will prioritize allowing the vehicle with low battery power to pass. When handling scenarios involving merging lanes or ramps, the system prioritizes vehicles based on their urgency, granting priority to vehicles with higher urgency and instructing those with lower urgency to slow down, yield, or temporarily stop. The resulting speed adjustment or yielding command is then sent directly to the control systems of the relevant vehicles via inter-vehicle communication. The vehicles adjust their throttle, brakes, or steering angles accordingly, completing a smooth, coordinated interaction.

[0058] In one embodiment, the step of determining the set of vehicles or tasks to be planned based on the type of the triggering event includes:

[0059] Analyze the type of the triggering event;

[0060] When the type of the triggering event is abnormal vehicle status or task execution obstruction, the vehicle corresponding to the triggering event is marked as an abnormal vehicle, and the current location of the abnormal vehicle and the remaining path of the unfinished task are obtained.

[0061] Centered on the current location of the abnormal vehicle, a spatial region is defined according to a preset spatial distance threshold. Vehicles within the spatial region that are not connected to the abnormal vehicle are removed by using the road topology structure to obtain a candidate set of neighboring vehicles.

[0062] Based on the remaining paths of the unfinished tasks of the abnormal vehicle and the predicted driving trajectory of each vehicle in the candidate set of neighboring vehicles, the overlap degree between the predicted trajectory of each neighboring vehicle and the abnormal vehicle is calculated.

[0063] Based on the calculation results, if the overlap of the predicted trajectory exceeds the preset overlap threshold, then the neighboring vehicle will be added to the set of neighboring vehicles to be planned.

[0064] The abnormal vehicles and the set of neighboring vehicles to be planned are jointly determined as the set of vehicles to be planned;

[0065] The unfinished tasks of the abnormal vehicles, and the tasks in the set of nearby vehicles to be planned that are associated with the unfinished tasks of the abnormal vehicles in terms of pick-up and delivery locations, are jointly determined as the set of tasks to be planned.

[0066] As mentioned above, Step One first determines the specific event triggering the replanning. If a vehicle reports low battery or mechanical failure, it's considered a vehicle status anomaly. If a vehicle is unable to deliver on time due to traffic congestion or road closures, it's considered a task execution obstruction. Although these two situations manifest differently, the subsequent processing logic is similar: both require identifying the abnormal vehicle and other vehicles substantially interfering with its operation. Distinguishing event types allows for different handling strategies for obvious global events like widespread road closures, but this approach focuses primarily on localized anomaly scenarios. Step Two, once a vehicle status anomaly or task execution obstruction is confirmed, marks the vehicle corresponding to the triggering event as an abnormal vehicle. Then, its current precise location and the remaining travel path for each segment of its unfinished delivery tasks are read. The remaining path is not a simple straight line, but an actual planned route composed of a series of time-stamped waypoints. This remaining path information plays a crucial role in subsequent trajectory overlap calculations, as knowing the abnormal vehicle's future route is essential to determining which road segments might conflict with it. Step 3: The system uses the current location of the abnormal vehicle as the center and, according to a preset spatial distance threshold, such as 200 meters or 500 meters, delineates a circular or rectangular spatial region outwards. Other vehicles falling within this region are potentially affected neighboring vehicles. However, delineating the region solely based on Euclidean distance has a significant drawback: some vehicles, although very close in a straight line—for example, located on opposite sides of a river without a bridge connection, or completely separated by an elevated road—cannot actually reach the abnormal vehicle. Blindly including these vehicles in the replanning set would waste computational resources. Therefore, this scheme further utilizes road topology, i.e., the actual road network connections on the map, to eliminate vehicles that are not connected to the abnormal vehicle by any road. Only vehicles that can reach the abnormal vehicle's location through the road network are retained in the neighboring vehicle candidate set. This filtering makes subsequent calculations more focused, avoiding wasting time on physically unrelated vehicles. Step 4: After obtaining the neighboring vehicle candidate set, it is necessary to determine which of these candidate vehicles are truly likely to have path conflicts or task coupling with the abnormal vehicle. The basis for this determination is the predicted trajectory overlap. The system predicts the trajectory of each vehicle in the candidate set for a period of time in the future, typically 30 to 60 seconds, based on its current location, speed, and navigation path. It also uses the remaining paths of abnormal vehicles with unfinished tasks as their future trajectories. Then, it calculates the degree of spatiotemporal overlap between the two predicted trajectories on the same road segment or intersection. Spatiotemporal overlap means that the two vehicles not only spatially pass through the same area, but also that their time windows for arriving in that area overlap.This degree of overlap can be represented by a value between 0 and 1, with a higher value indicating a greater likelihood of interference between the two vehicles. Conventional methods may ignore the time dimension, leading to the classification of vehicles passing through the same road segment with a long interval between them as conflicts. This solution, however, significantly reduces the false alarm rate through a dual spatiotemporal assessment. Step five sets a preset overlap threshold, such as 0.6 or 0.7. If the overlap between the predicted trajectory of a neighboring vehicle and the abnormal vehicle exceeds this threshold, it indicates that the vehicle is highly likely to be affected by the abnormal vehicle. For example, it may be following behind the abnormal vehicle, about to meet it at an intersection, or its travel path may highly overlap with the remaining delivery route of the abnormal vehicle. These vehicles are added to the set of neighboring vehicles to be planned. Conversely, vehicles with an overlap below the threshold are excluded and do not participate in this replanning. Through threshold control, the system can flexibly adjust the aggressiveness of the replanning according to actual operational needs: a lower threshold will include more vehicles, resulting in a more comprehensive replanning but with a larger computational load; a higher threshold will result in a lighter replanning but may miss some edge effects. Step six merges the abnormal vehicle itself and the set of neighboring vehicles to be planned obtained in the previous step, forming a unified set of vehicles to be planned. In other words, this replanning not only rearranges the tasks and paths of the abnormal vehicle but also adjusts the plans of neighboring vehicles whose trajectories highly overlap with itss. This avoids a situation where only the abnormal vehicle is adjusted while the surrounding vehicles remain unchanged, resulting in new conflicts between the abnormal vehicle's new path and neighboring vehicles after replanning. Step seven, after determining which vehicles need replanning, also needs to determine which tasks on these vehicles need to be rearranged. This scheme employs a task association-based filtering strategy. For the abnormal vehicle itself, all its unfinished tasks obviously need to be included in the set of tasks to be planned. For other vehicles in the set of neighboring vehicles to be planned, not all tasks need to be recalculated; only those tasks that have a pickup / delivery location association with the abnormal vehicle's unfinished tasks are extracted. Pickup / delivery location association is a common form of task coupling. For example, vehicle A's last delivery point happens to be vehicle B's next pickup point, or the tasks of two vehicles exchange goods at the same transfer station. This association can be understood as a spatial connection between tasks. By replanning only related tasks, rather than clearing the entire task list of neighboring vehicles, we can maximize the preservation of independent tasks that are not disturbed, and reduce the impact of replanning on overall operational efficiency.

[0067] In one embodiment, the step of determining the set of vehicles or tasks to be planned based on the type of the triggering event further includes:

[0068] Analyze the type of the triggering event;

[0069] When the preset trigger event is receiving a new delivery order, the order task information is extracted from the new delivery order. The order task information includes the task location, time window constraints, and task weight.

[0070] The operating status and existing task information of each vehicle are obtained from the collaborative delivery data. The operating status includes the current location, remaining energy and remaining capacity. The existing task information includes the execution time window of each task.

[0071] Based on the path between each vehicle's current location and the task location, estimate the energy and time required for each vehicle to execute the new delivery order;

[0072] Based on the estimation results, combined with the remaining energy, remaining capacity of each vehicle and the time window of the existing task, vehicles that simultaneously meet the energy constraint, capacity constraint and time window constraint are selected as the candidate vehicle set.

[0073] From the candidate vehicle set, the vehicle with the lowest overall weight of the current tasks is selected as the target vehicle. The overall weight is determined based on the task weight and number of tasks of each vehicle.

[0074] The target vehicles are identified as the set of vehicles to be planned, and the existing tasks of the target vehicles and the new delivery orders are jointly identified as the set of tasks to be planned.

[0075] As mentioned above, when a new delivery order is received, how to select the most suitable vehicle from the cluster to accept the order and determine the set of tasks that need to be re-planned? This solution incorporates actual energy consumption, physical capacity limitations, and strict time window constraints into the selection criteria, and introduces a task priority evaluation method based on comprehensive weights. This makes order allocation no longer simply pursue the closest distance or lightest load, but rather more closely reflects the multi-objective optimization needs of actual operations. Specifically: Step 1 first determines whether the event triggering the replanning belongs to a new delivery order. If so, it enters the dedicated processing flow defined in this solution. This step clarifies the applicable scenario of this solution, distinguishing it from other types of events such as vehicle malfunctions or changes in road conditions, ensuring that each event has targeted processing logic. Step 2: Each newly arrived delivery order carries several key pieces of information. The order task information is extracted from this information, mainly including the task location, time window constraints, and task weight. The task location includes the specific coordinates of the pickup and delivery points; the time window constraint specifies the customer's expected pickup and delivery time ranges, exceeding which may lead to customer dissatisfaction or additional delay costs; the task weight reflects the urgency or commercial value of the order, for example, expedited orders have higher weights and regular orders have lower weights. This information forms the basis for all subsequent decisions, and the absence of any one of these can lead to unreasonable allocation results. Step three: The system reads the real-time operating status and existing task information of all vehicles in the cluster from the pre-built collaborative delivery data. The operating status includes the vehicle's current location, remaining energy, and remaining capacity. Remaining energy generally refers to the remaining battery power, and remaining capacity refers to how much cargo the vehicle can still carry. Existing task information includes the execution time windows of each task that each vehicle has committed to performing, that is, the planned start and end times of each task. This data depicts the current resource distribution pattern of the cluster, providing an objective basis for determining which vehicle is suitable to take on new orders. Step Four: For each vehicle in the cluster, the system estimates the minimum energy required to complete the order and the shortest travel time based on its current location, distance to the pickup and delivery points of the new order, road network conditions, and the vehicle's average energy consumption model. Energy estimation considers the impact of the vehicle's current load on future energy consumption, as heavier vehicles consume significantly more electricity than lighter vehicles. Time estimation considers factors such as speed limits, traffic light waiting times, and real-time traffic flow. This step allows the system to determine in advance whether each vehicle's remaining energy is sufficient to support the entire order and whether there is enough time to complete the delivery within the order's time window. Step Five: The system compares the estimated required energy with each vehicle's remaining energy. Only when the remaining energy exceeds the required energy and a certain safety margin is the vehicle satisfied with the energy constraint. Simultaneously, the vehicle's remaining capacity must be greater than or equal to the volume or weight of the goods in the new order; otherwise, loading is impossible—this is the capacity constraint.Furthermore, the time window for adding a new order cannot severely conflict with the time windows of the vehicle's existing tasks. For example, the pickup time must be after the existing task is completed and still within the order's pickup time window, and the delivery time cannot be later than the order's delivery deadline. This is a time window constraint. Only vehicles that simultaneously meet these three constraints can enter the candidate vehicle set. This triple screening mechanism is much stricter than common methods that only consider distance or a single factor, eliminating vehicles that seem close but are actually unable to complete the task from the source of physical feasibility. Step Six: After the above screening, the remaining vehicles in the candidate vehicle set are all capable of completing the new order. However, further comparison is needed to determine which vehicle should receive the order. This solution introduces the concept of a comprehensive weight. Each task in each vehicle's current existing task set has a task weight, representing the importance or urgency of the task. The system combines these weights according to certain rules, such as calculating the average or highest weight of existing tasks, while also considering the number of existing tasks. If a vehicle already has many high-weight tasks, it means its workload is already heavy. Adding a new order to it may lead to a decline in the quality of all task completion. Therefore, the overall weight is generally defined as positively correlated with the weight of existing tasks and negatively correlated with the remaining available time, or simply represented by a function of the total weight of existing tasks multiplied by the number of tasks. The system selects the vehicle with the lowest overall weight of existing tasks from the candidate set as the target vehicle. The basic idea behind this is to prioritize vehicles with lighter workloads and lower priority tasks to take on new orders, thereby achieving load balancing and a smooth distribution of task priorities within the cluster. For example, a vehicle already carrying three regular deliveries is more suitable to take on a new regular order than a vehicle currently carrying two express deliveries, because express deliveries have greater time pressure and should not be burdened further. Step seven determines the selected target vehicle as the set of vehicles to be planned. That is, only the plan of this one vehicle needs to be readjusted, while the plans of other vehicles remain unchanged. At the same time, the set of tasks to be planned includes two parts: all the unfinished tasks currently existing on the target vehicle, and the newly added delivery orders. The reason for putting existing tasks and new orders together into the set of tasks to be planned is that these two tasks need to be uniformly ordered and aligned in time windows, rather than simply appending the new orders to the end of the task list. The system will then call the global collaborative solution generation module to reorder and plan the paths for these tasks of the target vehicle to obtain an optimal execution order.

[0076] It's worth noting that traditional allocation methods often only consider the straight-line distance between the vehicle's current location and the pickup point, selecting the closest vehicle to accept the order. This can result in a vehicle that, while close, has low battery remaining and cannot return to the charging station after completing the order, or a vehicle that, while close, is already fully loaded with goods and has no space to load the new order. Another common approach is to only consider load balancing, prioritizing the vehicle with the fewest current tasks, ignoring the differences in urgency between different tasks. This can lead to an urgent order being assigned to a vehicle already handling multiple regular orders, causing the urgent order to be delayed due to regular tasks cutting in line. This solution uses a combined selection based on energy, capacity, and time window constraints to ensure, from a physical feasibility standpoint, that the selected vehicle is indeed capable of completing the new order, avoiding situations where the vehicle cannot be executed after allocation. This is the first layer of protection. Then, through a comprehensive weighted comparison, the vehicle with the lightest current load and lowest task priority is selected as the final target, achieving a dual balance of load and priority, rather than simply comparing the number of tasks. Finally, by incorporating existing tasks and new orders for the target vehicle into the replanning process, the global optimal task execution order is ensured, rather than simply adding new orders to the end.

[0077] In one embodiment, the method for generating a global collaborative scheme further includes:

[0078] When generating the global collaboration scheme, a task dependency directed graph is constructed synchronously. Nodes in the task dependency directed graph represent individual delivery tasks, and directed edges represent the sequential execution dependencies between tasks. The dependencies include: the delivery location of the first task is the pickup location of the second task, or the completion time window of the first task must be earlier than the start time window of the second task.

[0079] Generate a chain identifier for the task undertaken by each vehicle. The chain identifier includes the index of the corresponding task chain and the sequence number of the task in the chain.

[0080] As mentioned above, in step one, when the system generates a new global coordination scheme for the cluster, it inevitably includes the allocation and sequencing of a series of delivery tasks. For example, the scheme might arrange for vehicle A to pick up goods from the warehouse and deliver them to a transit station, and vehicle B to pick up goods from the transit station and deliver them to the customer. In this multi-vehicle relay delivery mode, tasks are not isolated from each other, but rather have obvious sequential execution dependencies. This scheme requires that a directed graph of task dependencies be constructed simultaneously while generating the global scheme. Each node in this graph represents an independent delivery task, such as a single transport from a pickup point to a delivery point. The direction of the directed edges in the graph is used to represent the dependencies between tasks. Specifically, the dependencies include at least two typical cases. The first is spatial relay dependency, where the delivery location of the first task is exactly the pickup location of the second task. In this case, the second task can only begin after the first task is successfully completed; otherwise, there is no goods to pick up. For example, vehicle A delivers a batch of packages to a regional distribution center, and vehicle B's task is to pick up the same batch of packages from this distribution center and deliver them to the end customer. There is a directed edge between these two tasks, pointing from A to B. The second type is temporal dependency, where the completion window of the first task must be earlier than the start window of the second task. In some cases, two tasks do not have a direct spatial relationship for pickup and delivery, but due to customer requirements or operational strategies, one task must be started only after the other task is completely completed. A directed edge is also needed in this case. By constructing such a directed graph, the system, for the first time, connects the previously scattered tasks across the various vehicle task lists into a complete task chain according to actual business logic. This graph does not rely on any verbal agreement or manual annotation but is directly derived from the inherent attributes of the tasks. This allows the system to immediately identify which tasks belong to the same batch, the same customer order, or the same transportation relay sequence, rather than treating each task as an independent entity. Step two: After obtaining the task dependency directed graph, the next step is to generate a chain of identifiers for the tasks undertaken by each vehicle. Chain identifiers refer to two key pieces of information: the index of the task chain to which the task belongs, and the task's sequence number within that chain. A task chain is a continuous sequence of tasks linked together by directed edges, where each task is spatially or temporally dependent on the tasks preceding and following it. The task chain index assigns a unique number to each chain, allowing the system to quickly locate all tasks belonging to the same chain. The task's sequence number within the chain indicates which link in the chain it belongs to; for example, whether a task is the first segment of transportation before a transit station or the second segment of delivery after a transit station. These chain identifiers are ultimately stored in the metadata accompanying each vehicle's task list and do not affect the vehicle's navigation logic during normal task execution. However, their value becomes immediately apparent when dealing with abnormal situations.For example, when a vehicle breaks down near a transfer station, the system only needs to check the vehicle's task chain identifier to immediately know which other vehicles are performing other tasks on the same chain, and to know which part of the chain these vehicles are in, so as to make a targeted decision on whether to adjust the plans of those vehicles.

[0081] In one embodiment, determining the set of vehicles or tasks to be planned based on the type of the triggering event further includes:

[0082] When the type of the triggering event is abnormal vehicle status or task execution obstruction, and the task undertaken by the abnormal vehicle has a successor task in the task dependency directed graph, all affected task nodes are extracted from the unfinished tasks of the abnormal vehicle.

[0083] Starting from each affected task node, traverse all reachable successor task nodes along the directed edges of the task dependency directed graph, and mark these successor task nodes as chained dependent tasks.

[0084] Add the unfinished tasks of the abnormal vehicles and the chained dependent tasks to the set of tasks to be planned.

[0085] The vehicles that undertake the chain-dependent tasks, along with the abnormal vehicles, will be added to the set of vehicles to be planned.

[0086] When regenerating the global collaboration scheme, all tasks in the set of tasks to be planned are assigned to vehicles in the set of vehicles to be planned, and priority is given to assigning multiple consecutive tasks belonging to the same task chain to the same vehicle.

[0087] As mentioned above, when one link in the task chain fails, how can we prevent the entire chain from collapsing? In traditional scheduling systems, exceptions are often handled piecemeal, with only the problematic vehicle's task being reassigned. This strategy works fine in independent task mode, but it's fatal in task chain mode. Because the dependency of downstream vehicles on upstream vehicles is completely ignored, downstream vehicles will still proceed to the pickup point as planned, only to find there's no cargo to transport, forcing them to return empty or wait indefinitely, ultimately leading to widespread delays or even cancellations of customer orders. This solution fundamentally changes this passive situation by introducing chain-recursive replanning based on a directed graph of task dependencies. When an exception occurs, the system no longer treats the problematic vehicle in isolation, but automatically propagates it forward like dominoes, incorporating all affected subsequent tasks and their corresponding vehicles into the replanning. This whole-chain, integrated approach ensures that all tasks from the point of failure onward are adjusted synchronously, preventing information lag in downstream vehicles.

[0088] Specifically, in Step 1, once the system confirms that the triggered event is a vehicle status anomaly or task execution obstruction, it first checks whether the task currently being undertaken by the abnormal vehicle has a successor task in the task dependency directed graph. A successor task is one that depends on the completion of the current task before it can begin. For example, if vehicle A's battery is about to run out, and its task is to deliver goods to a transfer station, this task is a prerequisite for vehicle B to pick up goods and continue delivery. Therefore, vehicle B's task is a successor task of vehicle A. If the abnormal vehicle's task has no successor task, it means its task chain ends there, and it only needs to process its own task without initiating a chained recursion. Conversely, if a successor task exists, it means the entire task chain is at risk of breaking, and subsequent recursive steps need to be executed. In Step 2, after confirming the existence of a successor task, the system extracts all unfinished task nodes from the abnormal vehicle's list of incomplete tasks. It's important to note that a vehicle may be undertaking multiple tasks simultaneously, some of which may be completed, some in progress, and some not yet started. Only unfinished tasks will affect downstream tasks; therefore, only these task nodes are extracted as the starting point for the recursion. Step three begins with each affected task node extracted in step two. The system recursively traverses downstream along the directed edges representing sequential execution dependencies in the task dependency graph. For each directed edge encountered, the system finds its immediate successor task and continues searching from that successor task until no further successor task is found. All task nodes encountered during the traversal are marked as chained dependent tasks. This process is equivalent to following a chain, finding all subsequent links in the chain without missing a single one.

[0089] refer to Figure 5For example, suppose there's a task chain with five links: Task 1 is executed by vehicle A, Tasks 2 and 3 by vehicle B, and Tasks 4 and 5 by vehicle C. The completion of Task 1 is a prerequisite for starting Task 2, and the completion of Task 2 is a prerequisite for starting Task 3, and so on. If vehicle A malfunctions while executing Task 1, the system will start from Task 1 and sequentially find Tasks 2, 3, 4, and 5, marking them all as chain-dependent tasks. It won't just stop at Task 2, which is directly connected to Task 1. This recursive traversal design is precisely to handle relay delivery scenarios with long chains and multiple layers of dependencies. After step four, the system obtains two sets of tasks: one set consists of the incomplete tasks of the malfunctioning vehicle itself, and the other set consists of all chain-dependent tasks obtained by traversing from these tasks. These two sets of tasks are merged to form the set of tasks to be planned. This means that subsequent replanning will simultaneously adjust the tasks of the malfunctioning vehicle and the related tasks on every vehicle downstream in the chain, rather than just adjusting the malfunctioning vehicle itself. This avoids the awkward situation where downstream vehicles continue to run aimlessly according to the old plan. Step 5: Since the task set to be planned includes tasks from downstream vehicles, these downstream vehicles themselves should also be included in the replanning scope. The system identifies the vehicle responsible for each chain-dependent task and adds these vehicles, along with the abnormal vehicle, to the vehicle set to be planned. This step ensures that all vehicles potentially affected by abnormal events can adjust their plans synchronously, avoiding inconsistencies caused by fragmented processing. Step 6: After completing the above filtering, the system will re-invoke the method for generating a global coordination scheme, assigning all tasks in the task set to be planned to the vehicles in the vehicle set to be planned. Unlike ordinary allocation strategies, this scheme explicitly requires prioritizing the allocation of multiple consecutive tasks belonging to the same task chain to the same vehicle. For example, in a task chain containing five stages, if the task of an abnormal vehicle is interrupted, the system will, during reassignment, try to package stages 2, 3, 4, and even 5 and assign them to the same replacement vehicle, rather than breaking them down into multiple fragments and assigning them to different vehicles. There are two important considerations behind this priority allocation principle. First, assigning multiple consecutive tasks to the same vehicle can save the original multiple handover stages, reducing the time loss and uncertainty caused by transfers. Second, the more handover points there are, the greater the probability of subsequent anomalies. Concentrating the chain onto one vehicle can significantly reduce the risk of the entire chain breaking. Of course, priority does not equal mandatory. If the same vehicle cannot accommodate all continuous tasks in terms of battery power, cargo capacity, or time window, the system can still split it into two vehicles as appropriate. However, when splitting, it will try to choose links that naturally have buffers in time or space for division.

[0090] Reference Figure 3This application also provides a real-time planning system for collaborative delivery routes of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, including:

[0091] The first acquisition module 1 is used to acquire the real-time status information of each unmanned logistics vehicle in the cluster.

[0092] Module 2 is used to construct collaborative delivery data for global planning based on the real-time status information;

[0093] The first generation module 3 is used to generate a global collaborative solution in response to a preset trigger event and in combination with the collaborative delivery data;

[0094] The second acquisition module 4 is used to acquire the local environmental perception information of each vehicle and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently.

[0095] The judgment module 5 is used to determine whether the conflict triggering conditions are met between neighboring vehicles based on the local execution strategy of each vehicle.

[0096] The second generation module 6 is used to generate a collaborative interaction strategy to resolve the conflict based on the determination result if the conflict triggering conditions are met between adjacent vehicles.

[0097] Output module 7 is used to output the real-time delivery route planning results of each vehicle based on the execution results of the collaborative interaction strategy.

[0098] As described above, it is understood that each component of the real-time collaborative delivery path planning system for unmanned logistics vehicle clusters based on multi-agent reinforcement learning proposed in this application can realize the function of any of the real-time collaborative delivery path planning methods for unmanned logistics vehicle clusters based on multi-agent reinforcement learning as described above, and the specific structure will not be repeated.

[0099] Reference Figure 4 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores monitoring data and other data. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a real-time path planning method for collaborative delivery of unmanned logistics vehicle clusters based on multi-agent reinforcement learning.

[0100] The processor described above executes the real-time collaborative delivery path planning method for unmanned logistics vehicle clusters based on multi-agent reinforcement learning, including: acquiring real-time status information of each unmanned logistics vehicle in the cluster; constructing collaborative delivery data for global planning based on the real-time status information; generating a global collaborative scheme in response to a preset trigger event and in combination with the collaborative delivery data; acquiring local environmental perception information of each vehicle and generating a corresponding local execution strategy for each vehicle independently based on the local environmental perception information of each vehicle and the global collaborative scheme; determining whether neighboring vehicles meet the conflict triggering conditions based on the local execution strategy of each vehicle; generating a collaborative interaction strategy to resolve the conflict based on the determination result if neighboring vehicles meet the conflict triggering conditions; and outputting the real-time delivery path planning result of each vehicle based on the execution result of the collaborative interaction strategy.

[0101] One embodiment of this application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements a real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning. The method includes the following steps: acquiring real-time status information of each unmanned logistics vehicle in the cluster; constructing collaborative delivery data for global planning based on the real-time status information; generating a global collaborative scheme in response to a preset trigger event, combined with the collaborative delivery data; acquiring local environmental perception information of each vehicle, and generating a corresponding local execution strategy for each vehicle independently, combined with the local environmental perception information of each vehicle and the global collaborative scheme; determining whether neighboring vehicles meet conflict triggering conditions based on the local execution strategy of each vehicle; generating a collaborative interaction strategy for resolving the conflict based on the determination result if neighboring vehicles meet the conflict triggering conditions; and outputting the real-time delivery path planning result of each vehicle based on the execution result of the collaborative interaction strategy.

[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0104] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A real-time planning method for collaborative delivery routes of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, characterized in that, The method includes: Real-time status information of each unmanned logistics vehicle in the cluster; Based on the real-time status information, construct collaborative delivery data for global planning; In response to a preset trigger event, a global collaborative solution is generated by combining the collaborative delivery data. Acquire local environmental perception information of each vehicle, and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently. Based on the local execution strategy of each vehicle, determine whether the conflict triggering conditions are met between neighboring vehicles; If the conflict triggering conditions are met between adjacent vehicles, a collaborative interaction strategy for resolving the conflict is generated based on the determination result. Based on the execution results of the aforementioned collaborative interaction strategy, the real-time delivery route planning results for each vehicle are output.

2. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 1, characterized in that, The steps for generating a global collaborative solution in response to a preset trigger event and in conjunction with the collaborative delivery data include: Obtain the preset triggering event, and determine the set of vehicles or tasks to be planned based on the type of the triggering event; Extract the remaining battery power, current cargo load, and list of unfinished tasks for each vehicle from the collaborative delivery data; Based on the extracted results, a joint resource constraint vector is constructed; The graph attention network is invoked, and through the graph attention network, the joint resource constraint vector is used as the node feature, and the distance between vehicles and the task overlap are used as the edge weights to generate a resource dependency graph between vehicles. A multi-agent reinforcement learning algorithm is invoked, and the resource dependency graph is used as the basis for collaborative decision-making to generate the global collaborative scheme for the set of vehicles or task sets to be planned.

3. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 1, characterized in that, After the step of determining whether the conflict triggering conditions are met between adjacent vehicles, the method includes: Obtain real-time status information of nearby vehicles and local road topology information, wherein the local road topology information includes road segment width constraints and intersection location identifiers; The position and speed of each vehicle are determined based on the real-time status information; Combining the position, speed, and road width constraints of each vehicle, it is determined whether there is a spatial competition relationship between adjacent vehicles. Specifically, when the spatiotemporal overlap of the predicted driving trajectories of multiple vehicles at the same road segment or intersection exceeds a preset threshold, it is determined that there is a spatial competition relationship. When a spatial competition relationship is determined to exist, the conflict triggering condition is determined to be met. Based on the determination that the conflict triggering conditions are met, the passage urgency of each vehicle is calculated according to the remaining battery power and current cargo load of each vehicle. The passage urgency is negatively correlated with the remaining battery power and positively correlated with the current cargo load. Based on the urgency of passage and the right-of-way allocation order determined by the intersection location identifier, a collaborative interaction strategy including speed adjustment instructions or yielding actions is generated.

4. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 2, characterized in that, The step of determining the set of vehicles or tasks to be planned based on the type of the triggering event includes: Analyze the type of the triggering event; When the type of the triggering event is abnormal vehicle status or task execution obstruction, the vehicle corresponding to the triggering event is marked as an abnormal vehicle, and the current location of the abnormal vehicle and the remaining path of the unfinished task are obtained. Centered on the current location of the abnormal vehicle, a spatial region is defined according to a preset spatial distance threshold. Vehicles within the spatial region that are not connected to the abnormal vehicle are removed by using the road topology structure to obtain a candidate set of neighboring vehicles. Based on the remaining paths of the unfinished tasks of the abnormal vehicle and the predicted driving trajectory of each vehicle in the candidate set of neighboring vehicles, the overlap degree between the predicted trajectory of each neighboring vehicle and the abnormal vehicle is calculated. Based on the calculation results, if the overlap of the predicted trajectory exceeds the preset overlap threshold, then the neighboring vehicle will be added to the set of neighboring vehicles to be planned. The abnormal vehicles and the set of neighboring vehicles to be planned are jointly determined as the set of vehicles to be planned; The unfinished tasks of the abnormal vehicles, and the tasks in the set of nearby vehicles to be planned that are associated with the unfinished tasks of the abnormal vehicles in terms of pick-up and delivery locations, are jointly determined as the set of tasks to be planned.

5. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 4, characterized in that, The step of determining the set of vehicles or tasks to be planned based on the type of the triggering event further includes: Analyze the type of the triggering event; When the preset trigger event is receiving a new delivery order, the order task information is extracted from the new delivery order. The order task information includes the task location, time window constraints, and task weight. The operating status and existing task information of each vehicle are obtained from the collaborative delivery data. The operating status includes the current location, remaining energy and remaining capacity. The existing task information includes the execution time window of each task. Based on the path between each vehicle's current location and the task location, estimate the energy and time required for each vehicle to execute the new delivery order; Based on the estimation results, combined with the remaining energy, remaining capacity of each vehicle and the time window of the existing task, vehicles that simultaneously meet the energy constraint, capacity constraint and time window constraint are selected as the candidate vehicle set. From the candidate vehicle set, the vehicle with the lowest overall weight of the current tasks is selected as the target vehicle. The overall weight is determined based on the task weight and number of tasks of each vehicle. The target vehicles are identified as the set of vehicles to be planned, and the existing tasks of the target vehicles and the new delivery orders are jointly identified as the set of tasks to be planned.

6. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 2, characterized in that, The method for generating a global collaborative scheme further includes: When generating the global collaboration scheme, a task dependency directed graph is constructed synchronously. Nodes in the task dependency directed graph represent individual delivery tasks, and directed edges represent the sequential execution dependencies between tasks. The dependencies include: the delivery location of the first task is the pickup location of the second task, or the completion time window of the first task must be earlier than the start time window of the second task. Generate a chain identifier for the task undertaken by each vehicle. The chain identifier includes the index of the corresponding task chain and the sequence number of the task in the chain.

7. The real-time planning method for collaborative delivery paths of unmanned logistics vehicle clusters based on multi-agent reinforcement learning according to claim 6, characterized in that, The step of determining the set of vehicles or tasks to be planned based on the type of the triggering event further includes: When the type of the triggering event is abnormal vehicle status or task execution obstruction, and the task undertaken by the abnormal vehicle has a successor task in the task dependency directed graph, all affected task nodes are extracted from the unfinished tasks of the abnormal vehicle. Starting from each affected task node, traverse all reachable successor task nodes along the directed edges of the task dependency directed graph, and mark these successor task nodes as chained dependent tasks. Add the unfinished tasks of the abnormal vehicles and the chained dependent tasks to the set of tasks to be planned. The vehicles that undertake the chain-dependent tasks, along with the abnormal vehicles, will be added to the set of vehicles to be planned. When regenerating the global collaboration scheme, all tasks in the set of tasks to be planned are assigned to vehicles in the set of vehicles to be planned, and priority is given to assigning multiple consecutive tasks belonging to the same task chain to the same vehicle.

8. A real-time planning system for collaborative delivery routes of unmanned logistics vehicle clusters based on multi-agent reinforcement learning, characterized in that, include: The first acquisition module is used to acquire the real-time status information of each unmanned logistics vehicle in the cluster. A construction module is used to construct collaborative delivery data for global planning based on the real-time status information; The first generation module is used to generate a global collaborative solution in response to a preset trigger event and in combination with the collaborative delivery data. The second acquisition module is used to acquire the local environmental perception information of each vehicle, and combine the local environmental perception information of each vehicle with the global coordination scheme to generate a corresponding local execution strategy for each vehicle independently. The judgment module is used to determine whether the conflict triggering conditions are met between neighboring vehicles based on the local execution strategy of each vehicle. The second generation module is used to generate a collaborative interaction strategy to resolve the conflict based on the determination result if the conflict triggering conditions are met between adjacent vehicles. The output module is used to output the real-time delivery route planning results of each vehicle based on the execution results of the collaborative interaction strategy.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images