An unmanned vehicle-unmanned aerial vehicle heterogeneous collaborative distribution method for dynamic task requirements

Abstract

The application belongs to the technical field of unmanned vehicle-unmanned aerial vehicle heterogeneous collaborative distribution. The application provides a dynamic task demand-oriented unmanned vehicle-unmanned aerial vehicle heterogeneous collaborative distribution method. The embodiment of the disclosure predicts the package demand of each distribution area in the future by using historical end distribution data, regional graph structure and time sequence characteristics; the unmanned vehicle and the unmanned aerial vehicle are modeled respectively, so that the two are differentiated in terms of movement mode, load capacity, energy consumption model, service radius, weather sensitivity and action space; weather factors such as wind speed, wind direction, rainfall and temperature are introduced into energy consumption estimation, road passing time, unmanned aerial vehicle range and action feasibility judgment; a heterogeneous multi-agent reinforcement learning method is used to train the unmanned vehicle strategy and the unmanned aerial vehicle strategy respectively, and collaborative distribution is realized in the framework of centralized training and decentralized execution; in the online running stage, dynamic rescheduling is carried out according to real-time orders, weather changes and vehicle states, so as to realize continuous collaborative distribution of unmanned vehicles and unmanned aerial vehicles.

Description

Technical Field

[0001] This disclosure relates to the field of unmanned vehicle-drone heterogeneous collaborative delivery technology, and in particular to an unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic mission requirements. Background Technology

[0002] With the development of instant delivery, community group buying, e-commerce express delivery, and urban last-mile logistics, last-mile delivery systems face challenges such as highly uneven spatial and temporal distribution of orders, dynamic changes in order arrival, stringent service timeliness requirements, and frequent road congestion and weather disturbances. Traditional delivery methods relying primarily on human couriers or single vehicles struggle to simultaneously meet the requirements of high efficiency, low cost, and high robustness. Unmanned vehicles (UAVs) offer advantages such as large payload capacity, stable operation along road networks, and the ability to serve as mobile supply stations or mobile warehousing nodes; drones offer advantages such as high flight speed, the ability to traverse congested roads, and suitability for short-distance, high-efficiency last-mile delivery. Collaborating UAVs and drones for last-mile delivery can create a collaborative model of "ground mobile warehousing / supply + rapid aerial last-mile delivery," thereby improving delivery efficiency and coverage. Furthermore, urban delivery demand exhibits significant spatiotemporal correlation; the number of packages in different areas and at different times varies depending on weekdays, holidays, business district attributes, weather, historical orders, and regional connectivity. Therefore, spatiotemporal graph neural networks can be used to predict future regional demand, providing forward-looking information for vehicle pre-deployment, task allocation, and route scheduling. Multi-agent reinforcement learning is suitable for handling multi-vehicle, multi-task, dynamic arrival, and continuous decision-making problems. However, in the scenario of autonomous vehicle and drone collaborative delivery, different agents have significant differences in movement mode, load capacity, battery capacity, reachable space, weather sensitivity, service actions, and safety constraints, which is a typical heterogeneous multi-agent collaborative problem.

[0003] Existing last-mile delivery and unmanned vehicle-drone (V2V) collaborative scheduling technologies still have the following shortcomings: First, they mostly focus on static path planning or deterministic vehicle path problems, often pre-fixing the order set, customer location, number of vehicles, and distance matrix, and then using methods such as mixed integer programming, genetic algorithms, neighborhood search, or insertion heuristics to solve the problem. When faced with real-time new orders, order cancellations, regional demand fluctuations, and weather changes during operation, they usually need to be re-optimized, resulting in high computational costs and insufficient real-time performance. Second, they do not adequately characterize the heterogeneity of V2V and UAVs. Some methods abstract vehicles or delivery resources into homogeneous nodes or distinguish them only by a few parameters such as speed and load, failing to systematically reflect the differences between V2V and UAVs in the state space, action space, dynamic constraints, energy consumption model, service radius, take-off / landing / meeting constraints, and task handover constraints. This leads to the strategy failing to reflect the differences between V2V and UAVs in the real delivery cycle. The following issues exist: First, the feasibility of the method is insufficient. Second, there is a lack of real-time modeling and safety constraints for weather disturbances. Existing methods often use fixed speed or fixed energy consumption models, making it difficult to dynamically adjust the drone deployment, return battery threshold, rendezvous point selection, and task allocation strategies based on factors such as wind speed, wind direction, rainfall, temperature, road congestion, and road traffic conditions. Third, demand forecasting and scheduling decisions are disconnected. Existing delivery systems typically first predict parcel volume and then use an independent scheduling module for route planning. Prediction errors and demand uncertainties are not fully absorbed by the scheduling strategy. Fourth, homogeneous multi-agent reinforcement learning struggles to handle heterogeneous vehicle-machine collaboration. Common methods rely on parameter sharing or similar action spaces, which are suitable for collaboration between vehicles of the same type. However, when unmanned vehicles and drones differ significantly in action meaning, physical constraints, and service capabilities, parameter sharing may lead to insufficient strategy expression, unstable training, and decreased collaboration efficiency.

[0004] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0005] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0006] The purpose of this disclosure is to provide a heterogeneous collaborative delivery method for unmanned vehicles and drones oriented towards dynamic task requirements, thereby overcoming, to at least to some extent, one or more problems caused by the limitations and defects of related technologies.

[0007] According to embodiments of this disclosure, a heterogeneous collaborative delivery method for unmanned vehicles and drones oriented towards dynamic task requirements is provided, including: Step S1: Obtain delivery data for the delivery area and construct a delivery area map. Based on the delivery data and the delivery area map, predict the future parcel demand for each delivery area and obtain the demand forecast results. Step S2: Construct a weather-aware, heterogeneous delivery environment model for unmanned vehicles and drones; Step S3: Based on the real-time status of the unmanned vehicle and the drone, the order time window, the load, the battery level, the range, the road accessibility, the no-fly zone, the weather conditions, and the rendezvous conditions, generate a safety action mask for each unmanned vehicle and each drone. Step S4: Treat each unmanned vehicle and each drone as a heterogeneous intelligent agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, train the cooperative scheduling strategy model of the unmanned vehicle and drone to obtain the trained cooperative scheduling strategy model. Step S5: Obtain real-time order, weather, road and vehicle status data, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

[0008] Furthermore, step S1 specifically includes: The system acquires historical orders, real-time orders, regional / AOI data, road network data, weather data, and vehicle status data within the delivery area. It divides continuous time into multiple time slices and aggregates orders according to their region and order generation time to obtain the parcel demand, number of unfulfilled orders, number of completed orders, and corresponding weather and time characteristics for each region in each time slice, in order to generate delivery data. A delivery area map is formed by using delivery areas, AOIs, stations or grids as graph nodes, and spatial proximity, road connectivity, historical order flow correlation or business district association between areas as graph edges. Input delivery data and delivery area maps from several past time periods into the spatiotemporal map prediction model, and output the demand prediction results for each area in several future time periods.

[0009] Furthermore, in step S2, the unmanned vehicle-drone heterogeneous delivery environment model includes an unmanned vehicle model, a drone model, and a weather perception model; among which, The unmanned vehicle model is used to describe functions such as road driving, unmanned vehicle load capacity, unmanned vehicle battery power, loading and unloading, ground delivery, drone refueling, and mobile rendezvous point. The drone model is used to describe takeoff, flight delivery, landing, return, drone battery level, drone payload, range, no-fly zones, and takeoff and landing point constraints. The weather perception model maps wind speed, wind direction, rainfall, temperature, and visibility to UAV energy consumption, flight time, maximum range, return-to-home safety margin, and UAV road travel time and road accessibility, and sets return-to-home safety power constraints.

[0010] Furthermore, step S3 specifically includes: Generate candidate action sets for autonomous vehicles and drones respectively; Based on the constraints of autonomous vehicles and drones, each executable candidate action in the candidate action set is selected to generate a set of safe actions.

[0011] Furthermore, step S3 also includes: Candidate actions for autonomous vehicles include road movement, waiting and parking, ground delivery, loading and unloading packages, traveling to rendezvous points, and recharging drones; Candidate drone actions include takeoff, delivery, return, landing, waiting, and charging. The constraints for the autonomous vehicle are: determining whether the target road node is reachable, whether the remaining battery power is sufficient to complete the movement, whether the remaining capacity meets the loading requirements, whether the estimated arrival time meets the order time window, and whether it can rendezvous or hand over the task with the drone on time; among these, The constraints for drones are: whether the target mission is overloaded, whether the flight path passes through a no-fly zone, whether the remaining battery power meets the requirements for outbound, service, return and safety margins, whether the weather conditions allow takeoff, and whether the return point or rendezvous point is reachable.

[0012] Furthermore, step S4 also includes: Each autonomous vehicle and drone is assigned a separate policy network. A framework of centralized training and distributed execution is adopted. The centralized critic network receives the global state. Autonomous vehicle actors output actions based on their own observations, and drone actors output actions based on their own observations. The action selection of each actor is constrained by the action mask. The cooperative scheduling policy model of each type of agent is updated sequentially according to the heterogeneous agent update method of HAPPO.

[0013] Furthermore, the reward function of the cooperative scheduling strategy model is:

[0014] in, For time Team rewards To receive a reward for completing orders on time, For energy consumption costs, For overtime costs, Costs related to weather risk or low power supply risk, The cost of a failed rendezvous or mission failure, As a reward for resource utilization, As the first weighting coefficient, This is the second weighting coefficient. This is the third weighting coefficient. It is the fourth weighting coefficient. This is the fifth weighting coefficient. This is the sixth weighting coefficient; The policy update objective is represented as:

[0015] in, For intelligent agents The strategy update target, For intelligent agents The strategy network parameters, The probability ratio between the old and new strategies. This is the advantage estimate. For the clipping function, Update the magnitude limit parameter for the strategy. To determine the expected value of the sampling trajectory time step.

[0016] Furthermore, step S5 specifically includes: The trained strategy is applied to the actual or simulated operation process. At the beginning of each scheduling cycle, the current order pool, unmanned vehicle status, drone status, weather status and road status are updated. The demand prediction model is called to obtain the future regional demand. The action mask module is called to generate the current set of actionable actions. The trained collaborative scheduling strategy model is called to output collaborative delivery instructions. Dynamic rescheduling is triggered if new orders arrive, orders are canceled, weather changes occur, roads become inaccessible, the unmanned vehicle or drone runs out of power, the drone is unable to return as planned, the unmanned vehicle and drone are expected to fail to rendezvous, or the risk of order timeout increases during operation.

[0017] Furthermore, the online action selection is as follows:

[0018] in, For time The output of the joint scheduling action, For the completed heterogeneous multi-agent scheduling strategy, Given the current environmental state, Based on future demand forecasts, This is the action mask for all current agents; The risk of order timeout is:

[0019] in, For orders The risk of timeout For orders The estimated completion time, For orders The end time of service.

[0020] According to a second aspect of the present disclosure, an unmanned vehicle-drone heterogeneous collaborative delivery system for dynamic task requirements is provided, characterized in that it includes: The delivery data and demand forecasting module is used to acquire delivery data for delivery areas and construct delivery area maps. Based on the delivery data and delivery area maps, it forecasts the future parcel demand for each delivery area and obtains the demand forecast results. The module for building a heterogeneous delivery environment model for unmanned vehicles and drones is used to construct a weather-aware heterogeneous delivery environment model for unmanned vehicles and drones. The safety constraint and action mask module is used to generate safety action masks for each unmanned vehicle and each unmanned drone based on the real-time status, order time window, load, battery level, range, road accessibility, no-fly zone, weather conditions, and rendezvous conditions. The heterogeneous multi-agent cooperative scheduling module is used to treat each unmanned vehicle and each drone as a heterogeneous agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, it trains the cooperative scheduling strategy model of unmanned vehicles and drones to obtain the trained cooperative scheduling strategy model. The online scheduling and dynamic rescheduling module is used to acquire real-time order, weather, road and vehicle status, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

[0021] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the aforementioned method and system predict future regional parcel demand using a spatiotemporal graph prediction model, and incorporate the prediction results as part of the reinforcement learning scheduling state, enabling the delivery strategy to be forward-looking rather than passively scheduling based solely on current orders. State, action, energy consumption, load, accessibility, and safety constraints are established for both unmanned vehicles and drones, and the policy networks are trained separately using heterogeneous multi-agent reinforcement learning to avoid insufficient policy expression caused by homogeneous parameter sharing. Weather factors such as wind speed, wind direction, rainfall, and temperature are mapped to drone energy consumption, flight feasibility, unmanned vehicle road travel time, and safety constraints, allowing the system to dynamically adjust task allocation and vehicle scheduling under changing weather conditions. Action masks are used to mask inactive actions such as low battery, overload, exceeding range, no-fly zones, unmet rendezvous conditions, and unreachable roads, reducing ineffective exploration during reinforcement learning training and improving online scheduling security. During operation, rescheduling can be performed based on real-time order arrivals, weather changes, and vehicle status updates, making it suitable for handling dynamic last-mile delivery scenarios. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 The diagram illustrates the steps of a heterogeneous collaborative delivery method for unmanned vehicles and drones oriented towards dynamic task requirements in an exemplary embodiment of this disclosure. Figure 2 This diagram illustrates the construction of a delivery area map and demand forecasting in an exemplary embodiment of this disclosure. Figure 3 This diagram illustrates the weather-sensing heterogeneous environment modeling and safety action mask generation in an exemplary embodiment of this disclosure. Figure 4 This diagram illustrates the HAPPO heterogeneous multi-agent cooperative scheduling training in an exemplary embodiment of this disclosure. Figure 5 This diagram illustrates online scheduling and dynamic rescheduling in an exemplary embodiment of this disclosure. Figure 6 This illustration shows a schematic diagram of an unmanned vehicle-drone heterogeneous collaborative delivery system for dynamic mission requirements, as shown in an exemplary embodiment of this disclosure. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0025] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0026] This example implementation provides a heterogeneous collaborative delivery method between unmanned vehicles and drones to meet dynamic task requirements. (Reference) Figure 1 As shown, the method may include: Step S1: Obtain delivery data for the delivery area and construct a delivery area map. Based on the delivery data and the delivery area map, predict the future parcel demand for each delivery area and obtain the demand forecast results. Step S2: Construct a weather-aware, heterogeneous delivery environment model for unmanned vehicles and drones; Step S3: Based on the real-time status of the unmanned vehicle and the drone, the order time window, the load, the battery level, the range, the road accessibility, the no-fly zone, the weather conditions, and the rendezvous conditions, generate a safety action mask for each unmanned vehicle and each drone. Step S4: Treat each unmanned vehicle and each drone as a heterogeneous intelligent agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, train the cooperative scheduling strategy model of the unmanned vehicle and drone to obtain the trained cooperative scheduling strategy model. Step S5: Obtain real-time order, weather, road and vehicle status data, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

[0027] Using the methods described above, a spatiotemporal graph prediction model is employed to predict future regional parcel demand. The prediction results are then incorporated into the reinforcement learning scheduling state, enabling the delivery strategy to be proactive rather than passively scheduling based solely on current orders. State, action, energy consumption, payload, accessibility, and safety constraints are established for both autonomous vehicles and drones. Heterogeneous multi-agent reinforcement learning is used to train the policy networks separately, avoiding insufficient policy expression caused by homogeneous parameter sharing. Weather factors such as wind speed, wind direction, rainfall, and temperature are mapped to drone energy consumption, flight feasibility, autonomous vehicle road travel time, and safety constraints, allowing the system to dynamically adjust task allocation and vehicle scheduling under changing weather conditions. Action masks are used to mask inactive actions such as low battery, overload, exceeding range, no-fly zones, unmet rendezvous conditions, and unreachable roads, reducing ineffective exploration during reinforcement learning training and improving online scheduling security. During operation, rescheduling can be performed based on real-time order arrivals, weather changes, and vehicle status updates, making it suitable for handling dynamic last-mile delivery scenarios.

[0028] Below, we will refer to Figures 1 to 5 The steps of the method described above in this example embodiment will be explained in more detail.

[0029] In one embodiment, the technical solution adopted by this application to solve its technical problem includes the following steps: Step S1: Construct delivery spatiotemporal data and predict future regional demand; This application acquires historical orders, real-time orders, regional divisions, road networks, weather, airspace constraints, and the status data of unmanned vehicles and drones. The above data is cleaned, divided into time slices, aggregated, normalized in coordinates, and encoded with features to construct a delivery area map and a road map. In the delivery area map, nodes represent delivery areas, AOIs (Areas of Interest), stations, or grids, and edges represent spatial proximity, road connectivity, historical order flow correlations, or business district relationships. The road map represents roads accessible to unmanned vehicles, road distances, basic travel times, and road conditions. Based on the delivery area map, a spatiotemporal graph prediction model is used to predict the parcel demand, demand intensity, and / or demand uncertainty for each region within one or more future time windows. In one implementation, the spatiotemporal graph prediction model employs STGCN, capturing temporal changes in demand through temporal convolution and capturing spatial correlations between regions through graph convolution.

[0030] Step S2: Construct a weather-aware unmanned vehicle-drone heterogeneous delivery environment; Based on the data and prediction results obtained in step S1, an unmanned vehicle-drone collaborative delivery environment is constructed. The unmanned vehicle model describes functions such as road travel, load capacity, battery level, loading and unloading, ground delivery, drone refueling, and mobile rendezvous points. The drone model describes takeoff, flight delivery, landing, return, battery level, load capacity, range, no-fly zones, and takeoff / landing point constraints. Simultaneously, a weather perception model is introduced, mapping factors such as wind speed, wind direction, rainfall, temperature, and visibility to drone energy consumption, flight time, maximum range, return safety margin, and unmanned vehicle road travel time and road accessibility. This forms a heterogeneous delivery environment that simultaneously includes an order pool, unmanned vehicles, drones, stations, rendezvous points, charging points, road networks, airspace constraints, and weather disturbances.

[0031] Step S3: Generate a security action mask; Based on the real-time status of autonomous vehicles and drones, order time windows, vehicle load, battery level, drone range, road accessibility, no-fly zones, weather conditions, takeoff and landing conditions, and rendezvous conditions, a safety action mask is generated. For autonomous vehicles, the system determines whether the road target is reachable, whether the battery is sufficient, whether the capacity is adequate, and whether the estimated arrival time meets the mission and rendezvous requirements. For drones, the system determines whether the mission is overloaded, whether the flight path passes through no-fly zones, whether the battery level meets the requirements for outbound, service, return, and safety margins, whether the weather permits takeoff, and whether the rendezvous point is reachable. Any actions that do not meet the above conditions are masked, and the reinforcement learning strategy selects only actions from the set of possible actions.

[0032] Step S4: Train a heterogeneous multi-agent cooperative scheduling strategy; This application treats autonomous vehicles and drones as heterogeneous agents to construct a multi-agent decision-making process. The global state includes the current order pool, future demand forecasts, vehicle location, battery level, load, task status, weather status, road status, airspace constraints, rendezvous point status, and charging point status. Each agent's local observations include its own state, nearby orders, nearby vehicles, local weather, predicted demand intensity, and action masks. A heterogeneous multi-agent reinforcement learning algorithm is used to train a collaborative scheduling strategy. In one implementation, the HAPPO algorithm is used, with separate actor networks for autonomous vehicles and drones, and a centralized critic network is used to evaluate team collaboration benefits during the training phase. The reward function comprehensively considers on-time order completion, timeout penalties, unserved order penalties, autonomous vehicle and drone energy consumption, weather risk, low battery risk, rendezvous failure, and resource utilization. After training, the autonomous vehicle and drone strategies can output collaborative decisions such as road movement, waiting, task allocation, drone dispatch, return to base, charging replenishment, and rendezvous point selection based on their respective observations and action masks.

[0033] Step S5: Perform online scheduling and dynamic rescheduling; During the online operation phase, the system periodically receives real-time order, vehicle status, and weather information, and invokes previous steps to output collaborative delivery instructions for unmanned vehicles and drones. When a new order arrives, weather changes suddenly, roads become inaccessible, drone battery is low, vehicle malfunctions, rendezvous fails, or the risk of order timeout increases, the system updates the status and re-executes predictions, action mask generation, and strategy invocation to achieve dynamic rescheduling.

[0034] This application establishes a complete closed loop, encompassing real-world delivery data modeling, future demand forecasting, construction of heterogeneous environments based on weather perception, safety action constraints, HAPPO heterogeneous collaborative scheduling, and online dynamic rescheduling. This solution enables safe and efficient last-mile collaborative delivery even under conditions of dynamic orders, weather disturbances, and heterogeneous constraints between autonomous vehicles and drones.

[0035] In one specific embodiment, this application aims to construct an unmanned vehicle-drone collaborative delivery scheduling method for dynamic order and weather disturbance scenarios by combining spatiotemporal data modeling technology for last-mile delivery, spatiotemporal map demand prediction technology, unmanned vehicle-drone heterogeneous operation modeling technology, weather perception energy consumption and safety constraint modeling technology, and HAPPO heterogeneous multi-agent reinforcement learning scheduling technology. Through the above approach, this application can output decisions on unmanned vehicle movement, drone dispatch, task allocation, rendezvous point selection, charging replenishment, and dynamic rescheduling under conditions of dynamic order changes, weather condition changes, and differences in vehicle and drone capabilities. Its goal is to improve the on-time completion rate, resource utilization rate, and operational safety of last-mile delivery, while reducing the timeout rate, invalid actions, and delivery energy consumption.

[0036] To achieve the above objectives, this application adopts the following technical solution: The autonomous vehicle-drone heterogeneous collaborative delivery method for dynamic demand and weather disturbances includes the following steps: Step S1: As Figure 2 The diagram shown illustrates the construction of a delivery area map and demand forecasting.

[0037] Constructing delivery spatiotemporal data and predicting future regional demand: First, the system accesses historical orders, real-time orders, regional / AOI data, road network data, weather data, and vehicle status data within a specific delivery city or region, unifying this data to the same spatial region and temporal granularity. Specifically, the system divides continuous time into multiple time slices, aggregating orders according to their region and order creation time, thereby obtaining the parcel demand, number of unfulfilled orders, number of completed orders, and corresponding weather and time characteristics for each region within each time slice. Each order can be represented in the following form:

[0038] in Indicates the order number. Indicates the service location. Indicates the region or AOI. Indicates the order generation time. Indicates the service time window. Indicates the package weight or estimated weight. This indicates order priority. After obtaining the regional time series, the system further constructs a delivery area map. Delivery areas, AOIs, stations, or grids serve as graph nodes, while spatial proximity relationships, road connectivity relationships, historical order flow correlations, or business district association relationships between areas serve as graph edges, forming the delivery area map. Among them, the adjacency matrix It can be generated comprehensively based on regional distance, historical demand relevance, and road connectivity, for example, it can take the following form:

[0039] in, Indicates the similarity of distances between regions. Indicates the correlation of historical demand. Indicates the relationship between roads or services. , , These are the weighting coefficients. After constructing the regional graph, the system inputs the regional node features and adjacency matrix from several past time slices into the spatiotemporal graph prediction model. This model can employ STGCN, extracting demand change trends through temporal convolution and spatial relationships between regions through graph convolution, outputting the parcel demand volume, demand intensity, or demand uncertainty for each region in the next few time slices. These outputs are used to subsequently determine which areas require advance deployment of unmanned vehicles, which areas require advance refueling of drones, and which tasks require priority allocation of delivery resources.

[0040] Step S2: Constructing a Weather-Aware Unmanned Vehicle-Drone Heterogeneous Delivery Environment: This application constructs an unmanned vehicle-drone heterogeneous collaborative delivery environment. This environment does not simply treat unmanned vehicles and drones as the same type of vehicle, but rather describes their respective operating modes, service capabilities, and constraints. Unmanned vehicles move along a road map, suitable for tasks such as heavy-load delivery, ground delivery, mobile warehousing, drone refueling, and drone rendezvous. Drones can take off from stations, unmanned vehicles, or rendezvous points, suitable for short-distance, high-efficiency, low-load last-mile delivery tasks. In this environment, the unmanned vehicle status can include current location, remaining battery power, remaining capacity, loaded packages, current task, bound drone group, and estimated arrival time; the drone status can include current location, remaining battery power, load capacity, current payload, flight status, return point, and current task. The system also integrates stations, rendezvous points, charging points, road networks, airspace constraints, and order pools into the environmental status, enabling subsequent scheduling strategies to simultaneously consider orders, vehicles, roads, weather, and vehicle-drone rendezvous relationships.

[0041] To reflect the impact of weather, the system maps weather factors such as wind speed, wind direction, rainfall, temperature, and visibility into the operating parameters of unmanned vehicles (UAVs) and drones. For example, rainfall or road flooding will reduce the speed of UAVs on roads; strong winds or rainfall will increase the energy consumption per unit distance of drones, shorten their flight range, or directly render some aerial missions unusable. The travel time of the UAV from position i to position j can be simplified as follows:

[0042] in, Indicates the time of the driverless car The estimated time to travel from position i to position j, where d_road(i,j) represents the road distance, v_g represents the base speed of the drone, lambda_traffic(i,j,t) represents the traffic or road traffic coefficient, and lambda_weather_g(t) represents the influence coefficient of weather on the drone's travel speed. When a drone performs a delivery task, it needs to consider the flight distance, package weight, and weather effects simultaneously. Its task energy consumption can be summarized as follows:

[0043] in, Indicates the time of the drone From the starting point Go to order location And return to the return point Task energy consumption Indicates the corresponding flight distance. Indicates the order weight. This indicates the energy consumption or risk associated with wind speed and direction. This indicates the energy consumption or risk associated with rainfall. , , ,

[0044] This represents the energy consumption weighting coefficient. To ensure the safe return of the drone, the system further sets a safe return power constraint, namely, the remaining power of the drone should not be less than the sum of the mission energy consumption and the weather safety margin:

[0045] in, Indicates drone In time The remaining battery power, This indicates the energy consumption required for the drone to perform its current task. Indicates weather conditions The determined safe return-to-home battery capacity is then calculated. The output of this step is a weather-aware autonomous vehicle-drone heterogeneous delivery environment. This environment provides a unified interactive object for subsequent safety action selection and HAPPO strategy training.

[0046] Step S3: As Figure 3 The diagram shown illustrates the modeling of heterogeneous environments for weather perception and the generation of safety action masks.

[0047] Generate a set of feasible actions based on safety constraints: This application generates a set of feasible actions for each unmanned vehicle (UAV) and drone. The core purpose of this step is to eliminate obviously infeasible or unsafe actions before the reinforcement learning strategy makes a decision, avoiding tasks involving low battery, overload, exceeding range, no-fly zones, unreachable roads, inability to meet on time, or failure to meet the order time window. Specifically, the system first generates candidate actions for UAVs and drones. Candidate actions for UAVs include road movement, waiting, ground delivery, loading and unloading packages, heading to the rendezvous point, and recharging the drone; candidate actions for drones include takeoff, delivery, return, landing, waiting, and charging. Subsequently, the system judges the feasibility of each candidate action based on the constraints of different types of agents. For UAVs, the system mainly judges whether the target road node is reachable, whether the remaining battery is sufficient to complete the movement, whether the remaining capacity meets the loading requirements, whether the estimated arrival time meets the order time window, and whether it can meet the drone or complete the task handover on time. For drones, the system primarily determines whether the target mission is overloaded, whether the flight path passes through no-fly zones, whether the remaining battery power meets the requirements for outbound, service, return, and safety margins, whether weather conditions permit takeoff, and whether the return or rendezvous point is reachable. Let A_i be the set of candidate actions for agent i, then the filtered set of safe actions can be summarized as:

[0048] in, Represents intelligent agents In time A set of safe and feasible actions. Represents intelligent agents The set of candidate actions Indicates candidate actions, Indicates the current environmental state. Indicates action In state The system then uses a function to determine whether security and business constraints are met. The system further generates an action mask. The value is 1 if the action is safe and feasible, and 0 otherwise. When the scheduling strategy outputs the action probability, it only selects from actions with an action mask of 1, which can be simplified as follows:

[0049] in, This represents the probability of the security policy after the action mask is applied. Represents the original policy probability. Represents intelligent agents In time Local observations Indicates action The mask value, This represents the normalization factor.

[0050] This step injects physical constraints, security constraints, and business rules into the policy execution process in advance, ensuring that the HAPPO policy makes decisions only within the set of safe actions. The output of this step is the set of possible actions and action masks for each autonomous vehicle and each drone in its current state.

[0051] Step S4: As Figure 4 The diagram shown is a schematic of the HAPPO heterogeneous multi-agent collaborative scheduling training.

[0052] This application employs HAPPO to train a heterogeneous multi-agent cooperative scheduling strategy (i.e., a cooperative scheduling strategy model): Each autonomous vehicle (RV) and each drone is treated as an agent. Based on the future demand forecasts, heterogeneous delivery environment, and safety action masks obtained in the previous steps, a cooperative scheduling strategy for RVs and drones is trained. Due to significant differences between RVs and drones in action meaning, operating space, payload capacity, power constraints, and weather sensitivity, this application does not adopt a completely homogeneous parameter-sharing strategy but instead sets up separate policy networks for RVs and drones. During training, the system adopts a centralized training and distributed execution framework. The centralized critic network receives the global state during the training phase to evaluate the collaborative benefits of the entire vehicle-machine team. RV actors output actions such as road movement, parking, handover, recharging, and ground delivery based on their own observations. Drone actors output actions such as takeoff, delivery, return, waiting, and charging based on their own observations. The action selection of each actor is constrained by the action mask. The global state can include the current order pool, future regional demand forecasts, autonomous vehicle status, drone status, weather status, road status, airspace constraints, rendezvous point status, and charging point status. Local observations are the parts of the global state relevant to the current agent's execution. For example, an autonomous vehicle observes its own battery level, location, load, nearby orders, attached drones, and road status; a drone observes its own battery level, load, local weather, reachable orders, return point, and attached autonomous vehicle or station status. The reward function comprehensively considers delivery efficiency, energy consumption, safety, and collaborative effects, and can be summarized as follows:

[0053] in, Indicates time Team rewards This indicates a reward for completing orders on time. Indicates energy consumption cost, Indicates overtime cost, Indicates costs related to weather risk or low power consumption risk. This indicates the cost of a failed rendezvous or mission failure. Indicates a reward for resource utilization. to These correspond to the weighting coefficients. During training, the system collects the interaction trajectories of multiple agents, calculates the team reward and advantage function, and updates the policies of various agents sequentially according to HAPPO's heterogeneous agent update method. The range of change between old and new policies is limited during policy updates to avoid excessive policy fluctuations during training. The update concept can be simplified as follows:

[0054] in, Represents intelligent agents The strategy update target, Represents intelligent agents The strategy network parameters, This represents the ratio of the probabilities of the new and old strategies. This represents the advantage estimate. This represents the clipping function. This indicates the parameter limiting the policy update magnitude. This represents the expectation of the time step of the sampling trajectory.

[0055] After training, the system acquires an autonomous vehicle actor, a drone actor, and a centralized critic. During the online execution phase, the critic does not need to participate in the actual action selection; the autonomous vehicle and drone output collaborative delivery actions based on their respective observations and action masks.

[0056] Step S5: As Figure 5 The diagram shown illustrates online scheduling and dynamic rescheduling.

[0057] Online execution scheduling and dynamic rescheduling: This application applies the trained strategy to the actual or simulated operation process and periodically outputs scheduling instructions based on real-time orders, weather, road and vehicle status. At the beginning of each scheduling cycle, the system first updates the current order pool, unmanned vehicle status, drone status, weather status, and road status; then it calls the demand prediction model to obtain future regional demand; next, it calls the action mask module to generate the current set of actionable actions; finally, it calls the trained HAPPO strategy to output instructions such as unmanned vehicle movement, drone dispatch, order allocation, rendezvous point selection, return, charging and resupply, and waiting / staying. Online action selection can be summarized as follows:

[0058] in, Indicates time The output of the joint scheduling action, This indicates the heterogeneous multi-agent scheduling strategy after training is complete. Indicates the current environmental state. This indicates the results of future demand forecasts. This represents the action mask for all current agents. After executing scheduling actions, the system updates order completion status, vehicle location, battery level, load, task status, and operational metrics. If events occur during operation, such as new orders arriving, order cancellations, sudden weather changes, road inaccessibility, insufficient battery power for unmanned vehicles or drones, drones failing to return as planned, anticipated rendezvous between unmanned vehicles and drones failing, or increased risk of order timeouts, the system triggers dynamic rescheduling. Rescheduling does not start from scratch; instead, it re-executes demand forecasting, action mask generation, and strategy invocation after updating the current state, enabling the system to quickly adjust delivery plans during operation. Order timeout risk can be simplified as follows:

[0059] in, Indicates order The risk of timeout Indicates order The estimated completion time, Indicates order The system determines the deadline for service. When the risk of exceeding the preset threshold, the system prioritizes reassigning the order or adjusting the rendezvous and delivery sequence of unmanned vehicles and drones. During the online operation phase, the system can record indicators such as completion rate, timeout rate, and total energy consumption. The output of this step is online collaborative delivery instructions and, if necessary, rescheduling instructions, thereby enabling continuous collaborative operation of unmanned vehicles and drones in dynamic delivery scenarios.

[0060] Furthermore, this example embodiment provides a heterogeneous collaborative delivery system for unmanned vehicles and drones, designed to meet dynamic mission requirements. (Reference) Figure 6 As shown, the system may include: The delivery data and demand forecasting module is used to acquire delivery data for delivery areas and construct delivery area maps. Based on the delivery data and delivery area maps, it forecasts the future parcel demand for each delivery area and obtains the demand forecast results. The module for building a heterogeneous delivery environment model for unmanned vehicles and drones is used to construct a weather-aware heterogeneous delivery environment model for unmanned vehicles and drones. The safety constraint and action mask module is used to generate safety action masks for each unmanned vehicle and each unmanned drone based on the real-time status, order time window, load, battery level, range, road accessibility, no-fly zone, weather conditions, and rendezvous conditions. The heterogeneous multi-agent cooperative scheduling module is used to treat each unmanned vehicle and each drone as a heterogeneous agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, it trains the cooperative scheduling strategy model of unmanned vehicles and drones to obtain the trained cooperative scheduling strategy model. The online scheduling and dynamic rescheduling module is used to acquire real-time order, weather, road and vehicle status, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

[0061] In one embodiment, (1) the delivery data and demand forecasting module is used to access historical orders, real-time orders, regional / AOI, road network, weather and vehicle status data, and construct a delivery area map, and use a spatiotemporal map prediction model to predict future regional parcel demand; (2) the unmanned vehicle-drone heterogeneous environment modeling module is used to establish unmanned vehicle models, drone models, weather impact models, energy consumption models, load models, time window models and vehicle-machine collaborative constraint models respectively, forming a delivery environment that can be used for reinforcement learning interaction; (3) the safety constraint and action masking module is used to generate a set of feasible actions for unmanned vehicles and drones based on power, load, range, weather, no-fly zones, road accessibility, order time windows and vehicle-machine meeting conditions, and mask infeasible or unsafe actions; (4) the heterogeneous multi-agent collaborative scheduling module is used to adopt HAPPO The heterogeneous multi-agent reinforcement learning algorithm is used to train the scheduling strategy of unmanned vehicles and drones, and realize task allocation, vehicle movement, drone dispatch, meeting point selection and charging replenishment decision-making; (5) Online scheduling and dynamic rescheduling module, which is used to output scheduling instructions according to real-time orders, weather changes and vehicle status during operation, and to perform dynamic rescheduling when new orders, sudden weather changes, vehicle failures, road inaccessibility or increased risk of timeout occur.

[0062] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0063] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0064] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A heterogeneous collaborative delivery method between unmanned vehicles and unmanned aerial vehicles (UAVs) for dynamic task requirements, characterized in that, include: Step S1: Obtain delivery data for the delivery area and construct a delivery area map. Based on the delivery data and the delivery area map, predict the future parcel demand for each delivery area and obtain the demand forecast results. Step S2: Construct a weather-aware, heterogeneous delivery environment model for unmanned vehicles and drones; Step S3: Based on the real-time status of the unmanned vehicle and the drone, the order time window, the load, the battery level, the range, the road accessibility, the no-fly zone, the weather conditions, and the rendezvous conditions, generate a safety action mask for each unmanned vehicle and each drone. Step S4: Treat each unmanned vehicle and each drone as a heterogeneous intelligent agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, train the cooperative scheduling strategy model of the unmanned vehicle and drone to obtain the trained cooperative scheduling strategy model. Step S5: Obtain real-time order, weather, road and vehicle status, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

2. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 1, characterized in that, Step S1 specifically includes: The system acquires historical orders, real-time orders, regional / AOI data, road network data, weather data, and vehicle status data within the delivery area. It divides continuous time into multiple time slices and aggregates orders according to their region and order generation time to obtain the parcel demand, number of unfulfilled orders, number of completed orders, and corresponding weather and time characteristics for each region in each time slice, in order to generate delivery data. A delivery area map is formed by using delivery areas, AOIs, stations or grids as graph nodes, and spatial proximity, road connectivity, historical order flow correlation or business district association between areas as graph edges. Input delivery data and delivery area maps from several past time periods into the spatiotemporal map prediction model, and output the demand prediction results for each area in several future time periods.

3. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 2, characterized in that, In step S2, the unmanned vehicle-drone heterogeneous delivery environment model includes an unmanned vehicle model, a drone model, and a weather perception model; among them, The unmanned vehicle model is used to describe functions such as road driving, unmanned vehicle load capacity, unmanned vehicle battery power, loading and unloading, ground delivery, drone refueling, and mobile rendezvous point. The drone model is used to describe takeoff, flight delivery, landing, return, drone battery level, drone payload, range, no-fly zones, and takeoff and landing point constraints. The weather perception model maps wind speed, wind direction, rainfall, temperature, and visibility to UAV energy consumption, flight time, maximum range, return-to-home safety margin, and UAV road travel time and road accessibility, and sets return-to-home safety power constraints.

4. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 3, characterized in that, Step S3 specifically includes: Generate candidate action sets for autonomous vehicles and drones respectively; Based on the constraints of autonomous vehicles and drones, each executable candidate action in the candidate action set is selected to generate a set of safe actions.

5. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements according to claim 4, characterized in that, Step S3 also includes: Candidate actions for autonomous vehicles include road movement, waiting and parking, ground delivery, loading and unloading packages, traveling to rendezvous points, and recharging drones; Candidate drone actions include takeoff, delivery, return, landing, waiting, and charging. The constraints for the autonomous vehicle are: determining whether the target road node is reachable, whether the remaining battery power is sufficient to complete the movement, whether the remaining capacity meets the loading requirements, whether the estimated arrival time meets the order time window, and whether it can rendezvous or hand over the task with the drone on time; among these, The constraints for drones are: whether the target mission is overloaded, whether the flight path passes through a no-fly zone, whether the remaining battery power meets the requirements for outbound, service, return and safety margins, whether the weather conditions allow takeoff, and whether the return point or rendezvous point is reachable.

6. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements according to claim 4, characterized in that, Step S4 also includes: Each autonomous vehicle and drone is assigned a separate policy network. A framework of centralized training and distributed execution is adopted. The centralized critic network receives the global state. Autonomous vehicle actors output actions based on their own observations, and drone actors output actions based on their own observations. The action selection of each actor is constrained by the action mask. The cooperative scheduling policy model of each type of agent is updated sequentially according to the heterogeneous agent update method of HAPPO.

7. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 6, characterized in that, The reward function for the collaborative scheduling strategy model is: in, For time Team rewards To receive a reward for completing orders on time, For energy consumption costs, For overtime costs, Costs related to weather risk or low power supply risk, The cost of a failed rendezvous or mission failure, Incentives for resource utilization As the first weighting coefficient, This is the second weighting coefficient. This is the third weighting coefficient. It is the fourth weighting coefficient. This is the fifth weighting coefficient. This is the sixth weighting coefficient; The policy update objective is represented as: in, For intelligent agents The strategy update target, For intelligent agents The strategy network parameters, The probability ratio between the old and new strategies. This is the advantage estimate. For the clipping function, Update the magnitude limit parameter for the strategy. To determine the expected value of the sampling trajectory time step.

8. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 7, characterized in that, Step S5 specifically includes: The trained strategy is applied to the actual or simulated operation process. At the beginning of each scheduling cycle, the current order pool, unmanned vehicle status, drone status, weather status and road status are updated. The demand prediction model is called to obtain the future regional demand. The action mask module is called to generate the current set of actionable actions. The trained collaborative scheduling strategy model is called to output collaborative delivery instructions. Dynamic rescheduling is triggered if new orders arrive, orders are canceled, weather changes occur, roads become inaccessible, the unmanned vehicle or drone runs out of power, the drone is unable to return as planned, the unmanned vehicle and drone are expected to fail to rendezvous, or the risk of order timeout increases during operation.

9. The unmanned vehicle-drone heterogeneous collaborative delivery method for dynamic task requirements as described in claim 8, characterized in that, The online action selection is: in, For time The output of the joint scheduling action, For the completed heterogeneous multi-agent scheduling strategy, Given the current environmental state, Based on future demand forecasts, This is the action mask for all current agents; The risk of order timeout is: in, For orders The risk of timeout For orders The estimated completion time, For orders The end time of service.

10. A heterogeneous collaborative delivery system for unmanned vehicles and drones oriented towards dynamic task requirements, characterized in that: include: The delivery data and demand forecasting module is used to acquire delivery data for delivery areas and construct delivery area maps. Based on the delivery data and delivery area maps, it predicts the future parcel demand for each delivery area and obtains the demand forecasting results. The module for building a heterogeneous delivery environment model for unmanned vehicles and drones is used to construct a weather-aware heterogeneous delivery environment model for unmanned vehicles and drones. The safety constraint and action mask module is used to generate safety action masks for each unmanned vehicle and each drone based on the real-time status, order time window, load, battery level, range, road accessibility, no-fly zone, weather conditions, and rendezvous conditions. The heterogeneous multi-agent cooperative scheduling module is used to treat each unmanned vehicle and each drone as a heterogeneous agent. Based on the demand prediction results, the unmanned vehicle-drone heterogeneous delivery environment model and the safety action mask, it trains the cooperative scheduling strategy model of unmanned vehicles and drones to obtain the trained cooperative scheduling strategy model. The online scheduling and dynamic rescheduling module is used to acquire real-time order, weather, road and vehicle status, and input them into the trained collaborative scheduling strategy model to output scheduling instructions.

Images