Multi-uav and vehicle cooperative regional coverage path planning method and system

Abstract

This invention relates to the field of path planning technology, proposing a method and system for multi-UAV and vehicle collaborative area coverage path planning. The method includes acquiring road network data of the target area and modeling it as a graph structure, generating a candidate anchor point set, and discretizing the target area into grid task points. Task point features, allocation states, and current anchor point identifiers are constructed into a state vector, which is input into a pre-trained deep reinforcement learning network. Combined with an action masking mechanism, target anchor points are selected sequentially and task points are assigned, outputting an anchor point sequence and a task point set. For each anchor point and its task point set, an operation plan is planned according to UAV endurance constraints. Based on the graph structure, the vehicle driving path and stopping order connecting each anchor point are determined, generating a complete vehicle-UAV collaborative patrol scheme. This invention can effectively combine macro-scheduling and micro-planning, improving the generalization ability and operational efficiency of multi-UAV collaboration.

Description

Technical Field

[0001] This invention relates to the field of path planning technology, and in particular to a method and system for collaborative regional coverage path planning involving multiple UAVs and vehicles. Background Technology

[0002] Forest fires, characterized by their suddenness and rapid spread, pose a serious threat to the ecological environment and the safety of people's lives and property. Traditional forest patrol methods mainly rely on manual inspections, observation from lookout towers, and manned aerial patrols, which have significant drawbacks such as limited coverage, slow response speed, and difficulty in accessing high-risk areas. With the development of drone technology, using drones for forest fire prevention patrols has become an important means. However, a single drone is limited by its endurance and communication range, making it difficult to independently complete long-term patrol missions over large areas. Therefore, a vehicle-mounted multi-drone collaborative operation mode has emerged. The vehicle acts as a mobile charging station and communication relay, carrying multiple drones into the forest area. The drones take off from the vehicle to perform patrol missions, forming a vehicle-drone collaborative patrol system. However, the core of this type of collaborative patrol—the path planning problem—mathematically belongs to a constrained combinatorial optimization problem, involving several highly coupled sub-problems such as anchor point selection, task allocation, drone route planning, and load balancing. Existing technologies typically use traditional heuristic algorithms or metaheuristic algorithms for solving this problem, such as using spatial distance-based clustering methods for anchor point partitioning, followed by iterative optimization using genetic algorithms or particle swarm optimization. The performance of these methods is highly dependent on specific scene parameters and manual parameter tuning. When the shape, size, or task distribution of the forest area changes, the generalization ability is insufficient, and it is often necessary to re-optimize for each scene. At the same time, existing methods generally adopt a step-by-step optimization architecture, which lacks coordination between sub-problems. The error in the early anchor point selection is continuously amplified in the subsequent UAV route planning stage, making it difficult to obtain the globally optimal patrol plan. In addition, these algorithms have high computational complexity and high resource consumption, making it difficult to run in real time on vehicle-mounted edge computing units, and thus failing to meet the urgent need for rapid response in forest fire fighting. Summary of the Invention

[0003] To address the aforementioned shortcomings, the present invention aims to propose a multi-UAV and vehicle collaborative regional coverage path planning method and system. This method aims to construct a hierarchical architecture that integrates a macro-allocation layer and a micro-planning layer. It utilizes a deep reinforcement learning model to learn anchor point selection and task allocation strategies end-to-end from the original scene information, and combines this with traditional heuristic algorithms for rapid generation and load balancing of local flight routes. This achieves high-quality path planning that adapts to different forest geographical environments while meeting the constraints of actual road network and UAV endurance, and reduces reliance on edge computing devices for computational power.

[0004] To achieve this objective, the present invention adopts the following technical solution: A method for collaborative area coverage path planning involving multiple drones and vehicles includes: The road network data of the target area is acquired and modeled as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the field of view radius of the UAV, the target area is discretized into grid task points, and the distance between each grid task point and each candidate anchor point is calculated. The distribution features, allocation status, and current anchor point identifier of the grid task points are constructed into a state vector, which is input into a pre-trained deep reinforcement learning network. The deep reinforcement learning network is combined with an action masking mechanism to sequentially select target anchor points from the action space and assign grid task points to the selected target anchor points. The output is an anchor point sequence and a set of task points corresponding to each target anchor point. For each target anchor point in the anchor point sequence and its corresponding task point set, a multi-UAV operation plan is planned at each target anchor point according to the UAV's endurance constraints. The multi-UAV operation plan includes dividing the task point set into several single-flight routes and assigning the routes to multiple UAVs for execution. Based on the graph structure of the road network and the sequence of anchor points, the driving path and stopping order of vehicles connecting each target anchor point are determined; By integrating the vehicle's driving path, the parking sequence, and the single-trip flight routes of each drone at different anchor points, a complete vehicle-drone collaborative area coverage patrol plan is generated.

[0005] Preferably, the road network data of the target area is acquired and modeled as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, the target area is discretized into grid task points, including: The acquired road network data is modeled as a graph structure including road intersection nodes and road segments, and each road segment has a length attribute; Virtual points are inserted at preset intervals on the road segment as candidate anchor points. Each candidate anchor point includes road node coordinates and road network identifiers. The candidate anchor point set is composed of all the virtual points. Regular grid points are generated within the target area at intervals equal to twice the UAV's field of view radius. Grid points located outside the target area are removed, and the remaining regular grid points constitute the grid task points.

[0006] Preferably, calculating the distance between each grid task point and each candidate anchor point includes: Based on Dijkstra's algorithm, the shortest path distance between any two road intersection nodes or candidate anchor points in the graph structure is pre-calculated to obtain a road distance matrix, in which the vehicle travel time between any two candidate anchor points is calculated. Satisfying the relation: ; in, Indicates that the vehicle is from the candidate anchor point Drive to candidate anchor point Time, Indicates candidate anchor points With candidate anchors The shortest path distance between them Indicates the vehicle's speed; The grid task points to the candidate anchor point straight-line distance Satisfying the relation: ; in, Represents grid task points With candidate anchors The straight-line distance between them This indicates the preset maximum single-trip flight distance for the drone.

[0007] Preferably, the construction of the state vector includes: For each grid task point, its coordinates are normalized and combined with its assignment state to construct a task point feature vector. The following relation is satisfied: ; in, Represents the feature vector of the task point. and They represent the first The normalized x and y coordinates of the grid task points Indicates the first Access flags for the aforementioned grid task points This represents the total number of grid task points; One-hot encoding is used to represent the currently selected target anchor point, and an identifier vector for the current anchor point is constructed. The task point feature vector is then concatenated with the current anchor point identifier vector to obtain the state vector. The following relation is satisfied: ; in, This represents the state vector input to the deep reinforcement learning network. This represents the vector concatenation operator. Indicates length is One-hot encoded vector, This indicates the number of candidate anchor points in the candidate anchor point set.

[0008] Preferably, when using the deep reinforcement learning network in conjunction with the action masking mechanism to sequentially select target anchors from the action space and assign grid task points to the selected target anchors, the deep reinforcement learning network is guided by a reward function to minimize the total completion time. The construction of the reward function includes: If the current action is to assign task points, then calculate the immediate reward for each step. The following relation is satisfied: ; in, This indicates an instant reward for each step. This represents the preset first balance coefficient. Indicates the currently assigned grid task point With the currently selected target anchor point The straight-line distance between them; If the task assignment corresponding to the current target anchor point is completed and there are no remaining valid task points, then calculate the anchor point completion reward. The following relation is satisfied: ; in, This indicates a reward for completing the anchor point. This represents the preset second balance coefficient. Indicates the current target anchor point The estimated drone execution time is as follows. This represents the preset positive reward coefficient. This indicates that the current target anchor point is assigned. Number of task points; If all grid task points have been assigned, calculate the total completion time for the entire task. and the corresponding final reward The following relation is satisfied: ; ; in, Indicates the total completion time. This indicates the total travel time of the vehicle. Indicates the first The actual execution time of the drone at each target anchor point This represents the total number of target anchor points in the anchor point sequence. This indicates the final reward. This represents the preset third balance coefficient; Satisfying the relation: ; in, Indicates at the target anchor point The longest flight time among all drones. This indicates the total number of drones. Indicates the target anchor point The corresponding set of drone routes, Indicate route Total flight distance This indicates the route assignment indicator variable. Indicates the drone's flight speed; The total travel time of the vehicle Satisfying the relation: ; in, Indicates the vehicle's starting point. This represents the first target anchor point in the anchor point sequence. This represents the last target anchor point in the anchor point sequence. This represents the shortest path distance between two points. Indicates the vehicle's speed.

[0009] Preferably, the deep reinforcement learning network uses a multilayer perceptron as an approximator for the policy network and the value network, wherein: The multilayer perceptron accepts the current state vector as input, passes through two fully connected hidden layers, each containing 64 neurons, and uses a modified linear unit as the activation function. The output layer of the multilayer perceptron includes a policy branch and a value branch. The policy branch is used to output the original score of the action space, which is then processed by a normalized exponential function to obtain the action probability distribution. The value branch is used to output a scalar to represent the value estimate of the current state. During training and inference, an action masking technique is used to block illegal actions, setting the original score corresponding to the illegal action to negative infinity, so that its probability after being processed by the normalized exponential function is zero. Specifically, during the new anchor point selection phase, the action mask only allows the selection of unused candidate anchor points; during the task point allocation phase, the action mask only allows the selection of grid task points that have not been visited and are no more than half of the maximum single-flight distance of the UAV from the current target anchor point.

[0010] Preferably, the training process of the deep reinforcement learning network includes: At the beginning of each training round, a forest scene is randomly loaded from several pre-built scene data files and the environment is reset so that the deep reinforcement learning network can extract common patterns under different geographical features. Empirical data is collected through the interaction between the current policy network and the environment, and the advantage function is calculated using the generalized advantage estimation method. Among them, the timing difference error Satisfying the relation: ; in, Indicates the first One-step timing difference error at each time step Indicates the first Instant rewards for each time step The first value network output represents the value network output. State value estimation at each time step This represents the preset discount factor. Indicates the end of the process; The advantage function Satisfying the relation: ; in, Indicates the first The advantage value of each time step This represents the pre-defined generalized dominance estimation parameter, used to balance the bias and variance of the estimation; The dominance value is standardized to obtain the standardized dominance value. The following relation is satisfied: ; in, This represents the standardized advantage value. This represents the mean of the current batch's advantage value. This represents the standard deviation of the current batch's dominance value. This represents the preset zero-reduction constant; The policy network and the value network are updated using the standardized advantage value, wherein the optimization objective of the policy network is... Satisfying the relation: ; in, This represents the policy shearing loss function. Expressing expected experience, This represents the ratio of the probabilities of the new and old strategies. This represents the clipping function. This indicates the preset clipping parameters.

[0011] Preferably, generating a complete vehicle-machine collaborative area coverage patrol scheme includes: The trained policy network is deployed to the target forest scene. The state vector of the target forest scene is input, and the deep reinforcement learning network outputs the anchor point sequence and the task point set corresponding to each target anchor point. For each target anchor point and its corresponding task point set, a greedy algorithm is used to generate several single-trip flight routes. The nearest unvisited grid task point is selected sequentially from the target anchor point to accumulate the path, and the straight-line flight distance of a single path does not exceed the maximum single-trip flight distance of the UAV. A greedy bin packing algorithm is used to distribute the generated single flight routes to several drones. By sorting all the single flight routes in descending order of flight time and assigning the route with the longest current flight time to the drone with the shortest current total flight time, load balancing among the drones is achieved. Based on the anchor point sequence, a path planning algorithm is used on the road network to solve for the shortest driving path connecting the starting point, each of the target anchor points and the end point, thereby determining the actual driving path of the vehicle and the stopping order. The output includes the vehicle travel path of the road node sequence, the stopping sequence, and the operation plan for each UAV, including the takeoff anchor point, the grid task point sequence, and the return anchor point.

[0012] A multi-UAV and vehicle collaborative area coverage path planning system includes: The environmental modeling module is used to acquire road network data of the target area and model it as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, the target area is discretized into grid task points, and the distance between each grid task point and each candidate anchor point is calculated. The intelligent decision-making module is used to construct a state vector from the distribution characteristics, allocation status and current anchor point identifier of the grid task points, input it into a pre-trained deep reinforcement learning network, and use the deep reinforcement learning network combined with the action masking mechanism to select target anchor points sequentially from the action space and assign the grid task points to the selected target anchor points, and output the anchor point sequence and the task point set corresponding to each target anchor point. The planning module is used to plan a multi-UAV operation plan for each target anchor point in the anchor point sequence and its corresponding task point set, based on the UAV's endurance constraints. The multi-UAV operation plan includes dividing the task point set into several single-flight routes and assigning the routes to multiple UAVs for execution. The vehicle guidance module is used to determine the vehicle driving path and stopping order connecting each of the target anchor points based on the graph structure of the road network and the anchor point sequence. The scheme generation module is used to integrate the vehicle driving path, the parking sequence, and the single flight route of each UAV at different anchor points to generate a complete vehicle-drone cooperative area coverage patrol scheme.

[0013] One of the above technical solutions has the following advantages or beneficial effects: This invention models real road network data as a graph structure and generates candidate anchor points along the roads. Simultaneously, it discretizes the target area into grid task points based on the UAV's field of view radius, ensuring that vehicle parking positions are strictly limited to the actual road network to guarantee accessibility and that reconnaissance coverage is logically continuous and free of redundancy. Furthermore, it constructs the distribution characteristics, allocation status, and current anchor point identifier of the grid task points as state vectors input to a pre-trained deep reinforcement learning network. An action masking mechanism dynamically masks illegal actions at each decision step, enabling the agent to progressively select target anchor points and assign task points only within the legal action space, thus avoiding the anchor point selection issues inherent in traditional step-by-step optimization architectures. The problem of early decision-making errors accumulating due to the disconnect between selection, task allocation, and route planning is addressed. Subsequently, for each target anchor point and its corresponding task point set, single-trip flight routes for multiple drones are planned based on the drone's endurance constraints, and the routes are allocated to each drone for execution, ensuring that the operation time of each drone tends to be balanced and strictly meets the actual flight capability limits. At the same time, based on the graph structure of the road network and the output anchor point sequence, the shortest driving path and stopping order of the vehicle are determined, ensuring that the vehicle movement strictly follows the actual road topology rather than straight-line distance. Finally, the vehicle driving path, stopping order, and single-trip flight routes of each drone at different anchor points are integrated to generate a complete vehicle-drone cooperative area coverage patrol scheme. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This is a flowchart of the multi-UAV and vehicle collaborative area coverage path planning method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the structure of the multi-UAV and vehicle collaborative area coverage path planning system provided in an embodiment of the present invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] In this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0018] A method for collaborative area coverage path planning involving multiple UAVs and vehicles, such as Figure 1 As shown, a preferred embodiment of the present invention includes the following steps: S1: Acquire road network data of the target area and model it as a graph structure. Generate candidate anchor points on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, discretize the target area into grid task points and calculate the distance between each grid task point and each candidate anchor point. It should be noted that road network data refers to the actual road information within the target forest area, including geographical information such as the location of road intersections, road segment directions, and lengths; a graph structure is a mathematical model composed of a set of nodes and a set of edges. In this step, road intersections are abstracted as nodes, and road segments are abstracted as edges, thereby transforming the continuous road network into a discrete, computable structure; candidate anchor points are virtual docking points inserted along the road at preset intervals, used for vehicle parking and providing charging and communication support for the UAV. Each candidate anchor point contains the coordinates of road nodes and road network identifiers; the field of view radius is the radius of the ground area that the reconnaissance equipment carried by the UAV can effectively cover, used to determine the spacing of grid task points to ensure that the reconnaissance coverage is complete and without overlap; grid task points are regular grid points generated within the target area with a spacing of twice the field of view radius, and the set of points remaining after removing points located outside the area represents the specific location that the UAV needs to fly over for reconnaissance.

[0019] Understandably, the purpose of step S1 is to transform the real geographical environment into a standardized data structure that can be computed. By modeling the road network as a graph structure, it ensures that vehicle parking locations are strictly limited to actually accessible roads, avoiding the generation of driving plans that cannot be landed. At the same time, the target area is discretized based on the UAV's field of view radius, so that the task points are evenly distributed in space and have continuous coverage, ensuring the integrity of reconnaissance while avoiding resource waste caused by repeated coverage. By pre-calculating the distance between each grid task point and the candidate anchor point, it provides basic data support for the feasibility judgment of UAV take-off and landing and task allocation in subsequent steps, establishing the spatial relationship between vehicle movement and UAV operation from the data level.

[0020] S2: Construct a state vector from the distribution features, allocation status, and current anchor point identifier of the grid task points, input it into a pre-trained deep reinforcement learning network, and use the deep reinforcement learning network in conjunction with the action masking mechanism to sequentially select target anchor points from the action space and assign the grid task points to the selected target anchor points, and output the anchor point sequence and the task point set corresponding to each target anchor point. It should be noted that the distribution feature refers to the spatial location information of the grid task points within the target area. In step S2, normalized coordinates are used to describe the geometric distribution of the task points. The allocation state is a binary flag representing whether each grid task point has been assigned to a target anchor point, with 0 indicating unassigned and 1 indicating assigned, used to track the task allocation progress. The state vector is a fixed-length one-dimensional vector formed by concatenating the task point feature vector with the current anchor point identifier vector, which serves as the input to the deep reinforcement learning network, enabling it to perceive global environmental information. The deep reinforcement learning network is a type of network that uses an agent to interact with the environment. The machine learning model learns the optimal decision-making strategy through trial and error. In this step, a multilayer perceptron is used as an approximator for the policy network and the value network. The action masking mechanism is a technique that dynamically masks illegal actions at each decision step. By setting the network output corresponding to an illegal action to negative infinity, it makes the probability distribution approach zero, ensuring that the agent only selects legal actions. The action space is a set of actions that include two types: selecting new anchor points and assigning task points. The total number of actions is the sum of the number of candidate anchor points and the number of grid task points. The anchor point sequence is the set of target anchor points arranged in the order of selection, representing the order in which the vehicles stop.

[0021] Understandably, the purpose of step S2 is to transform the complex multi-UAV and vehicle cooperative patrol path planning problem into a sequential decision problem. A deep reinforcement learning network directly learns the mapping relationship from task point distribution to anchor point selection and task allocation from the original scene information. The state vector encodes the spatial distribution of task points, allocation progress, and current decision anchor point information into fixed-dimensional inputs, enabling the network to perceive the global state. The action masking mechanism dynamically constrains the legal action space at each step, preventing the agent from making invalid decisions such as repeatedly selecting used anchor points or allocating task points beyond the UAV's range, thus ensuring the stability of the learning process and the feasibility of the solution. Through an end-to-end learning approach, the error accumulation caused by the disconnect between anchor point selection and task allocation in traditional step-by-step optimization is avoided, allowing the macro-allocation strategy to consider global objectives such as subsequent UAV execution time and vehicle movement time.

[0022] S3: For each target anchor point in the anchor point sequence and its corresponding task point set, according to the drone's endurance constraints, plan a multi-drone operation plan for each target anchor point. The multi-drone operation plan includes dividing the task point set into several single-trip flight routes and assigning the routes to multiple drones for execution. It should be noted that endurance constraint refers to the maximum flight distance limit of a single takeoff of the drone, which is determined by battery capacity and energy consumption characteristics. It is used to limit the total length of a single flight route to ensure that the drone can safely return to the target anchor point. A single flight route is a closed path that starts and ends at the target anchor point, flies over several grid task points in sequence, and then returns. Its total straight-line flight distance does not exceed the maximum single flight distance of the drone. A multi-drone operation plan is a complete execution plan that includes multiple single flight routes and their correspondence with multiple drones. It is used to guide the specific patrol actions of each drone at the target anchor point. Load balancing is used to distribute different flight routes to multiple drones so that the total flight time of each drone is as close as possible, in order to reduce waiting time and improve overall operation efficiency.

[0023] Understandably, the purpose of step S3 is to transform the macroscopically allocated task point set into an executable specific flight plan; by dividing the task point set into several single-trip flight routes that meet the endurance constraints, it ensures that each drone will not lose contact due to power depletion during mission execution; a greedy algorithm is used to generate single-trip flight routes, starting from the target anchor point and selecting the nearest unvisited task point each time, accumulating the path without exceeding the endurance limit to form an actual executable flight trajectory; a greedy bin packing algorithm is used to allocate routes to multiple drones, prioritizing routes with longer flight times to drones with shorter current total flight times, achieving load balancing among drones, thereby minimizing the longest operation time at each anchor point and improving the overall efficiency of vehicle-drone collaboration.

[0024] S4: Based on the graph structure of the road network and the sequence of anchor points, determine the vehicle driving path and stopping order connecting each of the target anchor points; It should be noted that the vehicle travel path is the actual drivable route formed by the vehicle starting from the starting point, passing through each target anchor point in sequence, and finally returning to the starting point on the road network. It consists of continuous graph nodes and road segments. The stopping order is the order in which the vehicle visits each target anchor point, which is consistent with the anchor point sequence and is used to guide the movement of the vehicle on the actual road network. The shortest path is the path that connects two nodes in the weighted graph structure with the smallest total weight and is used to determine the optimal travel route between adjacent anchor points.

[0025] Understandably, the purpose of step S4 is to plan the actual driving scheme of the vehicle on the real road network based on the anchor point sequence output by the macro-allocation. Since the candidate anchor points are strictly limited to the road network, the vehicle can only travel along the road. Therefore, it is necessary to solve the shortest path between adjacent anchor points based on the graph structure. By modeling the road network as a weighted graph, the shortest path algorithm is used to calculate the optimal road connections from the starting point to the first anchor point, between each adjacent anchor point, and from the last anchor point back to the starting point. This yields the complete feasible driving path and stopping order of the vehicle, ensuring that the vehicle movement strictly follows the actual road topology and avoiding infeasible schemes caused by planning based on straight-line distance in traditional methods.

[0026] S5: Integrate the vehicle driving path, the parking sequence, and the single-trip flight route of each UAV at different anchor points to generate a complete vehicle-machine collaborative area coverage patrol scheme.

[0027] It should be noted that the vehicle-machine collaborative area coverage patrol scheme is a comprehensive execution scheme that includes vehicle driving plans and drone flight plans, specifying when and where vehicles will stop and the specific patrol tasks to be performed by each drone from each anchor point; a single flight route is a single closed flight trajectory with the target anchor point as the take-off and landing point, including take-off anchor point, mission point visit sequence and return anchor point information.

[0028] Understandably, the purpose of step S5 is to organically integrate the vehicle dispatching plan with the drone operation plan to form a complete plan that can be directly executed in actual forest fire patrols. By associating the multi-drone operation plan at each target anchor point with the vehicle's travel path and stopping sequence, the dwell time of the vehicle at each anchor point is specified (determined by the longest operation time of the drone at that anchor point), so that vehicle movement and drone operation are coordinated in time. The final output plan ensures that the vehicle can arrive at each stopping point in sequence according to the actual road network, and also ensures that each drone can complete its assigned patrol task and return safely after taking off from the corresponding anchor point, achieving seamless connection of vehicle-drone collaborative operation.

[0029] Preferably, the road network data of the target area is acquired and modeled as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, the target area is discretized into grid task points, including: The acquired road network data is modeled as a graph structure including road intersection nodes and road segments, and each road segment has a length attribute; Virtual points are inserted at preset intervals on the road segment as candidate anchor points. Each candidate anchor point includes road node coordinates and road network identifiers. The candidate anchor point set is composed of all the virtual points. Regular grid points are generated within the target area at intervals equal to twice the UAV's field of view radius. Grid points located outside the target area are removed, and the remaining regular grid points constitute the grid task points.

[0030] It should be noted that road network data refers to spatial information data of actual passable roads within the target forest area, usually derived from Geographic Information Systems (GIS) or satellite remote sensing imagery. This data includes the road's geographical location, direction, intersection location, and road grade attributes, serving as the basic input data for constructing the mathematical model. A graph structure is a mathematical abstraction, composed of a set of vertices and edges. Road intersections are abstracted as vertices (i.e., road intersection nodes), and the road segments connecting two intersections are abstracted as edges (i.e., road segments). Each edge has a length attribute representing the actual physical length of the road segment. This structure allows subsequent path calculations to be performed efficiently based on mature graph theory algorithms. Candidate anchor points refer to locations where vehicles can stop and provide charging and communication support for drones during patrol missions. These must be strictly limited to the road network to ensure vehicle accessibility and are generated by inserting virtual points at preset intervals (e.g., 2.5km) on road segments. Road node coordinates refer to the position coordinates of anchor points in a two-dimensional plane coordinate system, typically using longitude-latitude or... Cartesian coordinates are used for subsequent distance calculations and location identification; road network identifiers refer to the index numbers of anchor points in the graph structure, used to quickly locate the topological position of the anchor point in the road network, facilitating the subsequent use of Dijkstra's algorithm to calculate the shortest path between anchor points; the UAV's field of view radius refers to the radius of the ground area that the UAV can effectively cover when equipped with reconnaissance equipment, determined by the UAV's flight altitude, camera field of view angle, and ground resolution requirements, and is a core parameter for determining the spacing between task points; grid task points refer to the set of reconnaissance locations formed by discretizing a continuous forest area. The UAV needs to fly over these points to achieve full coverage reconnaissance of the area. Generating them with a spacing of twice the field of view radius ensures continuous and non-overlapping reconnaissance coverage; regular grid points refer to discrete point arrays arranged with fixed row and column spacing, using a square grid layout for easy calculation and storage; the elimination operation refers to using an algorithm to determine if a point is inside a polygon to filter out grid points located inside the boundary of the target forest area, excluding invalid points outside the area, ensuring that the distribution of task points matches the actual patrol area.

[0031] Understandably, by abstracting the real road network into a graph structure and generating candidate anchor points along the roads, the problem of anchor points potentially being located in roadless areas, rendering the proposed solutions infeasible in real-world geographical environments, is fundamentally solved in traditional path planning methods. A feasible domain constraint for vehicle movement is established, ensuring that vehicles can only travel and stop on existing roads. Simultaneously, by discretizing the grid based on the UAV's field of view radius, the continuous area coverage problem is transformed into a discrete task point allocation problem, enabling subsequent reinforcement learning models to make decisions within a finite and standardized action space. The node-edge model in graph theory accurately characterizes the topological relationships and geometric properties of the road network, and the spatial discretization theory decomposes the two-dimensional region into computable grid cells, thereby reducing problem complexity while ensuring full reconnaissance coverage. This provides standardized input for the macro-allocation layer and establishes a mapping bridge from real-world geographical space to a mathematical space that the algorithm can process, ensuring that the subsequently generated patrol plans have strict executability in real-world environments.

[0032] Preferably, calculating the distance between each grid task point and each candidate anchor point includes: Based on Dijkstra's algorithm, the shortest path distance between any two road intersection nodes or candidate anchor points in the graph structure is pre-calculated to obtain a road distance matrix, in which the vehicle travel time between any two candidate anchor points is calculated. Satisfying the relation: ; in, Indicates that the vehicle is from the candidate anchor point Drive to candidate anchor point Time, Indicates candidate anchor points With candidate anchors The shortest path distance between them Indicates the vehicle's speed; The grid task points to the candidate anchor point straight-line distance Satisfying the relation: ; in, Represents grid task points With candidate anchors The straight-line distance between them This indicates the preset maximum single-trip flight distance for the drone.

[0033] It should be noted that Dijkstra's algorithm is a classic shortest path algorithm used to calculate the shortest path from a source node to all other nodes in a weighted graph. This algorithm employs a greedy strategy, maintaining a distance-priority queue. It iteratively expands the set of nodes with determined shortest paths by selecting the nearest undetermined node to the source node for relaxation operations, until all nodes have been processed. This process is used to pre-calculate the shortest travel distance between any two candidate anchor points on the road network. The road distance matrix is ​​a two-dimensional matrix whose elements... Indicates candidate anchor points arrive The shortest path length matrix on the road network, which is symmetric (for undirected graphs) and has zero diagonal elements, serves as the basis for calculating vehicle travel time and evaluating the reward function; vehicle travel time This refers to vehicles traveling along the road network from anchor points Drive to the anchor point The required pure travel time, excluding stopping and loading / unloading time, is calculated by dividing the shortest path distance by the vehicle speed. It is a core parameter of the vehicle movement time component in the total completion time model; vehicle speed... This refers to the average speed of a vehicle traveling on forest roads, determined by road grade, road conditions, and safety regulations, and is input as a constant parameter for time calculation; straight-line distance. Refers to grid task points With candidate anchors The Euclidean distance (the straight-line distance between two points), disregarding terrain obstacles, is used to determine whether the drone can reach the mission point and return safely after taking off from the anchor point; the maximum single-flight distance of the drone. This refers to the maximum distance a drone can fly continuously on a full charge. It is determined by the drone model, battery capacity, payload, and flight environment, and is a key parameter constraining the range of a single drone flight. It is used to select task points that can be assigned to anchor points (meeting certain conditions). (Ensure round-trip travel is feasible).

[0034] Understandably, by pre-compiling the road distance matrix and establishing straight-line distance constraints, a precise distance metric foundation is provided for subsequent vehicle path planning and UAV task allocation. A quantitative mapping from spatial location to time cost is established, enabling the reinforcement learning reward function to accurately evaluate the time efficiency of different decision-making schemes. The global optimality of Dijkstra's algorithm ensures the accuracy of road distance calculations, while the geometric intuition of Euclidean distance is used to quickly determine UAV reachability. By transforming complex geospatial relationships into a rapidly queryable numerical matrix, the repetitive computation overhead in subsequent decision-making processes is significantly reduced. This provides efficient data support for reward calculation and action mask generation at the macro-allocation layer, ensuring that the reinforcement learning model has accurate numerical basis when evaluating the cost of transfers between anchor points and the feasibility of task point allocation.

[0035] Preferably, the construction of the state vector includes: For each grid task point, its coordinates are normalized and combined with its assignment state to construct a task point feature vector. The following relation is satisfied: ; in, Represents the feature vector of the task point. and They represent the first The normalized x and y coordinates of the grid task points Indicates the first Access flags for the aforementioned grid task points This represents the total number of grid task points; One-hot encoding is used to represent the currently selected target anchor point, and an identifier vector for the current anchor point is constructed. The task point feature vector is then concatenated with the current anchor point identifier vector to obtain the state vector. The following relation is satisfied: ; in, This represents the state vector input to the deep reinforcement learning network. This represents the vector concatenation operator. Indicates length is One-hot encoded vector, This indicates the number of candidate anchor points in the candidate anchor point set.

[0036] It should be noted that normalization refers to mapping the original coordinate values ​​to a linear transformation. The numerical processing of the interval involves subtracting the minimum coordinate of the region from the actual coordinates of the task point and then dividing by the region's coordinate span. This process eliminates the impact of differences in the size of different forest regions on the input scale of the neural network, ensuring that the model can generalize to scenes of different scales. The horizontal axis... and ordinate This refers to the position coordinates of the task point in the normalized two-dimensional plane coordinate system. It preserves the relative spatial distribution information of the task point and serves as the basic input for deep reinforcement learning networks to perceive the geometric layout of the task point; access flag. This is a binary variable; a value of 0 indicates that the task point has not yet been assigned to any anchor point (unvisited), and a value of 1 indicates that the task point has been assigned to an anchor point (visited). It is used to characterize the progress state of task assignment and is key information representing the decision history in the state space; task point feature vector. It is a one-dimensional vector formed by concatenating the normalized coordinates and access flags of all task points in a fixed order, and its dimension is... ( (Total number of task points), this vector encodes the current state of the entire task space in a compact numerical form; one-hot encoding is a method of converting discrete categorical variables into binary vectors, the length of which is equal to the number of categories, where only the positions corresponding to the category are 1 and the rest are 0, used to represent the currently selected anchor point, enabling the neural network to explicitly perceive the current decision context; current anchor point identifier vector. It is a length of ( A binary vector representing the number of candidate anchor points. If the current anchor point is the [number of anchor points], then [the vector is the first one]. The nth candidate anchor point, then the vector of the nth... The first bit is 1 and the rest are 0. If no anchor point has been selected, all bits are 0. This vector provides anchor point context information during the decision-making stage; vector concatenation operator This refers to the operation of concatenating two vectors end-to-end to form a longer vector, used to combine task point features and anchor point identification information; state vector It is the final complete state representation input to the deep reinforcement learning network, with dimensions of It contains global information on three aspects: task distribution, allocation progress, and current anchor point.

[0037] Understandably, by encoding the spatial distribution, allocation status, and current anchor point information of task points into fixed-dimensional state vectors, a complete and standardized environmental perception input is provided for deep reinforcement learning networks. This establishes a mapping from complex geographical scenes to numerical vectors that neural networks can process, enabling policy networks to make decisions on anchor point selection and task allocation based on global information. By using normalization to eliminate scene scale differences, using binary flags to encode discrete states, and using vector concatenation to integrate multi-source information, heterogeneous geographical and decision information is unified into a homogeneous numerical vector form, satisfying the requirement of multilayer perceptrons for fixed input dimensions. This constructs a perception system for reinforcement learning agents, enabling them to "see" the layout of task points, understand which tasks have been assigned, and clearly identify which anchor point they are currently making decisions at, laying the foundation for state representation in subsequent end-to-end learning.

[0038] Preferably, when using the deep reinforcement learning network in conjunction with the action masking mechanism to sequentially select target anchors from the action space and assign grid task points to the selected target anchors, the deep reinforcement learning network is guided by a reward function to minimize the total completion time. The construction of the reward function includes: If the current action is to assign task points, then calculate the immediate reward for each step. The following relation is satisfied: ; in, This indicates an instant reward for each step. This represents the preset first balance coefficient. Indicates the currently assigned grid task point With the currently selected target anchor point The straight-line distance between them; If the task assignment corresponding to the current target anchor point is completed and there are no remaining valid task points, then calculate the anchor point completion reward. The following relation is satisfied: ; in, This indicates a reward for completing the anchor point. This represents the preset second balance coefficient. Indicates the current target anchor point The estimated drone execution time is as follows. This represents the preset positive reward coefficient. This indicates that the current target anchor point is assigned. Number of task points; If all grid task points have been assigned, calculate the total completion time for the entire task. and the corresponding final reward The following relation is satisfied: ; ; in, Indicates the total completion time. This indicates the total travel time of the vehicle. Indicates the first The actual execution time of the drone at each target anchor point This represents the total number of target anchor points in the anchor point sequence. This indicates the final reward. This represents the preset third balance coefficient; Satisfying the relation: ; in, Indicates at the target anchor point The longest flight time among all drones. This indicates the total number of drones. Indicates the target anchor point The corresponding set of drone routes, Indicate route Total flight distance This indicates the route assignment indicator variable. Indicates the drone's flight speed; The total travel time of the vehicle Satisfying the relation: ; in, Indicates the vehicle's starting point. This represents the first target anchor point in the anchor point sequence. This represents the last target anchor point in the anchor point sequence. This represents the shortest path distance between two points. Indicates the vehicle's speed.

[0039] It should be noted that action masking is a technique in reinforcement learning that dynamically restricts the set of available actions. By setting the probability value of illegal actions to negative infinity in the current state, the probability of these actions in the softmax output is zero, thus ensuring that the agent only selects legal actions. The available actions are dynamically filtered based on the current decision stage and constraints. The action space refers to the set of all possible actions that the reinforcement learning agent can take in the current state, including selecting a new anchor point (…). (actions) and task allocation points ( There are two categories (actions) and a total number of actions. Instant rewards for every step This refers to the reward signal obtained immediately after performing a single task point assignment action. It is calculated by multiplying the negative straight-line distance by a balance coefficient and is used to encourage the assignment of task points to nearby anchor points, thereby reducing the drone's flight distance; the first balance coefficient... This is a hyperparameter used to adjust the magnitude of the immediate reward, usually set to a small value (e.g., 0.01) to ensure that the immediate reward matches the magnitude of subsequent rewards, preventing the strategy from becoming overly dominant in early allocation; Anchor point completion reward This refers to the phased reward obtained after completing all task assignments at an anchor point. It consists of a negative estimated execution time penalty and a positive task quantity reward, and is used to evaluate the overall assignment quality of that anchor point; the second balancing coefficient. This is a hyperparameter (e.g., 0.1) used to adjust the magnitude of the anchor point execution time penalty, ensuring that time cost has an appropriate weight in the reward; predicting the drone execution time. This refers to the anchor point. At this point, the time required to complete all assigned tasks is estimated using a fast heuristic algorithm. A greedy algorithm is then used to generate flight routes, and after load balancing, the longest UAV flight time is selected to approximate the operational efficiency of that anchor point. A positive reward coefficient is also used. This is a positive incentive parameter (e.g., 0.05) used to encourage individual anchors to cover more task points, resulting in higher rewards for anchors covering more task points and reducing the total number of anchors required; number of task points. This refers to the allocation of anchor points. task point set The base number, i.e., the number of task points that the anchor point needs to cover; total completion time. This refers to the total time required to complete the patrol mission across the entire forest area, consisting of the sum of vehicle travel time and drone execution time at each anchor point. It is the core objective of optimization; the final reward... This refers to the end-of-round reward obtained after completing all task point allocations, calculated by multiplying the negative total time by a coefficient, directly guiding the agent to minimize the global objective; the third balance coefficient. This is a parameter used to adjust the magnitude of the final reward (e.g., 1.0) to ensure that the final reward dominates the total reward for each round; actual execution time. This refers to the anchor point. At that location, after actual route planning and load balancing, The longest flight time required for a drone to complete its assigned task is obtained by dividing the total flight distance of the busiest drone at that anchor point by its flight speed; route assignment indicator variable. It is a binary variable, and a value of 1 indicates the route. Assigned to drones Execution, with a value of 0 indicating no allocation, satisfies the constraints that each route is assigned to exactly one drone and each drone can execute multiple routes.

[0040] Understandably, by constructing multi-layered reward functions and action masking mechanisms, complex combinatorial optimization objectives are transformed into reward signals that reinforcement learning can optimize. This guides the agent to learn strategies that minimize the total completion time, while ensuring the legality and feasibility of the decision-making process. Immediate rewards are used to encourage locally optimal allocation (proximity allocation), anchor point completion rewards are used to evaluate the quality of mid-term decisions (time efficiency and coverage density), and final rewards are used to optimize the global objective (total completion time). Furthermore, action masks enforce the satisfaction of constraints at each step (no repeated allocation, meeting endurance limits). This transforms the constrained combinatorial optimization problem into an unconstrained sequential decision-making problem, establishing a numerical feedback channel from decision actions to optimization objectives. This allows the agent to gradually master the strategies of "which anchor points to choose and how to allocate task points" through trial and error learning, without the need for manually designing complex heuristic rules.

[0041] Preferably, the deep reinforcement learning network uses a multilayer perceptron as an approximator for the policy network and the value network, wherein: The multilayer perceptron accepts the current state vector as input, passes through two fully connected hidden layers, each containing 64 neurons, and uses a modified linear unit as the activation function. The output layer of the multilayer perceptron includes a policy branch and a value branch. The policy branch is used to output the original score of the action space, which is then processed by a normalized exponential function to obtain the action probability distribution. The value branch is used to output a scalar to represent the value estimate of the current state. During training and inference, an action masking technique is used to block illegal actions, setting the original score corresponding to the illegal action to negative infinity, so that its probability after being processed by the normalized exponential function is zero. Specifically, during the new anchor point selection phase, the action mask only allows the selection of unused candidate anchor points; during the task point allocation phase, the action mask only allows the selection of grid task points that have not been visited and are no more than half of the maximum single-flight distance of the UAV from the current target anchor point.

[0042] It should be noted that a Multilayer Perceptron (MLP) is a feedforward artificial neural network architecture consisting of an input layer, one or more hidden layers, and an output layer. Each layer contains a number of neurons, and adjacent layers exchange information through fully connected layers (each neuron is connected to all neurons in the next layer). It serves as a unified function approximator for policy and value networks, used to map state vectors to action probabilities and value estimates. A fully connected hidden layer refers to a neural network layer where every neuron within the layer has a weighted connection to all neurons in the previous layer. Two hidden layers are used, each with 64 neurons, which allows for control of model complexity while maintaining expressive power. The Corrected Linear Unit (ReLU) is a commonly used activation function, mathematically expressed as: The output branch, or softmax branch, is a sub-network of the MLP output layer. It outputs 0 when the input is negative and equal to the input when the input is positive. This introduces nonlinear transformations, enabling the network to learn complex nonlinear mappings. The policy branch, a sub-network of the MLP output layer, outputs the raw scores (logits) of each action in the action space, reflecting the relative merit of the action. After processing by the softmax function, it is converted into a probability distribution for action sampling. The value branch, another sub-network of the MLP output layer, outputs a scalar value representing the value estimate of the current state, i.e., the expected cumulative reward from that state. This is used for advantage function calculation and policy evaluation. The normalized exponential function (softmax function) is a function that converts a real vector into a probability distribution, mapping each component of the input vector to a probability distribution. The interval and the sum of all components is 1, so that the output can be interpreted as the probability of action selection; the original score (logits) refers to the original output value of the softmax function input layer, usually in the real number range, representing the original preference of the neural network; the action masking technique refers to replacing the logits corresponding to illegal actions with extremely small negative numbers (such as negative infinity) before softmax calculation, so that the probability of these actions approaches zero, thereby forcing the agent to only select legal actions; the new anchor point selection stage refers to the decision stage where no anchor point has been selected or the current anchor point has been assigned and a new anchor point needs to be selected. At this time, the action space is the set of candidate anchor points; the task point assignment stage refers to the decision stage where the current anchor point has been selected and a specific task point needs to be assigned to the anchor point. At this time, the action space is the set of task points.

[0043] Understandably, by constructing a multilayer perceptron network structure and integrating an action masking mechanism, an end-to-end mapping from state vectors to action decisions is achieved. This provides a learnable decision function for the macro-allocation layer, enabling the agent to output reasonable anchor point selection or task allocation actions based on the current global state. It utilizes the universal approximation capability of MLP to learn complex nonlinear mappings from state to action, uses a dual-branch structure to simultaneously output policy (what to do) and value (state quality), introduces nonlinearity using the ReLU activation function to capture complex patterns, and embeds constraints at the architecture level using action masks. This achieves data-driven policy learning while ensuring the legality of decisions, constructing the "decision brain" of the reinforcement learning agent, enabling it to make autonomous decisions based on perceived states and continuously optimize decision strategies through interaction with the environment.

[0044] Preferably, the training process of the deep reinforcement learning network includes: At the beginning of each training round, a forest scene is randomly loaded from several pre-built scene data files and the environment is reset so that the deep reinforcement learning network can extract common patterns under different geographical features. Empirical data is collected through the interaction between the current policy network and the environment, and the advantage function is calculated using the generalized advantage estimation method. Among them, the timing difference error Satisfying the relation: ; in, Indicates the first One-step timing difference error at each time step Indicates the first Instant rewards for each time step The first value network output represents the value network output. State value estimation at each time step This represents the preset discount factor. Indicates the end of the process; The advantage function Satisfying the relation: ; in, Indicates the first The advantage value of each time step This represents the pre-defined generalized dominance estimation parameter, used to balance the bias and variance of the estimation; The dominance value is standardized to obtain the standardized dominance value. The following relation is satisfied: ; in, This represents the standardized advantage value. This represents the mean of the current batch's advantage value. This represents the standard deviation of the current batch's dominance value. This represents the preset zero-reduction constant; The policy network and the value network are updated using the standardized advantage value, wherein the optimization objective of the policy network is... Satisfying the relation: ; in, This represents the policy shearing loss function. Expressing expected experience, This represents the ratio of the probabilities of the new and old strategies. This represents the clipping function. This indicates the preset clipping parameters.

[0045] It should be noted that a training episode refers to a complete process of interaction between the reinforcement learning agent and the environment, starting from the initial state, going through a series of decision steps until the termination condition is reached (all task points have been assigned). Executing multiple episodes in diverse scenarios allows the network to learn generalization strategies. Scene data files refer to pre-constructed environmental data files containing geographical features (shape, size, road layout) of different forest regions. Each file contains a road network map, candidate anchor points, task points, and distance matrices for that scene, providing diverse training environments. Environment reset refers to restoring the environment to its initial state at the beginning of each episode, including clearing the task assignment state, resetting vehicle positions, and regenerating task point access flags, ensuring each episode is independent. Generalized advantage estimation (GAE) is an algorithm for estimating the advantage function. By weighted combining multi-step temporal difference errors, it achieves a balance between bias and variance, used to calculate the advantage estimate required for policy gradient updates. Temporal difference errors... This refers to the difference between the estimated value of the next state (after adding a discount to the single-step reward) and the estimated value of the current state, reflecting the immediate value change brought about by the current action; discount factor. This is a parameter between 0 and 1 (usually 0.99), used to calculate the present value of future rewards, representing the agent's emphasis on future rewards; termination flag. It is a binary variable, with a value of 1 indicating a time step. The environment reaches a termination state (all tasks assigned), with a value of 0 indicating continuation, used to truncate the bootstrap of value estimation; advantage value. This refers to the state Take action below The degree of superiority or inferiority relative to the average level; a positive value indicates that the action is better than the average, and a negative value indicates that it is worse than the average, which is the core signal for policy updates; generalized advantage estimation parameters It is a parameter between 0 and 1 (usually 0.95), which controls the weighting of multi-step errors in GAE. Only single-step error (high bias, low variance) is used. Use all future errors (low bias, high variance); standardization involves subtracting the batch mean from the odds value and dividing by the batch standard deviation to make the odds distribution mean 0 and variance 1, used to stabilize the training process; prevent zero constants. It is a very small positive number (e.g.) To prevent numerical errors caused by division by zero; strategy shearing loss function The core objective function of the PPO algorithm is to limit the probability ratio of the new and old strategies within a certain range. To prevent excessively large policy updates from causing training instability; empirical expectation. It refers to the sample average calculated from collected empirical data, approximating the true expectation; the probability ratio of the old and new strategies. The pruning function refers to the ratio of the probability of an action under the new strategy to the probability of an action under the old strategy, reflecting the degree of policy change; This refers to input Limited to Within the interval, values ​​outside the range are truncated to the boundary; clipping parameters It is a small positive number (such as 0.2) that defines the radius of the trust zone for policy updates.

[0046] Understandably, by designing diverse scene sampling mechanisms and stable policy update algorithms, effective training of deep reinforcement learning networks is achieved. This enables the policy network to learn generalized anchor selection and task allocation policies from a large number of different scenarios, while ensuring the stability and convergence of the training process. Diverse scene sampling enhances the model's generalization ability, GAE provides low-variance advantage estimation to reduce policy gradient noise, PPO's pruning mechanism limits the policy update magnitude to avoid training collapse, and advantage standardization stabilizes the learning dynamics. Thus, a stable learning path is found in complex combinatorial optimization problems, establishing a closed loop from environmental interaction to network parameter optimization. This allows the agent to gradually improve its decision-making ability through continuous trial and error, ultimately obtaining a deployable and efficient policy.

[0047] Preferably, generating a complete vehicle-machine collaborative area coverage patrol scheme includes: The trained policy network is deployed to the target forest scene. The state vector of the target forest scene is input, and the deep reinforcement learning network outputs the anchor point sequence and the task point set corresponding to each target anchor point. For each target anchor point and its corresponding task point set, a greedy algorithm is used to generate several single-trip flight routes. The nearest unvisited grid task point is selected sequentially from the target anchor point to accumulate the path, and the straight-line flight distance of a single path does not exceed the maximum single-trip flight distance of the UAV. A greedy bin packing algorithm is used to distribute the generated single flight routes to several drones. By sorting all the single flight routes in descending order of flight time and assigning the route with the longest current flight time to the drone with the shortest current total flight time, load balancing among the drones is achieved. Based on the anchor point sequence, a path planning algorithm is used on the road network to solve for the shortest driving path connecting the starting point, each of the target anchor points and the end point, thereby determining the actual driving path of the vehicle and the stopping order. The output includes the vehicle travel path of the road node sequence, the stopping sequence, and the operation plan for each UAV, including the takeoff anchor point, the grid task point sequence, and the return anchor point.

[0048] It should be noted that deployment refers to loading the trained neural network model into the inference environment, freezing the network parameters, performing only forward propagation calculations, and applying the policy capabilities gained during the training phase to generate patrol plans in real-world scenarios. A greedy algorithm is a heuristic algorithm that selects the current optimal solution (local optimum) at each step in the hope of achieving a global optimum. It is used to quickly generate drone flight routes that meet endurance constraints by constructing the route by selecting the nearest unvisited task point to the current location each time. A single flight route refers to a complete flight trajectory of the drone from an anchor point, visiting several task points, and returning to the same anchor point; its total flight distance must not exceed [a certain value]. Greedy bin packing algorithm is a heuristic allocation algorithm for load balancing. It mimics the greedy strategy of the bin packing problem, sorting tasks (routes) in descending order of their flight time and then allocating them sequentially to the lightest-loaded resource (drone). This achieves balanced operation time among multiple drones. Descending flight time sorting means arranging all single-trip routes from longest to shortest estimated flight time, ensuring that time-consuming routes are allocated first, avoiding overloading any drone by allocating the last route. The road node sequence is an ordered list of road intersections and candidate anchor points along the vehicle's path, reflecting the vehicle's trajectory on the actual road network. The stopping order refers to the order in which vehicles visit each anchor point, consistent with the anchor point sequence. The takeoff anchor point is the departure and return anchor point location of the drone when executing a single-trip flight. The job plan is the set of all single-trip flights that each drone needs to execute at each anchor point, including the task point sequence for each route.

[0049] Understandably, by deploying the trained policy network to the target scenario and combining it with traditional heuristic algorithms for micro-path planning and load balancing, a complete solution generation from macro-allocation to specific execution plans is achieved. The decision output of the reinforcement learning model is transformed into a practically executable vehicle-drone cooperative patrol solution. The trained policy network is used to quickly generate high-quality anchor point sequences and task allocations (macro-level). The local optima of the greedy algorithm are used to quickly construct flight routes that meet the endurance constraints (micro-level). The greedy bin packing algorithm is used to achieve multi-machine load balancing. The graph search algorithm is used to solve the shortest path for vehicles. Thus, the NP-hard joint optimization problem is decomposed into a layer that can be solved efficiently, taking into account both solution quality and computational efficiency. A bridge is established from intelligent decision-making to physical execution, ensuring that the output solution has both an intelligently optimized global structure and meets the constraints of actual execution. It can be directly used to guide the actual patrol operations of vehicles and drones.

[0050] A multi-UAV and vehicle collaborative area coverage path planning system, such as Figure 2 As shown, it includes: The environmental modeling module is used to acquire road network data of the target area and model it as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, the target area is discretized into grid task points, and the distance between each grid task point and each candidate anchor point is calculated. The intelligent decision-making module is used to construct a state vector from the distribution characteristics, allocation status and current anchor point identifier of the grid task points, input it into a pre-trained deep reinforcement learning network, and use the deep reinforcement learning network combined with the action masking mechanism to select target anchor points sequentially from the action space and assign the grid task points to the selected target anchor points, and output the anchor point sequence and the task point set corresponding to each target anchor point. The planning module is used to plan a multi-UAV operation plan for each target anchor point in the anchor point sequence and its corresponding task point set, based on the UAV's endurance constraints. The multi-UAV operation plan includes dividing the task point set into several single-flight routes and assigning the routes to multiple UAVs for execution. The vehicle guidance module is used to determine the vehicle driving path and stopping order connecting each of the target anchor points based on the graph structure of the road network and the anchor point sequence. The scheme generation module is used to integrate the vehicle driving path, the parking sequence, and the single flight route of each UAV at different anchor points to generate a complete vehicle-drone cooperative area coverage patrol scheme.

[0051] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0052] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for multi-UAV and vehicle cooperative regional coverage path planning, characterized in that, include: The road network data of the target area is acquired and modeled as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the field of view radius of the UAV, the target area is discretized into grid task points, and the distance between each grid task point and each candidate anchor point is calculated. The distribution features, allocation status, and current anchor point identifier of the grid task points are constructed into a state vector, which is input into a pre-trained deep reinforcement learning network. The deep reinforcement learning network is combined with an action masking mechanism to sequentially select target anchor points from the action space and assign grid task points to the selected target anchor points. The output is an anchor point sequence and a set of task points corresponding to each target anchor point. For each target anchor point in the anchor point sequence and its corresponding task point set, a multi-UAV operation plan is planned at each target anchor point according to the UAV's endurance constraints. The multi-UAV operation plan includes dividing the task point set into several single-flight routes and assigning the routes to multiple UAVs for execution. Based on the graph structure of the road network and the sequence of anchor points, the driving path and stopping order of vehicles connecting each target anchor point are determined; By integrating the vehicle's driving path, the parking sequence, and the single-trip flight routes of each drone at different anchor points, a complete vehicle-drone collaborative area coverage patrol plan is generated. 2.The method of claim 1, wherein, Acquire road network data of the target area and model it as a graph structure. Generate candidate anchor points on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, discretize the target area into grid task points, including: The acquired road network data is modeled as a graph structure including road intersection nodes and road segments, and each road segment has a length attribute; Virtual points are inserted at preset intervals on the road segment as candidate anchor points. Each candidate anchor point includes road node coordinates and road network identifiers. The candidate anchor point set is composed of all the virtual points. Regular grid points are generated within the target area at intervals equal to twice the UAV's field of view radius. Grid points located outside the target area are removed, and the remaining regular grid points constitute the grid task points. 3.The method of claim 2, wherein, Calculating the distance between each grid task point and each candidate anchor point includes: Based on Dijkstra's algorithm, the shortest path distance between any two road intersection nodes or candidate anchor points in the graph structure is pre-calculated to obtain a road distance matrix, in which the vehicle travel time between any two candidate anchor points is calculated. Satisfying the relation: ； in, Indicates that the vehicle is from the candidate anchor point Drive to candidate anchor point Time, Indicates candidate anchor points With candidate anchors The shortest path distance between them Indicates the vehicle's speed; The grid task points to the candidate anchor point straight-line distance Satisfying the relation: ； in, Represents grid task points With candidate anchors The straight-line distance between them This indicates the preset maximum single-trip flight distance for the drone.

4. The multi-UAV and vehicle collaborative area coverage path planning method according to claim 1, characterized in that, The construction of the state vector includes: For each grid task point, its coordinates are normalized and combined with its assignment state to construct a task point feature vector. The following relation is satisfied: ； in, Represents the feature vector of the task point. and They represent the first The normalized x and y coordinates of the grid task points Indicates the first Access flags for the aforementioned grid task points This represents the total number of grid task points; One-hot encoding is used to represent the currently selected target anchor point, and an identifier vector for the current anchor point is constructed. The task point feature vector is then concatenated with the current anchor point identifier vector to obtain the state vector. The following relation is satisfied: ； in, This represents the state vector input to the deep reinforcement learning network. This represents the vector concatenation operator. Indicates length is One-hot encoded vector, This indicates the number of candidate anchor points in the candidate anchor point set.

5. The multi-UAV and vehicle collaborative area coverage path planning method according to claim 1, characterized in that, When using the deep reinforcement learning network combined with an action masking mechanism to sequentially select target anchors from the action space and assign grid task points to the selected target anchors, the deep reinforcement learning network is guided by a reward function to minimize the total completion time. The construction of the reward function includes: If the current action is to assign task points, then calculate the immediate reward for each step. The following relation is satisfied: ； in, This indicates an instant reward for each step. This represents the preset first balance coefficient. Indicates the currently assigned grid task point With the currently selected target anchor point The straight-line distance between them; If the task assignment corresponding to the current target anchor point is completed and there are no remaining valid task points, then calculate the anchor point completion reward. The following relation is satisfied: ； in, This indicates a reward for completing the anchor point. This represents the preset second balance coefficient. Indicates the current target anchor point The estimated drone execution time is as follows. This represents the preset positive reward coefficient. This indicates that the current target anchor point is assigned. Number of task points; If all grid task points have been assigned, calculate the total completion time for the entire task. and the corresponding final reward The following relation is satisfied: ； ； in, Indicates the total completion time. This indicates the total travel time of the vehicle. Indicates the first The actual execution time of the drone at each target anchor point This represents the total number of target anchor points in the anchor point sequence. This indicates the final reward. This represents the preset third balance coefficient; Satisfying the relation: ; in, Indicates at the target anchor point The longest flight time among all drones. This indicates the total number of drones. Indicates the target anchor point The corresponding set of drone routes, Indicate route Total flight distance This indicates the route assignment indicator variable. Indicates the drone's flight speed; The total travel time of the vehicle Satisfying the relation: ； in, Indicates the vehicle's starting point. This represents the first target anchor point in the anchor point sequence. This represents the last target anchor point in the anchor point sequence. This represents the shortest path distance between two points. Indicates the vehicle's speed.

6. The multi-UAV and vehicle collaborative area coverage path planning method according to claim 1, characterized in that, The deep reinforcement learning network employs a multilayer perceptron as an approximator for the policy network and the value network, wherein: The multilayer perceptron accepts the current state vector as input, passes through two fully connected hidden layers, each containing 64 neurons, and uses a modified linear unit as the activation function. The output layer of the multilayer perceptron includes a policy branch and a value branch. The policy branch is used to output the original score of the action space, which is then processed by a normalized exponential function to obtain the action probability distribution. The value branch is used to output a scalar to represent the value estimate of the current state. During training and inference, an action masking technique is used to block illegal actions, setting the original score corresponding to the illegal action to negative infinity, so that its probability after being processed by the normalized exponential function is zero. Specifically, during the new anchor point selection phase, the action mask only allows the selection of unused candidate anchor points; during the task point allocation phase, the action mask only allows the selection of grid task points that have not been visited and are no more than half of the maximum single-flight distance of the UAV from the current target anchor point.

7. The multi-UAV and vehicle collaborative area coverage path planning method according to claim 1, characterized in that, The training process of the deep reinforcement learning network includes: At the beginning of each training round, a forest scene is randomly loaded from several pre-built scene data files and the environment is reset so that the deep reinforcement learning network can extract common patterns under different geographical features. Empirical data is collected through the interaction between the current policy network and the environment, and the advantage function is calculated using the generalized advantage estimation method. Among them, the timing difference error Satisfying the relation: ； in, Indicates the first One-step timing difference error at each time step Indicates the first Instant rewards for each time step The first value network output represents the value network output. State value estimation at each time step This represents the preset discount factor. Indicates the end of the process; The advantage function Satisfying the relation: ； in, Indicates the first The advantage value of each time step This represents the pre-defined generalized dominance estimation parameter, used to balance the bias and variance of the estimation; The dominance value is standardized to obtain the standardized dominance value. The following relation is satisfied: ； in, This represents the standardized advantage value. This represents the mean of the current batch's advantage value. This represents the standard deviation of the current batch's dominance value. This represents the preset zero-reduction constant; The policy network and the value network are updated using the standardized advantage value, wherein the optimization objective of the policy network is... Satisfying the relation: ； in, This represents the policy shearing loss function. Expressing expected experience, This represents the ratio of the probabilities of the new and old strategies. This represents the clipping function. This indicates the preset clipping parameters.

8. The multi-UAV and vehicle collaborative area coverage path planning method according to claim 7, characterized in that, The complete vehicle-machine collaborative area coverage patrol plan includes: The trained policy network is deployed to the target forest scene. The state vector of the target forest scene is input, and the deep reinforcement learning network outputs the anchor point sequence and the task point set corresponding to each target anchor point. For each target anchor point and its corresponding task point set, a greedy algorithm is used to generate several single-trip flight routes. The nearest unvisited grid task point is selected sequentially from the target anchor point to accumulate the path, and the straight-line flight distance of a single path does not exceed the maximum single-trip flight distance of the UAV. A greedy bin packing algorithm is used to distribute the generated single flight routes to several drones. By sorting all the single flight routes in descending order of flight time and assigning the route with the longest current flight time to the drone with the shortest current total flight time, load balancing among the drones is achieved. Based on the anchor point sequence, a path planning algorithm is used on the road network to solve for the shortest driving path connecting the starting point, each of the target anchor points and the end point, thereby determining the actual driving path of the vehicle and the stopping order. The output includes the vehicle travel path of the road node sequence, the stopping sequence, and the operation plan for each UAV, including the takeoff anchor point, the grid task point sequence, and the return anchor point.

9. A multi-UAV and vehicle collaborative area coverage path planning system, characterized in that, include: The environmental modeling module is used to acquire road network data of the target area and model it as a graph structure. Candidate anchor points are generated on the roads of the graph structure at preset intervals to form a set of candidate anchor points. Based on the UAV's field of view radius, the target area is discretized into grid task points, and the distance between each grid task point and each candidate anchor point is calculated. The intelligent decision-making module is used to construct a state vector from the distribution characteristics, allocation status and current anchor point identifier of the grid task points, input it into a pre-trained deep reinforcement learning network, and use the deep reinforcement learning network combined with the action masking mechanism to select target anchor points sequentially from the action space and assign the grid task points to the selected target anchor points, and output the anchor point sequence and the task point set corresponding to each target anchor point. The planning module is used to plan a multi-UAV operation plan for each target anchor point in the anchor point sequence and its corresponding task point set, based on the UAV's endurance constraints. The multi-UAV operation plan includes dividing the task point set into several single-flight routes and assigning the routes to multiple UAVs for execution. The vehicle guidance module is used to determine the vehicle driving path and stopping order connecting each of the target anchor points based on the graph structure of the road network and the anchor point sequence. The scheme generation module is used to integrate the vehicle driving path, the parking sequence, and the single flight route of each UAV at different anchor points to generate a complete vehicle-drone cooperative area coverage patrol scheme.

Images