Flight path planning method and system based on multi-unmanned aerial vehicle cooperative inspection

Abstract

The invention relates to a flight path planning method and system based on multi-unmanned aerial vehicle cooperative inspection, and the method comprises the following steps: S1, carrying out the combined modeling of LiDAR and oblique photography, generating a point cloud map of an inspection region, and generating a voxel map with obstacle labels through TSDF fusion; s2, according to the voxel map with the obstacle label, based on Voronoi iteration of an improved Lloyd algorithm, dynamically allocating sub-regions in combination with the performance of the unmanned aerial vehicle, and ensuring load balance; s3, performing path planning based on a bilayer planning framework according to the unmanned aerial vehicles after the sub-regions are allocated; s4, performing routing inspection according to a path planning result, and realizing distributed conflict resolution by adopting a contract network protocol; and S5, in combination with 5G communication, an ad-hoc network and an ultra wide band, enhancing data communication between the unmanned aerial vehicles and between the unmanned aerial vehicles and the aircraft nest. According to the invention, comprehensive optimization of multi-unmanned aerial vehicle cooperative inspection is realized.

Description

Technical Field

[0001] The present invention relates to the field of multi-UAV collaborative control, and in particular to a track planning method and system based on multi-UAV collaborative inspection. Background Art

[0002] With the development of drone technology, drones are increasingly being used in inspections, particularly in scenarios such as power line inspections, oil and gas pipeline monitoring, farmland surveillance, and emergency rescue operations, demonstrating their advantages of efficiency, intelligence, and low cost. However, traditional single-drone inspection models struggle to meet the demands of large-scale, highly complex scenarios due to issues such as limited mission coverage, low inspection efficiency, and poor adaptability to dynamic environments. Therefore, multi-drone collaborative inspections are becoming a research hotspot. Through collaboration among multiple drones, efficient coverage of the inspection area, balanced allocation of mission resources, and coordinated optimization of path planning can be achieved.

[0003] Collaborative multi-UAV inspections present a host of technical challenges: how to obtain high-precision environmental modeling in complex 3D environments to ensure efficient path planning and flight safety; how to efficiently allocate tasks in dynamic and uncertain environments to ensure collision avoidance and collaboration among multiple UAVs; and how to ensure robust inter-UAV communication and real-time task monitoring. These challenges place greater demands on environmental perception, path planning, communication coordination, and algorithm optimization, and represent the key technical challenges addressed in this research. Summary of the Invention

[0004] In order to solve the above problems, the purpose of the present invention is to provide a trajectory planning method and system based on multi-UAV collaborative inspection, which realizes the comprehensive optimization of multi-UAV collaborative inspection.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A trajectory planning method based on multi-UAV collaborative inspection includes the following steps:

[0007] S1: Use LiDAR and oblique photography to jointly model and generate a point cloud map of the inspection area, and then generate a voxel map with obstacle labels through TSDF fusion;

[0008] S2: Based on the voxel map with obstacle labels, the improved Lloyd algorithm and Voronoi iteration are used to dynamically allocate sub-areas in combination with the performance of the UAV to ensure load balancing;

[0009] S3: Path planning is performed for each UAV after the sub-area is assigned based on a two-layer planning framework;

[0010] S4: Carry out inspections based on the path planning results and use the contract network protocol to achieve distributed conflict resolution;

[0011] S5: Combining 5G communication, ad hoc networks, and ultra-wideband, it enhances data communication between drones and between drones and the drone nest. Each drone periodically reports its location, power information, and mission progress to the drone nest, enabling flexible monitoring and scheduling.

[0012] Furthermore, LiDAR and oblique photography are combined to generate a point cloud map of the inspection area, and TSDF fusion is used to generate a voxel map with obstacle labels, as follows:

[0013] Obtain the 3D point cloud of the inspection area through LiDAR:

[0014] Plidar={p a ∣p ia =(x a ,y a ,z a ,I a )}

[0015] Among them, p a is the a-th point cloud, (x a ,y a ,z a ) is the position of the a-th point cloud, I a is the reflection intensity;

[0016] Obtain a multi-view image collection through oblique photography, along with POS data;

[0017] Align oblique photography with a 3D point cloud using collinearity equations:

[0018] The inspection area is divided into cubic voxels with a side length of δ. For each voxel v, the center point c of the voxel v is calculated. v Truncated signed distance to the nearest surface TSDF(v):

[0019]

[0020] Among them, d(c v ) is the Euclidean distance from the voxel center to the nearest surface; τ is the cutoff distance; sgn is the sign function;

[0021] Fuse the point cloud data reconstructed by LiDAR and oblique photography, and calculate the TSDF value TSDF of the fused voxel v by weighted average fused (v):

[0022]

[0023] Among them, w lida 、w photo is the weight coefficient; TSDFlidar (v) is the TSDF value generated by the three-dimensional point cloud data of voxel v; TSDF photo (v) TSDF value generated for the oblique photography data of voxel v;

[0024] Determine obstacle type based on point cloud distribution within voxels:

[0025] Calculate the variance of the normal vector of the point cloud within the voxel like θ vertical If the threshold is set to a preset value, it is marked as a vertical obstacle, which includes towers and trees. The ground plane is fitted by RANSAC, excluding ground voxels.

[0026] Combined with the RGB information of the oblique photography image, a lightweight convolutional network is used to classify obstacles and obtain the semantic label Label(v) and confidence:

[0027]

[0028] Among them, I v is the oblique photographic image block corresponding to the voxel projection; category c∈{wires, trees, buildings, others}; Softmax represents the Softmax function, f CNN Represents a lightweight convolutional network;

[0029] Finally, we get a voxel map M with obstacle labels voxel , each element contains TSDF value, semantic label and confidence.

[0030] Furthermore, based on the voxel map with obstacle labels, the sub-regions are dynamically allocated based on the Voronoi iteration of the improved Lloyd algorithm and the performance of the drone, as follows:

[0031] Assume that the drone set U={u1,u2,…,u i ,…,u N Each drone u i Equipped with performance parameters: battery life T i , load capacity L 0i and flight speed v i ; Where i∈[1,N], N represents the total number of drones;

[0032] From the voxel map M voxel Eliminate obstacle areas:

[0033] S free ={v∈M voxel |Label(v)=free};

[0034] Among them, S freeis a set of non-obstruction voxels; Label(v)=free indicates that voxel v is free space;

[0035] Randomly select N voxels as the initial position of the drone;

[0036] Based on the performance of drones, define drone u i Weight w i for:

[0037]

[0038] Among them, T max , L max 、v max are the maximum endurance time, load capacity and flight speed in the UAV set respectively; α, β and γ are weight coefficients;

[0039] Each free voxel v is assigned to the nearest drone u whose distance to the free voxel v is adjusted by weight k

[0040]

[0041] in, is the drone u at the tth iteration k location;

[0042] For each drone u i , the corresponding Voronoi subregion at the tth iteration is updated to V i (t) :

[0043]

[0044] Update the center point position of the drone at the tth iteration to the weighted center of mass of the sub-region:

[0045]

[0046] When the region division converges, that is, the change amplitude of all drone sub-regions at the tth iteration When it is less than the threshold ∈, stop the iteration:

[0047]

[0048] Among them, |V i (t) ΔV i (t-1) | represents the symmetric difference of sets;

[0049] After the sub-area division, the mission load L of each UAV is calculated i :

[0050]

[0051] Among them, d(c v ,p i ) is the distance from the UAV to the center of the voxel; s(v) is the complexity of the inspection task; p i is the current position coordinate vector of UAV i;

[0052] Adjust the sub-region division according to the equilibrium objective function LoadVar;

[0053]

[0054] in, is the average task load;

[0055] According to the adjusted sub-area division, the sub-area allocation V corresponding to the drone is generated. i …,V N}.

[0056] Furthermore, when a sudden task or drone failure occurs in the multi-drone collaborative inspection system, the originally divided drone sub-area V needs to be dynamically adjusted and the sub-areas are reallocated through a bidding mechanism, as follows:

[0057] Define emergency tasks as newly added urgent tasks falling in a sub-area V j , some areas need to be reallocated to ensure that the task is executed as soon as possible; the drone failure is a drone u j Failure, the sub-area V j The task in cannot be completed, you need to change V j Divide and reallocate to other drones;

[0058] UAV i Pair Area V j The affordability is determined by the sub-region redistribution cost function C ij Sure:

[0059]

[0060] Among them, p i For drone u i Current location; c Vj For sub-region V j The geometric center position of Indicates drone u i To sub-area V j Distance; T Vj For sub-region V j Estimated task working time; T i -Tused,i Indicates the remaining flight time of the drone; ΔL i The change of UAV load after allocating the area; B min,Vj To complete sub-area V j The minimum power required for operation; λ1, λ2, λ3, λ4 are weight parameters;

[0061] The nest broadcasts the sub-area or the area where the emergency task is located that needs to be reallocated. Each drone u i Calculate the pair region V j The bid value B ij :

[0062]

[0063] For sub-region V j , select the drone with the highest bidding value.

[0064] Furthermore, the two-layer planning framework includes a global layer and a local layer. The global layer uses the improved RRT* to generate a collision-free rough path P global , the local layer, according to the collision-free coarse path P global , whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance.

[0065] Furthermore, the improved RRT* is used to generate the collision-free rough path P global , as follows:

[0066] The improved RRT* introduces a heuristic potential field for guidance, and preferentially samples points close to the target area:

[0067]

[0068] Among them, p bias is a biased sampling point; δ is the target biased sampling probability; p rand is the coordinate of the random sampling point; rand() means returning a uniformly distributed random number between 0 and 1; p uniform is a point sampled uniformly and randomly in space;

[0069] Perform collision detection on obstacle labels in the voxel map, rejecting points in the obstacle area:

[0070]

[0071] Among them, CollisionFree(v) indicates whether voxel v can pass collision detection; Label(v) is the label attribute of voxel v; obstacle indicates that the voxel is an obstacle area; free indicates that the voxel is free space;

[0072] Construct the initial tree T and set the starting point p start Insert T;

[0073] Iteration tree extension:

[0074] Under the obstacle constraint, sample a random point p rand , find the nearest point p in the tree near , generate a new point p new :

[0075]

[0076] Where η is the control step size; ||p rand -p near || is the Euclidean distance between two points;

[0077] P new Perform collision detection, if there is no collision, add it to tree T;

[0078] According to the path cost optimization, the tree is pruned and the path is optimized by reconnecting;

[0079] The path node encounters the target position p goal When , return to the coarse path P global , use the stereo space spline interpolation to calculate the coarse path P global Smoothing, reducing redundancy.

[0080] Furthermore, according to the collision-free rough path P global Whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance, as follows:

[0081] Define speed barrier VO:

[0082]

[0083] Among them, p u (t′) is the trajectory of the UAV at the current time t′; p o (t′) is the position trajectory of the dynamic obstacle;

[0084] Guaranteed planning speed v * Not in a speed barrier area:

[0085]

[0086] Integrating the speed constraints proposed by VO Generate the optimal trajectory through MPC to avoid dynamic obstacles and minimize the path cost function J:

[0087]

[0088] Among them, ||p t′ -p goal || 2 is the distance from the current point to the target point; CollisionPenality is the collision cost term; w1, w2, w3 are the weight coefficients of the cost function, and N' is the cruising time.

[0089] Furthermore, S4 is specifically:

[0090] Each UAV determines whether there is a space-time conflict with other UAVs on the path based on the path planning results;

[0091] The initiator of conflict detection broadcasts conflict information to other UAVs, including: the area, location, preliminary time window, current path priority or mission status of the conflict;

[0092] Each potentially conflicting drone calculates its bidding cost based on its own status and submits the bid value to the initiator;

[0093] Based on all bid values, the initiator selects the drone with the highest bid value to cooperate in adjusting the path. Conflicting drones avoid conflicts by adjusting the time window and introduce sequence adjustment using the time window synchronization mechanism to ensure that drones are scheduled in an orderly manner in the conflict area.

[0094] Furthermore, S5 is specifically:

[0095] The main communication link between drones and nests is built based on 5G, and the auxiliary communication link between drones is built based on self-organizing network and ultra-wideband; each drone u i per ton c Upload UAV status to the drone nest or neighboring drones in seconds i (t') includes:

[0096] S i (t')={p i (t'),E i (t'),P i (t'),Q i (t')}

[0097] Among them, p i (t′) is the current position of the UAV; E i (t′) is the current remaining power percentage; P i (t′) is the current waypoint number or route progress; Q i (t′) is the current task queue status;

[0098] The self-organizing network dynamically maintains the communication links between drones to ensure the adaptability of the network topology.i and u j The communication signal is interrupted, triggering path reconstruction.

[0099] A trajectory planning system based on collaborative inspection of multiple unmanned aerial vehicles (UAVs) includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the trajectory planning method based on collaborative inspection of multiple unmanned aerial vehicles as described above.

[0100] The present invention has the following beneficial effects:

[0101] 1. This invention improves task efficiency through load balancing and dynamic path planning, ensures the coordinated safety of the UAV system through distributed conflict resolution and efficient communication, and achieves comprehensive optimization of multi-UAV collaborative inspection from environmental perception to task execution;

[0102] 2. This invention combines LiDAR and oblique photography data sources to generate a high-precision voxel map with obstacle labels through TSDF fusion, effectively integrating the geometric and texture characteristics of the point cloud to fully capture the spatial information of the inspection area. The map with obstacle labels provides precise perception capabilities for route design and obstacle avoidance, significantly improving planning accuracy.

[0103] 3. This solution, through an improved Lloyd algorithm and Voronoi iteration method, implements dynamic sub-region allocation based on drone performance parameters, ensuring balanced mission load. Combined with global path planning and local dynamic avoidance strategies (a two-level planning framework), drones can execute missions with both efficiency and safety, significantly improving collaborative performance and adapting to complex mission requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0105] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0106] refer to Figure 1 In this embodiment, a trajectory planning method based on multi-UAV collaborative inspection is provided, comprising the following steps:

[0107] S1: Use LiDAR and oblique photography to jointly model and generate a point cloud map of the inspection area, and then generate a voxel map with obstacle labels through TSDF fusion;

[0108] S2: Based on the voxel map with obstacle labels, the improved Lloyd algorithm and Voronoi iteration are used to dynamically allocate sub-areas in combination with the performance of the UAV to ensure load balancing;

[0109] S3: Path planning is performed for each UAV after the sub-area is assigned based on a two-layer planning framework;

[0110] S4: Carry out inspections based on the path planning results and use the contract network protocol to achieve distributed conflict resolution;

[0111] S5: Combining 5G communication, ad hoc networks, and ultra-wideband, it enhances data communication between drones and between drones and the drone nest. Each drone periodically reports its location, power information, and mission progress to the drone nest, enabling flexible monitoring and scheduling.

[0112] In this embodiment, LiDAR and oblique photography are combined to generate a point cloud map of the inspection area, and TSDF fusion is used to generate a voxel map with obstacle labels, as follows:

[0113] Obtain the 3D point cloud of the inspection area through LiDAR:

[0114] Plidar={p a ∣p ia =(x a ,y a ,z a ,I a )}

[0115] Among them, p a is the a-th point cloud, (x a ,y a ,z a ) is the position of the a-th point cloud, I a is the reflection intensity;

[0116] Obtain a multi-view image collection through oblique photography, along with POS data;

[0117] Align oblique photography with a 3D point cloud using collinearity equations:

[0118] The inspection area is divided into cubic voxels with a side length of δ. For each voxel v, the center point c of the voxel v is calculated. v Truncated signed distance to the nearest surface TSDF(v):

[0119]

[0120] Among them, d(c v ) is the Euclidean distance from the voxel center to the nearest surface (calculated by 3D point cloud); τ is the cutoff distance; sgn is the sign function;

[0121] Fuse the point cloud data reconstructed by LiDAR and oblique photography, and calculate the TSDF value TSDF of the fused voxel v by weighted average fused(v):

[0122]

[0123] Among them, w lida 、w photo is the weight coefficient; TSDF lidar (v) is the TSDF value generated by the three-dimensional point cloud data of voxel v; TSDF photo (v) TSDF value generated for the oblique photography data of voxel v;

[0124] Determine obstacle type based on point cloud distribution within voxels:

[0125] Calculate the variance of the normal vector of the point cloud within the voxel like θ vertical If the threshold is set, vertical obstacles such as towers and trees are marked; the ground plane is fitted through RANSAC to exclude ground voxels;

[0126] Combined with the RGB information of the oblique photography image, a lightweight convolutional network (such as MobileNet) is used to classify the obstacles and obtain the semantic label Label (v) and confidence:

[0127]

[0128] Among them, I v is the oblique photographic image block corresponding to the voxel projection; category c∈{wires, trees, buildings, others}; Softmax represents the Softmax function, f CNN Represents a lightweight convolutional network;

[0129] Finally, we get a voxel map M with obstacle labels voxel , each element contains the TSDF value (surface distance), semantic label (obstacle type) and confidence (fusion weight and classification probability).

[0130] In this embodiment, based on the voxel map with obstacle labels and the Voronoi iteration of the improved Lloyd algorithm, sub-regions are dynamically allocated in combination with the performance of the drone, as follows:

[0131] Assume that the drone set U={u1,u2,…,u i ,…,u N Each drone u i Equipped with performance parameters: battery life T i , load capacity L 0i and flight speed v i ; Where i∈[1,N], N represents the total number of drones;

[0132] From the voxel map M voxel Eliminate obstacle areas:

[0133] S free ={v∈M voxel |Label(v)=free};

[0134] Among them, S free is a set of non-obstruction voxels; Label(v)=free indicates that voxel v is free space;

[0135] Randomly select N voxels as the initial position of the drone;

[0136] Based on the performance of drones, define drone u i Weight w i for:

[0137]

[0138] Among them, T max , L max 、v max are the maximum endurance time, load capacity and flight speed in the UAV set respectively; α, β and γ are weight coefficients;

[0139] Each free voxel v is assigned to the nearest drone u whose distance to the free voxel v is adjusted by weight k

[0140]

[0141] in, is the drone u at the tth iteration k location;

[0142] For each drone u i , the corresponding Voronoi subregion at the tth iteration is updated to V i (t) :

[0143]

[0144] Update the center point position of the drone to the weighted centroid of the sub-area:

[0145]

[0146] When the region division converges, that is, the change amplitude of all drone sub-regions at the tth iteration When it is less than the threshold ∈, stop the iteration:

[0147]

[0148] Among them, |Vi (t) ΔV i (t-1) | represents the symmetric difference of sets;

[0149] After the sub-area division, the mission load L of each UAV is calculated i :

[0150]

[0151] Among them, d(c v ,p i ) is the distance from the UAV to the center of the voxel; s(v) is the complexity of the inspection task; p i is the current position coordinate vector of UAV i;

[0152] Adjust the sub-region division according to the equilibrium objective function LoadVar;

[0153]

[0154] in, is the average task load;

[0155] According to the adjusted sub-area division, the sub-area allocation V corresponding to the drone is generated. i …,V N}.

[0156] In this embodiment, when an emergency task or drone failure occurs in the multi-drone collaborative inspection system, the previously divided drone sub-areas V need to be dynamically adjusted and the sub-areas are reallocated through a bidding mechanism, as follows:

[0157] The definition of emergency tasks is that new urgent tasks (such as detailed inspection of specific targets or temporary collection tasks) fall into a certain sub-area V j , some areas need to be reallocated to ensure that the task is executed as soon as possible; the drone failure is a drone u j Failure, the sub-area V j The task in cannot be completed, you need to change V j Divide and reallocate to other drones;

[0158] UAV i Pair Area V j (or part thereof j' ∈Vj) is redistributed through the sub-region cost function C ij Sure:

[0159]

[0160] Among them, p iFor drone u i Current location; c Vj For sub-region V j The geometric center position of Indicates drone u i To sub-area V j Distance; T Vj For sub-region V j Estimated task working time (calculated by voxel task set); T i -T used,i Indicates the remaining flight time of the drone; ΔL i The change of UAV load after allocating the area; B min,Vj To complete sub-area V j The minimum power required for operation; λ1, λ2, λ3, λ4 are weight parameters;

[0161] The nest broadcasts the sub-area or the area where the emergency task is located that needs to be reallocated. Each drone u i Calculate the pair region V j The bid value B ij :

[0162]

[0163] For sub-region V j , select the drone with the highest bidding value.

[0164] In this embodiment, the two-layer planning framework includes a global layer and a local layer. The global layer uses the improved RRT* to generate a collision-free rough path P global , the local layer, according to the collision-free coarse path P global ,Whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance.

[0165] In this embodiment, the improved RRT* is used to generate the collision-free rough path P global , as follows:

[0166] The improved RRT* introduces a heuristic potential field for guidance, and preferentially samples points close to the target area:

[0167]

[0168] Among them, p bias is a biased sampling point; δ is the target biased sampling probability; p rand is the coordinate of the random sampling point; rand() means returning a uniformly distributed random number between 0 and 1; p uniform is a point sampled uniformly and randomly in space;

[0169] Perform collision detection on obstacle labels in the voxel map, rejecting points in the obstacle area:

[0170]

[0171] Among them, CollisionFree(v) indicates whether voxel v can pass collision detection; Label(v) is the label attribute of voxel v; obstacle indicates that the voxel is an obstacle area; free indicates that the voxel is free space;

[0172] Construct the initial tree T and set the starting point p start Insert T;

[0173] Iteration tree extension:

[0174] Under the obstacle constraint, sample a random point p rand , find the nearest point p in the tree near , generate a new point p new :

[0175]

[0176] Where η is the control step size; ||p rand -p near || is the Euclidean distance between two points;

[0177] P new Perform collision detection, if there is no collision, add it to tree T;

[0178] According to the path cost optimization, the tree is pruned and the path is optimized by reconnecting;

[0179] The path node encounters the target position p goal When , return to the rough path P global , use the cubic spline interpolation to calculate the coarse path P global Smoothing, reducing redundancy.

[0180] In this embodiment, according to the collision-free coarse path P global Whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance, as follows:

[0181] Define speed barrier VO:

[0182]

[0183] Among them, p u (t′) is the trajectory of the UAV at the current time t′; p o (t′) is the position trajectory of the dynamic obstacle;

[0184] Guaranteed planning speed v * Not in a speed barrier area:

[0185]

[0186] Integrating the speed constraints proposed by VO Generate the optimal trajectory through MPC to avoid dynamic obstacles and minimize the path cost function J:

[0187]

[0188] Among them, ||p t′ -p goal || 2 is the distance from the current point to the target point; CollisionPenality is the collision cost term; w1, w2, w3 are the weight coefficients of the cost function, and N' is the cruising time.

[0189] In this embodiment, S4 is specifically:

[0190] Each UAV determines whether there is a space-time conflict with other UAVs on the path based on the path planning results;

[0191] The initiator of conflict detection broadcasts conflict information to other UAVs, including: the area, location, preliminary time window, current path priority or mission status of the conflict;

[0192] Each potentially conflicting drone calculates its bidding cost based on its own status and submits the bid value to the initiator;

[0193] Based on all bid values, the initiator selects the drone with the highest bid value to cooperate in adjusting the path. Conflicting drones avoid conflicts by adjusting the time window and introduce sequence adjustment using the time window synchronization mechanism to ensure that drones are scheduled in an orderly manner in the conflict area.

[0194] In this embodiment, S5 is specifically:

[0195] The main communication link between drones and nests is built based on 5G, and the auxiliary communication link between drones is built based on self-organizing network and ultra-wideband; each drone u i per ton c Upload UAV status to the drone nest or neighboring drones in seconds i (t') includes:

[0196] S i (t′)={p i (t′),E i (t′),P i (t′),Q i (t′)}

[0197] Among them, p i (t′) is the current position of the UAV (three-dimensional coordinates obtained by GPS / IMU); E i (t′) is the current remaining power percentage; P i (t′) is the current waypoint number or route progress; Q i (t′) is the current task queue status (pending, executing, or completed task set);

[0198] The self-organizing network dynamically maintains the communication links between drones to ensure the adaptability of the network topology. i and u j If the communication signal is interrupted (for example, due to an obstacle or out of communication range), path reconstruction is triggered.

[0199] A trajectory planning system based on multi-UAV collaborative inspection includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the above-mentioned trajectory planning method based on multi-UAV collaborative inspection.

[0200] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0202] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0204] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A trajectory planning method based on multi-UAV collaborative inspection, characterized in that: The following steps are involved: S1: Use LiDAR and oblique photography to jointly model and generate a point cloud map of the inspection area, and then generate a voxel map with obstacle labels through TSDF fusion; S2: Based on the voxel map with obstacle labels, the improved Lloyd algorithm and Voronoi iteration are used to dynamically allocate sub-areas in combination with the performance of the UAV to ensure load balancing; S3: Path planning is performed for each UAV after the sub-area is assigned based on a two-layer planning framework; S4: Carry out inspections based on the path planning results and use the contract network protocol to achieve distributed conflict resolution; S5: Combining 5G communication, ad hoc networks, and ultra-wideband, it enhances data communication between drones and between drones and the drone nest. Each drone periodically reports its location, power information, and mission progress to the drone nest, enabling flexible monitoring and scheduling.

2. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 1 is characterized in that: The method uses LiDAR and oblique photography to jointly build a model to generate a point cloud map of the inspection area, and generates a voxel map with obstacle labels through TSDF fusion, as follows: Obtain the 3D point cloud of the inspection area through LiDAR: Plidar={p a ∣p ia =(x a ,y a ,z a ,I a )} Among them, p a is the a-th point cloud, (x a ,y a ,z a ) is the position of the a-th point cloud, I a is the reflection intensity; Obtain a multi-view image collection through oblique photography, along with POS data; Align oblique photography with a 3D point cloud using collinearity equations: The inspection area is divided into cubic voxels with a side length of δ. For each voxel v, the center point c of the voxel v is calculated. v Truncated signed distance to the nearest surface TSDF(v): Among them, d(c v ) is the Euclidean distance from the voxel center to the nearest surface; τ is the cutoff distance; sgn is the sign function; Fuse the point cloud data reconstructed by LiDAR and oblique photography, and calculate the TSDF value TSDF of the fused voxel v by weighted average fused (v): Among them, w lida 、w photo is the weight coefficient; TSDF lidar (v) is the TSDF value generated by the three-dimensional point cloud data of voxel v; TSDF photo (v) TSDF value generated for the oblique photography data of voxel v; Determine obstacle type based on point cloud distribution within voxels: Calculate the variance of the normal vector of the point cloud within the voxel like If the threshold is set to a preset value, it is marked as a vertical obstacle, which includes towers and trees. The ground plane is fitted by RANSAC, excluding ground voxels. Combined with the RGB information of the oblique photography image, a lightweight convolutional network is used to classify obstacles and obtain the semantic label Label(v) and confidence: Among them, I v is the oblique photographic image block corresponding to the voxel projection; category c∈{wires, trees, buildings, others}; Softmax represents the Softmax function, f CNN Represents a lightweight convolutional network; Finally, we get a voxel map M with obstacle labels voxel , each element contains TSDF value, semantic label and confidence.

3. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 2 is characterized in that: The sub-regions are dynamically allocated based on the voxel map with obstacle labels, the Voronoi iteration of the improved Lloyd algorithm, and the performance of the drone, as follows: Assume that the drone set U={u1,u2,…,u i ,…,u N Each drone u i Equipped with performance parameters: battery life T i , load capacity L 0i and flight speed v i ; Where i∈[1,N], N represents the total number of drones; From the voxel map M voxel Eliminate obstacle areas: S free ={v∈M voxel |Label(v)=free}; Among them, S free is a set of non-obstruction voxels; Label(v)=free indicates that voxel v is free space; Randomly select N voxels as the initial position of the drone; Based on the performance of drones, define drone u i Weight w i for: Among them, T max , L max 、v max are the maximum endurance time, load capacity and flight speed in the UAV set respectively; α, β and γ are weight coefficients; Each free voxel v is assigned to the nearest drone u whose distance to the free voxel v is adjusted by weight k in, is the drone u at the tth iteration k location; For each drone u i , the corresponding Voronoi subregion at the tth iteration is updated to Update the center point position of the drone at the tth iteration to the weighted center of mass of the sub-region: When the region division converges, that is, the change amplitude of all drone sub-regions at the tth iteration When it is less than the threshold ∈, stop the iteration: in, represents the symmetric difference of sets; After the sub-area division, the mission load L of each UAV is calculated i : Among them, d(c v ,p i ) is the distance from the UAV to the center of the voxel; s(v) is the complexity of the inspection task; p i is the current position coordinate vector of UAV i; Adjust the sub-region division according to the equilibrium objective function LoadVar; in, is the average task load; According to the adjusted sub-area division, the sub-area allocation V corresponding to the drone is generated. i …,V N }.

4. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 3 is characterized in that: When a sudden task or drone failure occurs in the multi-drone collaborative inspection system, the originally divided drone sub-area V needs to be dynamically adjusted and the sub-areas are reallocated through a bidding mechanism, as follows: Define emergency tasks as newly added urgent tasks falling in a sub-area V j , some areas need to be reallocated to ensure that the task is executed as soon as possible; the drone failure is a drone u j Failure, the sub-area V j The task in cannot be completed, you need to change V j Divide and reallocate to other drones; UAV i Pair Area V j The affordability is determined by the sub-region redistribution cost function C ij Sure: Among them, p i For drone u i Current location; c Vj For sub-region V j The geometric center position of Indicates drone u i To sub-area V j Distance; T Vj For sub-region V j Estimated task working time; T i -T used,i Indicates the remaining flight time of the drone; ΔL i The change of UAV load after allocating the area; B min,Vj To complete sub-area V j The minimum power required for operation; λ1, λ2, λ3, λ4 are weight parameters; The nest broadcasts the sub-area or the area where the emergency task is located that needs to be reallocated. Each drone u i Calculate the pair region V j The bid value B ij : For sub-region V j , select the drone with the highest bidding value.

5. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 4 is characterized in that: The two-layer planning framework includes a global layer and a local layer. The global layer uses improved RRT* to generate a collision-free rough path P. global , the local layer, according to the collision-free coarse path P global , whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance.

6. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 5 is characterized in that: The improved RRT* is used to generate a collision-free rough path P global , as follows: The improved RRT* introduces a heuristic potential field for guidance, and preferentially samples points close to the target area: Among them, p bias is a biased sampling point; δ is the target biased sampling probability; p rand is the coordinate of the random sampling point; rand() means returning a uniformly distributed random number between 0 and 1; p uniform is a point sampled uniformly and randomly in space; Perform collision detection on obstacle labels in the voxel map, rejecting points in the obstacle area: Among them, CollisionFree(v) indicates whether voxel v can pass collision detection; Label(v) is the label attribute of voxel v; obstacle indicates that the voxel is an obstacle area; free indicates that the voxel is free space; Construct the initial tree T and set the starting point p start Insert T; Iteration tree extension: Under the obstacle constraint, sample a random point p rand , find the nearest point p in the tree near , generate a new point p new : Where η is the control step size; ||p rand -p near || is the Euclidean distance between two points; P new Perform collision detection, if there is no collision, add it to tree T; According to the path cost optimization, the tree is pruned and the path is optimized by reconnecting; The path node encounters the target position p goal When , return to the rough path P global , use the cubic spline interpolation to calculate the coarse path P global Smoothing, reducing redundancy.

7. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 6 is characterized in that: The collision-free rough path P global Whenever there are dynamic obstacles or environmental interference, VO and MPC are used to perform local trajectory optimization within a fixed prediction time window to achieve dynamic obstacle avoidance, as follows: Define speed obstacle VO: Among them, p u (t′) is the trajectory of the UAV at the current time t′; p o (t′) is the position trajectory of the dynamic obstacle; Guaranteed planning speed v * Not in a speed barrier area: Integrating the speed constraints proposed by VO Generate the optimal trajectory through MPC to avoid dynamic obstacles and minimize the path cost function J: Among them, ||p t ′-p goal || 2 is the distance from the current point to the target point; CollisionPenality is the collision cost term; w1, w2, w3 are the weight coefficients of the cost function, and N' is the cruising time.

8. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 1 is characterized in that: The S4 is specifically: Each UAV determines whether there is a space-time conflict with other UAVs on the path based on the path planning results; The initiator of conflict detection broadcasts conflict information to other UAVs, including: the area, location, preliminary time window, current path priority or mission status of the conflict; Each potentially conflicting drone calculates its bidding cost based on its own status and submits the bid value to the initiator; Based on all bid values, the initiator selects the drone with the highest bid value to cooperate in adjusting the path. Conflicting drones avoid conflicts by adjusting the time window and introduce sequence adjustment using the time window synchronization mechanism to ensure that drones are scheduled in an orderly manner in the conflict area.

9. The method for trajectory planning based on multi-UAV collaborative inspection according to claim 1, characterized in that: The S5 is specifically: The main communication link between drones and nests is built based on 5G, and the auxiliary communication link between drones is built based on self-organizing network and ultra-wideband; each drone u i per ton c Upload UAV status to the drone nest or neighboring drones in seconds i (t') includes: S i (t′)={p i (t′),E i (t′),P i (t′),Q i (t′)} Among them, p i (t′) is the current position of the UAV; E i (t′) is the current remaining power percentage; P i (t′) is the current waypoint number or route progress; Q i (t′) is the current task queue status; The self-organizing network dynamically maintains the communication links between drones to ensure the adaptability of the network topology. i and u j The communication signal is interrupted, triggering path reconstruction.

10. A trajectory planning system based on multi-UAV collaborative inspection, characterized in that: It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the trajectory planning method based on multi-UAV collaborative inspection as described in any one of claims 1 to 9.

Images