A multi-unmanned aerial vehicle cooperative inspection track planning method and device

Abstract

The application provides a multi-unmanned aerial vehicle cooperative inspection route planning method and device, and relates to the technical field of unmanned aerial vehicles. The method comprises the following steps: constructing a digital twin of an inspection scene, synchronizing real-time dynamic state information and environment perception data to each unmanned aerial vehicle; generating an inspection task based on the digital twin and distributing it to each unmanned aerial vehicle, generating a planned route according to the inspection task; detecting space-time conflicts between multiple unmanned aerial vehicles, controlling each unmanned aerial vehicle to perform avoidance according to the detection result, synchronizing the updated route to the digital twin; after the task is completed, obtaining actual flight data and inspection data of each unmanned aerial vehicle, evaluating the route planning effect, optimizing parameters and strategies for indicators that do not meet the preset standard, and updating the optimization results to the digital twin and synchronizing them to each unmanned aerial vehicle. The application can adapt to dynamically changing inspection scenes in real time, significantly improving the safety, inspection quality and work efficiency of multi-unmanned aerial vehicle cooperative inspection.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for collaborative inspection trajectory planning of multiple unmanned aerial vehicles (UAVs). Background Technology

[0002] Currently, multi-UAV collaborative inspection trajectory planning mainly adopts a pre-built static environment model, completing task allocation and trajectory planning offline. When the environment changes abruptly, map deviations can easily lead to trajectory failure, collisions, or missed detections. Task allocation is mostly based on physical parameters such as endurance and speed to achieve load balancing, without introducing semantic priority and heterogeneous payload adaptation, making it difficult to match high-value tasks with the optimal UAV. Trajectory planning decouples flyability from perception requirements, only aiming for the shortest path, without considering constraints such as shooting distance and field of view, often resulting in substandard data collection requiring secondary flights. Conflict resolution is prone to negotiation loss and deadlock under weak network conditions, and high-priority tasks are easily squeezed out by low-priority tasks. Fixed communication links cannot be dynamically adjusted according to environmental quality, making it difficult for closed-loop control to operate stably under weak network conditions.

[0003] Ensuring the safe, efficient, and high-quality execution of multi-drone collaborative inspections in dynamic scenarios is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This invention provides a method and apparatus for planning flight paths for multi-UAV collaborative inspections. This method can achieve safe, efficient and high-quality execution of multi-UAV collaborative inspections in dynamic scenarios.

[0005] Firstly, a method for collaborative inspection trajectory planning using multiple unmanned aerial vehicles (UAVs) is provided, including: A digital twin of the inspection scenario is constructed. The digital twin includes the real-time dynamic status information of each drone. The real-time dynamic status information and the real-time collected environmental perception data are spatiotemporally registered with the digital twin and then synchronized to each drone. Each inspection task is generated based on the inspection target entity in the digital twin, and each inspection task is assigned to each UAV. The planned flight path is generated by the digital twin according to the inspection task. Based on the planned flight path and the real-time dynamic status information of each UAV, the system detects spatiotemporal conflicts between multiple UAVs, controls each UAV to perform avoidance based on the detection results of the spatiotemporal conflicts, and synchronizes the updated flight path to the digital twin after the avoidance is completed. After the mission is completed, the actual flight data and inspection data of each UAV are obtained. The flight path planning effect is evaluated based on the actual flight data and inspection data. For indicators that do not meet the preset standards in the evaluation results, the parameters and strategies are optimized. The optimization results are updated to the digital twin and synchronized to each UAV.

[0006] In this way, by constructing a digital twin and establishing a spatiotemporal registration and synchronization mechanism for real-time dynamic status information and environmental perception data, each UAV has a globally consistent dynamic environmental cognition, providing a unified digital foundation for subsequent task allocation, trajectory planning, conflict detection and avoidance. On this basis, through full-process task allocation, trajectory planning, conflict resolution and closed-loop optimization, dynamic closed-loop control of multi-UAV collaborative inspection is realized, enabling the inspection system to adapt to environmental changes in real time, effectively suppressing trajectory failure, collision and missed detection problems caused by sudden environmental changes, and improving the adaptability and reliability of collaborative inspection in complex dynamic scenarios.

[0007] In some possible implementations, inspection tasks are generated based on the inspection target entities in the digital twin, and these tasks are assigned to various drones, including: Read each inspection target entity from the digital twin and generate corresponding inspection tasks for the inspection target entities; Each drone calculates its suitability score for each inspection task based on its own payload adaptability, endurance, and spatial position adaptability, and uses one or more inspection tasks with the highest scores as the bidding price. Tasks are allocated based on the bidding prices of each drone. Tasks with higher semantic priority are assigned to the drone with the highest suitability score. The total estimated time of inspection tasks assigned to a single drone does not exceed a preset proportion of its remaining flight time, and the allocation results minimize the variance of the total task time among all drones.

[0008] In this way, by introducing an auction-style task allocation mechanism that adapts semantic priority to heterogeneous payloads, high-value inspection tasks can be matched with drones of optimal performance. At the same time, dynamic balance of task load is achieved by minimizing endurance constraints and load variance, which fully leverages the hardware performance of heterogeneous drone clusters and improves the value density of inspection data.

[0009] In some possible implementations, a planned flight path is generated based on the inspection task using a digital twin, including: Starting from the current position of the UAV, and ending with a sequence of target positions arranged in descending order of semantic priority for each inspection task, a path search algorithm is used in a 3D grid map to search for a path and generate a global reference track covering the target position sequence. The grid markers in the 3D grid map include flyable areas, no-fly zones, or obstacles. Using the global reference trajectory as a reference, local trajectory rolling optimization is performed. In each control cycle, the current state of the UAV is used as the initial condition, and the optimal control sequence is solved with flight safety constraints, perception constraints and UAV maneuver constraints as boundary conditions. Based on the optimal control sequence, the actual planned trajectory is generated and stored in a digital twin.

[0010] In this way, by using a two-layer architecture of global reference trajectory planning and local trajectory rolling optimization, flight safety constraints and inspection perception constraints are embedded into the optimization objective, so that the generated trajectory simultaneously meets the requirements of flight maneuver safety and inspection imaging. This fundamentally avoids secondary reflights due to unqualified imaging, thereby improving inspection efficiency and data quality.

[0011] In some possible implementations, spatiotemporal conflicts between multiple UAVs are detected based on planned flight paths and real-time dynamic status information of each UAV, including: The planned flight paths and real-time dynamic status information of each UAV are read from the digital twin, and the position sequence of each UAV in the future time period is predicted based on the planned flight paths and real-time dynamic status information of each UAV. The system determines whether there is a spatiotemporal conflict between multiple drones based on their position sequence. Specifically, if the distance between the predicted positions of two or more drones at the same time is less than a preset safety interval, then a spatiotemporal conflict is determined to exist.

[0012] In this way, by performing spatiotemporal conflict prediction and detection of multi-aircraft flight paths in a unified digital twin, conflict prediction from a global perspective is achieved, providing accurate information on conflict type, conflict area, and time window for subsequent avoidance decisions.

[0013] In some possible implementations, each UAV is controlled to perform avoidance maneuvers based on the detection results of spatiotemporal conflicts. After the avoidance is completed, the updated flight path is synchronized to the digital twin, including: Based on the detection results of spatiotemporal conflict, the drones involved in the conflict and the location area where the conflict occurred are identified; based on the mission semantic priority of each drone involved in the conflict, the avoidance priority of each drone in this avoidance is determined. The right of way for each drone is determined according to the priority of avoidance. Among them, drones with lower task semantic priority give way to drones with higher task semantic priority; if the task semantic priority is the same, the drone with less remaining flight time has priority to pass. Based on the right-of-way determination, each drone executes an avoidance strategy, which includes any one or more combinations of the following methods: By adjusting flight speed, the time taken to reach the conflict area can be altered to avoid time conflicts; Generate offset waypoints near the conflict location area so that the UAV can bypass the conflict location area and return to its original flight path; Local replanning is performed on the flight path after the conflict location area to regenerate a detour flight path; In the process of implementing the above avoidance strategy, the communication status between the nest and the drone determines the scheduling method: when the communication is normal, the nest coordinates all drones to perform avoidance in a unified manner; when the communication is abnormal, each drone independently performs avoidance according to the preset autonomous avoidance rules based on the digital twin data stored locally. When three or more drones are detected to form a cyclical avoidance relationship during the avoidance process, the deadlock resolution mechanism is immediately triggered, and the drone with the lowest avoidance priority is forced to hover and wait, while the other drones continue to pass according to the original avoidance rules. The avoidance is restored after the hovering drone meets the passage conditions. After all the drones involved in the conflict have completed their avoidance maneuvers, each drone will synchronize its updated flight path to the digital twin.

[0014] In this way, the execution timeliness of high-priority inspection tasks is guaranteed by semantic priority-driven avoidance priority determination; the dual-mode switching of unified scheduling of the drone nest and autonomous avoidance of drones realizes global coordination optimization when communication is normal and local autonomy in weak network environment, solving the problem of negotiation failure of existing solutions in weak network environment; through the deadlock resolution mechanism, the lowest priority drone is forced to hover and wait, breaking the circular avoidance relationship and fundamentally avoiding scheduling deadlock.

[0015] In some possible implementations, after the mission is completed, the actual flight data and inspection data of each UAV are acquired, and the effectiveness of the flight path planning is evaluated based on the actual flight data and inspection data, including: After the mission is completed, actual flight data and inspection data are obtained from each drone. Based on actual flight data and inspection data, flight safety index values, inspection quality index values, execution efficiency index values, and collaborative performance index values ​​are obtained. The evaluation results of the flight path planning effect are obtained based on the flight safety index, inspection quality index, execution efficiency index, and collaborative performance index.

[0016] In this way, by constructing a multi-dimensional quantitative evaluation index system covering flight safety, inspection quality, execution efficiency and collaborative performance, a comprehensive quantitative evaluation of the trajectory planning effect is achieved, providing objective data support for subsequent iterative optimization of parameters and strategies.

[0017] In some possible implementations, parameters and strategies are optimized for indicators that do not meet the preset standards in the evaluation results, and the optimization results are updated to the digital twin and synchronized to each drone, including: Obtain the evaluation results and evaluation index values ​​for each dimension. If the evaluation index value for any dimension fails to meet the preset standard, the optimization process for the corresponding dimension will be triggered. If the flight safety indicators fail to meet the preset flight safety standards, the safe distance threshold and obstacle expansion coefficient of the flight path planning will be optimized, and the attribute information of the obstacles in the digital twin will be updated. If the inspection quality indicators do not meet the preset inspection quality standards, the shooting distance range and shooting pose constraint range of the inspection perception constraints will be adjusted, and the weight of the perception constraint penalty item in the local planning will be iteratively optimized. If the execution efficiency index does not meet the preset execution efficiency standard, then optimize the cost function weight of the global planning and adjust the load balancing parameters of task allocation. If the collaborative performance indicators do not meet the preset collaborative performance standards, then optimize the preset safety interval for spatiotemporal conflict detection, adjust the weight coefficient of task semantic priority, and update the conflict resolution rules and deadlock avoidance strategies. The optimization results are summarized and updated in the digital twin and synchronized to each drone.

[0018] In this way, by triggering a differentiated closed-loop optimization mechanism based on the evaluation results of different dimensions, the trajectory planning method achieves self-learning and self-adaptation, eliminating the need for repeated manual parameter tuning and continuously improving the execution effect of subsequent inspection tasks.

[0019] Secondly, a multi-UAV collaborative inspection trajectory planning device is provided, including: The first processing module is used to construct a digital twin of the inspection scenario. The digital twin includes the real-time dynamic status information of each UAV. The real-time dynamic status information and the real-time collected environmental perception data are spatiotemporally registered with the digital twin and then synchronized to each UAV. The second processing module is used to generate each inspection task based on the inspection target entity in the digital twin, allocate each inspection task to each UAV, and generate a planned flight path through the digital twin according to the inspection task. The third processing module is used to detect spatiotemporal conflicts between multiple drones based on the planned flight path and the real-time dynamic status information of each drone. Based on the detection results of the spatiotemporal conflict, it controls each drone to perform avoidance. After the avoidance is completed, the updated flight path is synchronized to the digital twin. The fourth processing module is used to acquire the actual flight data and inspection data of each UAV after the mission is completed, evaluate the flight path planning effect based on the actual flight data and inspection data, optimize the parameters and strategies for indicators that do not meet the preset standards in the evaluation results, update the optimization results to the digital twin and synchronize them to each UAV.

[0020] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; and one or more programs, wherein the one or more programs are stored in the one or more memories, and the one or more programs include instructions that, when executed by the one or more processors, cause the configured device to perform the method as described in the first aspect.

[0021] Fourthly, a computer storage medium is provided, the computer storage medium storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or the second aspect.

[0022] Fifthly, a computer program product is provided, the computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or the second aspect.

[0023] The beneficial effects of the second to fifth aspects can be referred to the introduction of the beneficial effects of the first aspect above, and will not be repeated here. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the multi-UAV collaborative inspection trajectory planning method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a multi-UAV collaborative inspection trajectory planning method according to another embodiment of the present invention; Figure 3 This is a schematic diagram of a multi-UAV collaborative inspection trajectory planning device that can be used to implement the method of the present invention; Figure 4 This is a schematic diagram of an electronic device provided by the present invention. Detailed Implementation

[0025] The solutions provided by the embodiments of the present invention will now be described with reference to the accompanying drawings. In the embodiments of the present invention, "multiple" refers to two or more objects, and "various kinds" refers to two or more types. Terms such as "first," "second," etc., are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.

[0026] Currently, multi-UAV collaborative inspection technology generally adopts an open-loop serial architecture of "static environment modeling—offline task allocation—fixed trajectory planning—centralized conflict resolution—pre-set link communication." This paradigm can complete basic inspection operations in conventional inspection scenarios such as power transmission lines, oil and gas pipelines, and rail transit. However, in complex scenarios such as emergency rescue, disaster investigation, and mountainous canyons where the environment is dynamically changing or communication conditions are limited, there is a significant deviation between the pre-built environmental map and the real-time environment. The task requirements do not match the characteristics of the UAV payload, trajectory planning and perception requirements are disconnected, and the communication link lacks dynamic scheduling methods. This leads to a series of problems such as trajectory planning failure, increased UAV collision risk, missed inspection targets, and inability to transmit key data, which seriously restrict the reliability and operational efficiency of UAV inspection.

[0027] Existing technologies primarily employ static environment modeling and fixed trajectory planning. This approach uses LiDAR and oblique photogrammetry to jointly model and generate point cloud maps of the inspection area. Sub-region allocation is performed based on Voronoi iteration, and a two-layer planning framework is used to generate trajectories. This type of solution uses an open-loop architecture of "model once, use throughout," making it suitable only for routine inspection scenarios with fixed environments. In scenarios with dynamic and rapidly changing environments, such as emergency inspections or disaster sites, significant discrepancies arise between the pre-built map and the real-time environment, making trajectory planning prone to failure and increasing collision risks. Furthermore, trajectory planning only optimizes for collision avoidance and shortest paths, neglecting inspection perception constraints such as shooting distance, pitch angle, illumination angle, imaging resolution, and field of view coverage. While the generated trajectories may meet flight safety requirements, they often fail to collect inspection data that meets quality inspection standards, necessitating a second flight.

[0028] The second existing technology is a task allocation scheme based on physical parameters. This type of scheme uses physical parameters such as the drone's endurance, flight speed, and payload weight as the basis for allocation, achieving load balancing through clustering algorithms or market mechanisms. The problem is that it only focuses on the physical capabilities of the drone, neglecting the semantic priority of the inspection targets and failing to adapt the payload characteristics of heterogeneous drones. For example, optical zoom is suitable for detailed inspection, thermal imaging for temperature sensing, and LiDAR for 3D modeling; these differences are ignored in task allocation. As a result, high-value tasks cannot be assigned to the drones with the best performance match, data value density is low, and the hardware performance of heterogeneous drones cannot be fully utilized.

[0029] The third existing technology is a distributed conflict resolution scheme based on the Contract Network protocol. This type of scheme relies on negotiation messages between UAVs to achieve distributed resolution of spatiotemporal conflicts among multiple UAVs. In weak network environments such as mountainous areas and canyons, negotiation messages are easily lost, causing conflict resolution to time out or fail. When multiple UAVs form a cyclical avoidance relationship, the system lacks deadlock detection and avoidance mechanisms, and scheduling deadlocks occur frequently. Furthermore, avoidance decisions are unrelated to task semantic priority, and it is common for high-priority tasks to be squeezed out and delayed by low-priority tasks.

[0030] To address the shortcomings of existing technologies, such as insufficient dynamic environmental adaptability, disconnect between task allocation and payload characteristics, decoupling of trajectory planning and perception requirements, failure of conflict resolution in weak network environments, lack of dynamic scheduling of communication links, and lack of a closed-loop iterative optimization mechanism in the overall architecture, this invention proposes a multi-UAV collaborative inspection trajectory planning method. This method first constructs a three-layer semantic digital twin: a static base layer stores digital elevation models, orthophotos, and point cloud maps; a semantic entity layer records the label attributes of inspection targets, obstacles, and no-fly zones; and a dynamic state layer stores real-time UAV status and environmental change information. Globally modeled UAVs collect point clouds and images, assign semantic labels through instance segmentation, and calculate priority weights based on industry standards and historical defect data to complete the twin initialization. The twin is updated using a combination of periodic and event-triggered updates, and synchronized to all UAVs after spatiotemporal registration via Kalman filtering, providing a unified digital foundation for subsequent stages. Based on this, a task pool is constructed with semantic priority as the core. Scores are calculated by comprehensively considering payload, endurance, and spatial adaptability. Optimal task allocation is achieved through bidirectional auction, matching high-priority tasks with the optimal UAV model, minimizing the overall load variance. After allocation, the trajectory planning phase begins, employing a two-layer architecture combining global semantic guidance and local perception constraints. The global layer generates a coarse trajectory, with a cost function incorporating semantic guidance costs to prioritize high-value targets and avoid risk areas. The local layer, based on model predictive control, embeds perception constraints such as shooting distance, pose, illumination angle, and coverage requirements as penalty terms into the objective function. It then performs rolling optimization with flight safety, perception requirements, and maneuver constraints as boundary conditions to generate a trajectory that simultaneously satisfies both flight safety and imaging quality.

[0031] During execution, a semantic priority-driven conflict resolution mechanism coordinates spatiotemporal conflicts among multiple drones. Drones with higher total priority are given priority to pass, while those with lower priority avoid them. In case of equal priority, drones with less remaining flight time are given priority. Avoidance strategies include speed adjustment, waypoint offset, and track replanning. In the event of communication interruption, drones autonomously avoid conflicts based on local twin data. When three or more drones form a cyclical avoidance mechanism, in nested mode, the lowest priority drone is forced to hover. In distributed mode, each drone generates a unique sorting based on its ID and priority to achieve automatic avoidance. After the mission is completed, the drones are quantitatively evaluated from four dimensions: flight safety, inspection quality, execution efficiency, and collaborative performance. The planning parameters and strategies are automatically optimized and synchronized to all drones, forming a closed-loop process.

[0032] In summary, by constructing a digital twin and establishing a spatiotemporal registration and synchronization mechanism for real-time dynamic status information and environmental perception data, each UAV possesses a globally consistent dynamic environmental awareness, providing a unified digital foundation for subsequent task allocation, trajectory planning, conflict detection, and avoidance. Based on this, through end-to-end task allocation, trajectory planning, conflict resolution, and closed-loop optimization, dynamic closed-loop control of multi-UAV collaborative inspection is achieved. This enables the inspection system to adapt to environmental changes in real time, effectively suppressing trajectory failures, collisions, and missed detections caused by sudden environmental changes, and improving the adaptability and reliability of collaborative inspection in complex dynamic scenarios.

[0033] The following is a detailed description of a multi-UAV heterogeneous collaborative inspection trajectory planning method based on semantic digital twin closed loop, as illustrated in the accompanying drawings.

[0034] Figure 1 This is a flowchart illustrating a multi-UAV collaborative inspection trajectory planning method, such as... Figure 1 As shown, the specific steps include: In step S1, a semantic digital twin of the inspection scenario is constructed and a real-time closed-loop update mechanism is established.

[0035] In this embodiment of the invention, the three-layer semantic digital twin is divided as follows: the static base layer stores the digital elevation model, digital orthophoto, and initial point cloud map of the inspection scene; the semantic entity layer records the semantic tags and attribute information of the inspection targets, obstacles, and no-fly zones, including the unique ID, geographic coordinates, outline size, inspection standards, priority weight, and historical defect probability of the inspection target entity; the obstacle entity includes the unique ID, geographic coordinates, outline size, obstacle type, and movement attributes; and the no-fly zone entity includes the unique ID, geographic boundary, restricted flight altitude, and restricted flight time period; the dynamic state layer stores the real-time position, speed, remaining battery power, payload status, and task completion rate of the UAV cluster, as well as the real-time position, speed, and motion trajectory prediction data of dynamic obstacles.

[0036] In this embodiment of the invention, a global modeling UAV equipped with LiDAR and a global shutter camera performs a full-coverage flight over the inspection scene, collecting 3D laser point clouds and multi-view images. Point cloud stitching and georeferencing are completed using POS data and RTK positioning information to generate an initial point cloud map and a digital elevation model. Taking a 110kV transmission line corridor in a mountainous area as an example, the line is 25km long and includes 32 transmission towers. After UAV No. 1, equipped with LiDAR and a wide-angle camera, completes a full-coverage flight, the point cloud map accuracy is 0.2m, and the digital elevation model grid accuracy is 0.5m. An instance segmentation network performs entity segmentation on the multi-view images, combines point cloud geometric features to complete entity matching and contour extraction, and assigns a unique semantic label and attribute information to each entity.

[0037] Semantic priority weights are calculated using the following formula: in, This represents the semantic priority weight; the higher the value, the higher the priority. The inspection importance coefficient is specified in industry standards, and its value is [value missing]. ; This represents the historical probability of defects occurring for this type of target. For real-time risk coefficients, a value of 1 is used for routine inspections, and a value of [missing value] is used for emergency inspections or disaster scenarios. ; For the weighting coefficients, satisfying In power transmission line inspection scenarios, tension towers... Take 5 for insulators, 4 for ordinary vegetation, and 1 for general vegetation. The default value is 0.5. Take 0.3, Take 0.2.

[0038] Twin updates employ a combination of periodic and event-triggered methods. Periodic trigger interval... Every 200ms, all UAVs report their own status and real-time environmental data to the nest. Event triggering conditions include the addition of obstacles, abnormal inspection targets, sudden environmental changes, or UAV malfunctions. After receiving the reported data, the nest edge computing nodes use a Kalman filter-based spatiotemporal registration algorithm to register the real-time point cloud and image data with the initial twin model, updating the position and trajectory of dynamic obstacles, the status of inspection targets, and the real-time status of the UAV cluster. New entities are added to the semantic entity layer by generating semantic labels and attribute information in real time using a lightweight semantic segmentation network. The updated twin data is synchronized to the onboard computing units of all online UAVs via a multi-mode communication link.

[0039] In step S2, distributed collaborative allocation of multiple UAV missions is performed based on semantic priority and heterogeneous payload adaptation.

[0040] In this embodiment of the invention, the drone swarm is denoted as... , This represents the total number of drones. Each drone... The attribute matrix is ​​defined as follows: in For maximum battery life, For cruising flight speed, For load type, For airborne computing power level, This shows the real-time remaining battery percentage. This indicates the current progress of the task.

[0041] Taking four heterogeneous drones as an example: No. 1 is equipped with a LiDAR and a wide-angle camera, and is a global modeling drone; No. 2 and No. 3 are equipped with 30x optical zoom cameras and RTK modules, and are optical detail inspection drones; No. 4 is equipped with an infrared thermal imaging camera, and is an infrared temperature measurement inspection drone.

[0042] A global inspection task pool is constructed based on the inspection target entities of semantic digital twins. Each task corresponds to an inspection target entity, and its attributes include a unique ID and semantic priority weight. Geographic coordinates, inspection and sensing requirements, compatible payload types, estimated execution time, and task coverage. The global task pool is divided according to spatial distribution and priority. Task subsets: High-priority tasks are grouped separately, similar tasks with a spatial distance of less than 50m are merged into one group, and tasks that cross no-fly zones are split into different subsets.

[0043] Each drone calculates a suitability score for each task subset. : in For load fit, the value is 1 if the load type matches, otherwise it is 0. ,when hour, = in, For the task The estimated execution time, For drones The remaining battery life; ,in, From the drone's current location to the mission The straight-line distance to the target location. The maximum spatial span of the scene; For the task Semantic priority weight normalization; For the weighting coefficients, satisfying , The default value is 0.4.

[0044] Each drone submits its three highest-performing task subset scores as bidding prices to the drone nest. The drone nest uses a two-way auction algorithm for allocation: high-priority tasks are assigned to the drone with the highest performance score, the total estimated duration of a single task does not exceed 70% of its remaining flight time, and the load variance of all drones is minimized. The load balancing objective function is: in For drones Total estimated duration of assigned tasks This is used to calculate the average task load. The allocation results are synchronized to all drones and semantic digital twins, and each drone generates a task execution queue sorted from high to low semantic priority.

[0045] When a drone malfunctions, its battery level drops below a threshold, its payload becomes abnormal, a new high-priority emergency mission is added, the data collection pass rate falls below a preset threshold, or there is a sudden change in the environment, the drone nest initiates dynamic reallocation, rebroadcasting mission information through a two-way auction mechanism, and the remaining available drones recalculate their suitability scores and bid for them.

[0046] In step S3, a two-layer multi-objective optimization trajectory planning with perceptual constraint embedding is performed.

[0047] In this embodiment of the invention, flight safety constraints include: the distance between all waypoints on the flight path and any obstacle entity is not less than the minimum safe distance. The flight altitude complies with airspace management requirements; the maximum turning angle does not exceed the maximum maneuvering turning angle of the drone. The maximum rate of ascent or descent shall not exceed the maneuverability limits. .

[0048] Inspection perception constraints include: the straight-line distance between the waypoint and the inspection target. Within the specified range; the pitch, roll, and heading angles must be within the preset range to ensure the inspected target is completely within the sensor's field of view; the angle between the shooting direction and the solar altitude angle must be no less than 30° to avoid backlighting; at least one waypoint of all detected parts of the target must meet the above constraints.

[0049] The track efficiency constraint requires the shortest total track length, the total flight time not exceeding 60% of the remaining flight time, and the fewest waypoints.

[0050] The global layer constructs a 3D grid map based on the flyable area of ​​the semantic digital twin, with a grid resolution of 1m. Each grid is marked with flyable, no-fly, or obstacle attributes. Starting from the UAV's current position, and taking the optimal shooting pose of the highest priority inspection target in the task queue as the first target point, a sequence of target points is constructed sequentially. The cost function of the improved A algorithm is: in From the starting point to the current node The actual cost of the flight path is the length of the path already flown. The heuristic cost from the current node to the target point is the three-dimensional Euclidean distance. The semantic guidance cost is set to 0 when approaching high-priority targets, and its value is inversely proportional to the distance when approaching obstacles, no-fly zones, or low-priority areas without tasks. After completing the trajectory planning for all target points in sequence, the trajectories are stitched together to form a global coarse trajectory, which is then smoothed using cubic B-spline interpolation to eliminate inflection points and abrupt changes.

[0051] Local layer sets prediction time window The control cycle is 2 seconds. The time is 50ms. Each control cycle collects the current state of the UAV, real-time environmental data, and the relative position of the inspection target to construct a local trajectory optimization objective function: in For drones The predicted state at any given moment; For the overall rough track in Reference status at any given moment; For control inputs, including throttle, roll angle, pitch angle, and yaw angle; As a collision penalty, it increases exponentially when the distance between the predicted trajectory and the obstacle is less than the safe distance; The penalty term for perception constraints is 0 when the shooting waypoint is reached and the pose satisfies the perception constraints, and increases exponentially when the constraints are not satisfied. For the weighting coefficients, satisfying The objective function is minimized by using flight safety constraints, perception constraints, and UAV maneuver constraints as boundary conditions. After obtaining the optimal control sequence, the control quantity for the first control cycle is output to the flight control system, and the rolling optimization is repeated for each control cycle.

[0052] In step S4, semantic priority-driven multi-machine spatiotemporal conflict distributed resolution and deadlock avoidance are performed.

[0053] A spatiotemporal conflict detection model is constructed based on the planned trajectories and real-time states of all UAVs in a semantic digital twin. Conflicts are classified into three types: spatial conflict refers to two or more UAVs operating at the same spatial location with a distance less than the minimum safe interval. Time conflict refers to two or more drones entering the same conflict zone within the same time window; spatiotemporal coupling conflict refers to the simultaneous existence of collision risks in both time and space dimensions. Each drone performs autonomous conflict detection based on synchronized twin data within each control cycle, while the drone nest simultaneously performs global conflict detection.

[0054] The conflict detection initiator broadcasts conflict information to all drones involved in the conflict, including the location of the conflict area, the conflict time window, the drone IDs involved, and the sum of the semantic priority weights of the corresponding tasks for each drone. The avoidance rules are as follows: drones with lower total semantic priority avoid those with higher priority; if priorities are the same, drones with less remaining battery life have priority; emergency tasks have higher priority than regular inspections.

[0055] There are three avoidance strategies: speed adjustment, which maintains the original track but adjusts the flight speed to change the arrival time in the conflict area; waypoint offset, which generates an offset waypoint that meets safety constraints near the conflict area, detours around the conflict area, and returns to the original track; and track replanning, which performs local replanning when the first two cannot be resolved.

[0056] Taking the time conflict between the No. 2 and No. 3 optical detail inspection drones near the same base tower as an example: after broadcasting the conflict information to the two drones, the task semantic priority is compared. If the priorities are the same, No. 2, which has less remaining battery life, passes first. No. 3 enters the conflict area by adjusting its speed by 2 seconds, thus resolving the conflict.

[0057] When communication between the nest and the drone is interrupted or the link is unstable, the drone involved in the conflict will autonomously execute the avoidance strategy according to the preset avoidance rules based on the locally synchronized twin data, without waiting for negotiation messages. After the avoidance is completed, the flight path will be synchronized to the nest and the twin after communication is restored.

[0058] The deadlock condition is: three or more drones form a cyclic avoidance relationship for more than three control cycles. The deadlock resolution strategy is as follows: In the global scheduling mode, the drone with the lowest priority is directly assigned to hover and avoid the drone based on the semantic priority of the task, thus breaking the cyclic avoidance relationship; in the distributed weak network mode, all conflicting drones generate a unique avoidance order based on their own ID and task priority. The drone with the smallest ID and the lowest priority automatically hovers and avoids the drone, and the others pass in turn.

[0059] In step S5, scenario-adaptive multi-mode communication link dynamic scheduling and hierarchical data transmission are performed.

[0060] In this embodiment of the invention, the three types of communication links are divided as follows: the 5G / 4G public network main link is used in areas with good public network signal coverage, undertaking global twin data synchronization and high-definition inspection data backhaul; the Mesh self-organizing network auxiliary link is used in areas with weak public network signal, undertaking state synchronization, conflict negotiation, and control command transmission between UAVs; the UWB communication link is used in short-range multi-UAV collaborative scenarios, undertaking relative positioning and obstacle avoidance command transmission. Each interval The time interval is 200ms. The nest and each UAV monitor the signal strength, bandwidth, transmission delay, packet loss rate and bit error rate of all available links.

[0061] Link switching is performed based on the following conditions: public network signal strength. And packet loss rate At that time, the 5G public network is the main link, the Mesh self-organizing network is the auxiliary link, and UWB is in standby mode; the public network signal strength is... And the packet loss rate is At that time, the Mesh self-organizing network is used as the main link, the 5G public network as the auxiliary link, and UWB is used for short-range coordination; the public network signal strength And packet loss rate At that time, the entire network adopts Mesh self-organizing network and UWB link, and the public network link is turned off to reduce power consumption; the distance between drones The UWB link is activated at the appropriate time. A smooth transition mechanism is adopted for the switchover: first, a new link connection is established and communication is verified to be normal; then, data transmission services are gradually switched over; and finally, the original link is shut down.

[0062] In this embodiment of the invention, data is divided into four levels according to value density and real-time requirements: Level L1 consists of core control data, such as UAV status data, control commands, conflict resolution messages, and key data for twin updates, with real-time requirements. The reliability requirement is 100%; Level 2 data consists of high-value inspection data. This includes defect images of high-priority inspection targets, abnormal environment data, and emergency scenario data, with real-time requirements. Reliability requirements Level L3 data is for routine inspections and requires real-time performance. Reliability requirements Level 4 data consists of non-real-time data, such as global modeling point cloud data, complete inspection videos, and historical archived data. It does not have strong real-time requirements or high reliability requirements. High-level data takes priority in consuming communication bandwidth, and L1 data is guaranteed to be transmitted under all circumstances. In weak network environments, the transmission rate and frequency of L3 and L4 data are automatically reduced, while reserved bandwidth ensures the real-time and reliable transmission of L1 and L2 data. When bandwidth is sufficient, all levels of data are transmitted at full speed and local backups are performed simultaneously.

[0063] Taking mountain inspection as an example: After the drone enters an area with weak public network signal, the system automatically switches to Mesh self-organizing network as the main link, reducing the frequency of regular inspection data transmission, reserving bandwidth to ensure the transmission of core control data and high-priority defect images, and using UWB link for relative positioning and obstacle avoidance command transmission when multiple drones conduct close-range collaborative inspections.

[0064] In step S6, the trajectory planning effect evaluation and dynamic iterative optimization of the digital twin closed loop are performed.

[0065] In this embodiment of the invention, the quantitative evaluation index system includes four dimensions: flight safety indicators, including the number of collision risks, the safety distance compliance rate, and the number of no-fly zone incursions, with a weight of 30%; inspection quality indicators, including the inspection target coverage rate, the inspection data qualification rate, the missed inspection rate, and the retake rate, with a weight of 40%; execution efficiency indicators, including the task completion time, the total flight path length, the endurance utilization rate, and the number of retake flights, with a weight of 20%; and coordination performance indicators, including the number of conflicts, the conflict resolution success rate, the number of deadlocks, and the task load balance, with a weight of 10%. After a single inspection task is completed, the nest calculates a comprehensive score, with a maximum score of 100 points.

[0066] Taking the inspection of power transmission lines in mountainous areas as an example: the overall score after completion was 92 points. The pass rate of inspection data was 95%. Root cause analysis revealed that the pass rate of some backlit images was low. Based on this, the lighting constraints of the flight path planning were optimized, and the positions and heading angles of the shooting waypoints were adjusted to avoid backlit shooting.

[0067] The optimization strategies for scores below the threshold are as follows: When flight safety indicators fail to meet the standards, optimize the safety distance threshold and obstacle expansion coefficient, and update the twin obstacle attributes; when inspection quality indicators fail to meet the standards, adjust the perception constraint parameters and shooting pose constraint range, and iteratively optimize the weight of perception constraint penalty terms; when execution efficiency indicators fail to meet the standards, optimize the global trajectory planning cost function and adjust the load balancing weight to reduce redundant waypoints and ineffective maneuvers; when collaborative performance indicators fail to meet the standards, optimize the conflict detection threshold and conflict resolution rules, adjust the task priority weight coefficient, and improve the deadlock avoidance mechanism. The optimized trajectory planning parameters and strategies are synchronously updated to the nest and all UAV onboard computing units and take effect in the next inspection mission.

[0068] The following describes the specific implementation process of this invention in detail using the example of heterogeneous UAV collaborative inspection of power transmission line corridors in mountainous areas. Those skilled in the art should understand that this scenario is merely an exemplary application of the invention and does not constitute a limitation on the scope of application of the invention.

[0069] The inspection scenario in this example is a 110kV transmission line corridor in a mountainous area. The line is 25km long and has 32 transmission towers along its route. The inspection area has significant terrain undulations, dense mountains and trees, and weak public network signal coverage in some sections, making it a typical complex terrain inspection scenario.

[0070] The drone swarm consists of four heterogeneous drones: Drone 1, equipped with a LiDAR and a wide-angle camera, is responsible for global modeling; Drones 2 and 3, equipped with 30x optical zoom cameras and RTK modules, are responsible for optical detail inspection; and Drone 4, equipped with an infrared thermal imaging camera, is responsible for infrared temperature measurement. The drone cluster is deployed within the substation at the starting point of the line, and is equipped with an edge computing server for deploying semantic digital twins and core algorithm modules.

[0071] The No. 1 global modeling UAV conducted a full-coverage flight over a 25km power transmission line corridor, collecting 3D laser point clouds and multi-view images along the line. It then used RTK positioning data to complete point cloud stitching and georegistration, generating an initial point cloud map and digital elevation model for the scene, forming a static base layer.

[0072] An instance segmentation network is employed to segment entities from multi-view images, extracting entities such as transmission towers, insulators, conductors, mountains, trees, and houses. Entity matching and contour extraction are then performed using point cloud geometric features. Each entity is assigned a unique semantic label and attribute information, generating a semantic entity layer. Key targets such as tension towers, insulators, and conductor joints are assigned high semantic priority according to industry standards for transmission line inspection, while ordinary vegetation and ground roads are assigned low semantic priority.

[0073] Each UAV reports its own status data and real-time environmental perception data to the nest every 200ms. The nest edge computing node uses a spatiotemporal registration algorithm based on Kalman filtering to register the real-time collected point cloud and image data with the initial twin model, update the position and trajectory of dynamic obstacles, the status of the inspected targets and the real-time status of the UAV cluster, and synchronize the updated twin data to the onboard computing units of all online UAVs to complete the initial construction and real-time synchronization of the semantic digital twin.

[0074] Based on the twin-like inspection target entities, a global inspection task pool is constructed. According to the spatial distribution and priority of semantic entities, the global task pool is divided into four task subsets: tower foundation inspection, insulator detail inspection, conductor infrared temperature measurement, and channel tree obstacle inspection. High-priority tasks are grouped separately, and tasks of the same type with a spatial distance of less than 50m are merged into one group.

[0075] As the auction initiator, the drone broadcasts task information from the global task pool to the four drones, including the semantic priority, compatible payload type, estimated execution time, and spatial location of each task. Each drone calculates its own compatibility score with each task subset, with payload compatibility as the first priority weight. If the drone's payload type does not match the task, the compatibility score is directly set to zero. At the same time, endurance compatibility and spatial compatibility are evaluated. If the estimated execution time of a task exceeds the drone's remaining endurance, the compatibility score is negative, and the drone automatically abandons the bidding for that task.

[0076] Calculations show that UAV 1 is suited for obstacle inspection and global modeling tasks, UAVs 2 and 3 for insulator detail inspection tasks, and UAV 4 for conductor infrared temperature measurement tasks. After collecting bids from all UAVs, the system uses a two-way auction algorithm to allocate tasks, assigning the insulator inspection and conductor temperature measurement tasks with the highest semantic priority to UAVs 2, 3, and 4, respectively, based on their suitability scores. This ensures that the total estimated time for tasks assigned to a single UAV does not exceed 70% of its remaining flight time and minimizes the variance in total task time among the four UAVs. After allocation, the results are synchronized to the task queues of the twin system and each UAV.

[0077] Each UAV, for its assigned task queue, arranges the task execution order from high to low according to semantic priority, and performs trajectory planning in sequence.

[0078] Global Layer: Starting from the UAV's current position, a target position sequence is constructed based on the optimal shooting poses of each inspection target in the task queue, ordered by semantic priority. An improved A algorithm can also be used to search paths within the 3D grid map. The grid markers in the 3D grid map have attributes such as flyable, no-fly, or obstacle. In addition to path length cost and heuristic cost, the cost function includes a semantic guidance cost, which prioritizes high semantic priority targets and avoids no-fly zones and obstacles during the search process. Track searches for all target positions are completed sequentially, and the tracks are stitched together to generate a global reference track. This track is then smoothed using cubic B-spline interpolation to eliminate inflection points and abrupt changes, ensuring it meets the UAV's maneuver constraints.

[0079] Local Layer: Using the global reference trajectory as a reference, model predictive control is employed for local trajectory rolling optimization. The prediction time window is set to 2 seconds, and the control cycle to 50 ms. Within each control cycle, using the current state of the UAV as the initial condition, and taking flight safety constraints, inspection perception constraints, and UAV maneuver constraints as boundary conditions, an objective function is constructed that includes tracking accuracy, collision penalties, and perception constraint penalties. The optimal control sequence is solved, and the first control variable is output to the flight control system for execution. This optimization process is repeated in each control cycle, rolling forward until the imaging tasks for all inspection targets are completed.

[0080] During the inspection, the drone nest and each drone perform real-time spatiotemporal conflict detection based on the synchronized flight paths and real-time dynamic status information of each drone in the digital twin. When it is detected that the time interval between drones No. 2 and No. 3 entering the same conflict location area near the same base tower in the future time period is less than the preset safety interval, a spatiotemporal conflict is determined to exist.

[0081] The nest broadcasts conflict information to UAVs 2 and 3. Based on the mission semantic priority of the two UAVs, if the priorities are the same, UAV 2, with less remaining battery life, has priority to pass. UAV 3 adjusts its flight speed and enters the conflict area with a 2-second delay, thus resolving the conflict.

[0082] When a drone enters an area without a public network signal, it automatically switches to a distributed autonomous conflict resolution mode. Each drone performs autonomous conflict detection based on locally stored digital twin data and independently executes avoidance according to preset avoidance rules, without waiting for negotiation messages, thus avoiding negotiation failure in weak network environments.

[0083] When three or more drones are detected forming a cyclical avoidance relationship that lasts for more than three control cycles, deadlock resolution is triggered: In the global scheduling mode, the drone with the lowest avoidance priority is forced to hover and wait; in the distributed mode, each drone automatically generates a unique avoidance order based on its own ID and task priority, with the drone with the lowest priority automatically hovering and waiting, and the remaining drones passing in sequence, thus avoiding deadlock at its source. After the avoidance is completed, each drone synchronizes its updated flight path to the digital twin.

[0084] In areas with good public network signal, the drone uses the 5G public network as the main link and the Mesh self-organizing network as the auxiliary link to transmit drone status data and high-definition inspection images in real time.

[0085] When the drone enters a mountainous area with weak public network signal, it automatically switches to the Mesh self-organizing network as the main link, and the UWB link is used for short-range collaboration. At the same time, it reduces the transmission frequency of routine inspection data and reserves bandwidth to ensure reliable transmission of core control data and high-priority defect images.

[0086] When multiple machines conduct close-range collaborative inspections, the UWB link is enabled for real-time transmission of relative positioning and obstacle avoidance commands, leveraging its low latency and high precision to enhance the safety of close-range collaboration.

[0087] The link switching adopts a smooth transition mechanism. After establishing a new link communication connection and verifying its normal operation, the data transmission services are gradually switched to the new link to avoid data loss during the switching process.

[0088] After the inspection mission is completed, the data center acquires actual flight data from each UAV, including actual flight trajectory, flight duration, and battery consumption, as well as actual inspection images and their shooting pose information for each inspection target. Based on this, evaluation index values ​​are calculated from four dimensions: flight safety, inspection quality, execution efficiency, and collaborative performance. The weighted sum of the index values ​​from each dimension yields a comprehensive score.

[0089] This embodiment received a comprehensive score of 92. Regarding the 95% pass rate for inspection data, root cause analysis revealed a low pass rate for some backlit images, indicating that the inspection quality indicator did not meet the preset standard. This triggered an optimization process for the inspection quality dimension: adjusting the shooting distance range and shooting pose constraints, optimizing lighting constraints, and iteratively updating the weight of the perception constraint penalty in the local planning, enabling the optimized trajectory planning to automatically avoid backlit shooting postures.

[0090] The optimized parameters are summarized and updated in the digital twin, and synchronized to all UAVs via communication links, automatically taking effect in the next inspection mission. The above evaluation and optimization process is repeated after each inspection mission, enabling continuous self-learning and self-adaptation of the trajectory planning method.

[0091] This invention constructs a semantic digital twin end-to-end closed-loop architecture. Using the digital twin as a unified data foundation, this architecture breaks down data barriers between environmental modeling, task allocation, trajectory planning, perception execution, twin updates, and iterative optimization through spatiotemporal registration and synchronization mechanisms of real-time dynamic status information and environmental perception data, forming a complete closed-loop process. Compared to the static open-loop model where environmental modeling and trajectory planning are separated in existing technologies, this architecture can adapt to dynamically changing inspection scenarios in real time. In complex scenarios such as emergency inspections and disaster reconnaissance, environmental adaptability and flight safety are significantly improved.

[0092] This invention proposes a distributed task allocation mechanism based on semantic priority and heterogeneous payload adaptation. This mechanism centers on the semantic value of the inspection target and uses a bidirectional auction algorithm to prioritize high-semantic-priority tasks for the UAVs with the highest payload adaptability, while also considering endurance adaptability, spatial adaptability, and load balancing. This mechanism fully leverages the differentiated hardware performance of heterogeneous UAV clusters, ensuring optimal flight performance for high-priority inspection tasks, while achieving dynamic load balancing and effectively improving the value density of inspection data and the overall operational efficiency of the cluster.

[0093] This invention embeds inspection and perception constraints into the entire trajectory planning process. Through a two-layer linkage of global-level semantic-guided search and local-level perception constraint optimization, perception requirements such as shooting distance, shooting pose, lighting conditions, and field of view coverage are unified with flight safety and maneuver constraints and incorporated into the trajectory planning optimization objective. This solution overcomes the limitation of existing technologies where trajectory planning and perception requirements are decoupled, achieving joint optimization of flight safety, inspection coverage, and data quality. The generated trajectory simultaneously meets maneuver constraints and imaging requirements, fundamentally avoiding secondary flights due to unqualified imaging.

[0094] This invention proposes a semantic priority-driven distributed conflict resolution mechanism. This mechanism uses task semantic priority as the primary basis for avoidance determination, ensuring the right-of-way for high-priority tasks. Under normal communication conditions, global coordination is optimized through unified scheduling by the drone nesting system. In the event of communication failure, each drone autonomously avoids obstacles based on its local twin data, without waiting for negotiation messages, thus preventing negotiation failures in weak network environments. Furthermore, for multi-drone cyclical avoidance scenarios, deadlock is resolved by forcibly designating the lowest-priority drone to hover and wait, improving the conflict resolution success rate and system robustness of multi-drone collaboration.

[0095] This invention implements scenario-adaptive multi-mode communication dynamic scheduling and hierarchical data transmission. By monitoring the status parameters of three types of communication links—public network, Mesh self-organizing network, and UWB—in real time, the main link is automatically switched according to signal strength and inspection scenario. Simultaneously, transmitted data is divided into four levels based on data value density, prioritizing bandwidth resources for core control data and high-value inspection data in weak network environments. This solution maintains the stability of digital twin closed-loop control even under limited communication conditions, ensuring the real-time and reliable transmission of critical data.

[0096] This invention constructs a quantitative evaluation system and a closed-loop iterative optimization mechanism for trajectory planning effectiveness. The system quantitatively scores a single inspection task from four dimensions: flight safety, inspection quality, execution efficiency, and collaborative performance. It automatically identifies the root causes of unmet criteria, triggers parameter and strategy optimization for the corresponding dimensions, updates the optimization results to the digital twin, and synchronizes them to each UAV. This mechanism enables the trajectory planning method to possess self-learning, self-optimization, and self-adaptation capabilities, eliminating the need for repeated manual parameter tuning and lowering the barrier to engineering application.

[0097] Figure 2 This is a flowchart illustrating a multi-UAV collaborative inspection trajectory planning method, such as... Figure 2 As shown, it specifically includes: Step 210: Construct a digital twin of the inspection scenario. The digital twin includes the real-time dynamic status information of each UAV. After spatiotemporal registration of the real-time dynamic status information and the real-time collected environmental perception data with the digital twin, the data is synchronized to each UAV.

[0098] Step 220: Generate each inspection task based on the inspection target entity in the digital twin, assign each inspection task to each UAV, and generate a planned flight path through the digital twin according to the inspection task.

[0099] Step 230: Detect spatiotemporal conflicts between multiple UAVs based on the planned flight path and the real-time dynamic status information of each UAV. Control each UAV to perform avoidance according to the detection results of the spatiotemporal conflict. After the avoidance is completed, synchronize the updated flight path to the digital twin.

[0100] Step 240: After the task is completed, obtain the actual flight data and inspection data of each UAV, evaluate the flight path planning effect based on the actual flight data and inspection data, optimize the parameters and strategies for indicators that do not meet the preset standards in the evaluation results, update the optimization results to the digital twin and synchronize them to each UAV.

[0101] This invention, through the construction of a digital twin and the establishment of a spatiotemporal registration and synchronization mechanism for real-time dynamic status information and environmental perception data, enables each UAV to have a globally consistent dynamic environmental perception, providing a unified digital foundation for subsequent task allocation, trajectory planning, conflict detection, and avoidance. On this basis, through the entire process of task allocation, trajectory planning, conflict resolution, and closed-loop optimization, dynamic closed-loop control of multi-UAV collaborative inspection is realized, enabling the inspection system to adapt to environmental changes in real time, effectively suppressing trajectory failure, collisions, and missed detections caused by sudden environmental changes, and improving the adaptability and reliability of collaborative inspection in complex dynamic scenarios.

[0102] Figure 3 This is a schematic diagram of a multi-UAV collaborative inspection trajectory planning device, such as... Figure 3 As shown, it specifically includes: The first processing module is used to construct a digital twin of the inspection scenario. The digital twin includes the real-time dynamic status information of each UAV. The real-time dynamic status information and the real-time collected environmental perception data are spatiotemporally registered with the digital twin and then synchronized to each UAV. The second processing module is used to generate inspection tasks based on the inspection target entities in the digital twin, allocate each inspection task to each UAV, and generate planned flight paths through the digital twin according to the inspection tasks.

[0103] The third processing module is used to detect spatiotemporal conflicts between multiple UAVs based on the planned flight path and the real-time dynamic status information of each UAV. Based on the detection results of the spatiotemporal conflict, it controls each UAV to perform avoidance. After the avoidance is completed, the updated flight path is synchronized to the digital twin.

[0104] The fourth processing module is used to acquire the actual flight data and inspection data of each UAV after the mission is completed, evaluate the flight path planning effect based on the actual flight data and inspection data, optimize the parameters and strategies for indicators that do not meet the preset standards in the evaluation results, update the optimization results to the digital twin and synchronize them to each UAV.

[0105] Those skilled in the art will readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the present invention.

[0106] It should be noted that, Figure 3 The division of modules / units is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single data acquisition module. The integrated modules described above can be implemented either in hardware or as software functional modules.

[0107] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned multi-UAV collaborative inspection trajectory planning methods. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the multi-UAV collaborative inspection trajectory planning method shown in any embodiment of the present invention by calling the computer program.

[0108] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 400 includes a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, for example, via a bus 402. Optionally, the electronic device 400 may further include a transceiver 404, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 404 is not limited to one type, and the structure of the electronic device 400 does not constitute a limitation on the embodiments of the present invention.

[0109] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0110] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0111] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any one of the above-mentioned multi-UAV collaborative inspection trajectory planning methods.

[0112] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0113] It should be noted that the terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of the invention described herein can be implemented in an order other than that shown or described.

[0114] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product in one or more computer-readable media, the computer-readable media including computer-readable program code.

[0115] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for collaborative inspection trajectory planning using multiple unmanned aerial vehicles (UAVs), characterized in that, include: A digital twin of the inspection scenario is constructed. The digital twin includes the real-time dynamic status information of each UAV. The real-time dynamic status information and the real-time collected environmental perception data are spatiotemporally registered with the digital twin and then synchronized to each UAV. Each inspection task is generated based on the inspection target entity in the digital twin, and each inspection task is assigned to each UAV. The planned flight path is generated by the digital twin according to the inspection task. Based on the planned flight path and the real-time dynamic status information of each UAV, the spatiotemporal conflict between multiple UAVs is detected. According to the detection result of the spatiotemporal conflict, each UAV is controlled to perform avoidance. After the avoidance is completed, the updated flight path is synchronized to the digital twin. After the task is completed, the actual flight data and inspection data of each UAV are obtained. The flight path planning effect is evaluated based on the actual flight data and inspection data. For indicators that do not meet the preset standards in the evaluation results, the parameters and strategies are optimized. The optimization results are updated to the digital twin and synchronized to each UAV.

2. The method according to claim 1, characterized in that, The process of generating inspection tasks based on the inspection target entities in the digital twin and assigning each inspection task to each of the drones includes: Each inspection target entity is read from the digital twin, and a corresponding inspection task is generated for each inspection target entity; Each UAV calculates its compatibility score with each inspection task based on its own payload adaptability, endurance, and spatial location adaptability, and uses one or more inspection tasks with the highest scores as the bidding price. Tasks are allocated based on the bidding prices of each UAV, with tasks of higher semantic priority being assigned to the UAV with the highest suitability score. The total estimated duration of inspection tasks assigned to a single UAV does not exceed a preset proportion of its remaining flight time, and the allocation result minimizes the variance of the total task duration among the UAVs.

3. The method according to claim 2, characterized in that, The step of generating a planned flight path based on the inspection task using the digital twin includes: Starting from the current position of the UAV, and ending with a sequence of target positions arranged in descending order of semantic priority for each inspection task, a path search algorithm is used in a 3D grid map to search for a path and generate a global reference track covering the sequence of target positions; wherein, the grid markers of the 3D grid map include flyable areas, no-fly areas, or obstacles. Using the global reference trajectory as a reference, local trajectory rolling optimization is performed. In each control cycle, the current state of the UAV is used as the initial condition, and the optimal control sequence is solved with flight safety constraints, perception constraints, and UAV maneuver constraints as boundary conditions. Based on the optimal control sequence, the actual planned trajectory is generated and stored in the digital twin.

4. The method according to claim 1, characterized in that, The detection of spatiotemporal conflicts between multiple UAVs based on the planned flight path and the real-time dynamic status information of each UAV includes: The planned flight paths and real-time dynamic status information of each UAV are read from the digital twin, and the position sequence of each UAV in the future time period is predicted based on the planned flight paths and real-time dynamic status information of each UAV. The location sequence is used to determine whether there is a spatiotemporal conflict between multiple drones. If the distance between the predicted positions of two or more drones at the same time is less than a preset safety interval, then a spatiotemporal conflict is determined to exist.

5. The method according to claim 1, characterized in that, The process of controlling each UAV to perform avoidance based on the detection results of the spatiotemporal conflict, and synchronizing the updated flight path to the digital twin after the avoidance is completed, includes: Based on the detection results of the spatiotemporal conflict, the UAVs involved in the conflict and the location area where the conflict occurred are determined; based on the task semantic priority of each UAV involved in the conflict, the avoidance priority of each UAV in this avoidance is determined. The right of way for each drone is determined according to the avoidance priority. Drones with lower task semantic priority give way to drones with higher task semantic priority. If the task semantic priorities are the same, the drone with less remaining flight time has priority to pass. Based on the right-of-way determination, each drone executes an avoidance strategy, wherein the avoidance strategy includes any one or more combinations of the following: By adjusting flight speed, the time taken to reach the conflict area can be altered to avoid time conflicts; Generate offset waypoints near the conflict location area so that the UAV can bypass the conflict location area and return to its original flight path; Local replanning is performed on the flight path after the conflict location area to regenerate a detour flight path; In the process of implementing the above avoidance strategy, the communication status between the nest and the drone determines the scheduling method: when the communication is normal, the nest coordinates all drones to perform avoidance in a unified manner; when the communication is abnormal, each drone independently performs avoidance according to the preset autonomous avoidance rules based on the digital twin data stored locally. When three or more drones are detected to form a cyclical avoidance relationship during the avoidance process, the deadlock resolution mechanism is immediately triggered, and the drone with the lowest avoidance priority is forced to hover and wait, while the other drones continue to pass according to the original avoidance rules. The avoidance is restored after the hovering drone meets the passage conditions. After all the drones involved in the conflict have completed the avoidance, each drone will synchronize its updated flight path to the digital twin.

6. The method according to claim 1, characterized in that, After the task is completed, the actual flight data and inspection data of each UAV are obtained. The effectiveness of the flight path planning is evaluated based on the actual flight data and inspection data, including: After the mission is completed, actual flight data and inspection data are obtained from each of the aforementioned drones; Based on the actual flight data and the inspection data, the flight safety index value, inspection quality index value, execution efficiency index value, and collaborative performance index value are obtained: The evaluation results of the flight path planning effect are obtained based on the flight safety index value, the inspection quality index value, the execution efficiency index value, and the collaborative performance index value.

7. The method according to claim 6, characterized in that, The process of optimizing parameters and strategies for indicators that do not meet the preset standards in the evaluation results, updating the optimization results to the digital twin, and synchronizing them to each drone includes: Obtain the evaluation results and evaluation index values ​​for each dimension. If the evaluation index value for any dimension fails to meet the preset standard, the optimization process for the corresponding dimension is triggered. If the flight safety indicators fail to meet the preset flight safety standards, the safe distance threshold and obstacle expansion coefficient of the flight path planning are optimized, and the attribute information of the obstacles in the digital twin is updated. If the inspection quality indicators do not meet the preset inspection quality standards, the shooting distance range and shooting pose constraint range of the inspection perception constraints will be adjusted, and the weight of the perception constraint penalty term of the local planning will be iteratively optimized. If the execution efficiency index fails to meet the preset execution efficiency standard, the cost function weight of the global planning is optimized and the load balancing parameters of task allocation are adjusted. If the collaborative performance index does not meet the preset collaborative performance standard, then optimize the preset safety interval for spatiotemporal conflict detection, adjust the weight coefficient of task semantic priority, and update the conflict resolution rules and deadlock avoidance strategy. The optimization results are summarized and updated in the digital twin, and synchronized to each of the drones.

8. A multi-UAV collaborative inspection trajectory planning device, characterized in that, include: The first processing module is used to construct a digital twin of the inspection scenario. The digital twin includes the real-time dynamic status information of each UAV. The real-time dynamic status information and the real-time collected environmental perception data are spatiotemporally registered with the digital twin and then synchronized to each UAV. The second processing module is used to generate each inspection task based on the inspection target entity in the digital twin, allocate each inspection task to each UAV, and generate a planned flight path through the digital twin according to the inspection task. The third processing module is used to detect spatiotemporal conflicts between multiple UAVs based on the planned flight path and the real-time dynamic status information of each UAV, control each UAV to perform avoidance according to the detection result of the spatiotemporal conflict, and synchronize the updated flight path to the digital twin after the avoidance is completed. The fourth processing module is used to acquire the actual flight data and inspection data of each UAV after the task is completed, evaluate the flight path planning effect based on the actual flight data and inspection data, optimize the parameters and strategies for indicators that do not meet the preset standards in the evaluation results, update the optimization results to the digital twin and synchronize them to each UAV.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the multi-UAV collaborative inspection trajectory planning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the multi-UAV collaborative inspection trajectory planning method as described in any one of claims 1 to 7.

Images