A multi-heterogeneous unmanned aerial vehicle cooperative inspection method, device and equipment

Abstract

This invention belongs to the field of UAV inspection technology and discloses a method, device, and equipment for collaborative inspection of multiple heterogeneous UAVs. The method includes: acquiring environmental point cloud data collected by an exploratory UAV and generating an initial occupancy map; extracting target occupancy voxels and generating an inspection waypoint set within the effective line-of-sight of the onboard gimbal camera of a shooting UAV; constructing path selection variables based on this waypoint set and obtaining a comprehensive observation quality score based on its corresponding ambiguity and resolution indices; establishing a model with the deployment status and facility allocation relationship of automatic battery swapping stations as the main problem and the path selection variables as sub-problems, and using the comprehensive observation quality score being greater than the engineering quality threshold as an observation quality constraint; under the condition of satisfying this constraint, calling the logical basis Benders decomposition algorithm in a closed-loop iterative solution to generate a global path set; extracting a rolling waypoint set from this set to perform local spatial optimization, generating local path segments to drive the flight of the shooting UAV. This invention breaks the traditional separation between path planning and optical imaging physical processes, reducing motion blur and sampling errors, and ensuring high fidelity of the inspection images.

Description

Technical Field

[0001] This invention relates to the field of drone inspection technology, specifically to a method, apparatus, and equipment for collaborative inspection of multiple heterogeneous drones, applicable to infrastructure inspection in indoor and outdoor 3D environments. Background Technology

[0002] Three-dimensional collaborative inspection of infrastructure is a core component of ensuring the safe operation of large-scale projects. With the evolution of unmanned aerial vehicle (UAV) technology, swarm collaborative operations are gradually becoming mainstream. When dealing with three-dimensional workspaces characterized by large scale, high complexity, and dynamic unknown obstacles, traditional solutions generally employ a homogeneous swarm model, requiring each UAV to simultaneously carry both three-dimensional sensing equipment for environmental perception and high-definition optical payloads for target observation. To manage such swarms, this field typically introduces operations research models such as the Logistic Fundamental Decomposition Algorithm (RDD), breaking down the task into a main problem of automatic battery swapping station site selection and allocation, and a sub-problem of UAV path planning, supplemented by rolling time-domain algorithms for dynamic obstacle avoidance.

[0003] However, homogeneous configurations inevitably lead to extremely high physical loads, severely limiting the effective battery endurance margin of the flight platform and making it extremely difficult for the fleet to perform long-term hovering and detailed observations of points of interest far from automatic battery swapping stations. To overcome the physical load limits of a single aircraft, a heterogeneous fleet architecture with separation of sensing and imaging is introduced. Although decoupling at the physical hardware level can unleash the platform's endurance potential, existing collaborative inspection systems still have fatal technical blind spots at the underlying algorithm and control logic levels. Specifically, when existing technologies handle the multi-aircraft collaborative path planning sub-problem and generate feedback logic cutting planes for the main problem, the construction of mathematical evaluation dimensions is severely limited, usually confined to conventional kinematic and resource constraints such as geometric flight distance, time consumption, or battery capacity limits. This purely geometric and energy-oriented operations research planning mechanism completely severs the necessary connection between macroscopic path planning and microscopic optical imaging physical processes at the underlying logic level.

[0004] In practical engineering inspection scenarios, high-quality image acquisition is strictly controlled by the instantaneous coupling of multiple optical and kinematic physical conditions. This not only requires the aircraft to reach the target position in spatial topology, but is also rigidly constrained by the aircraft's instantaneous relative velocity, the camera's single-frame exposure time, and the actual physical observation distance. Because existing path planning algorithm frameworks lack the ability to quantify and derive these continuous variables, they cannot effectively transform the continuous fidelity index characterizing optical imaging properties into hard constraint boundaries in discrete operations research node evaluation. This lack of underlying logical constraint directly leads to a very high risk of failure when the globally optimal path, calculated mathematically by existing systems, is executed in physical space. Specifically, the flight platform often experiences severe transient motion blur in the imaging system due to excessively high relative speeds upon approach, or the actual resolution is far below engineering standards due to excessive sampling distances in physical space. Ultimately, this renders the inspection data, acquired at great computational and flight energy costs, worthless for engineering evaluation. Summary of the Invention

[0005] In the first aspect, the present invention provides a collaborative inspection method for multiple heterogeneous UAVs, which aims to reduce the risk of image failure caused by motion blur or sampling distance exceeding tolerance due to excessive relative speed of the aircraft.

[0006] To address the above problems, the present invention provides the following technical solution: A collaborative inspection method for multiple heterogeneous unmanned aerial vehicles (UAVs) includes: Acquire environmental point cloud data by performing flight operations of an exploratory drone, and generate an initial occupied map based on the environmental point cloud data; Extract the set of points of interest to be inspected from the initial occupied map. The associated target occupies a voxel within the effective line-of-sight range of the onboard gimbal camera of the shooting drone. Generate a set of inspection waypoints within the range ; Based on the inspection waypoint set Construct path selection variables ; Based on the path selection variables The ambiguity index of the corresponding inspection waypoint With resolution indicators Obtain the overall observation quality score ; Establish based on the deployment status of automatic battery swapping stations Relationship with facility allocation The main problem is the allocation of location as the decision variable for the main problem, and the path selection variable is used as the main problem. Sub-problems are the decision variables for sub-problems; based on the comprehensive observation quality score. Greater than the engineering quality threshold As an observation quality constraint, under the condition that the observation quality constraint is satisfied, the logical basis Benders decomposition algorithm is invoked to perform closed-loop iterative solution on the main problem and the subproblems, generating a global path set. ; From the global path set Extract a rolling waypoint set, and perform spatial optimization within the rolling waypoint set to generate local path segments. Drive the shooting drone along the local path segment flight.

[0007] Secondly, the present invention provides a multi-heterogeneous UAV collaborative inspection device, which aims to reduce the risk of image failure caused by motion blur or sampling distance exceeding tolerance due to excessive relative speed of the aircraft.

[0008] To address the above problems, the present invention provides the following technical solution: A multi-heterogeneous unmanned aerial vehicle (UAV) collaborative inspection device includes: The initial mapping and waypoint generation module is used to acquire environmental point cloud data obtained during the flight of the exploratory UAV, generate an initial occupied map based on the environmental point cloud data, and extract the set of points of interest to be inspected from the initial occupied map. The associated target occupies a voxel within the effective line-of-sight range of the onboard gimbal camera of the shooting drone. Generate a set of inspection waypoints within the range ; The quality score acquisition module is used to obtain a score based on the inspection waypoint set. Construct path selection variables ; Variables selected based on the path The ambiguity index of the corresponding inspection waypoint With resolution indicators Obtain the overall observation quality score ; The processing module is used to establish the deployment status of automated battery swapping stations. Relationship with facility allocation The main problem is the allocation of location as the decision variable for the main problem, and the path selection variable is used as the main problem. Sub-problems are the decision variables for sub-problems; based on the comprehensive observation quality score. Greater than the engineering quality threshold As an observation quality constraint, under the condition that the observation quality constraint is satisfied, the logical basis Benders decomposition algorithm is invoked to perform closed-loop iterative solution on the main problem and the subproblems, generating a global path set. ; Output module, used to extract from the global path set Extract a rolling waypoint set, and perform spatial optimization within the rolling waypoint set to generate local path segments. Drive the shooting drone along the local path segment flight.

[0009] Thirdly, the present invention provides an electronic device for reducing the risk of image failure caused by motion blur or sampling distance exceeding tolerance due to excessive relative speed of the aircraft.

[0010] To address the above problems, the present invention provides the following technical solution: An electronic device includes a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the multi-heterogeneous unmanned aerial vehicle collaborative inspection method as described in the first aspect.

[0011] Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: This application introduces a heterogeneous swarm collaborative operation architecture, in which exploratory UAVs acquire environmental point cloud data first and generate an initial occupancy map, completely removing the heavy 3D spatial mapping load from the shooting UAVs. This avoids the physical contradiction between high-precision perception and long endurance faced by single-configuration UAVs, enabling the swarm to perform long-term hovering and detailed image observation at complex infrastructure points of interest far from automatic battery swapping stations.

[0012] 2. This application integrates the ambiguity index of the transient relative velocity of the associated UAV with the resolution index of the associated physical observation distance during the closed-loop iterative solution process of the logical basis Benders decomposition algorithm, constructs a comprehensive observation quality score, and uses it as a rigid observation quality constraint for the sub-problem. This mechanism forces the mathematical solver to take into account optical and dynamic constraints such as camera exposure time and relative motion speed when planning the global path. Compared with traditional operations research models that are only guided by geometric flight distance or battery consumption, this scheme ensures that the optimal path solved by the system in the mathematical space has absolute optical fidelity when executed in the physical space. It fundamentally eliminates the risk of image failure caused by motion blur or sampling distance deviation due to excessive relative velocity of the aircraft, and ensures that the collected inspection data has high standard engineering evaluation value. 3. This application extracts the rolling waypoint set from the global path set for local spatial optimization, and combines it with the system state space equation to transform the local path segment into the underlying control input, giving the UAV real-time dynamic obstacle avoidance capability in the unknown three-dimensional working space. It also more strictly locks the transient dynamic performance of the aircraft when approaching the target, ensuring that the optical quality baseline demonstrated in the early mathematical planning stage can be accurately realized in the actual inspection environment accompanied by real physical disturbances, which greatly improves the robustness and success rate of the entire automated inspection system. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the overall operation of the multi-heterogeneous UAV collaborative inspection system according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the LBBD algorithm framework according to an embodiment of the present invention; Figure 3 This is a flowchart of the D-RHLP dynamic path execution strategy according to an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0015] Please see Figure 1 This invention provides a multi-heterogeneous UAV collaborative inspection system. This system, through the collaborative operation of its internally configured functional modules, executes a dynamic constraint optimization decision-making closed-loop process, specifically including: (1) Heterogeneous drone swarms: including Exploratory drones and A camera drone, M represents the total number of drones; the exploratory drones are equipped with rotating lidar and gimbal cameras, and the shooting drones are equipped with gimbal cameras; the exploratory drones are used for environmental mapping and preliminary observation of points of interest, and the shooting drones are used for high-quality observation of points of interest, adapted to ABSS endurance support mode.

[0016] Specifically, for unknown areas to be inspected or 3D work spaces with sudden obstacles, deploying one or two exploratory drones for preliminary environmental reconnaissance can meet the needs of full-domain environmental scanning and mapping in most engineering scenarios. This not only avoids the waste of system hardware costs caused by too many drones carrying high-value sensors, but also reduces the computational complexity of obstacle avoidance and communication within the drone swarm. In addition, setting the number of shooting drones to a dynamically adjustable range limited by the total number of drones allows the entire collaborative inspection system to flexibly expand its cluster nodes according to the size of the infrastructure to be inspected, the spatial distribution density of points of interest, and the specific task time constraints.

[0017] The exploratory UAV is equipped with a rotating lidar and a gimbal camera. During system operation, the exploratory UAV is used for environmental mapping and preliminary observation of points of interest. Its implementation principle is as follows: the exploratory UAV uses its onboard rotating lidar for active environmental perception, transmitting and receiving laser beams to penetrate the limitations of complex lighting conditions, acquiring real-time 3D point cloud data of obstacles and infrastructure surfaces within the work space, and thus independently completing high-precision environmental mapping. Simultaneously, the exploratory UAV uses its synchronously equipped gimbal camera to perform preliminary image acquisition of points of interest on infrastructure surfaces along its flight path for environmental mapping, providing a basic spatial coordinate reference and initial image feature reference for subsequent refined inspections.

[0018] Unlike exploratory drones, the photographic drone is equipped only with a gimbal camera. Because it lacks the rotating lidar, the photographic drone significantly reduces its takeoff weight and total power consumption, resulting in a longer effective flight time and battery power margin. Leveraging this extended flight time, the photographic drone is specifically designed for high-quality observation of points of interest (POIs) and is compatible with automatic battery swapping stations for extended operation. During high-quality observation missions, the photographic drone can directly receive and utilize pre-mapped environmental data shared by the exploratory drone for safe navigation, approaching the POI to be measured. Thanks to the energy efficiency advantage of having no radar payload and the physical battery swapping network provided by the automatic battery swapping stations, large numbers of photographic drones can approach the POI at extremely low relative speeds or maintain a stable hovering flight attitude near the POI for extended periods of detailed imaging. This low-speed or hovering observation state strictly controls the relative physical displacement of the gimbal camera during exposure, thus ensuring low blur and high spatial resolution of the acquired images.

[0019] By separating the physical payloads of the exploratory drone and the photographic drone and coordinating their work in inspection tasks, this embodiment effectively resolves the dynamic conflicts faced by traditional single drones when simultaneously carrying heavy sensing equipment and performing long-endurance hovering and photographing tasks. Under the premise of ensuring the integrity of complex three-dimensional mapping, it achieves continuous, stable inspection operations for large-scale infrastructure that meet the rigid constraints of optical quality.

[0020] (2) Perception module: integrated into the UAV swarm, used for environmental data acquisition and point of interest observation; the lidar of the exploratory UAV collects environmental point cloud data to construct an occupancy map, and the gimbal camera is used to observe points of interest on the surface of infrastructure. (For a set of points of interest), the gimbal camera of the shooting drone is used for high-quality observation of the points of interest, and the observation quality meets the following requirements: in, As an ambiguity index, This refers to the resolution specification.

[0021] Specifically, the perception module is divided into two highly interconnected operational levels based on the physical configuration of the flight platform. In the first operational level, the exploratory UAV, positioned as the leader, uses its onboard rotating lidar to emit high-frequency laser detection beams into the surrounding physical space, collecting environmental point cloud data in real time that reflects the geometric appearance of infrastructure and the distribution of sudden spatial obstacles. This environmental point cloud data is further converted into an occupancy map representing the three-dimensional spatial passage status.

[0022] While performing panoramic scanning of laser point clouds, the exploratory UAV simultaneously utilizes its onboard gimbal camera. Using the camera's wide-angle or zoom optical system, it performs preliminary visual anchoring and coordinate extraction of the surface of the infrastructure to be inspected within the three-dimensional physical space. Here, the system defines effective areas on the target infrastructure surface that are preliminarily determined to have observational value or potential defects as points of interest, and sets the spatial coordinate parameters of a single point of interest as follows: Furthermore, all points of interest together constitute a set of points of interest enclosed within a specific three-dimensional space. The set satisfies the mathematical subset relation. .

[0023] After the exploratory UAV completes the initial spatial anchoring of the occupied map and target coordinates, the data flow from the perception module transitions to the second operational level, led by the camera UAV. Because the heavy 3D active perception equipment is completely removed from the physical structure, the camera UAV can approach the set of points of interest with a more stable, fine-tuned flight attitude and ample battery power margin. Spatial coordinates in the space.

[0024] To ensure that the image data acquired during the inspection mission truly meets the stringent standards for subsequent engineering defect identification and grading analysis, this application specifically requires that the gimbal camera carried by the shooting drone be aimed at any given point of interest. At a specific time step The final observation quality output must strictly satisfy a specific penalty product relationship, expressed as: In this quantitative evaluation model of observation quality, For the first The drone in At a specific physical point of interest at a discrete control time The overall observation quality score obtained; The blur index is a physical measure of the degree of image motion distortion caused by dynamic and optical coupling factors such as the instantaneous relative motion speed of the UAV platform itself, the high-frequency mechanical vibration of the gimbal, and the pixel displacement of the camera sensor within a given exposure time. The resolution metric is designed to quantify the real physical size of the infrastructure surface that a single pixel in the camera's image sensor array can map, i.e., the ground space sampling rate. Its calculation results are directly controlled by the absolute focal length parameter of the camera's optical lens and the three-dimensional straight-line distance between the optical center of the aircraft and the target surface.

[0025] To facilitate unified normalized penalty calculation and threshold comparison in subsequent complex operations research algorithms and microscopic obstacle avoidance models, the system maps and constrains both the ambiguity index characterizing relative motion interference and the resolution index characterizing spatial distance signal attenuation to... Within the set of closed intervals.

[0026] (3) Communication module: Supports point-to-point communication within line of sight between UAVs and between UAVs and ABSS, used for exchanging location information. Occupy map data Waypoint status information and ABSS operational data; opportunistic data sharing is only possible when there is no obstruction at line of sight, and the communicable set... Indicates time drones Within line-of-sight neighbors, ensuring real-time data interaction.

[0027] Considering that it is often difficult to maintain stable global centralized network coverage in large infrastructure such as inside bridges or dense building complexes, the communication module is strictly designed at the underlying physical link to support line-of-sight point-to-point communication architecture between drones and between drones and distributed automated battery swapping stations (ABSS).

[0028] Defines a dynamically evolving set of communicable sets. This communicable set It accurately characterizes the state at any discrete control moment from the physical spatial geometric dimension. Specific drones The set of all neighboring nodes that can be directly observed through the physical line of sight and whose connection is not obstructed by any physical obstacles.

[0029] The communication module is only activated and establishes a local high-bandwidth data transmission link when the receiving node and the sending node are within each other's unobstructed line of sight in three-dimensional physical space, i.e., when the conditions defined above for the communicable set are met, thereby triggering an opportunistic data sharing and synchronization process.

[0030] By introducing an opportunistic data sharing mechanism that occurs "only when the receiving node and the transmitting node are within each other's unobstructed line-of-sight range in three-dimensional physical space, i.e., satisfying the conditions defined above for the communicable set," this invention weaves distributed, heterogeneous flight platforms into a highly self-healing, locally relay-style digital network. The direct technical effect of this mechanism is that it not only significantly reduces the continuous radio frequency transmission power consumption of a single aircraft but also completely avoids the risk of system paralysis caused by global communication blind spots.

[0031] Simultaneously, this allows shooting drones without heavy environmental perception radar to instantly acquire the latest occupancy map and obstacle distribution status of the entire area through occasional spatial encounters with exploration drones or ABSS stations that have synchronized data during flight. This relay-style information synchronization based on spatial encounters maximizes the real-time performance and reliability of the underlying core data interaction under extremely harsh communication physical boundaries, ensuring the absolute flight safety of the entire heterogeneous fleet and the continuity of inspection workflow in complex obstructed environments.

[0032] (4) Path planning module: used to generate mapping paths and inspection paths; the mapping path is generated along the longest axis of the workspace, and the inspection path adopts a two-level strategy of "global optimization + local dynamic adjustment". The global path is generated by solving subproblems of the LBBD algorithm, and the local path is generated by the D-RHLP strategy, based on the currently occupied map. and drone status Ensure collision-free operation and adaptability to dynamic environments.

[0033] For exploratory unmanned aerial vehicles (UAVs) responsible for preliminary environmental reconnaissance, the path planning module controls the mapping path to be generated strictly along the longest axis of the pre-determined three-dimensional operational space, based on the geometric boundaries of that space. This linear reciprocating scan along the longest physical axis maximizes the radial coverage volume of the rotating lidar during a single unidirectional flight, effectively reducing additional rotor power loss and inertial trajectory drift induced by frequent large-angle yaw turns in confined spaces. This ensures that the exploratory UAV completes the construction of the underlying three-dimensional environmental topology with optimal spatial propulsion efficiency in the early stages of the inspection mission.

[0034] During or after the exploratory unmanned aerial vehicle (UAV) performs sweep mapping, in response to the high-precision image acquisition requirements of the photographic UAV, the path planning module adopts a two-level strategy of "global optimization + local dynamic adjustment" to generate the final inspection path used for actual observation tasks.

[0035] In the macro-level global path generation phase, the system relies on the sub-problem-solving framework of the basic logical decomposition algorithm. The solution process of this global path is entirely controlled by the boundary constraints such as the image observation quality set in advance by the system. In essence, it searches for an optimal topology connection that runs through the assigned infrastructure points of interest and satisfies the lowest theoretical total flight energy consumption on offline or semi-online computing nodes. That is, the system locks in the overall mission direction of the shooting UAV swarm at the macro level, avoiding the disorderly scheduling of limited flight battery resources.

[0036] Considering that the global optimal solution in operations research typically assumes a static environment within the mathematical space, making it difficult to handle unknown and sudden spatial obstructions or moving obstacles in the real physical operating space, the two-level planning strategy further incorporates a dynamic rolling temporal local path planning strategy at the micro-physical execution level to generate locally dynamically adjusted flight paths. This dynamic generation process of the micro-local path highly depends on real-time updates at the system's underlying layers: firstly, the currently occupied map data reflecting the external physical environment's obstruction boundaries, obtained through line-of-sight communication relay. Secondly, the UAV state vector is deeply bound to the instantaneous nonlinear physical kinematics equations of the aircraft itself. Specifically, the local path solution control model introduces a method based on three-dimensional spatial position coordinates. 3D yaw angle 3D linear velocity and three-dimensional yaw rate The eight-dimensional dynamic state vector of the UAV, formed by the combined forces, is mathematically expressed as follows: .

[0037] By expanding the parameter dimension to an eight-dimensional real space In this embodiment, the original pure three-dimensional geometric obstacle avoidance problem is transformed into a high-order dynamic closed-loop control problem that includes time derivatives and platform maneuver limits. This is achieved by using map data reflecting the occupancy of external obstacles. With the state vector reflecting the motion performance of the aircraft itself By coupling the spatial and temporal domains in a multidimensional manner, the dynamic rolling temporal local path planning strategy can, within a pre-defined finite-time look-ahead window, extract and reconstruct a continuous flight trajectory segment that is guaranteed to be collision-free and fully conforms to the actual aerodynamic limits of multirotor aircraft in real time from the abstract topological skeleton of the global planning. This asymmetric, two-layer architecture design enables the photographic unmanned aerial vehicle (UAV) without an active sensing radar payload to not only strictly follow the overall optimal task allocation logic of the system, but also possess the underlying survivability to perform smooth collision avoidance maneuvers in a highly complex and real-time evolving dynamic three-dimensional environment.

[0038] (5) Task scheduling module: The LBBD (Logical Basis Benders Decomposition) algorithm is used to achieve coordinated optimization of ABSS site selection, facility allocation and UAV path planning, and generate a path set. To meet full coverage constraints, enable collaborative operations of drone swarms, and maximize inspection scores. In this application, the selection of deployment nodes for automated battery swapping stations and the initial docking of facility sets are separated from the main problem for preliminary delimitation; while the generation of specific micro-pathways for each unmanned aerial vehicle (UAV) is treated as a sub-problem, independently optimized based on the allocation results from the main problem and the pre-set hard constraints on optical observation quality. Through rigorous solution and iterative convergence of the operations research logic cutting plane in this module, the system finally outputs a set of paths (denoted as ) to the fleet to guide the actual flight of the vehicles. This set of paths The system is required to meet the full coverage constraint at the system boundary, which means that every physical target node in the pre-calibrated set of points of interest on the infrastructure surface can be covered by the three-dimensional flight trajectory planned by the fleet and perform at least one effective high-fidelity observation, thereby completely eliminating potential blind spots in the spatial topology dimension.

[0039] To further optimize the combined hardware performance of the drone swarm while ensuring full coverage without blind spots, the task scheduling module sets the ultimate algorithmic control objective as achieving collaborative operation of the drone swarm and maximizing the global inspection score. Specifically, the system constructs a quantitative global inspection score model, based on the total physical score. This indicates the global score. The system evaluation criteria do not simply linearly accumulate the quality of all images captured by the aircraft in the air, but rather follow the following: That is, for the set of facilities points of interest Any single entity point of interest in The system introduces a double-level maximum value when performing quality assessment. Extraction rules. First, perform a vertical traversal in the time domain to extract individual unmanned aerial vehicles. The total execution cycle (i.e., total duration) of the entire task With discrete sampling interval (Within the quotient range) All observation quality scores obtained for this point This represents the highest score ever achieved. Subsequently, the algorithm performs a horizontal comparison across the spatial entity domain, starting from the total number of cluster configurations. Among all flight platforms, select those targeting this specific point of interest. The highest achievable observation quality score is selected and locked as the final effective evaluation score for the infrastructure defect. Finally, the task scheduling module linearly sums the final effective scores of nodes within all interest sets to form a global total score that directly characterizes the overall operational efficiency of the system. .

[0040] Through the aforementioned "dual maximum value" screening mechanism based on both time and entity dimensions, this embodiment no longer blindly guides single or multiple unmanned aerial vehicles (UAVs) to perform inefficient and redundant photography of the same target point. Instead, it allows a heterogeneous fleet to ensure, through deep task collaboration, that at least one photographic UAV in the large fleet can acquire a single high-quality image that meets stringent clarity thresholds at the optimal approach time and with the best optical and physical attitude. This operations research-driven scheduling and quantitative evaluation mechanism, dominated by the task scheduling module, not only significantly reduces ineffective computing power consumption and fleet power waste during aerial inspections but also effectively guarantees the high fidelity of the engineering inspection baseline and maximizes the benefits of comprehensive inspections at the level of mathematical constraints.

[0041] The specific implementation phase includes the following steps: Step 1: Determine the workspace based on the set of points of interest surrounding the area to be inspected. bounding box set and the initial position set of the drone The boundary of the workspace is calculated using a formula, forming a minimum cuboid that includes infrastructure and the drone swarm, thus dividing the workspace boundary into... Using voxels of rice to construct a 3D mesh The set of edges The formula for calculating the boundary of the workspace is: Initialization parameters: ABSS parameters (fixed cost) Yuan / day, runtime / minute, service radius / meter, number of batteries UAV parameters (flight speed) Flight power consumption rate Hovering power consumption Battery capacity Sampling interval ; Observation quality threshold LBBD algorithm parameters (stable column generation step size) Length of the rolling time domain window ).

[0042] Specifically, when implementing the multi-heterogeneous UAV collaborative inspection method, the system first enters the step of determining the workspace. The system acquires the set of points of interest surrounding the area to be inspected. bounding box set and read the initial position set of the drone. The initial position of a single drone , This represents the total number of drones. Based on the aforementioned bounding box set... and the initial position set of the drone The system calculates the boundary of the workspace using a formula to form a minimum cuboid that includes infrastructure and drone swarms.

[0043] The specific extraction method or generation rule for the bounding box set α in three-dimensional space is as follows: Based on the set of points of interest XS to be inspected, the axis-aligned bounding box (AABB) algorithm is used to extract the minimum bounding boundary. The formula for calculating the three-dimensional boundary is completely consistent with the formula for the work space boundary in step 1 of this application, that is: The boundary set α is the set of vertices and edges of a closed cuboid formed by the above 6 boundary planes, which completely encloses the spatial distribution range of all points of interest to be inspected.

[0044] The formula for calculating the boundary of the workspace is: the minimum boundary along the X-axis. Maximum Boundary Minimum boundary of the Y-axis Maximum Boundary Minimum boundary of the Z-axis Maximum Boundary .

[0045] After forming the minimum cuboid containing infrastructure and drone swarms, the system divides the operational boundary into... Voxels of rice are used to construct a 3D mesh. In this 3D mesh diagram In the middle, edge set Limited to That is, only if any two different voxel nodes in the 3D mesh graph and The spatial Euclidean distance between them is strictly smaller than the voxel size. When the two are connected, it is determined that there is a connected edge between them.

[0046] After the workspace is determined and the mesh diagram is constructed, the system loads initialization parameters. These initialization parameters include: ABSS parameters, which contain fixed costs. Yuan / day, runtime / minute, service radius / meter and number of batteries Drone parameters, including flight speed Flight power consumption rate Hovering power consumption Battery capacity and sampling interval ; Observation quality threshold ; and LBBD algorithm parameters, including the stable column generation step size. and the length of the scrolling time domain window .

[0047] After parameter initialization is complete, the system executes the environment mapping step. The system generates... A mapping path is defined along the longest axis of the operational space, and the exploratory UAV executes this mapping path. During flight, the exploratory UAV collects radius data using lidar. Use environmental point cloud data within the range to construct and update the respective occupancy maps. The occupied map Generate by marking voxels containing obstacle point cloud coordinates as "occupied".

[0048] The specific quantization rules or point cloud density determination conditions for mapping continuous obstacle point cloud coordinates to discrete voxels are as follows: 1. Spatial quantization rule: Discretize the workspace into a three-dimensional mesh according to the voxel size ${{V}}$. 3D coordinates of arbitrary point cloud The grid index mapped to the corresponding voxel is ,in To achieve the minimum value of the workspace boundary in step 1 of this application, the normalization mapping from continuous coordinates to discrete voxels is completed; 2. Point cloud density determination criteria: If the cumulative number of laser point clouds within a single voxel is ≥3, then that voxel is determined to be a valid obstacle voxel, corresponding to its map occupancy. The logic for marking the "occupation" state.

[0049] To eliminate blind spots for individual drones, drones within line of sight share and merge their updated occupancy maps. An initial occupation map is generated. Finally, based on the merged initial occupation map, the system removes grid maps. The inner edges occupying vertices are used to ensure that the inspection paths planned based on this mesh graph can avoid known physical obstacles from the underlying topology.

[0050] Step 2: Environment Mapping. Generate... The mapping path is along the longest axis of the workspace. The exploratory UAV executes the mapping path and collects the radius data using lidar. Use environmental point cloud data within the range to construct and update the respective occupancy maps. Within line of sight, drones share and merge Generate an initial occupied map and remove grid maps. The inward edge occupying the vertex in the map; Generate by marking voxels containing obstacle point cloud coordinates as "occupied".

[0051] The sensing range of a lidar is its radius. The spherical region contains point cloud data that corresponds to the surface of the nearest obstacle in the environment.

[0052] The drones exchange data only when there is no obstruction at line of sight, and can communicate with each other. Indicates time drones Within line-of-sight neighbors, data is exchanged including location. Occupy the map Including waypoint status information, to achieve dynamic information synchronization.

[0053] After completing step 1, the system proceeds to step 2. Specifically, the system first generates a quantity of... A mapping path that extends along the longest axis of the workspace. Among them, regarding the aforementioned... The specific array distribution rules of the strip map paths in three-dimensional physical space, or the basis for setting the spacing between adjacent routes, are as follows: The path mapping uses an equidistant parallel array distribution, with all paths extending parallel to the longest axis of the workspace. The spacing between adjacent paths is set based on the effective sensing radius of the exploratory UAV's lidar. Take the distance between adjacent routes This ensures that the point cloud collection areas of adjacent routes have a 20% overlap, avoiding blind spots in mapping.

[0054] Subsequently, the exploratory UAV was controlled to execute the mapping path, and continuously collected environmental point cloud data using an onboard rotating lidar during flight propulsion. The physical sensing range of the lidar was strictly calibrated to a radius of [missing information]. The spherical region from which the point cloud data is collected precisely corresponds to the surface of the physical obstacle closest to the aircraft in the operating environment.

[0055] Based on continuously acquired laser point cloud data, each exploratory UAV independently constructs and updates its own local occupancy map in real time within its onboard computing unit. The occupied map The underlying generation and update logic lies in: transforming the 3D mesh map Discrete voxel nodes containing the spatial coordinates of physical obstacle point clouds are marked as "occupied"; the specific point cloud quantity threshold or probability discrimination formula for mapping and normalizing continuous obstacle point cloud spatial coordinates to the corresponding discrete voxels, and determining whether a voxel is identified as "occupied", is as follows: 1. Coordinate mapping normalization rule: For any laser point cloud 3D coordinates The formula for calculating the grid index of the corresponding discrete voxel is: ,in This is the minimum value of the workspace boundary. Given the voxel side length, a one-to-one mapping from continuous physical space to discrete voxel mesh is achieved; 2. Occupation Status Determination Rules: A voxel is considered "occupied" only if two conditions are met simultaneously: first, the cumulative number of laser point cloud hits within a single voxel is ≥3; second, the laser ray hit rate of that voxel is ≥70%, i.e. ,in voxels Number of laser point cloud hits within the area The number of times a laser ray misses a voxel is recorded. Voxels meeting the specified conditions are ultimately marked as "occupied," corresponding to the occupied map. The generation logic.

[0056] To overcome the spatial perception limitations of a single UAV and achieve dynamic information synchronization among a swarm, the system restricts low-level data exchange between the UAVs to only under unobstructed physical line-of-sight conditions. Therefore, the system strictly defines a communicable set in the spatial geometric dimension. This communicable set... Used to accurately represent at any discrete time Specific drones The line-of-sight neighbor nodes. Only when the sending and receiving ends meet the line-of-sight connectivity conditions of the above-mentioned communicable set will neighboring UAVs trigger an opportunistic handshake and exchange underlying core data. The exchanged data strictly includes the current spatial location information of each UAV. Each maintains its own occupied map data for subsequent obstacle avoidance. And the current waypoint status information.

[0057] Based on the aforementioned line-of-sight data sharing mechanism, each UAV receives multi-source local occupancy maps. Spatial registration and fusion are performed.

[0058] The specific algorithm rules or conflict resolution mechanism for globally consistent fusion of multiple locally occupied maps are as follows: An occupation probability overlay fusion algorithm based on spatial registration is adopted, with the specific rules and conflict resolution mechanism as follows: 1. Spatial registration: based on the real-time location of the UAV and To divide the map into localized areas occupied by different drones and The coordinates are uniformly converted to the global operation space coordinate system defined in step 1 of this application to ensure the spatial alignment of voxel grids in different local maps; 2. Probabilistic fusion: For voxels at the same spatial location The multi-source data overlay method is used for fusion. If the point cloud data of two or more UAVs are marked as occupied, then it is confirmed as a globally occupied voxel. 3. Conflict resolution: If two local maps have opposite determinations of the occupation status of the same voxel, the determination of the local map with the closer LiDAR acquisition distance and more point cloud data shall prevail, while retaining the status information of the conflicting voxel, and correcting it again after subsequent updates of new point cloud data.

[0059] This generates an initial occupation map covering the entire area. Finally, based on this fused initial occupation map, the system applies the previously constructed 3D mesh map... Perform low-level topology safety corrections, specifically by forcibly removing mesh graphs. The inward edges corresponding to all vertices marked as occupied.

[0060] Step 3: Inspection waypoint generation: The drone swarm, based on the updated occupation map, moves around the bounding box. The voxel generation inspection waypoint within the area Each waypoint Associated direction vector pointing to the occupying voxel Used to calculate the tilt angle of the gimbal camera. and azimuth The field of view of the gimbal camera is a rectangular pyramid structure, consisting of a horizontal field of view angle. Vertical field of view and effective distance The gimbal rotation angle is defined as satisfying , .

[0061] After completing step 2, the system proceeds to step 3. In this stage, the system controls the drone swarm to perform bounding box operations based on the updated occupation map. The voxels marked as occupied (i.e., voxels representing physical obstacles or target infrastructure surfaces) are used to generate a set of inspection waypoints for performing close-range observations. .

[0062] Specifically, regarding how the UAV swarm establishes safe and effective waypoint 3D spatial coordinates around the occupied voxel, the specific spatial sampling algorithm, envelope surface generation logic, or safe distance offset rule adopts an equidistant envelope surface sampling algorithm, with the following specific rules: 1. Envelope surface generation: Based on the infrastructure surface composed of target-occupied voxels, offset outward by a safe distance along the surface normal vector. (2x voxel side length) to generate a safety envelope surface equidistant from the infrastructure surface, ensuring that the drone maintains an absolutely safe distance from obstacles; 2. Spatial sampling: Along the horizontal and vertical directions of the envelope plane, the sampling interval is set according to 80% coverage of the effective field of view of the gimbal camera to ensure that there is a 20% overlap of the field of view of the cameras at adjacent waypoints and avoid blind spots in the inspection. 3. Waypoint Filtering: Finally, waypoints are selected that are collision-free and whose targets occupy the entire voxel surface within the camera's effective line of sight. Waypoints within the range constitute the inspection waypoint set. .

[0063] In the above-generated set of inspection waypoints, each specific discrete waypoint Each is forcibly associated by the system with a direction vector in three-dimensional physical space that points directly from the waypoint to the corresponding voxel. That is, the system calculates the tilt angle of the gimbal camera at the corresponding waypoint using this direction vector. and azimuth ; wherein, based on the three-dimensional direction vector Derive and calculate the tilt angle of the gimbal camera. and azimuth The specific mathematical transformation formula or spatial geometric mapping relationship is as follows: Let the three-dimensional direction vector be... It is a unit vector pointing from the waypoint position to the target's occupied voxel, and the gimbal camera tilt angle. (Pitch angle) and azimuth angle The formula for the spatial geometric mapping of (horizontal rotation angle) is: in The azimuth angle of the gimbal must be within the range specified in step 3 of this application. Inside; The range of values ​​for the gimbal tilt angle must be constrained within [specific range]. If the calculation result exceeds the mechanical limit, the corresponding boundary value is taken as the final gimbal control command.

[0064] To ensure that the calculated observation posture can be accurately executed by the physical device and conforms to the physical boundaries of actual optical imaging characteristics, the system strictly defines the optical sensing range and mechanical rotation limits of the gimbal camera. Specifically, the spatial field of view of the gimbal camera is precisely defined in three-dimensional geometry as a rectangular pyramid structure. The physical envelope of this rectangular pyramid structure is defined by the camera's horizontal field of view angle. Vertical field of view And the effective distance to ensure image fidelity Common quantization definition. Under this physical field-of-view structure, the servo control commands generated by the system require that the mechanical rotation angle of the gimbal camera must strictly converge within its hardware limits, that is, the tilt angle executed by the gimbal must satisfy... And the azimuth angle to be executed must satisfy .

[0065] Step 4: Solving the LBBD master problem. Establish a mathematical model for the master problem, with decision variables including the ABSS deployment status. (1 indicates deployment, 0 indicates no deployment), facility allocation relationship (1 indicates facility) Assigned to ABSS (0 indicates unassigned); the objective function is: The constraints include: The ABSS deployment scheme and facility allocation results are obtained by solving the problem using the branch-cut (BC) algorithm. Generate the Benders cut optimization solution process.

[0066] After completing step 3, the system proceeds to step 4. To this end, the system establishes a mathematical model of the main problem for overall planning. The system identifies two key types of binary decision variables in the model: the first type is the deployment state variable of the Automated Swapping Station (ABSS). ,when "Time" indicates that physical deployment is performed at a given candidate node location. The first category indicates no deployment; the second category is facility allocation relationship variables. ,when The time indicates the specific facilities to be inspected. Logically divide and allocate to the corresponding automatic battery swapping stations. To obtain its physical support for endurance, when The time indicates that it is unallocated.

[0067] Based on the two types of decision variables mentioned above, the system constructs a main problem objective function aimed at minimizing the integrated infrastructure investment and air travel costs. Its specific mathematical expression is as follows: ; This represents the fixed daily deployment cost of a single automated battery swapping station. A preset weighting coefficient is used to balance hardware investment and time cost; among which, regarding this weighting coefficient The specific value range or dynamic adjustment logic within the system is: weighting coefficient. To balance the pre-defined weights of the fixed hardware investment cost of ABSS and the time cost of drone inspections, its fixed value range is: The specific value will be dynamically adjusted based on the project's operational goals: - When the project prioritizes controlling the fixed costs of ABSS deployment hardware, the value will be... Reduce the weight of flight time cost in the objective function; - When the project prioritizes reducing inspection operation time and improving inspection efficiency, take Increase the weight of flight time cost in the objective function; - In routine inspection scenarios where cost and efficiency are balanced, take the baseline value. This setting perfectly matches the objective function system of the LBBD master problem in step 4 of this application.

[0068] This is a set of candidate automatic battery swapping stations. Let be the set of path edges within the workspace; where is the set of variables involved in the objective function. Path flight time and hovering time The exact physical meaning of these, and how they were estimated or obtained in the main problem stage before specific route planning was carried out, are explained in the following logic: 1. Physical meaning of the variable: The variable is a 0-1 decision variable. A value of 1 indicates that the inspection path of ABSS station i uses the flight arc from node m to node j, and a value of 0 otherwise. The flight time of the UAV from node m to node j is calculated by the Euclidean distance between the two points and the steady-state flight speed of the UAV set in step 1 of this application. Calculated, i.e. ; 1. The hovering service time of the UAV at node m, i.e., the duration of a single inspection operation of the target facility by the UAV; 2. Main problem stage prediction logic: In the main problem stage before specific route planning, The shortest flight time corresponding to the straight-line distance between nodes is used as the lower limit of the estimate; The fixed inspection duration for a single facility is used as a fixed value. The appropriate flight segment is determined by iterating through mixed-integer programming of the main problem.

[0069] To ensure that the solution to the above objective function has strict executability in the physical world, the system imposes the following constraints on the main problem model: Specifically, This constraint mandates that the system only operates on nodes. The physical deployment of automated battery swapping stations has indeed been implemented (i.e. Only under the premise of [specific conditions] can a set of facilities to be inspected be assigned to that station. Specific inspection tasks within the scope; This formula ensures that every facility to be inspected within the workspace is accounted for. Each is assigned to a single, unique automatic battery swapping station, which completely eliminates the redundancy of inspection blind spots or repeated assignment of multiple units at the macro task scheduling level. The system utilizes the steady-state flight speed of the aircraft. Expected flight time between nodes The product of these factors is used to calculate the actual physical span, which is then mandated to be allocated to specific stations. facilities It must be strictly within the maximum service radius of the automatic battery swapping station. Within the physical envelope; This is designed to ensure that the cumulative operation time cost of a single automated battery swapping station in a single service network does not exceed its specified maximum operating time. ; This constraint, from the perspective of the upper limit of physical hardware resources, limits the total number of variables related to all unmanned aerial vehicles allocated to a certain site, which must absolutely not exceed the number of batteries that the internal mechanical structure of the battery swapping station can store. Among them, regarding decision variables The specific referential meaning and its relationship with sets The corresponding extraction rule is: decision variables These are variables ranging from 0 to 1, with a value of 1 representing the set of feasible paths for ABSS site i. The p-th inspection path in the list is enabled; otherwise, it is 0. Corresponding extraction rule: any feasible path for ABSS site i. The path p contains a unique sequence of nodes, flight arcs, and operation duration, satisfying battery capacity constraints, ABSS runtime constraints, and service radius constraints. After the main problem is solved, based on the facility allocation results, from... Extract all feasible paths that meet the coverage requirements, and determine the branch-pricing algorithm for the subproblems. The final value of .

[0070] This complex nonlinear constraint will affect the facility's hovering power consumption rate. Absolute battery capacity Coupled with extreme time parameters, it directly eliminates any invalid allocation schemes that exceed the site's power turnover cycle capacity; Among them, in the formula and The accurate definition and derivation calculation basis for time is: The precise time is defined as: the shortest flight time between any two nodes within the workspace (including ABSS sites and facilities to be inspected), and its derivation and calculation basis is: ,in Let u and v be the three-dimensional coordinates. The steady-state flight speed of the UAV set in step 1 of this application; The precise time is defined as the operation time required for the UAV to complete one battery replacement at ABSS site i, and its derivation and calculation are based on the rated mechanical battery swapping time of the ABSS equipment.

[0071] After rigorously constructing the objective function, multidimensional space, and physical constraints, the system control unit uses the branch-cut (BC) algorithm at the lower level to precisely iteratively solve the mixed-integer programming master problem model. After the algorithm's closed-loop convergence calculation, the system outputs the global-level automatic battery swapping station deployment scheme and facility allocation results, mathematically denoted as... Generate the Benders cut optimization solution process.

[0072] Step 5: Solving LBBD Subproblems (Inspection Path Optimization) Each ABSS corresponds to a subproblem, with the objective of minimizing the cost of the UAV inspection path. The decision variable is the path selection variable. The constraints include: The Branch-and-Price (BP) algorithm is used to solve the problem. The initial path is constructed using the cheapest insertion (CI) heuristic, and a stable column generation technique is employed to solve the pricing problem. A branching strategy ensures the integer nature of the solution, generating the optimal path set. ; Generate logical cuts and feed them back to the main problem to accelerate convergence.

[0073] The system outputs a global-level deployment plan and facility allocation results for automated battery swapping stations by solving the main problem. Subsequently, the system triggers the sub-problem-solving step of the Logical Basic Decomposition (LBBD) algorithm downstream, i.e., performing micro-level inspection path optimization. In this cascaded optimization architecture, each Automated Battery Swapping Station (ABSS) deployed by the established entity in the pre-allocation scheme independently corresponds to a sub-problem to be solved mathematically. The core optimization objective of this sub-problem is strictly defined as minimizing the cost of the UAV inspection path, and its core decision variable is the path selection variable representing the activation status of a specific route. Among them, regarding the path set Specific spatial construction boundaries and decision variables The detailed mapping and referencing relationships are as follows: 1. Path set Spatial construction boundaries: Geographic boundaries: The starting and ending points of all paths must be ABSS site i, and all facilities covered by the path must be within the service radius R of site i; Constraints: The total power consumption of a single path shall not exceed the UAV battery capacity Cap set in step 1 of this application, and the total duration of a single path (flight + hovering) shall not exceed the maximum duration S limit of a single operation under ABSS. 2. Mapping and referencing relationship: Path set For the complete set of all feasible paths to ABSS site i that satisfy the above boundaries, the decision variables are... This is the enabled state variable for path p, and the two correspond one-to-one; This indicates that path p is included in the final inspection plan for station i. This indicates that the path is not enabled.

[0074] To ensure the legitimacy and engineering effectiveness of the micro-route in the physical execution dimension, the solution space of this sub-problem model is strictly limited by three core constraints. The first is the task coverage connectivity constraint, expressed as: This constraint mandates that as long as the primary issue concerns facilities at the macro level... Assigned to the site (Right now If the value is valid, then the set of paths selected at the bottom level by the subproblem must traverse and cover the facility node in the spatial topology; where, regarding the parameters of the correlation matrix... The specific numerical extraction rules are as follows: Correlation matrix parameters The parameter is 0-1, and its specific value extraction rule is as follows: For the p-th feasible path of ABSS site i, if the path covers the j-th facility to be inspected (i.e., the path node sequence contains facility j), then... ,otherwise This matrix is ​​used to constrain the paths of subproblems to completely cover all facilities assigned to site i by the main problem.

[0075] The second term is the continuity constraint of dynamic charge, and its expression is: This inequality uses a sufficiently large constant. To achieve logical decoupling, the power consumption of the aircraft is transferred between nodes. ) and power consumption of hovering operations at nodes ( The cumulative effect closely tracks and restricts the aircraft's movement along continuous topological edges. Transient battery power This effectively prevents the risk of a plane crash due to overdrawn battery power at the underlying level.

[0076] The third term is the rigid constraint of optical imaging, expressed as: This constraint requires that for all planned route nodes, the product score of transient ambiguity and resolution must be higher than the minimum engineering quality threshold. .

[0077] After establishing the aforementioned multidimensional physical and logical boundaries, the system control unit invokes the branch-pricing (BP) algorithm to precisely solve this complex integer programming subproblem. In the initial startup phase, the system first uses the cheapest insertion (CI) heuristic to quickly search and construct initial feasible paths across all nodes, serving as a warm-start baseline for subsequent iterations. Subsequently, the system introduces a stable column generation technique to solve the underlying pricing problem, continuously adding high-quality path variables with marginal optimization potential to the master control pool. Given that a pure column generation process might output non-integer solutions (i.e., fractional states) that are not executable in the physical world, the algorithm further incorporates a specific branching strategy to ensure the absolute integer nature of the final solution.

[0078] Specifically, the branching strategy includes a two-layer spatial dimensionality reduction operation: the first is a path count branch, where the system monitors the total number of currently active paths in real time. If the total quantity is fractional, the algorithm rigidly divides it into two parts. and Two mutually exclusive search subtrees; the second is a finer-grained arc branch, where the system locks non-integer flow in the topology graph. free edge Forced to separate its branches into (i.e., forcibly using this side in the physical flight path) and (That is, the use of this side is absolutely prohibited in the physical flight path). After the closed-loop convergence of the above branch-pricing algorithm, the system finally outputs the set of optimal paths that completely satisfy the integer execution properties. .

[0079] Furthermore, to break the isolation of the master-slave problem and significantly improve global optimization efficiency, at the end of each iteration of the subproblem solution, the system generates specific logical cut planes based on the feasibility and cost lower bound of the current solution and feeds them back to the master problem to accelerate overall convergence. The generated logical cuts are strictly divided into two categories: the first category is logically feasible cuts, mathematically expressed as: ; When a subproblem determines that the currently assigned subset of tasks exceeds the physical execution limit, the cut plane will be submitted to the main control engine as a blacklist, absolutely prohibiting the main problem from using the exact same subset of tasks in subsequent iterations. All of them will be redistributed to the current site; Among them, the bottleneck task subset that makes the subproblem infeasible The specific extraction and recognition mechanism is as follows: 1. Infeasibility determination: When the solution to the subproblem shows that the set of facilities assigned to ABSS site i cannot generate a feasible inspection path under the constraints of battery capacity Cap and runtime S, the allocation scheme is determined to be infeasible. 2. Bottleneck Subset Extraction: An incremental elimination-verification method is used to extract the bottleneck subset: from the complete set of facilities assigned to site i, facilities are eliminated one by one, and the subproblem is solved again. If eliminating a facility makes the subproblem feasible, then that facility is the bottleneck facility; the final set of all bottleneck facilities is the bottleneck subset. ; 3. Blacklist mechanism: [This will...] As an infeasible combination, it is submitted to the main problem. Through the logical feasibility cut constraint in step 5 of this application, the main problem is prohibited from allocating the entire subset to the current site again in subsequent iterations, which is completely aligned with the cut generation logic of the patented LBBD algorithm.

[0080] The second type is the logically optimal cut, which is mathematically expressed as: ; This cutting plane introduces a dynamic time compensation factor. This provides a lower bound on the cost that closely approximates the actual physical consumption in the main problem. The time compensation in the formula is rigorously derived as follows: The extreme time parameters are defined as follows: as well as This optimal logical cut, which includes hovering penalties and round-trip costs, enables the system's macro-scheduling model to accurately predict global energy fluctuations caused by changes in local facility allocation without performing a full recalculation of underlying paths. This fundamentally achieves the reduction of computing power and the locking of physical optimal solutions in extremely high dimensions for heterogeneous multi-machine collaborative inspection systems.

[0081] Step 6: The collaborative inspection drone executes the inspection path using the D-RHLP strategy. The specific process includes: (1) Scrolling window settings: The system presets the length of the scrolling time domain window. The drone operates based on the length of the window. From its currently assigned global path In the middle, the distance from the current position of the drone to the position ahead is extracted. A sequence of waypoints is used as the rolling waypoint set for the current execution cycle; where, regarding how to proceed from discrete global paths... The above are accurately extracted or interpolated. The specific spatial sampling rules for each local rolling waypoint are as follows: 1. Window Capture: Starting from the drone's current position, capture along the global path. The node sequence is sequentially truncated to extract the next L consecutive global waypoints to form the initial rolling waypoint set, where L is the rolling time-domain window length set in step 1 of this application. 2. Iso-interpolation: If the straight-line distance between the intercepted global waypoints is greater than twice the voxel side length. Then, linear interpolation is used to supplement waypoints between the two points to ensure that the distance between adjacent rolling waypoints does not exceed [the specified value]. Finally, the L waypoints closest to the current position are retained as the local rolling waypoint set for the current execution cycle; 3. Collision check: Perform collision checks on the interpolated waypoints, remove waypoints that are within the occupied voxel, and add collision-free alternative waypoints to ensure the executability of rolling waypoints.

[0082] (2) Dijkstra path solution: The system is based on the current control time. Real-time updated capture map Based on the current three-dimensional spatial position of the drone Using the starting point of the topology search and the last waypoint in the rolling waypoint set as the search endpoint, the system control unit invokes Dijkstra's algorithm to perform spatial optimization between the aforementioned starting and ending points, generating a local path segment with the shortest spatial distance and no collisions within the local field of view. .

[0083] (3) Local path execution: The UAV is controlled to follow the local path segment generated by the above solution. Performing flight missions. During flight along the trajectory, the UAV collects real-time external environmental data to continuously update the occupied map. Meanwhile, to ensure collision avoidance safety in a multi-drone collaborative environment, the system forcibly marks the current spatial voxel node of the drone as "occupied" in real time to avoid spatial collisions with other collaborative drones; Among them, the specific quantitative formulas or rules for how to expand and map the physical contour of the UAV body to a discrete voxel mesh, and for establishing the safety redundancy boundary of the self-collision avoidance mark, are as follows: 1. Aircraft contour expansion mapping rule: The UAV airframe is equivalent to a shape with a side length of... The cube-shaped safety bounding box expands outwards in all three-dimensional directions from the drone's current location. The distance is used to form a safe expansion zone for the UAV; all discrete voxels covered by this safe expansion zone are marked as temporarily "occupied"; 2. Safety Redundancy Boundary Quantification Rule: The safety redundancy distance is taken as 1 voxel side length. That is, the minimum safe distance between the drone body and the obstacle is not less than This ensures absolute collision avoidance safety in multi-machine collaborative scenarios.

[0084] (4) Rolling update and replanning: When the UAV arrives at the current local path segment When the system reaches the destination, it triggers a sliding window update mechanism. Specifically, the system slides the rolling time-domain window to remove executed historical waypoints and add subsequent unexecuted waypoint sequences to the current window. After completing the window update, the system repeats steps 6(2) to 6(3) until the UAV safely reaches the global path. The final waypoint. Throughout the entire dynamic rolling execution cycle described above, the real-time iterative update of the UAV's state strictly follows the discrete dynamics model, and its state equation is: in, For a moment The state vector; For a moment Control input; The sampling interval of the control system; and They are respectively The identity matrix and the zero matrix.

[0085] Regarding the control input The specific solution to the constraints or the mapping model for transforming the underlying action execution mechanism is as follows: 1. Solving for constraints: control input The three-axis acceleration and yaw acceleration control variables of the UAV must satisfy the following constraints when solved: Acceleration constraints: , , ,in This is the maximum flight acceleration of the drone; Angular acceleration constraints: ,in - Maximum yaw acceleration of the UAV; - Velocity constraint: UAV flight speed updated iteratively as control input. Must meet ,in This is the maximum flight speed of the drone; 2. Low-level motion mapping model: The solved three-axis acceleration control quantity is mapped to the rotational speed difference of the four rotors of the UAV to realize the level flight and take-off control of the UAV; the yaw angle acceleration control quantity is mapped to the differential speed control of the UAV rotors to realize the yaw steering control of the UAV, thus completing the transformation from control input to low-level flight motion.

[0086] Step 7: Observation Quality Verification and Data Feedback. During the collaborative inspection flight and image acquisition process described above, the system simultaneously executes an observation quality verification procedure for the target infrastructure points of interest.

[0087] Specifically, the system determines the point of interest of a certain entity. The rigid prerequisite for effective observation is: the point of interest It must physically fall into the first Unmanned aerial vehicles in discrete control moments Camera field of view The interior, that is, satisfying the spatial geometric containment relationship. Furthermore, the system calculates a comprehensive observation quality score for this observation. It must be strictly greater than the preset engineering quality threshold. After confirming that the above-mentioned effective observation conditions are met, the UAV will feed back the acquired effective inspection images and related status data to its affiliated Automated Battery Swapping Station (ABSS) in real time, and further collect and transmit the full-domain inspection data back to the ground control center through the underlying distributed communication module for the backend system; Specifically, the image processing algorithm or intelligent judgment logic for the backend control center to perform defect identification and grading analysis is as follows: 1. Image preprocessing: The images collected during the inspection are processed sequentially by distortion correction, noise reduction and enhancement, and histogram equalization to improve the contrast and clarity of the defect area. The correction model adopts the camera intrinsic parameter matrix and lens distortion compensation model in step 7 of this application. 2. Defect Feature Extraction: Convolutional Neural Network (CNN) algorithm is used to extract typical infrastructure defect features such as cracks, corrosion, deformation, and missing bolts from the image. The spatial location and size of the defects are located by anchor frame regression algorithm. 3. Defect Classification Analysis: Based on the type, size, spatial location, and development trend of defects, the safety level is classified according to the national infrastructure operation and maintenance standards, and divided into four levels: general defects, relatively serious defects, severe defects, and emergency defects. 4. Output Results: Generate an inspection report containing defect location, type, level, and maintenance recommendations. Simultaneously, the defect location coordinates are fed back to the route planning module for waypoint generation in subsequent key review inspections. This completes the final infrastructure defect identification and analysis.

[0088] To accurately quantify the overall observation quality in the above-mentioned judgment criteria, a set of underlying physical optics evaluation quantification models was constructed. Among these, the blurriness index is used to characterize the degree of image motion distortion. The ambiguity scoring is calculated using a specific image plane displacement penalty mechanism. The formula is established as follows: When solving the above scoring formula, the system first introduces the physical projection formula of pixel position in the camera coordinate system to calculate the transient image plane coordinates before and after exposure. The camera intrinsic parameter matrix transformation relationship or lens distortion compensation model for projecting three-dimensional physical coordinates onto the two-dimensional camera pixel plane is as follows: 1. Camera intrinsic parameter matrix transformation relationship: Let the 3D coordinates of the point of interest in the camera coordinate system be... Its two-dimensional coordinates projected onto the pixel plane are The conversion formula is obtained through the camera intrinsic parameter matrix. accomplish: , in For camera focal length, The principal pixel coordinates of the camera are perfectly aligned with the pixel position formula in step 7 of this application. 2. Lens Distortion Compensation Model: The Brownian distortion model is used to compensate for the radial and tangential distortions of the lens. The compensation formula is as follows: in The radial distortion coefficient is... The tangential distortion coefficients are obtained through camera calibration to correct distorted pixel coordinates to ideal pixel coordinates. The specific image plane coordinate system transformation equation is: , , , .

[0089] Meanwhile, the system defines the instantaneous physical displacement of the target point of interest in the camera coordinate system during the camera exposure as: In the above series of optical and kinematic coupling formulas The total horizontal pixel width of the camera sensor; The absolute physical focal length of the camera lens; Precise representation of interest points Initial 3D spatial position in the camera's local coordinate system; It represents the instantaneous three-dimensional relative motion velocity vector of the point of interest relative to the camera's optical center at the moment the shutter is triggered; This represents the physical exposure time of a single frame of the camera.

[0090] Wherein, the relative velocity vector The specific forward kinematic transfer equations can be obtained by jointly solving the motion state vector of the UAV body and the instantaneous rotation angular velocity of the gimbal: The three-dimensional relative velocity vector of the point of interest with respect to the camera's optical center The forward kinematic transfer equation is obtained by jointly calculating the linear velocity, angular velocity, and gimbal rotation angular velocity of the UAV body: in: The three-dimensional flight linear velocity of the UAV body as defined in step 6 of this application; The three-dimensional angular velocity of the UAV body is given by the yaw angular velocity in the UAV's state vector. It is composed of pitch and roll angular velocities; This is the position vector from the center of the UAV body to the optical center of the camera; The instantaneous rotational angular velocity of the gimbal is determined by the gimbal tilt angle. With azimuth The rate of change was calculated. From the camera's optical center to the point of interest The position vector.

[0091] In addition to the aforementioned ambiguity index, the observation quality verification model also includes a resolution index for characterizing spatial sampling accuracy. This resolution metric is quantified by evaluating the real-world physical size mapped by a single pixel at the current observation distance, and its mathematical expression is strictly defined as follows: In this resolution evaluation equation, This represents the engineering desired resolution index required to meet specific defect detection standards; while and These represent the actual spatial sampling resolution achieved by the camera imaging system in the horizontal and vertical directions under the specific spatial observation pose. Among them, regarding the actual resolution in the horizontal and vertical directions Based on the current UAV observation distance, camera target size, and effective field of view of the lens, the specific spatial geometric conversion formula is derived as follows: The actual resolution in the horizontal and vertical directions refers to the real-world physical size corresponding to a single pixel, with the unit being mm / pixel, and its spatial geometric conversion formula is as follows: in: From the optical center to the point of interest of a drone camera The straight-line observation distance; , These are the camera's horizontal field of view and vertical field of view, as defined in step 3 of this application; These represent the total number of horizontal pixels and the total number of vertical pixels of the camera's image sensor, respectively. The conversion result is completely consistent with the resolution index calculation formula in step 7 of this application, and is used to evaluate the image spatial sampling accuracy under the current observation pose.

Claims

1. A method for collaborative inspection of multiple heterogeneous unmanned aerial vehicles (UAVs), characterized in that, include: Acquire environmental point cloud data by performing flight operations of an exploratory drone, and generate an initial occupied map based on the environmental point cloud data; Extract the set of points of interest to be inspected from the initial occupied map. The associated target occupies a voxel within the effective line-of-sight range of the onboard gimbal camera of the shooting drone. Generate a set of inspection waypoints within the range ; Based on the inspection waypoint set Construct path selection variables ; Based on the path selection variables The ambiguity index of the corresponding inspection waypoint With resolution indicators Obtain the overall observation quality score ; Establish based on the deployment status of automatic battery swapping stations Relationship with facility allocation The main problem is the allocation of location as the decision variable for the main problem, and the path selection variable is used as the main problem. Subproblems that are decision variables for subproblems; Based on the overall observation quality score Greater than the engineering quality threshold As an observation quality constraint, under the condition that the observation quality constraint is satisfied, the logical basis Benders decomposition algorithm is invoked to perform closed-loop iterative solution on the main problem and the subproblems, generating a global path set. ; From the global path set Extract a rolling waypoint set, and perform spatial optimization within the rolling waypoint set to generate local path segments. Drive the shooting drone along the local path segment flight.

2. The method according to claim 1, characterized in that, The generation of the initial occupied map based on the environmental point cloud data includes: Obtain the set of interest points surrounding the points to be inspected. The bounding box set, and the preset initial position set of the exploratory drone; In response to the bounding box set and the initial position set of the exploratory UAV, the maximum and minimum boundaries of the generated workspace are calculated; Obtain the preset voxel size, and discretize the maximum and minimum boundaries of the working space based on the voxel size to generate a 3D mesh map; The three-dimensional coordinates extracted from the environmental point cloud data are mapped to the discrete voxels corresponding to the 3D mesh map; Obtain the preset occupation discrimination conditions. In response to the cumulative number of laser point cloud hits and the hit ratio within the discrete voxel satisfying the occupation discrimination conditions, mark the corresponding discrete voxel as occupied and output the local occupation map. Extract the spatial locations of neighboring drones within the communicable set, perform spatial registration and data fusion on the local occupancy map based on the spatial locations of the neighboring drones, remove the inward edges corresponding to the vertices marked as occupied in the 3D mesh map, and output the initial occupancy map; Preferably, the extreme value boundary of the workspace is generated based on the extreme value function, and the algebraic expression is limited as follows: in For the set of bounding boxes, The initial position set of the exploratory UAV; the step of mapping the three-dimensional coordinates extracted from the environmental point cloud data to the discrete voxels corresponding to the 3D mesh map, the mapping index formula is as follows: ,in For the voxel size, The components of the extracted three-dimensional coordinates; the occupancy discrimination condition is limited to the cumulative number of laser point cloud hits within a single discrete voxel being greater than or equal to 3, and the hit ratio satisfying the inequality. ,in This represents the number of laser point cloud hits within a voxel. This represents the number of times the laser beam missed the voxel.

3. The method according to claim 1, characterized in that, The effective line-of-sight of the onboard gimbal camera of the shooting drone. Generate a set of inspection waypoints within the range ,include: A preset safety offset distance is obtained, and a safety envelope surface is generated by offsetting the surface normal vector outward along the surface normal vector using the surface formed by the target voxel as a reference. The horizontal and vertical field of view angles preset by the airborne gimbal camera are obtained, and spatial sampling is performed on the safety envelope plane according to the overlap of the horizontal and vertical field of view angles to generate a discrete sampling point set. In response to the fact that the sampling points within the set of discrete sampling points do not overlap with the occupying voxel and the linear spatial distance is within the effective line-of-sight. Within the specified range, perform retention processing and output the set of inspected waypoints containing discrete waypoints. ; Preferably, the range of the safety offset distance is limited to twice the voxel size. The mapping formula for converting the three-dimensional direction vector pointing to the corresponding target voxel into servo control commands for the airborne gimbal camera is limited to: and ,in The coordinate components of the three-dimensional direction vector are... For the azimuth angle control of the airborne gimbal camera, This refers to the tilt angle control value of the airborne gimbal camera.

4. The method according to claim 1, characterized in that, The path selection variable The ambiguity index of the corresponding inspection waypoint With resolution indicators Obtain the overall observation quality score ,include: Extract the three-dimensional linear velocity, three-dimensional angular velocity, and instantaneous rotational angular velocity of the onboard gimbal camera of the shooting drone, and generate a relative velocity vector based on the forward kinematic transfer equation; The preset single-frame physical exposure time and total horizontal pixel width of the airborne gimbal camera are obtained. The relative velocity vector and the single-frame physical exposure time are mapped to the image plane coordinate system to generate an image plane displacement. Based on the algebraic ratio between the image plane displacement and the total horizontal pixel width, the blur index is generated. ; The system acquires the preset physical focal length and desired resolution index of the airborne gimbal camera. Based on the acquired effective field of view, the physical focal length, and the current physical observation distance, it generates the actual sampling resolution of the image pixels mapped to physical space. Then, based on the ratio of the actual sampling resolution to the desired resolution index, it generates the resolution index. ; Execute the ambiguity index With the resolution index The product operation outputs the comprehensive observation quality score. ; Preferably, the ambiguity index The generation formula is limited to ,in The total width of the horizontal pixels of the image plane. These are the displacement coordinates of the image plane in two frames, caused by the relative velocity vectors. The resolution index The generation formula is limited to ,in For the desired resolution specification, These represent the horizontal and vertical resolutions of the actual sampling, respectively.

5. The method according to claim 1, characterized in that, The establishment is based on the deployment status of automatic battery swapping stations. Relationship with facility allocation The main problem of site allocation for decision variables includes: Obtain the preset fixed deployment cost of the automated battery swapping station and the dynamic inspection time cost generated by adding the flight time and hovering service time between nodes; obtain preset weighting coefficients, and perform a weighted summation operation on the fixed deployment cost of the automated battery swapping station and the dynamic inspection time cost according to the weighting coefficients to generate a minimized objective function; obtain the preset maximum service radius, maximum runtime, and maximum battery capacity of the automated battery swapping station, and adjust the deployment status of the automated battery swapping station according to the maximum service radius, the maximum runtime, and the maximum battery capacity. Relationship with facility allocation Apply algebraic boundary constraints and output the constraints of the main problem of the location allocation; Preferably, the objective function formula for minimizing the main problem of location allocation is limited to: ,in To fix deployment costs, These are the weighting coefficients. This is a set of candidate automatic battery swapping stations. For the set of path edges, For 0-1 decision variables that characterize path selection, For flight time, For hovering service time.

6. The method according to claim 1, characterized in that, The Benders decomposition algorithm, based on logical basis, is invoked to perform closed-loop iterative solutions to the main problem and the subproblems, generating a global path set. ,include: In response to the subproblem's solution generating a set of allocated facilities exceeding the maximum runtime or maximum battery capacity, a subset of bottleneck tasks leading to infeasibility is extracted, and a logical feasibility cut plane containing this subset is generated. This logical feasibility cut plane is then input into the main problem of the location allocation. In response to the subproblem completing a single path optimization calculation, node hovering penalties and round-trip costs are extracted to generate a dynamic time compensation. This dynamic time compensation is then introduced into the relaxation variable iteration formula of the main problem of the location allocation to generate a logical optimality cut plane. This logical optimality cut plane is then input into the main problem of the location allocation to execute the next round of calculation. In response to the algorithm convergence flag, the global path set is output. .

7. The method according to claim 1, characterized in that, The global path set Extract a rolling waypoint set, and perform spatial optimization within the rolling waypoint set to generate local path segments. ,include: Based on a preset rolling temporal window length, the nodes are selected from the global path set in sequence. Extracting adjacent global waypoints; responding to situations where the spatial distance between adjacent global waypoints is greater than twice the preset voxel size, performing linear interpolation between the adjacent global waypoints and outputting the rolling waypoint set; obtaining the preset equivalent safe bounding box boundary size of the UAV body, performing dilation mapping on the occupied voxels in the initial occupied map using the algebraic sum of the equivalent safe bounding box boundary size and the voxel size, generating a boundary-updated occupied map; using the current spatial position of the shooting UAV as the starting point of the topology search, and the end sequence node in the rolling waypoint set as the search endpoint, performing graph search calculation based on the boundary-updated occupied map, and outputting the local path fragment. .

8. The method according to claim 1, characterized in that, The driving-type shooting drone along the local path segment Flight, including: Extract the current state vector of the shooting drone, which includes its three-dimensional spatial position, three-dimensional yaw angle, three-dimensional linear velocity, and three-dimensional yaw angular velocity; and extract the local path segment. The mapping is to include control inputs that include triaxial acceleration and yaw angle acceleration; Obtain the preset control system sampling interval, substitute the state vector and the control input quantity into the system state space equation according to the control system sampling interval, and generate the underlying flight control command; The underlying flight control commands are output to drive the camera-type drone. Preferably, the algebraic expansion of the system state-space equations is limited to: ,in The sampling interval of the control system is... It is a 4th order identity matrix. It is a 4th order zero matrix. Let be the state vector from the previous moment. The control input from the previous moment. This is the current state vector.

9. A multi-heterogeneous unmanned aerial vehicle (UAV) collaborative inspection device, characterized in that, include: The initial mapping and waypoint generation module is used to acquire environmental point cloud data obtained by the exploratory UAV during its flight, and to generate an initial occupied map based on the environmental point cloud data. Extract the set of points of interest to be inspected from the initial occupied map. The associated target occupies a voxel within the effective line-of-sight range of the onboard gimbal camera of the shooting drone. Generate a set of inspection waypoints within the range ; The quality score acquisition module is used to obtain a score based on the inspection waypoint set. Construct path selection variables ; Variables selected based on the path The ambiguity index of the corresponding inspection waypoint With resolution indicators Obtain the overall observation quality score ; The processing module is used to establish the deployment status of automated battery swapping stations. Relationship with facility allocation The main problem is the allocation of location as the decision variable for the main problem, and the path selection variable is used as the main problem. Sub-problems are the decision variables for sub-problems; based on the comprehensive observation quality score. Greater than the engineering quality threshold As an observation quality constraint, under the condition that the observation quality constraint is satisfied, the logical basis Benders decomposition algorithm is invoked to perform closed-loop iterative solution on the main problem and the subproblems, generating a global path set. ; Output module, used to extract from the global path set Extract a rolling waypoint set, and perform spatial optimization within the rolling waypoint set to generate local path segments. Drive the shooting drone along the local path segment flight.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the multi-heterogeneous UAV collaborative inspection method as described in any one of claims 1 to 8.

Images