An unmanned aerial vehicle cluster cooperative perception-based steel trestle intelligent inspection method

Abstract

The present application belongs to the technical field of steel trestle intelligent inspection, specifically a steel trestle intelligent inspection method based on unmanned aerial vehicle cluster collaborative sensing, comprising: collecting multi-source physical sensing data of the steel trestle area, and determining sensing calibration anchor blocks according to voxel complexity, dynamically adjusting unmanned aerial vehicle observation pose, generating initial inspection flight path and physical sensing basic data packet; identifying sensing quality attenuation trend according to multi-modal sensing flow data, generating flight control compensation and load adjustment instructions, and triggering multi-machine collaborative compensation and resource redistribution when a single machine node fails; extracting multi-machine physical feature anchor points, performing space-time registration and feature fusion on global inspection data, completing structure disease identification and safety state evaluation, generating a digital intelligent inspection report, and incrementally updating the steel trestle physical space model; the present application realizes fine spatial expression of the steel trestle geometric structure by adopting steel trestle physical space voxel modeling technology.

Description

Technical Field

[0001] This invention belongs to the field of intelligent inspection technology for steel trestle bridges, specifically a method for intelligent inspection of steel trestle bridges based on collaborative perception by a cluster of unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of infrastructure construction, the safety maintenance and inspection of large structures such as steel trestle bridges have become increasingly important. Traditional inspection methods for steel trestle bridges mainly rely on manual labor, which is not only inefficient and time-consuming, but also poses significant safety hazards when facing high-altitude, narrow, or dangerous areas, making it difficult to conduct a comprehensive and detailed inspection of all critical parts. To overcome the limitations of manual inspection, drone technology has gradually been introduced into infrastructure inspection work. Drones can be equipped with high-resolution cameras, infrared thermal imagers, and other sensors to achieve non-contact, multi-angle photography and data collection of the steel trestle bridge structure surface, effectively compensating for the blind spots and high-risk areas of manual inspection.

[0003] However, existing technologies for achieving collaborative perception by UAV swarms and refined inspection of steel trestle bridges are mainly limited by the logical coupling constraints between the dynamic nature of multi-UAV collaborative path planning and the real-time fusion accuracy of multi-source heterogeneous data. This makes it difficult to balance coverage integrity (blind spot elimination and track overlap control) and spatiotemporal consistency of perception data (multi-UAV data synchronization accuracy and feature registration quality) in complex steel structure environments in practical applications. Ultimately, this affects the accuracy of structural defect identification and the overall execution efficiency of inspection tasks, thus restricting the engineering application and promotion of UAV swarm collaborative perception technology in the field of intelligent operation and maintenance of large steel trestle bridges. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an intelligent inspection method for steel trestle bridges based on collaborative perception of UAV swarms, which solves the problems of logical coupling conflict between multi-machine collaborative path planning and multi-source heterogeneous data, poor coverage integrity in complex steel structure environments, and low collaborative efficiency of multi-machine inspection in existing technologies.

[0005] A method for intelligent inspection of steel trestle bridges based on collaborative perception of UAV swarms includes the following steps: collecting multi-source physical perception data of the target steel trestle bridge area, dividing the target steel trestle bridge area into multiple physical space voxel units, determining perception calibration anchor blocks based on the physical perception complexity index of each voxel unit, dynamically adjusting the observation distance and pitch angle of the corresponding UAV based on the determined perception calibration anchor blocks, collecting the physical arrangement results after each attitude adjustment, generating an initial inspection trajectory sequence after all observation attitude points have been processed, performing perception load sampling rate equalization processing on each node unit in the initial inspection trajectory sequence, determining an effective initial configuration scheme, and determining the physical perception basic data package based on the effective initial configuration scheme, together with the hardware perception parameter matrix, three-dimensional physical coordinates, attitude data, and obstacle avoidance safety field threshold data of each node.

[0006] The system receives multimodal perception stream data transmitted back by the UAV cluster and constructs the real-time perception quality state vector of each UAV node using a determined perception calibration anchor block as a physical reference. It also identifies the multidimensional attenuation gradient of the perception quality on the physical surface of the steel trestle in real time. Based on the identified multidimensional attenuation gradient, it generates corresponding flight control compensation commands and load adjustment commands. When a single node cannot meet the current inspection requirements, it triggers collaborative replacement and resource reallocation among multiple UAVs.

[0007] Physical feature anchor points of different UAVs in the same time series are extracted, and high-precision spatial registration and feature-level fusion are performed on the full-domain high-fidelity inspection dataset. Based on the fused full-domain inspection data, structural defects identification and safety status assessment are performed to generate digital intelligent inspection reports. The physical space voxel model of the steel trestle bridge is incrementally updated according to the inspection results.

[0008] Preferably, the determination of the perception calibration anchor block based on the physical perception complexity index of each voxel unit is as follows:

[0009] Using an airborne 3D laser scanner and a 4K high-magnification zoom camera, point cloud spatial distribution data, surface texture physical feature data, and real-time pose feedback data of the target steel trestle area were collected. Based on the collected multi-source physical perception data, the target steel trestle area was divided into sections according to the rate of change of geometric curvature and spatial connectivity. physical space voxel unit And calculate each voxel unit Physical perception complexity index ;

[0010] The calculated physical perception complexity index As an evaluation index for the inspection priority of each voxel unit, the voxel unit with the largest physical perception complexity index is selected as the perception calibration anchor block from all voxel units.

[0011] Preferably, the dynamic adjustment of the observation distance and pitch angle of the corresponding UAV based on the determined perception calibration anchor block is as follows:

[0012] Based on the determined sensing calibration anchor block, multiple initial observation pose points are selected for the complex structural region radiated by it. The selected initial observation pose points are then compared with the sensing resolution fidelity index of the corresponding positions in the physical sensing reference model. A comparison is made, and adjustments to the observation distance and pitch angle are performed based on the comparison results, specifically as follows:

[0013] Set upper and lower limits for the perceptual resolution deviation threshold and And calculate the initial observation pose point. Perceptual resolution difference at the same location as the physical perception reference model Based on the calculated difference in perceptual resolution, the arrangement state of the current observation pose point is determined, and thus:

[0014] If the comparison results satisfy the formula If the current observation pose point captures a physical feature resolution that meets the preset accuracy requirements, then no pose adjustment will be performed. Instead, the current observation pose point will be marked as a steady-state observation point, and the process will proceed directly to the next observation pose point.

[0015] If the comparison results satisfy the formula If the current observation pose point is in a fluctuating range, it is marked as an observation point to be calibrated and pose adjustment is not performed for the time being. After all other observation pose points have been processed, a second judgment is made based on the trend of the cluster perception field strength. If the second judgment result still cannot determine the adjustment direction, the current observation pose point is kept unchanged.

[0016] If the comparison results satisfy the formula If this occurs, it indicates that the current observation pose point has experienced a severe degradation in perception accuracy, and hardware pose dynamic compensation will be performed.

[0017] Preferably, the execution of hardware pose dynamic compensation is specifically as follows:

[0018] Set occlusion probability threshold If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula And the current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, radial inward adjustment of the observation distance is performed until the adjusted perception resolution difference satisfies the formula when compared with the set threshold. until;

[0019] If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula or current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, the observation angle pitch and yaw adjustment is performed until the difference in perceived resolution after adjustment satisfies the formula when compared with the set threshold. until.

[0020] Preferably, the determination of the physical sensing basic data packet is as follows:

[0021] Collect the physical layout results after each pose adjustment. Once all observed pose points have been processed, generate the initial inspection track sequence. Based on preset flight constraints, the initial inspection track sequence is... Each node unit performs sensing payload sampling rate equalization processing, and the effective initial configuration scheme after equalization processing, together with the hardware sensing parameter matrix, three-dimensional physical coordinates and attitude data of each node, and obstacle avoidance safety field threshold data, are used as the physical sensing basic data package.

[0022] Preferably, the construction of the real-time perceived quality state vector for each UAV node is as follows:

[0023] By utilizing edge computing units to synchronously receive multimodal perception stream data transmitted back by the UAV cluster, and using a defined perception calibration anchor block as a physical reference, a real-time perception quality state vector for each UAV node is constructed. Then we have:

[0024] ;

[0025] in, This represents the signal fidelity based on image gradient structure similarity. This represents the local sampling density based on the octree-based point cloud filtering distribution. This represents the effective view coverage based on the projection geometry overlap algorithm. This indicates the pointing accuracy of the gimbal motor based on encoder feedback. This indicates the reliability of the communication link based on electromagnetic environment monitoring.

[0026] Preferably, the real-time identification and sensing of the multidimensional attenuation gradient of the perceived quality on the physical surface of the steel trestle is specifically as follows:

[0027] By perceiving the quality state vector Hardware sensing parameter matrix in the physical sensing base data package Perform correlation mapping to identify the multidimensional attenuation gradient of perceived mass on the physical surface of the steel trestle in real time. Then we have:

[0028] The derivative of the perceived quality state vector in the three-dimensional physical space and time dimension is obtained as the corresponding multidimensional decay gradient.

[0029] Preferably, the generation of corresponding flight control compensation commands and load adjustment commands based on the identified multidimensional attenuation gradient is as follows:

[0030] Using the Lie group manifold mapping algorithm to sense the mass decay gradient Real-time conversion into thrust distribution correction for the UAV power system Three-axis angle compensation of the gimbal motor ;

[0031] When the multidimensional decay gradient Display local point density When a nonlinear drop occurs, the system immediately initiates a radial distance compensation closed loop, specifically:

[0032] By adjusting the pulse width modulation duty cycle of the electronic speed controller of the drone's brushless motor, the motor speed is adjusted to shorten the physical distance between the drone and the steel structure. At the same time, the pulse repetition frequency of the lidar and the angular velocity of the scanning mirror stepper motor are adjusted in conjunction with the hardware interrupt signal to ensure that the number of effective sampling points per unit physical area is restored to the set reference threshold.

[0033] When the view coverage is detected When affected by the geometric obstruction of the steel structure, the optimal obstacle avoidance deflection is calculated in real time in the confined space, the gimbal motor is driven to perform tangential view smoothing compensation, and the camera's autofocus actuator is adjusted simultaneously to ensure that the sensor focal plane is always locked on the surface of the steel structure during dynamic pitch change.

[0034] Preferably, the collaborative backup and resource reallocation among multiple machines are specifically as follows:

[0035] Real-time monitoring of single nodes using distributed consensus protocols Real-time perceived quality state vector ;

[0036] If the quality state vector is sensed in real time If the drone falls below the set obstacle avoidance safety field threshold, the cooperative gain calculation operator is immediately activated. Under the premise of satisfying the dynamic nonlinear constraints, neighboring drones are dispatched. Perform coordinated track overlap compensation;

[0037] The hardware-level precision time protocol performs nanosecond-level timestamp alignment on heterogeneous data streams collected by multiple machines, and dynamically adjusts the direct memory access priority and data transmission priority register values ​​of neighboring machine positions through the sensing parameter matrix. At the same time, the communication transmission power is dynamically adjusted through the radio frequency front-end to compensate for the lack of sensing of failed nodes through the reallocation of hardware resources.

[0038] When the overlap of the field of view of multiple drones meets the preset feature registration criteria, a synchronization pulse signal is automatically sent to the sensor trigger pins of all drones in the cluster, forcing the shutter actuators of all cameras and the clock counters of the lidar to operate at the same physical moment, thereby acquiring heterogeneous raw data with strong spatiotemporal correlation.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. This invention achieves a refined spatial representation of the complex geometry, obstructed areas, and sparsely characterized areas of steel trestle bridges by adopting voxel modeling of the physical space of the steel trestle bridge and adaptive determination technology of the perception calibration anchor block. The area with the highest complexity is used as the benchmark for the cluster perception performance, enabling the UAV cluster to dynamically adjust the allocation of inspection resources according to the complexity of the structure. This effectively reduces the blind spot problem caused by fixed path planning in complex truss areas and improves the coverage integrity and rationality of flight path planning in complex steel structure environments.

[0041] 2. This invention achieves adaptive and collaborative optimization of UAV observation distance, pitch angle and payload sampling parameters by adopting observation pose dynamic adjustment technology based on perception resolution feedback. This enables the UAV to correct its observation attitude in real time according to the degree of local occlusion and perception degradation, avoiding feature loss caused by structural occlusion, limited viewing angle and insufficient sampling density, and improving the effective perception coverage and defect detail acquisition capability in complex steel trestle environments.

[0042] 3. This invention adopts a dynamic collaborative control technology for UAV swarms based on perception-action manifold mapping, which realizes a direct correlation mapping between the perception quality decay state and flight control behavior. This enables UAVs to dynamically adjust thrust distribution, gimbal attitude, and payload operating parameters according to real-time changes in perception quality. This breaks the traditional control mode where path planning and data acquisition are independent, reduces the logical coupling constraints between multi-UAV collaborative path dynamic adjustment and perception accuracy, and improves the autonomous collaborative capability and mission execution efficiency of UAV swarms in complex environments.

[0043] 4. This invention adopts a multi-machine collaborative supplementation and resource reallocation technology based on a distributed consensus protocol. When a local UAV node fails to perceive due to obstruction, multipath effect, or communication attenuation, it automatically triggers neighboring UAVs to perform collaborative trajectory compensation, data acquisition takeover, and dynamic reconstruction of communication resources. This effectively eliminates the perception gaps and data loss caused by single-machine failure in traditional multi-machine inspection processes, and improves the continuity, robustness, and coverage integrity of inspection tasks in complex steel structure areas.

[0044] 5. This invention achieves nanosecond-level time synchronization and sub-pixel-level spatial registration of multi-source heterogeneous data, point cloud data, and pose data by adopting a multi-source heterogeneous data full-domain linkage fusion technology based on unified spatiotemporal constraints. It also completes adaptive fusion of texture features and geometric features by combining a cross-modal attention mechanism. This effectively solves the problems of data misalignment, feature mismatch, and low fusion accuracy caused by asynchronous acquisition of multiple machines in the prior art. It significantly improves the accuracy of structural defect identification, the accuracy of physical measurement, and the credibility of digital inspection results, providing high-precision data support for the intelligent operation and maintenance of large steel trestle bridges. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall method steps of the intelligent inspection method for steel trestle bridges based on collaborative perception of UAV swarms according to the present invention. Detailed Implementation

[0046] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0047] Example 1

[0048] Reference Figure 1 As an embodiment of the present invention, a method for intelligent inspection of steel trestle bridges based on collaborative perception of unmanned aerial vehicle (UAV) swarms is provided, comprising the following steps:

[0049] S1: Physical space voxel modeling of steel trestle bridge and adaptive decomposition of inspection tasks.

[0050] Specifically, the voxel modeling of the physical space of the steel trestle bridge and the adaptive decomposition of the inspection task involve collecting multi-source physical sensing data of the target steel trestle bridge area and constructing a voxel map of the steel trestle bridge's physical space based on the collected data. This enables the initial planning of the sensing accuracy ratio, UAV observation pose, and cooperative flight path points for the inspection task in the physical space, outputting an initial configuration scheme for the steel trestle bridge inspection and providing a basic data package for physical sensing for subsequent steps. The specific implementation is as follows:

[0051] Using an airborne 3D laser scanner, a 4K high-magnification zoom camera, and a high-precision inertial measurement unit (IMU), point cloud spatial distribution data, surface texture physical feature data, and real-time pose feedback data of the UAV were collected for the target steel trestle area.

[0052] Based on the collected multi-source physical sensing data, the physical space voxels of the steel trestle bridge are constructed, specifically as follows:

[0053] The target steel trestle area is divided into sections based on the rate of change of geometric curvature and spatial connectivity. physical space voxel unit And calculate each voxel unit Physical perception complexity index Then we have:

[0054] ;

[0055] in, Represents the local geometric curvature coefficients, calculated from voxel elements. The average Gaussian curvature of all point clouds within the cloud is obtained. This represents the occlusion probability evaluation coefficient, obtained by analyzing voxel units. Three-dimensional ray tracing analysis was performed to obtain the proportion of rays that were blocked from the preset observation point to the voxel surface. The sparsity coefficient of feature points is represented by the voxel unit. The ratio of the number of SIFT feature points to the voxel surface area is obtained. , , This represents the physical characteristic weighting factor. The specific value is dynamically set by the inspection system based on the structural type of the steel trestle and the lighting environment, and satisfies the formula... , Voxel unit representing computation The physical perception complexity index;

[0056] The calculated physical perception complexity index As an evaluation index for the inspection priority of each voxel unit, the voxel unit with the highest physical perception complexity index is selected from all voxel units. As a sensing calibration anchor block ;

[0057] It should be noted that the sensing calibration anchor block The selection criteria are as follows: the physical spatial geometry of the current voxel unit is the most complex and the risk of sensor signal attenuation is the highest. By setting it as the benchmark, a lower limit performance reference for physical perception can be provided for subsequent multi-UAV collaborative inspections, and it can also serve as a feedback benchmark for adaptive adjustment of cluster collaborative perception parameters, ensuring that the fidelity of sensor data acquisition can still be maintained in complex occlusion environments. Specifically:

[0058] When multiple voxel units have the same maximum complexity index, the voxel unit closest to the centroid of the current UAV cluster is selected as the perception calibration anchor block.

[0059] If the complexity index of the sensing calibration anchor block is significantly lower than that of the suboptimal voxel unit due to structural damage or drastic environmental changes during the inspection process, the sensing calibration anchor block will be dynamically switched to the new voxel unit with the highest complexity.

[0060] Based on the determined sensing calibration anchor block For the complex structural regions radiated by it, multiple initial observation pose points are randomly selected. And the selected initial observation pose point is compared with the corresponding position in the physical perception reference model to obtain the perception resolution fidelity index. For comparison, the perceptual resolution fidelity index is the product of the effective feature points within a single physical area and the image signal-to-noise ratio. Based on the comparison results, differential observation distance and pitch angle adjustments are performed, specifically as follows:

[0061] Set upper and lower limits for the perceptual resolution deviation threshold and And calculate the initial observation pose point. Perceptual resolution difference at the same location as the physical perception reference model The arrangement status of the current observation pose point is determined based on the calculated difference in perceptual resolution. Specifically:

[0062] If the comparison results satisfy the formula If the current observation pose point captures a physical feature resolution that meets the preset accuracy requirements, then no pose adjustment will be performed. Instead, the current observation pose point will be marked as a steady-state observation point, and the process will proceed directly to the next observation pose point.

[0063] If the comparison results satisfy the formula If the current observation pose point is in a fluctuating range, it is marked as an observation point to be calibrated and pose adjustment is not performed for the time being. After all other observation pose points have been processed, a second judgment is made based on the trend of the cluster perception field strength. If the second judgment result still cannot determine the adjustment direction, the current observation pose point is kept unchanged.

[0064] If the comparison results satisfy the formula This indicates that the current observation pose point has experienced a severe degradation in perception accuracy. Performing hardware pose dynamic compensation will then yield the following:

[0065] Set occlusion probability threshold ;

[0066] If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula And the current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, the observation distance is adjusted radially inward (by controlling the drone to reduce the physical distance to the steel structure to increase the sampling point density per unit area) until the adjusted perception resolution difference satisfies the formula when compared with the set threshold. until;

[0067] If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula or current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, the observation angle pitch and yaw adjustment is performed (by controlling the gimbal motor to change the shooting pitch angle to avoid shadow areas caused by physical obstruction) until the adjusted perceived resolution difference meets the set threshold according to the formula. until.

[0068] Collect the physical layout results after each pose adjustment. Once all observed pose points have been processed, generate the initial inspection track sequence. Based on the industrial drone flight safety standards and the physical specifications for non-destructive testing of steel structures, Each node unit performs sensing payload sampling rate equalization processing (including dynamic matching of camera shutter speed, LiDAR scanning frequency, and IMU sampling frequency), and submits the effective initial configuration scheme after equalization processing, along with the hardware sensing parameter matrix of each node. 3D physical coordinates and attitude data and obstacle avoidance safety field threshold data As a basic data package for physical sensing Pass it on to the next step.

[0069] S2: Dynamic collaborative control and refined inspection of UAV swarms based on perception-action manifold mapping.

[0070] Specifically, dynamic collaborative control and refined inspection of UAV swarms based on perception-action manifold mapping.

[0071] It is an intelligent inspection system that uses a swarm of drones deployed on the drone's onboard terminal and ground station to collaboratively perceive and inspect basic data packets of physical perception. Mapped to the cluster collaborative control manifold space, the hardware control compensation vector is calculated in real time, driving the UAV cluster to perform perception-guided dynamic trajectory reconstruction and synchronous adjustment of load parameters, achieving seamless coverage of the complex structure of the steel trestle bridge. The specific implementation is as follows:

[0072] A multi-dimensional real-time evaluation system for hardware sensing throughput is established. Edge computing units synchronously receive multimodal sensing stream data transmitted from the UAV cluster. Using a defined sensing calibration anchor block as a physical reference, a real-time sensing quality state vector for each UAV node is constructed. Then we have:

[0073] ;

[0074] in, This represents the signal fidelity based on image gradient structural similarity (SSIM). This represents the local sampling density based on the octree-based point cloud filtering distribution. This represents the effective view coverage based on the projection geometry overlap algorithm. This indicates the pointing accuracy of the gimbal motor based on encoder feedback. Indicates the reliability of communication links based on electromagnetic environment monitoring;

[0075] By perceiving the quality state vector Hardware sensing parameter matrix in the physical sensing base data package Perform dynamic tensor product operations to identify the multidimensional attenuation gradient of the sensed mass on the physical surface of the steel trestle in real time. Then we have:

[0076] The derivative of the perceived quality state vector in three-dimensional physical space and time dimension is obtained as the corresponding multidimensional decay gradient.

[0077] Furthermore, the evolution trend of perceptual performance is represented using tangent vectors in the Lie Algebra space, specifically:

[0078] Using the Lie Group manifold mapping algorithm to sense the mass decay gradient Real-time conversion into thrust distribution correction for the UAV power system Three-axis angle compensation of the gimbal motor ;

[0079] When the multidimensional decay gradient Display local point density When a nonlinear drop occurs, the system immediately initiates a radial distance compensation closed loop: by driving the pulse width modulation (PWM) duty cycle of the electronic speed controller (ESC) of the drone's brushless motor, the motor speed is adjusted to shorten the physical distance between the drone and the steel structure. At the same time, the pulse repetition frequency (PRF) of the lidar and the angular velocity of the scanning mirror stepper motor are adjusted in conjunction with the hardware interrupt signal to ensure that the number of effective sampling points per unit physical area is restored to the set reference threshold.

[0080] When the view coverage is detected When affected by the geometric obstruction of the steel structure, the optimal obstacle avoidance deflection is calculated in real time in the confined space, the gimbal motor is driven to perform tangential viewpoint smoothing compensation, and the camera's autofocus (AF) actuator is adjusted simultaneously to ensure that the sensor focal plane is always locked on the surface of the steel structure during dynamic pitch change.

[0081] It should be noted that when the adjustment amount of the hardware actuator of a single node reaches the preset physical travel threshold, or the local sensing quality... If the system still cannot restore the position to the baseline requirements through single-machine pose adjustment, it automatically transforms the local perceived conflict into a resource scheduling requirement at the cluster level, thereby triggering multi-machine collaborative compensation and resource reallocation logic, specifically:

[0082] Real-time monitoring of single nodes using a distributed consensus protocol Real-time perceived quality state vector ;

[0083] If the multipath effect caused by the intersection of internal trusses of the steel trestle bridge or physical signal blockage causes the real-time sensing quality state vector to be affected... If the fall falls below the set obstacle avoidance safety field threshold, the system immediately activates the cooperative gain calculation operator to schedule neighboring UAVs while satisfying the dynamic nonlinear constraints. It performs collaborative track overlap and fill-in; performs nanosecond-level timestamp alignment on heterogeneous data streams collected by multiple machines through the hardware-level Precision Time Protocol (PTP), and dynamically adjusts the Direct Memory Access (DMA) priority and data transfer priority register values ​​of neighboring machine positions through the sensing parameter matrix. At the same time, it dynamically adjusts the communication transmit power through the radio frequency front-end (RFE) to compensate for the lack of sensing of failed nodes by reallocating hardware resources.

[0084] Furthermore, by utilizing ultra-wideband (UWB) relative positioning data among drone swarms, the overlap of the field of view of multiple drones can be calculated in real time.

[0085] When the overlap of the field of view of multiple drones meets the preset feature registration criteria, a synchronization pulse signal is automatically sent to the sensor trigger pins of all drones in the cluster, forcing the shutter actuators of all cameras and the clock counters of the lidar to operate at the same physical moment, thereby acquiring heterogeneous raw data with strong spatiotemporal correlation and resolving feature misalignment and logical coupling conflicts caused by asynchronous movement of multiple drones.

[0086] By transforming the originally static inspection trajectory into an adaptive sensing flow field that dynamically evolves with the physical geometry of the steel trestle and the electromagnetic properties of the environment, the logical coupling conflict between multi-machine collaborative path planning and multi-source heterogeneous data in existing technologies is fundamentally resolved. This significantly improves the coverage integrity and collaborative efficiency of cluster inspection in complex steel structure environments. Finally, a full-domain high-fidelity inspection dataset with hardware-level pose labeling, spatiotemporal consistency constraints, and environmental physical metadata is output and passed to subsequent steps.

[0087] S3: Multi-source heterogeneous data full-domain linkage fusion and inspection result output based on spatiotemporal constraint alignment.

[0088] Specifically, the multi-source heterogeneous data full-domain linkage fusion and inspection result output based on spatiotemporal constraint alignment is achieved by extracting physical feature anchor points of different UAVs in the same time series, establishing a multi-source heterogeneous data association model based on a unified spatiotemporal coordinate system, performing high-precision spatial registration and feature-level fusion on the full-domain high-fidelity inspection dataset, and using a deep learning network to perform defect identification and structural safety assessment on the fused full-domain data, ultimately generating a digital intelligent inspection report. The specific implementation is as follows:

[0089] Using multidimensional feature extraction operators, spatially invariant feature anchors are identified from high-fidelity image sequences and dense point cloud data. In conjunction with the physical space voxel map of the steel trestle bridge, each drone node is located in... Hardware pose data at any given moment As initial values, a global spatiotemporal consistency constraint cost function is constructed, specifically as follows:

[0090] ;

[0091] in, This represents the perspective projection transformation operator. Indicates the sensing calibration anchor block Mapped global reference coordinates The regularization constraint term represents the nanosecond-level clock residual of the PTP protocol. This represents a consistency penalty term for the hardware payload sampling frequency. , , Let represent the spatiotemporal balance weighting factors, and satisfy the formula. ;

[0092] The cost function is minimized by utilizing the Levenberg-Marquardt (LM) optimization algorithm. This enables subpixel-level spatiotemporal manifold alignment of heterogeneous data streams, eliminating feature displacement deviations caused by asynchronous multi-machine motion.

[0093] Based on the aligned spatiotemporal reference, multimodal feature-level linkage fusion is performed, as follows:

[0094] A dual-path feature pyramid network (Bi-FPN) is used to extract high-frequency surface textures from images and low-frequency geometric topology from point clouds in parallel.

[0095] By constructing a cross-modal interaction attention mechanism, the fusion weights are adjusted in real time based on the physical perception complexity index:

[0096] In regions of geometric singularity in complex trusses, point cloud curvature features are automatically invoked for structural reconstruction.

[0097] In areas with minute cracks on a smooth wall surface, gradient features from a high-magnification zoom image are automatically used for texture enhancement.

[0098] Meanwhile, by utilizing the gain parameters of the underlying hardware sensors, illumination consistency correction and noise suppression are performed on the fused feature map to generate a global linkage perception map containing three-dimensional geometry, surface texture and material properties.

[0099] The Multi-Scale Residual Recognition Network (MSRN) is used to extract disease features and perform physical quantification assessment on the global interconnected sensing map. The specific implementation is as follows:

[0100] The Mask R-CNN method is used to automatically segment typical defects in steel trestle structures, such as fatigue cracks, localized corrosion, loose bolts, and member deformation.

[0101] By combining the hardware-level pose marking data, the identified pixel-level disease features are back-projected into three-dimensional physical space to calculate the actual physical size of the disease (such as crack width Wcrack and corrosion depth Dcorr).

[0102] Subsequently, the structural reliability analysis engine is invoked, and the physical parameters of the defects are input into the preset finite element mechanical evaluation model to calculate the stress concentration factor and residual bearing capacity index of the current structural node under the current working condition in real time.

[0103] Finally, the multi-dimensional assessment results are digitally packaged to automatically generate a digital intelligent inspection report that includes a cloud map of the structural health status of the steel trestle bridge, a list of high-risk defects, and preventive maintenance strategies.

[0104] The digital intelligent inspection report is pushed to the cloud operation and maintenance platform through an encrypted data link and the physical space voxel map is updated simultaneously, providing incremental data support for the next cycle of inspection and realizing closed-loop management of the entire life cycle of inspection tasks.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of protection claimed by the present invention.

Claims

1. A method for intelligent inspection of steel trestle bridges based on collaborative perception by unmanned aerial vehicle (UAV) swarms, characterized in that: Includes the following steps: Multi-source physical perception data of the target steel trestle area is collected, and the target steel trestle area is divided into multiple physical space voxel units. The perception calibration anchor block is determined according to the physical perception complexity index of each voxel unit. Based on the determined perception calibration anchor block, the observation distance and pitch angle of the corresponding UAV are dynamically adjusted. The physical layout results after each attitude adjustment are collected. After all observation attitude points are processed, an initial inspection trajectory sequence is generated, and the perception load sampling rate equalization processing is performed on each node unit in the initial inspection trajectory sequence to determine an effective initial configuration scheme. Based on the effective initial configuration scheme, together with the hardware perception parameter matrix, three-dimensional physical coordinates, attitude data and obstacle avoidance safety field threshold data of each node, the physical perception basic data package is determined. The system receives multimodal perception stream data transmitted back by the UAV cluster and constructs the real-time perception quality state vector of each UAV node using a determined perception calibration anchor block as a physical reference. It also identifies the multidimensional attenuation gradient of the perception quality on the physical surface of the steel trestle in real time. Based on the identified multidimensional attenuation gradient, it generates corresponding flight control compensation commands and load adjustment commands. When a single node cannot meet the current inspection requirements, it triggers collaborative replacement and resource reallocation among multiple UAVs. Physical feature anchor points of different UAVs in the same time series are extracted, and high-precision spatial registration and feature-level fusion are performed on the full-domain high-fidelity inspection dataset. Based on the fused full-domain inspection data, structural defects identification and safety status assessment are performed to generate digital intelligent inspection reports. The physical space voxel model of the steel trestle bridge is incrementally updated according to the inspection results.

2. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The determination of the perception calibration anchor block based on the physical perception complexity index of each voxel unit is as follows: Using an airborne 3D laser scanner and a 4K high-magnification zoom camera, point cloud spatial distribution data, surface texture physical feature data, and real-time pose feedback data of the target steel trestle area were collected. Based on the collected multi-source physical perception data, the target steel trestle area was divided into sections according to the rate of change of geometric curvature and spatial connectivity. physical space voxel unit And calculate each voxel unit Physical perception complexity index ; The calculated physical perception complexity index As an evaluation index for the inspection priority of each voxel unit, the voxel unit with the largest physical perception complexity index is selected as the perception calibration anchor block from all voxel units.

3. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The dynamic adjustment of the observation distance and pitch angle of the corresponding UAV based on the determined perception calibration anchor block is as follows: Based on the determined sensing calibration anchor block, multiple initial observation pose points are selected for the complex structural region radiated by it. The selected initial observation pose points are then compared with the sensing resolution fidelity index of the corresponding positions in the physical sensing reference model. A comparison is made, and adjustments to the observation distance and pitch angle are performed based on the comparison results, specifically as follows: Set upper and lower limits for the perceptual resolution deviation threshold and And calculate the initial observation pose point. Perceptual resolution difference at the same location as the physical perception reference model Based on the calculated difference in perceptual resolution, the arrangement state of the current observation pose point is determined, and thus: If the comparison results satisfy the formula If the current observation pose point captures a physical feature resolution that meets the preset accuracy requirements, then no pose adjustment will be performed. Instead, the current observation pose point will be marked as a steady-state observation point, and the process will proceed directly to the next observation pose point. If the comparison results satisfy the formula If the current observation pose point is in a fluctuating range, it is marked as an observation point to be calibrated and pose adjustment is not performed for the time being. After all other observation pose points have been processed, a second judgment is made based on the trend of the cluster perception field strength. If the second judgment result still cannot determine the adjustment direction, the current observation pose point is kept unchanged. If the comparison results satisfy the formula If this occurs, it indicates that the current observation pose point has experienced a severe degradation in perception accuracy, and hardware pose dynamic compensation will be performed.

4. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 3, characterized in that: The specific details of performing hardware pose dynamic compensation are as follows: Set occlusion probability threshold If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula And the current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, radial inward adjustment of the observation distance is performed until the adjusted perception resolution difference satisfies the formula when compared with the set threshold. until; If the initial observation pose point The difference in perceptual resolution at the same location as the physical perception reference model satisfies the formula or current voxel unit The comparison between the occlusion probability evaluation coefficient and the occlusion probability threshold satisfies the formula. Then, the observation angle pitch and yaw adjustment is performed until the difference in perceived resolution after adjustment satisfies the formula when compared with the set threshold. until.

5. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The determination of the physical sensing basic data packet is as follows: Collect the physical layout results after each pose adjustment. Once all observed pose points have been processed, generate the initial inspection track sequence. Based on preset flight constraints, the initial inspection track sequence is... Each node unit performs sensing payload sampling rate equalization processing, and the effective initial configuration scheme after equalization processing, together with the hardware sensing parameter matrix, three-dimensional physical coordinates and attitude data of each node, and obstacle avoidance safety field threshold data, are used as the physical sensing basic data package.

6. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The construction of the real-time perceived quality state vector for each UAV node is as follows: By utilizing edge computing units to synchronously receive multimodal perception stream data transmitted back by the UAV cluster, and using a defined perception calibration anchor block as a physical reference, a real-time perception quality state vector for each UAV node is constructed. Then we have: ； in, This represents the signal fidelity based on image gradient structure similarity. This represents the local sampling density based on the octree-based point cloud filtering distribution. This represents the effective view coverage based on the projection geometry overlap algorithm. This indicates the pointing accuracy of the gimbal motor based on encoder feedback. This indicates the reliability of the communication link based on electromagnetic environment monitoring.

7. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The real-time identification and sensing of the multidimensional attenuation gradient of the perceived mass on the physical surface of the steel trestle bridge is as follows: By perceiving the quality state vector Hardware sensing parameter matrix in the physical sensing base data package Perform correlation mapping to identify the multidimensional attenuation gradient of perceived mass on the physical surface of the steel trestle in real time. Then we have: The derivative of the perceived quality state vector in the three-dimensional physical space and time dimension is obtained as the corresponding multidimensional decay gradient.

8. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The process of generating corresponding flight control compensation commands and load adjustment commands based on the identified multidimensional attenuation gradient is as follows: Using the Lie group manifold mapping algorithm to sense the mass decay gradient Real-time conversion into thrust distribution correction for the UAV power system Three-axis angle compensation of the gimbal motor ; When the multidimensional decay gradient Display local point density When a nonlinear drop occurs, the system immediately initiates a radial distance compensation closed loop, specifically: By adjusting the pulse width modulation duty cycle of the electronic speed controller of the drone's brushless motor, the motor speed is adjusted to shorten the physical distance between the drone and the steel structure. At the same time, the pulse repetition frequency of the lidar and the angular velocity of the scanning mirror stepper motor are adjusted in conjunction with the hardware interrupt signal to ensure that the number of effective sampling points per unit physical area is restored to the set reference threshold. When the view coverage is detected When affected by the geometric obstruction of the steel structure, the optimal obstacle avoidance deflection is calculated in real time in the confined space, the gimbal motor is driven to perform tangential view smoothing compensation, and the camera's autofocus actuator is adjusted simultaneously to ensure that the sensor focal plane is always locked on the surface of the steel structure during dynamic pitch change.

9. The intelligent inspection method for steel trestle bridges based on UAV swarm collaborative perception as described in claim 1, characterized in that: The collaborative backup and resource reallocation among multiple machines are as follows: Real-time monitoring of single nodes using distributed consensus protocols Real-time perceived quality state vector ; If the quality state vector is sensed in real time If the drone falls below the set obstacle avoidance safety field threshold, the cooperative gain calculation operator is immediately activated. Under the premise of satisfying the dynamic nonlinear constraints, neighboring drones are dispatched. Perform coordinated track overlap compensation; The hardware-level precision time protocol performs nanosecond-level timestamp alignment on heterogeneous data streams collected by multiple machines, and dynamically adjusts the direct memory access priority and data transmission priority register values ​​of neighboring machine positions through the sensing parameter matrix. At the same time, the communication transmission power is dynamically adjusted through the radio frequency front-end to compensate for the lack of sensing of failed nodes through the reallocation of hardware resources. When the overlap of the field of view of multiple drones meets the preset feature registration criteria, a synchronization pulse signal is automatically sent to the sensor trigger pins of all drones in the cluster, forcing the shutter actuators of all cameras and the clock counters of the lidar to operate at the same physical moment, thereby acquiring heterogeneous raw data with strong spatiotemporal correlation.

Images