Unmanned aerial vehicle bridge detection method and device

Abstract

The invention discloses an unmanned aerial vehicle bridge detection method and device, and the method comprises the steps: S1, collecting multi-modal data of a bridge according to a preset route, and transmitting the multi-modal data to a cloud in real time, the multi-modal data comprising a real-time image of the bridge; s2, pixel-level difference calculation and feature-level matching are carried out on the real-time image and a historical reference image, a structure change area is identified as an abnormal area, and the change condition of the structure change area is graded to obtain an anomaly detection result; s3, on the basis of the anomaly detection result, extracting a sub-route containing an abnormal area and a protection area from the preset route, and generating a recheck task queue to obtain a recheck path; and S4, integrating multiple periods of anomaly detection results, and generating a visual report and / or a structured data report containing an anomaly position, an evolution trend and a maintenance suggestion. The efficiency and progress of bridge detection can be improved, and full-period management can be realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection and artificial intelligence technology for unmanned aerial vehicles (UAVs), and in particular to a method and apparatus for UAV bridge inspection. Background Technology

[0002] The existing bridge inspection technology has the following problems: 1. Manual analysis is inefficient. Traditional bridge inspection relies on manual interpretation of massive amounts of images captured by drones. The inspection of a single bridge requires dozens of hours of manual image interpretation and is significantly affected by the experience of the inspectors, making it difficult to meet the high-frequency inspection needs of 2-4 times a year for bridges on the highway network.

[0003] 2. High risk of missed detections and misjudgments. Static preset routes cannot be dynamically adjusted based on real-time detection results, making it difficult to effectively identify newly emerging minute cracks (width < 0.1 mm) or slow expansion of historical anomalies (such as annual expansion < 0.5 m), resulting in a false negative rate as high as 15%-20%.

[0004] 3. Insufficient utilization of data from multiple periods. The lack of intelligent correlation analysis of historical monitoring data makes it impossible to automatically track abnormal evolution trends through image comparison at different times, resulting in a lack of data support for maintenance decisions.

[0005] 4. Lack of flexibility in route planning The fixed flight path design results in drones repeatedly flying over areas without any abnormalities, with 40%-60% of the flight distance being invalid. This is especially problematic during secondary inspections, which require covering the entire bridge and waste approximately 50% of the drone's battery life.

[0006] Current UAV bridge inspection systems primarily focus on anomaly identification from single-period images, such as crack detection algorithms based on YOLOv5. However, research on comparative analysis of multi-period images and dynamic flight path optimization is insufficient. Some systems introduce 3D modeling techniques (such as bridge point cloud reconstruction based on SfM), but fail to integrate AI algorithms to achieve feature-level correlation of historical data, making it difficult to meet the long-term needs of bridge structural health monitoring. Therefore, there is an urgent need for a detection method and system that can integrate AI image comparative analysis and dynamic flight path planning to achieve closed-loop management of bridge anomalies through "detection-comparison-tracking-optimization".

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] This invention provides a method and apparatus for bridge inspection by unmanned aerial vehicles (UAVs), which can solve at least one of the problems in the background art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for bridge inspection using unmanned aerial vehicles (UAVs) includes the following steps: S1: Collecting multimodal data of the bridge along a preset flight path and transmitting it to the cloud in real time, wherein the multimodal data includes real-time images of the bridge; S2: Performing pixel-level difference calculation and feature-level matching between the real-time images and historical reference images to identify structural change areas as abnormal areas and classifying the changes in the structural change areas to obtain anomaly detection results; S3: Based on the anomaly detection results, extracting sub-flight paths containing abnormal areas and protected areas from the preset flight path, and generating a re-inspection task queue to obtain a re-inspection path; S4: Integrating the anomaly detection results from multiple periods to generate a visual report and / or structured data report containing anomaly locations, evolution trends, and maintenance suggestions.

[0010] Preferably, the pixel-level difference calculation between the real-time image and the historical reference image includes: S21: performing grayscale normalization processing on the real-time image and the historical reference image to convert them to the same illumination reference; S22: calculating the grayscale difference value ΔG=|Gt-G0| for each pixel, where Gt is the current pixel grayscale value and G0 is the reference value; S23: performing morphological closing operation on the difference matrix to eliminate noise and extract continuous difference regions; S24: setting a dual threshold judgment: when ΔG>15 and the continuous region area>0.2m², it is marked as a valid difference region, and the pixel-level difference calculation result is obtained.

[0011] Preferably, the method further includes: optimizing the re-inspection path using a graph theory algorithm.

[0012] Preferably, optimizing the re-inspection path includes: modeling the abnormal area and the protected area as a weighted complete graph, where the node weights include: flight distance cost, sensor view coverage, and anomaly level coefficient; using an improved A* algorithm to solve for the optimal path, with the heuristic function being: h(n) = α·min_distance + (1-α)·max_coverage; where α is an empirical coefficient; and adding dynamic constraints: the length of the emergency abnormal path ≤ 50% of the remaining flight time; and the pitch angle change between adjacent waypoints ≤ 45°.

[0013] Preferably, the changes in the structural change area are classified into three levels according to their severity: Level I anomaly > Level II anomaly > Level III anomaly. The priority ranking rule for the re-inspection route is: Level I anomaly > Level II anomaly > Level III anomaly, and the flight time of the re-inspection route does not exceed 50% of the remaining range of the UAV.

[0014] Preferably, the sub-route generation rule includes: taking the abnormal area as the center, extracting the waypoints of the protected area containing the core waypoint and two adjacent detection units above and below, forming a minimum connected subgraph; removing waypoints unrelated to the abnormal area, so that the length of the sub-route does not exceed 30% of the preset route.

[0015] This invention also provides a UAV bridge inspection device for implementing any of the methods described above, comprising: a multi-sensor data acquisition module for acquiring multimodal bridge data according to a preset route and transmitting it to the cloud in real time, wherein the multimodal data includes real-time images; a cloud-based AI comparison and analysis module for performing pixel-level difference calculation and feature-level matching on the real-time images and historical benchmark images, identifying structural change areas as abnormal areas, and classifying the changes in the structural change areas to obtain anomaly detection results; a dynamic route planning module for extracting sub-routes containing abnormal areas and protected areas from the preset route based on the anomaly detection results, generating a re-inspection task queue according to the classification to obtain a re-inspection path; and an inspection report generation module for integrating the anomaly detection results from multiple periods to generate a visual report and / or structured data report containing anomaly location, evolution trend, and maintenance suggestions.

[0016] Preferably, the multi-sensor data acquisition module includes a visible light camera, a lidar, an infrared thermal imager, and an environmental sensor. The visible light camera is used to acquire visible light images, which are the real-time images. The lidar is used to acquire lidar point clouds, and the environmental sensor is used to acquire infrared thermal images. The visible light images, the lidar point clouds, and the infrared thermal images constitute the multimodal data.

[0017] Preferably, the feature extraction unit of the cloud-based AI comparative analysis module adopts an improved ResNet convolutional neural network. The input of the convolutional neural network is the fused multimodal data, and the output is a combination of texture feature vectors and geometric feature vectors.

[0018] Preferably, the multi-sensor data acquisition module supports an emergency encrypted acquisition mode, which automatically adjusts sensor parameters to perform high-precision re-inspection of the target area when an anomaly level is detected to be upgraded; the detection report generation module supports three-dimensional model annotation function, which can dynamically display the spatial location, evolution trajectory and thermal distribution of the health status of the abnormal area on the bridge building information model.

[0019] The present invention has the following beneficial effects: This invention improves the accuracy of anomaly identification by acquiring multi-modal data fusion to obtain multi-dimensional features of bridge structures; it also accurately identifies structural changes through automated comparison of real-time images with historical baseline images; it dynamically adjusts inspection paths based on anomaly detection results, prioritizing the re-inspection of high-risk areas and reducing unnecessary flights; and it constructs a fully intelligent system from data acquisition to report generation, reducing the cost of manual intervention and optimizing flight efficiency. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a UAV bridge inspection method according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of a method for calculating pixel-level differences between a real-time image and a historical reference image according to an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of a UAV bridge inspection device according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of another UAV bridge inspection device according to an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the visualization interface for a bridge structural health inspection report according to an embodiment of the present invention. Detailed Implementation

[0025] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0026] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component. Furthermore, a connection can be used for fixing, coupling, or communication.

[0027] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] This invention provides a method and device for bridge inspection using unmanned aerial vehicles (UAVs), which is applicable to the automated inspection, anomaly identification, and evolution tracking of key components (main towers, cables, and bridge deck beams) of highway and railway bridges (especially complex structures such as cable-stayed bridges and suspension bridges).

[0030] See Figure 1 One embodiment of the present invention provides a method for bridge inspection by unmanned aerial vehicles (UAVs), comprising the following steps: S1: Collect multimodal data of the bridge according to a preset route and transmit it to the cloud in real time. The multimodal data includes real-time images of the bridge. S2: Perform pixel-level difference calculation and feature-level matching between the real-time image and the historical reference image, identify the structural change region as an abnormal region, and classify the changes in the structural change region to obtain the anomaly detection result; S3: Based on the anomaly detection results, extract sub-routes containing anomaly areas and protection areas from the preset route, and generate a re-inspection task queue to obtain the re-inspection path; S4: Integrate the anomaly detection results from multiple phases to generate a visual report and / or structured data report that includes the anomaly location, evolution trend, and maintenance recommendations.

[0031] This invention acquires multi-dimensional features of bridge structures through multimodal data fusion, improving the accuracy of anomaly identification; it accurately identifies structural changes through automated comparison of real-time images with historical benchmark images; it dynamically adjusts inspection paths based on anomaly detection results, prioritizing the re-inspection of high-risk areas and reducing unnecessary flights; and it constructs a fully intelligent system from data acquisition to report generation, reducing the cost of manual intervention.

[0032] In one embodiment of the invention, a drone collects multimodal data of the entire bridge and constructs a historical benchmark database by following a grid flight path generated based on the bridge's BIM (Building Information Modeling) model. The drone performs periodic inspections, and a cloud-based AI model compares real-time data with historical benchmarks, marking areas of abnormal evolution and classifying them into severity levels (Level I / II / III). Differentiated re-inspection flight paths are generated based on the anomaly level; Level I anomalies generate immediate re-inspection instructions, while Level II / III anomalies are inserted into the task queue according to preset time windows. Multiple inspection results are integrated to generate a report, updating the historical benchmark database and marking key monitoring areas.

[0033] In one embodiment of the present invention, the high-definition visible light camera, lidar, infrared thermal imager and environmental sensors carried by the UAV are controlled to collect multimodal data of the bridge according to a preset route and transmit it to the cloud in real time.

[0034] like Figure 2 As shown, the pixel-level difference calculation between the real-time image and the historical reference image includes: S21: Perform grayscale normalization processing on the real-time image and the historical reference image to convert them to the same illumination reference; S22: Calculate the grayscale difference value ΔG=|Gt-G0| for each pixel, where Gt is the grayscale value of the current pixel and G0 is the reference value; S23: Perform morphological closing operations on the difference matrix to eliminate noise and extract continuous difference regions; S24: Set dual threshold judgment: when ΔG>15 and the area of ​​the continuous region>0.2m², it is marked as a valid difference region, and pixel-level difference calculation results are obtained.

[0035] In one embodiment of the present invention, the method further includes: optimizing the re-inspection path using a graph theory algorithm.

[0036] In one specific embodiment, optimizing the re-inspection path includes: The abnormal region and the protected region are modeled as a weighted complete graph, and the node weights include: Flight distance cost, sensor field coverage, anomaly level coefficient; In one specific embodiment, flight distance cost can account for 60%, sensor field of view coverage for 30%, and anomaly level coefficient (Level I × 1.5, Level II × 1.2, Level III × 1.0).

[0037] An improved A* algorithm is used to find the optimal path, with the heuristic function being: h(n)=α·min_distance+(1-α)·max_coverage Wherein, α is an empirical coefficient; in one specific embodiment, α = 0.7; Add dynamic constraints: Emergency abnormal path length ≤ 50% of endurance; pitch angle change between adjacent waypoints ≤ 45°.

[0038] In one embodiment of the present invention, the changes in the structural change region are classified into three levels according to their severity: Level I anomaly > Level II anomaly > Level III anomaly. The priority ranking rule for the re-inspection routes is: Level I anomaly > Level II anomaly > Level III anomaly, and the flight time of the re-inspection routes shall not exceed 50% of the remaining range of the UAV.

[0039] In one embodiment of the present invention, the anomaly classification criteria include: Pixel difference values ​​of visible light images (Δ gray level > 15 and area > 0.2㎡); The surface curvature change rate of lidar point clouds (>0.01 mm / m²). Temperature gradient of infrared thermography (ΔT > 0.5℃).

[0040] The classification of changes in the structural change region includes the following: Level I anomaly (urgent) meets any of the following criteria: Δ gray level > 25 and area > 0.5m² Curvature change rate > 0.03 mm / m² Crack propagation rate >1mm / month Level II anomalies (concerns) meet any two of the following criteria: Δ gray level > 15 and area > 0.2m² Rate of curvature change > 0.01 mm / m² Temperature gradient > 0.8℃ Level III anomaly (normal): Only one threshold condition is met.

[0041] In one embodiment of the present invention, the rules for generating the sub-routes include: Centered on the abnormal region, extract the waypoints of the protected region, which include the core waypoint and the two adjacent detection units above and below, to form a minimum connected subgraph; Waypoints unrelated to the abnormal area are removed, ensuring that the length of the sub-route does not exceed 30% of the preset route.

[0042] like Figure 3 As shown, the present invention also provides a UAV bridge inspection device for implementing any of the methods described above, comprising: Multi-sensor data acquisition module: used to collect multimodal data of the bridge according to a preset route and transmit it to the cloud in real time. The multimodal data includes real-time images. Cloud-based AI comparison and analysis module: used to perform pixel-level difference calculation and feature-level matching between the real-time image and the historical benchmark image, identify structural change areas as abnormal areas, and classify the changes in the structural change areas to obtain anomaly detection results; Dynamic route planning module: Based on the anomaly detection results, it extracts sub-routes containing anomaly areas and protection areas from the preset routes, and generates a re-inspection task queue according to the hierarchy to obtain the re-inspection path; Inspection report generation module: Used to integrate the anomaly detection results from multiple periods and generate a visual report and / or a structured data report that includes the location of the anomaly, its evolution trend, and maintenance suggestions.

[0043] like Figure 4 As shown, the multi-sensor data acquisition module includes a visible light camera, a lidar, an infrared thermal imager, and an environmental sensor. The visible light camera is used to acquire visible light images, which are the real-time images. The lidar is used to acquire lidar point clouds, and the environmental sensor is used to acquire infrared thermal images. The visible light images, the lidar point clouds, and the infrared thermal images constitute the multimodal data.

[0044] In a more specific embodiment, the hardware of the multi-sensor data acquisition module is as follows: Drone platform: Industrial-grade drone (endurance ≥ 60 minutes, wind resistance ≥ level 5). Multi-sensor data acquisition module includes: High-definition visible light camera: resolution ≥ 48 million pixels, supports automatic zoom (focal length 5-100mm).

[0045] LiDAR: Measurement accuracy ≤5mm, point cloud density ≥50 points / cm².

[0046] Infrared thermal imager: Temperature resolution ≤0.1℃, temperature measurement range -20℃ to 150℃.

[0047] Environmental sensors: collect data on temperature, humidity, wind speed and direction, and light intensity in real time.

[0048] The hardware components of the multi-sensor data acquisition module perform a full-area scan of the bridge according to a preset initial route, collect multimodal data (images, point clouds, thermal images, environmental parameters), and transmit them to the cloud in real time via a 5G / 4G link; it also supports an emergency trigger mode: when the cloud detects an emergency anomaly (such as a sudden expansion of cracks), it automatically adjusts the drone's route and performs encrypted data collection on the target area.

[0049] The method and apparatus of this invention can combine AI algorithms to achieve feature-level association of historical data. This feature-level association of historical data can be achieved through a spatiotemporal feature database, specifically including: (1) Multi-period feature encoding: The 1024-dimensional texture features and 512-dimensional geometric features extracted from each period of detection data are stored as a time-series feature sequence according to the detection timestamp; (2) Dynamic weight matching: An attention mechanism is used to calculate the association weights between real-time features and historical features. αt = softmax(Wa·tanh(Wq·Q + Wk·Kt)) Where Q represents the real-time feature, K_t represents the historical feature at period t, and Wa / Wq / Wk are trainable parameters. (3) Abnormal evolution modeling: The trend of feature change is predicted by LSTM network. When the real-time feature deviates from the predicted value by more than the threshold (cosine similarity <0.7), an abnormal alarm is triggered.

[0050] In one embodiment of the present invention, the cloud-based AI comparison analysis module includes an image preprocessing unit, a feature extraction unit, and a comparison analysis unit, which are described below: Image preprocessing unit: performs noise reduction (median filtering), geometric correction (homography transformation based on bridge BIM model), multispectral fusion (RGB and infrared band overlay) on visible light images, and unifies the coordinate system (WGS84 geographic coordinate system) and scale benchmark (1 pixel = 0.1mm).

[0051] Feature extraction unit: Employs an improved ResNet-50 convolutional neural network, including: Texture feature branch: captures two-dimensional surface features such as crack direction and rust distribution.

[0052] Geometric feature branch: Analyze the three-dimensional structural features of lidar point clouds, such as surface curvature and deformation displacement.

[0053] Output feature vector dimensions: 1024 dimensions (texture) + 512 dimensions (geometry).

[0054] Comparison Analysis Unit: Temporal Difference Method: Calculates the pixel-level difference (Δ gray level, Δ temperature) between the real-time image and the historical baseline image, and marks suspicious areas where the difference exceeds the threshold (e.g., Δ gray level > 15 and area > 0.2㎡).

[0055] Feature matching method: Feature points of images at different times are extracted using the SIFT algorithm, and the correspondence is established using the FLANN matcher. The displacement of feature points (unit: mm) and matching similarity (threshold ≥ 0.7) are calculated.

[0056] Anomaly grading model: Inputs: pixel difference value, feature matching degree, lidar deformation (Δd), infrared temperature gradient (ΔT).

[0057] Output: Abnormality level (Level I - Urgent: Requires handling within 24 hours; Level II - Attention: Requires re-inspection within 15 days; Level III - Routine: Included in the next inspection).

[0058] In one embodiment of the present invention, the feature extraction unit of the cloud AI comparative analysis module adopts an improved ResNet convolutional neural network. The input of the convolutional neural network is the fused multimodal data, and the output is a combination of texture feature vector and geometric feature vector.

[0059] In one specific embodiment, the improved ResNet network includes: (1) Two-branch input layer: Branch 1: Visible light image input with a standard ResNet-50 7×7 convolutional layer Branch 2: The LiDAR point cloud is voxelized and then input into a 3D convolutional layer. (2) Feature fusion layer: A cross-attention module is introduced after stage 3, and the calculation formula is: Attention(Q,K,V)=softmax((W_qQ)^T(W_kK) / √d)V Where Q represents image features, K represents point cloud features, and d=512 represents the feature dimension. Output layer: Parallel 1×1 convolutional layers output 1024-dimensional texture features and 512-dimensional geometric features respectively.

[0060] In one embodiment of the present invention, the multi-sensor data acquisition module supports an emergency encrypted acquisition mode. When an anomaly level is detected to be upgraded, the sensor parameters are automatically adjusted to perform a high-precision re-inspection of the target area.

[0061] In one specific embodiment, the data input and constraints of the dynamic route planning module are as follows: Input data: AI comparative analysis results (abnormal location, level), drone real-time status (battery level, sensor status), environmental data (wind speed ≤10m / s, precipitation probability <30%), bridge BIM model (including no-fly zones and height-restricted areas).

[0062] Constraints: Flight distance ≤ 80% of the UAV’s maximum range, flight time for a single sub-route ≤ 20 minutes, waypoint spacing ≥ 1 meter.

[0063] In one specific embodiment, the route generation strategy includes: Initial route generation: Based on the bridge BIM model, the main tower, cables, and bridge deck beams are divided into several detection units (e.g., each 5-meter section of the main tower is a unit), and a zigzag grid route is generated to ensure that each unit covers three perspectives: upward, side, and downward.

[0064] Waypoint parameters are predefined: flight altitude (5-10 meters above the structure surface), shooting angle (±45° tilt), and sensor configuration (e.g., enabling high-density LiDAR scanning mode when detecting cables).

[0065] Dynamic splitting and recombining: Sub-route extraction: Centered on the abnormal area, extract the core waypoint and the protected waypoints of the two adjacent units above and below to form the minimum connected subgraph.

[0066] Priority sorting: Level I abnormal sub-route has the highest priority, triggering the UAV to immediately return to base for re-inspection; Level II and Level III abnormalities are inserted into the inspection task queue according to time windows (e.g., Level II within 3 days, Level III within 7 days).

[0067] Path optimization: The A* algorithm is used to plan the execution order of sub-routes, with the objective function being "shortest flight distance + most complete view coverage", while avoiding no-fly zones.

[0068] In one embodiment of the present invention, the test report generation module includes: Data integration dimensions: Spatial dimensions: abnormal locations (latitude and longitude, bridge mileage markers), 3D model coordinates.

[0069] Time dimension: the time of first discovery, and the evolution data of each detection period (such as the crack length and width change curves).

[0070] Attribute dimensions: anomaly type (cracks, rust, deformation), severity, and repair recommendations (such as crack repair process and rust removal grade).

[0071] In one specific embodiment, the report can be in the form of a visual report: Bridge structural health heat map: different colors are used to mark the health status of each area (green - normal, yellow - attention, red - emergency).

[0072] 3D labeled model of abnormal area: Dynamic labels are superimposed on the BIM model to show the range and evolution trend of the abnormality (as shown in Figure 4).

[0073] In one specific embodiment, the report format can be a structured data report: Exception log: in Excel spreadsheet format, containing fields such as location, type, level, and processing status.

[0074] Detection index statistics table: summarizes the statistical values ​​of the number, distribution frequency, and evolution rate of each type of abnormality.

[0075] In one embodiment of the present invention, the detection report generation module supports three-dimensional model annotation function, which can dynamically display the spatial location, evolution trajectory and thermal distribution of the health status of the abnormal area on the bridge architectural information model.

[0076] This invention utilizes drones equipped with high-definition cameras, LiDAR, and other sensors to acquire multi-dimensional data on bridges. It then employs a cloud-based AI model to perform feature-level comparisons between real-time images and historical baseline images, intelligently identifying areas of structural change and dynamically generating re-inspection routes. The system achieves automated detection, evolution tracking, and precise re-inspection of bridge anomalies, significantly improving detection efficiency and accuracy. It is suitable for high-frequency monitoring of the structural health of various bridge types.

[0077] Based on comparative testing on 20 bridges (test sample size = 1,200 detection units), this system performed well in the following indicators: Crack detection false negative rate: 3.2% (18.7% with traditional methods) False alarm rate for rusted areas: 2.8% (15.3% for traditional methods) Deformation detection accuracy: 0.8mm (3.5mm for traditional methods).

[0078] The following is a specific embodiment of the detection process for crack evolution in the main tower of a cable-stayed bridge: 1. Initial data acquisition and benchmark library construction (time T0) (1) Flight parameters of the UAV: Flight path type: Full-area grid flight path of the main tower, covering elevation 0-50 meters, with a total of 10 detection units (each unit is 5 meters).

[0079] Flight altitude: 8 meters (from the surface of the main tower), flight speed: 3 m / s, shooting interval: 1 meter (i.e., 1 image is captured per meter).

[0080] (2) Data collection results: Visible light images: 500 images per main tower, resolution 8192×6144, including upward (top of the tower), side (side of the main tower), and downward (bottom of the tower) views.

[0081] LiDAR point cloud: density 100 points / cm², generating a 3D point cloud model of the main tower (accuracy ≤2mm).

[0082] (3) Cloud processing: The initial data is stored in the historical benchmark database, and benchmark feature templates for each detection unit are established (such as the concrete surface texture feature vector of unit 30-35 meters).

[0083] Construct a geographic reference model of the main tower and associate the detection unit with the BIM model coordinates (e.g., unit 30-35 meters corresponds to section ID-TA06 in the BIM model).

[0084] 2. Periodic inspection in period N (at time TN, N≥1) Historical data correlation analysis process: (1) Feature backtracking: retrieve the feature vectors of the detection unit for the most recent 6 periods (V_{t-1} to V_{t-6}) from the benchmark library; (2) Trend prediction: The LSTM network outputs the predicted feature V'_t, and the residual between the predicted feature V_t and the measured feature V_t is calculated: δ = ||V_t - V'_t||_2 > 0.25 (the threshold is determined by training with 500 sets of samples). (3) Relationship visualization: Use color bands to display the trajectory of feature evolution in the 3D report, for example, use red to indicate the region where δ exceeds the limit; a. Data Acquisition: The UAV flies along the same initial route as T0 and transmits the Nth phase image to the cloud in real time.

[0085] b. AI Comparative Analysis Steps: Preprocessing: Geometric correction is performed on the Nth period image to align it with the T0 reference image, with an error ≤ 0.5 pixels.

[0086] Feature extraction: The cosine similarity between the texture feature vector Vn of the ResNet model output unit (30-35 meters) and the baseline vector V0 is 0.68 (threshold 0.8), indicating the presence of texture anomalies.

[0087] Multimodal verification: The lidar point cloud shows that the surface curvature change rate in this area is 0.02 mm / m² (normal threshold ≤ 0.01 mm / m²), indicating structural deformation.

[0088] The infrared thermal imager detected a temperature gradient ΔT = 0.8℃ (normal ≤ 0.5℃), which may indicate abnormal heat conduction caused by a crack.

[0089] Anomaly Classification: Overall assessment classifies it as Level II anomaly (crack propagation requires attention), marked location: main tower elevation 32-34 meters, area 0.6㎡.

[0090] 3. Dynamic route generation and review execution (1) Route splitting logic: The original route contains 50 waypoints (0-50 meters, 1 waypoint per meter), and the anomaly is located at 32-34 meters, belonging to the 30-35 meter unit.

[0091] Sub-route extraction: Core waypoints 32-34 meters (3 points), protection waypoints 28-30 meters and 35-37 meters (2 points each), for a total of 7 waypoints, and the remaining 43 irrelevant waypoints are removed.

[0092] Sub-flight length: shortened from 50 meters to 9 meters, flight time reduced from 17 minutes to 3 minutes (flight speed 3m / s).

[0093] (2) Re-inspection parameters: Camera mode: Telephoto mode (focal length 80mm), resolution increased to 12000×8000 pixels, achieving a detection accuracy of 0.05mm / pixel.

[0094] LiDAR mode: High-density scanning (point cloud density 200 points / cm²), scanning interval 0.5 meters.

[0095] Drone execution: After receiving the sub-route instruction, the drone returns from its current location to the main tower area and completes the re-inspection according to the optimized 7 waypoints, which takes 5 minutes (including route switching time).

[0096] 4. Test report generation and historical data update (1) Confirmation of re-inspection results: Visible light images show that the crack length expanded from 2.3 meters at T0 to 3.1 meters at TN, and the width increased from 0.1 mm to 0.25 mm.

[0097] The lidar measurement showed a crack depth of 2.8cm (exceeding the thickness of the concrete protective layer by 2cm), which was determined to be a structural stress crack and upgraded to Level I anomaly.

[0098] (2) Report output: like Figure 5 As shown, the visualization report of this invention marks the location of cracks in the three-dimensional model of the main tower, for example, dynamically displaying the expansion trajectory from T0 to TN using a red curve.

[0099] Maintenance recommendations: Immediately perform pressure grouting on the cracks, and conduct a special inspection once a week for the next 3 months.

[0100] (3) Benchmark database update: Store the TN period data into the historical database as the benchmark for the next round of comparison, and mark the area as a "key monitoring unit".

[0101] This invention has the following advantages: I. Efficiency Improvement Reduced labor costs: AI-automated analysis reduces manual map interpretation time by 70%, shortening the inspection cycle for a single bridge from 3 days to 1 day.

[0102] Flight efficiency optimization: Dynamic flight routes reduce invalid flight distance by 60%-80%, and the number of bridges inspected by drones per day increases from 2 to 5.

[0103] II. Precision Improvement Anomaly detection capability: Multimodal data fusion can identify cracks as small as 0.1 mm and temperature anomalies as small as 0.5 °C, reducing the false negative rate from 20% to <5%.

[0104] Evolution tracking accuracy: Feature-level comparative analysis achieves anomaly location error ≤2mm and deformation measurement accuracy ≤1mm, meeting the millimeter-level accuracy requirements for bridge structural health monitoring.

[0105] III. Full-cycle management capability Historical data is traceable: an anomaly log for bridges is established, supporting the comparison of detection data at any time period, providing full-cycle data support for maintenance decisions.

[0106] Emergency response capability: For Level I anomalies, an immediate re-inspection command is generated, with the time interval from detection to response ≤30 minutes, meeting the needs of emergency disease treatment.

[0107] IV. Project Applicability Bridge type compatibility: Supports various structural forms such as beam bridges, arch bridges, and cable-stayed bridges, and can be quickly deployed by adjusting the BIM model and inspection parameters.

[0108] Drone compatibility: Compatible with mainstream industrial drones such as DJI Matrice 350 RTK and Parrot Anafi AI, with highly expandable sensor interfaces.

[0109] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for bridge inspection using unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1: Collect multimodal data of the bridge according to a preset route and transmit it to the cloud in real time. The multimodal data includes real-time images of the bridge. S2: Perform pixel-level difference calculation and feature-level matching between the real-time image and the historical reference image, identify the structural change region as an abnormal region, and classify the changes in the structural change region to obtain the anomaly detection result; S3: Based on the anomaly detection results, extract sub-routes containing anomaly areas and protection areas from the preset route, and generate a re-inspection task queue to obtain the re-inspection path; S4: Integrate the anomaly detection results from multiple phases to generate a visual report and / or structured data report that includes the anomaly location, evolution trend, and maintenance recommendations.

2. The UAV bridge inspection method as described in claim 1, characterized in that, Calculating the pixel-level difference between the real-time image and the historical reference image includes: S21: Perform grayscale normalization processing on the real-time image and the historical reference image to convert them to the same illumination reference; S22: Calculate the grayscale difference value ΔG=|Gt-G0| for each pixel, where Gt is the grayscale value of the current pixel and G0 is the reference value; S23: Perform morphological closing operations on the difference matrix to eliminate noise and extract continuous difference regions; S24: Set dual threshold judgment: when ΔG>15 and the area of ​​the continuous region>0.2m², it is marked as a valid difference region, and pixel-level difference calculation results are obtained.

3. The UAV bridge inspection method as described in claim 2, characterized in that, Also includes: The re-inspection path is optimized using a graph theory algorithm.

4. The UAV bridge inspection method as described in claim 3, characterized in that, Optimizing the re-inspection path includes: The abnormal region and the protected region are modeled as a weighted complete graph, and the node weights include: Flight distance cost, sensor field coverage, anomaly level coefficient; An improved A* algorithm is used to find the optimal path, with the heuristic function being: h(n)=α·min_distance+(1-α)·max_coverage Where α is an empirical coefficient; Add dynamic constraints: Emergency abnormal path length ≤ 50% of endurance; pitch angle change between adjacent waypoints ≤ 45°.

5. The UAV bridge inspection method as described in claim 4, characterized in that, The changes in the structural alteration areas are classified into three levels according to severity: Level I anomaly > Level II anomaly > Level III anomaly. The priority ranking rule for the re-inspection routes is: Level I anomaly > Level II anomaly > Level III anomaly, and the flight time of the re-inspection routes shall not exceed 50% of the remaining range of the UAV.

6. The UAV bridge inspection method as described in claim 1, characterized in that, The rules for generating the sub-routes include: Centered on the abnormal region, extract the waypoints of the protected region, which include the core waypoint and the two adjacent detection units above and below, to form a minimum connected subgraph; Waypoints unrelated to the abnormal area are removed, ensuring that the length of the sub-route does not exceed 30% of the preset route.

7. A bridge inspection device for unmanned aerial vehicles (UAVs), characterized in that, For implementing the method as described in any one of claims 1-6, comprising: Multi-sensor data acquisition module: used to collect multimodal data of the bridge according to a preset route and transmit it to the cloud in real time. The multimodal data includes real-time images. Cloud-based AI comparison and analysis module: used to perform pixel-level difference calculation and feature-level matching between the real-time image and the historical benchmark image, identify structural change areas as abnormal areas, and classify the changes in the structural change areas to obtain anomaly detection results; Dynamic route planning module: Based on the anomaly detection results, it extracts sub-routes containing anomaly areas and protection areas from the preset routes, and generates a re-inspection task queue according to the hierarchy to obtain the re-inspection path; Inspection report generation module: Used to integrate the anomaly detection results from multiple periods and generate a visual report and / or a structured data report that includes the location of the anomaly, its evolution trend, and maintenance suggestions.

8. The UAV bridge inspection device as described in claim 7, characterized in that, The multi-sensor data acquisition module includes a visible light camera, a lidar, an infrared thermal imager, and an environmental sensor. The visible light camera is used to acquire visible light images, which are the real-time images. The lidar is used to acquire lidar point clouds, and the environmental sensor is used to acquire infrared thermal images. The visible light images, the lidar point clouds, and the infrared thermal images constitute the multimodal data.

9. The UAV bridge inspection device as described in claim 8, characterized in that, The feature extraction unit of the cloud-based AI comparative analysis module uses an improved ResNet convolutional neural network. The input of the convolutional neural network is the fused multimodal data, and the output is a combination of texture feature vectors and geometric feature vectors.

10. The UAV bridge inspection device as described in claim 9, characterized in that, The multi-sensor data acquisition module supports an emergency encrypted acquisition mode. When an anomaly level is detected to be upgraded, the sensor parameters are automatically adjusted to perform a high-precision re-inspection of the target area. The detection report generation module supports 3D model annotation, which can dynamically display the spatial location, evolution trajectory, and thermal distribution of the health status of the abnormal area on the bridge architectural information model.

Images