Structural crack intelligent quantitative diagnosis method integrating unmanned aerial vehicle inspection and deep learning

Abstract

The invention belongs to the technical field of structural health monitoring and artificial intelligence crossing, and particularly relates to a structural crack intelligent quantitative diagnosis method fusing unmanned aerial vehicle inspection and deep learning, and the method comprises the steps: carrying a high-resolution camera and an RTK module through an unmanned aerial vehicle, and automatically collecting a structural surface image with geographic coordinates; pixel-level crack segmentation is carried out in combination with an improved U-Net + + network, and sub-millimeter-level quantization of crack length, trend and width (error is less than or equal to + / -0.05 mm) is realized based on accurate pixel-physical size mapping by using a skeleton extraction and distance transformation algorithm; furthermore, industry specifications and historical data are connected, risk level assessment and maintenance suggestions are automatically generated, and a visual report is output. The method is suitable for various infrastructures such as bridges, tunnels, dams, buildings and the like, has the advantages of high efficiency, safety, high precision, strong robustness and the like, and can support intelligent operation and maintenance and digital twinning application.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of structural health monitoring and artificial intelligence, and in particular relates to an intelligent quantitative diagnosis method for structural cracks that integrates UAV inspection and deep learning. It is applicable to the automatic identification, geometric parameter quantification and safety status assessment of surface cracks in large infrastructure such as bridges, tunnels, dams and high-rise buildings. Background Technology

[0002] With the continuous expansion of my country's infrastructure construction scale and the sustained increase in service life, the safety operation and maintenance of major engineering structures such as bridges, tunnels, dams, and high-rise buildings are becoming increasingly prominent issues. Surface cracks, as one of the most common and representative forms of damage, directly affect the load-bearing capacity and service life of structures due to their location, length, width, and development trend. Therefore, timely, accurate, and efficient detection and quantitative assessment of cracks are crucial for ensuring the safe operation of infrastructure.

[0003] Traditional crack detection mainly relies on manual inspection, which has significant drawbacks such as low efficiency, high subjectivity, high risks associated with working at heights or in complex environments, and difficulty in covering large areas. In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, automated inspection methods based on UAVs equipped with high-definition cameras have gradually become a new means of structural health monitoring. UAVs can flexibly approach areas that are difficult for humans to reach, enabling rapid, non-contact image acquisition, significantly improving inspection efficiency and safety.

[0004] However, relying solely on raw images acquired by UAVs still requires extensive manual interpretation, which is insufficient to meet the needs of intelligent operation and maintenance of large-scale infrastructure. Meanwhile, deep learning technologies, particularly convolutional neural networks (CNNs) and semantic segmentation models (such as U-Net and DeepLab), have made breakthroughs in image recognition and target detection, and have been widely applied to automatic crack identification tasks. However, existing deep learning-based crack identification methods mostly focus on image processing under laboratory conditions, and generally suffer from the following problems: (1) insufficient robustness to complex lighting, background interference, and image blurring in actual engineering scenarios; (2) lack of precise quantification capabilities for crack geometric parameters (especially crack width); and (3) lack of deep integration with front-end data acquisition systems (such as UAV platforms), making it difficult to form an integrated closed-loop intelligent diagnostic process of "perception-recognition-quantification-diagnosis." Therefore, there is an urgent need for an intelligent diagnostic method that organically integrates the efficient inspection capabilities of UAVs with the high-precision analysis capabilities of deep learning. Summary of the Invention

[0005] (a) Technical problems to be solved The purpose of this invention is to overcome the problems of low efficiency of manual inspection, reliance on manual interpretation of UAV images, weak generalization ability of deep learning models, and insufficient accuracy of crack quantization in the prior art, and to provide an end-to-end intelligent diagnostic method that realizes an integrated process of "automatic inspection - highly robust identification - pixel-level quantization - structural safety assessment".

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent quantitative diagnosis of structural cracks that integrates UAV inspection and deep learning, comprising the following steps: S1. Based on the 3D model or BIM information of the structure to be inspected, plan the autonomous flight path of the UAV, control the UAV equipped with a high-resolution camera and RTK positioning module to perform the inspection task, and acquire high-definition image sequences with geographic coordinates. S2. Perform distortion correction, illumination equalization, and multi-scale enhancement processing on the image sequence to generate a preprocessed image; S3. Input the preprocessed image into a pre-trained improved U-Net++ deep learning model and output a crack pixel-level semantic segmentation mask; the improved U-Net++ model includes nested skip connections, channel attention modules, and multi-scale feature fusion units; S4. Calculate the ground sampling distance (GSD) based on the UAV flight altitude, camera focal length and sensor parameters, convert the crack mask into actual physical size, and quantify the length, average width, maximum width and direction of each crack through skeleton extraction and distance transformation algorithm; S5. Compare the quantitative parameters with the preset structural safety assessment standard thresholds, and combine the crack location and historical development data to generate risk levels and maintenance recommendations; S6. Automatically generate a visual diagnostic report and upload it to the structural health monitoring cloud platform.

[0007] Preferably, in step S1, the UAV flight path is dynamically adjusted according to the structure type: for bridges, a straight path parallel to the web or flange is used; for tunnels, a spiral or reciprocating path along the central axis is used; and for high-rise buildings, a segmented circular path is used.

[0008] Preferably, the multi-scale enhancement processing in step S2 includes: constructing a Gaussian pyramid on the original image, performing contrast-limited adaptive histogram equalization (CLAHE) at different scales, and then fusing the images using a Laplacian pyramid to generate the enhanced image. The loss function of the improved U-Net++ model adopts a combined loss. L=α⋅Dice Loss+β⋅Focal Loss+γ⋅Boundary-aware Loss Where α, β, and γ are weighting coefficients, and Boundary-aware Loss is used to enhance the gradient response of pixels at the crack edge.

[0009] Preferably, in step S4, when the flight altitude is greater than 30 meters, subpixel interpolation and edge fitting algorithms are used to perform super-resolution optimization of the crack width so that the quantization error does not exceed ±0.05 mm.

[0010] Preferably, the structural safety assessment standard in step S5 includes at least one of the following: the "Technical Condition Assessment Standard for Highway Bridges" (JTG / T H21), the "Durability Assessment Standard for Concrete Structures", or the "Railway Tunnel Lining Condition Assessment Standard".

[0011] Preferably, the method further includes step S7: aligning the current diagnostic results with historical inspection data in time and space, calculating the crack development rate, and predicting the expansion trend over the next 6–12 months.

[0012] Preferably, the diagnostic report includes: a spatial distribution map of cracks on the structure, a table of quantitative parameters for each crack, a heat map of risk levels, a ranking of maintenance priorities, and a link to a 3D visualization.

[0013] Preferably, the UAV is equipped with an active supplementary lighting device in low-light environments, and the supplementary lighting intensity is adjusted in real time according to the ambient light sensor to ensure that the image signal-to-noise ratio is ≥20 dB.

[0014] Preferably, the structural health monitoring cloud platform supports API interfaces, which can push diagnostic results to the City Information Modeling (CIM) system or digital twin platform to achieve full life cycle management of infrastructure.

[0015] (III) Beneficial Effects Compared with existing technologies, this invention provides an intelligent quantitative diagnosis method for structural cracks that integrates UAV inspection and deep learning, and has the following beneficial effects: This invention integrates UAV autonomous inspection with an improved deep learning model to achieve efficient and high-precision intelligent diagnosis of structural cracks. Compared with traditional manual inspection, it significantly improves inspection efficiency and operational safety. By introducing an attention mechanism and a U-Net++ network that fuses multi-scale features, it effectively overcomes lighting, noise, and background interference in complex engineering scenarios, achieving high crack recognition accuracy and robustness. By combining RTK positioning and camera parameters to establish a pixel-physical size mapping model, and using skeleton extraction and distance transformation algorithms, it achieves sub-millimeter-level quantization of crack length, direction, and width (error ≤ ±0.05 mm). Furthermore, by connecting with industry standards and historical data, it automatically generates risk level assessments and maintenance suggestions, and outputs visualized diagnostic reports. It supports access to cloud platforms or digital twin systems, providing reliable and closed-loop technical support for intelligent infrastructure operation and maintenance. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of the overall process of this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Specific implementation examples are given below.

[0019] Example 1: Intelligent Diagnosis of Web Cracks in the Main Girder of a Cable-Stayed Bridge Scenario Description: A cable-stayed bridge spanning a river has been in service for 15 years, and suspected longitudinal cracks have been found in the web of its main girder. Traditional manual inspection requires scaffolding, which is costly and time-consuming.

[0020] Implementation steps: Import the BIM model into the UAV ground station software, set a parallel flight path along the web of the main beam, a flight height of 8m, a speed of 2m / s, and a camera vertically downward. The drone is equipped with a Sony RX1R II camera (42 megapixels, 35 mm focal length), with RTK positioning accuracy of ±1 cm, and captures 327 images. The image was corrected for lens distortion using OpenCV, and the CLAHE algorithm was used to enhance local contrast. Input the data into the trained improved U-Net++ model (the training set contains 5,000 labeled crack images, covering rain, fog, shadows, and rust interference), and output a crack probability map; After binarizing the probability graph, morphological closing operations are performed to extract connected components and obtain three main cracks. According to the formula: Where H=8m is the flight altitude, d=5.9μm is the pixel size, and f=35mm is the focal length, the ground sampling distance (GSD) is 1.35 mm / pixel; The length of crack 1 was calculated to be 2.1 m, the average width was 0.28 mm, and the maximum width was 0.41 mm using the skeleton method. According to the JTG / T H21 standard, a longitudinal crack width of <0.3 mm in the web is considered "minor". However, the maximum width exceeds the limit, and the location is in the shear zone, so it is judged as "medium risk". It is recommended to re-inspect within 3 months. The system automatically generates a PDF report containing an overlay map of crack locations, a quantitative table, and maintenance recommendations.

[0021] Example 2: Diagnosis of Crack Clusters in Mountain Tunnel Lining Scenario description: A highway tunnel has been in operation for 10 years, and multiple network cracks have appeared in the arch and sidewalls, which are difficult to fully cover by manual inspection.

[0022] Implementation steps: The DJI Matrice 300 RTK drone, equipped with a ring light, automatically flew along the central axis of the tunnel at a height of 3 m and a speed of 1.5 m / s. Due to uneven lighting inside the tunnel, some areas of the captured images are too dark. The preprocessing stage employs Retinex+Gamma combined enhancement to significantly improve the visibility of cracks in dark areas; Deep learning models employ multi-scale input (original image + 2x downsampled image) to improve recall rate for small cracks; A total of 27 cracks were identified, with the densely populated area of ​​the arch forming a "cracking" pattern; GSD = 0.51 mm / pixel (H = 3 m), the maximum crack width is quantified to be 0.62 mm, located in the tension zone of the arch waist; According to the "Standard for Assessment of the Condition of Railway Tunnel Lining", if the width is >0.5 mm and it is located in the stress zone, it is judged as "high risk" and reinforcement design should be initiated immediately. The system generates a 3D heat map of crack distribution, which can be viewed in VR.

[0023] Example 3: Monitoring of Cracks in the Facade of Super High-Rise Buildings Scenario description: Vertical cracks have appeared in the concrete structure behind the curtain wall of a 300-meter-high office building, posing a danger to working at height.

[0024] Implementation steps: The drone uses a wind-resistant hexacopter platform, flies at an altitude of 50–120 m, and performs segmented inspections. Shoot at a 45° angle to avoid glare from glass reflections. Image stitching uses SfM (Structure of Motion) to reconstruct local 3D point clouds, aided by orthorectification; An edge-aware loss function is added to the crack segmentation model to improve the integrity of slender cracks; Three penetrating vertical cracks were identified, the longest being 8.7 m; GSD achieves a width quantization accuracy of ±0.04 mm at a height of 3.2 mm / pixel at 80 m, optimized through sub-pixel interpolation. Based on the building structure drawings, the crack is located in the core tube shear wall, with a width of 0.35 mm, and is classified as "medium risk". It is recommended to implant fiber optic grating sensors for long-term monitoring. The data is synchronized to the city's CIM platform and incorporated into the digital twin.

[0025] Example 4: Intelligent inspection of cracks on the surface of a concrete gravity dam (including near-water area) Scenario Description: A large-scale water conservancy project's concrete gravity dam has been in operation for 25 years. Suspected horizontal cracks have appeared in the near-water area of ​​the downstream dam face. Traditional inspection methods require drainage or underwater operations, which are costly and time-consuming.

[0026] Implementation steps: Drone selection and path planning: The DJI Matrice 350 RTK drone with IP54 protection was adopted, equipped with a Zenmuse P1 full-frame camera (45 megapixels, 35 mm fixed-focus lens) and a laser rangefinder module; based on the dam BIM model, the vertical segmented flight path along the dam surface was set in the ground station software, with the minimum flight altitude only 2.5 m above the water surface to avoid the splash zone; Image acquisition and environmental adaptation: The inspection was carried out in the early morning to reduce water surface reflection. The UAV used polarization filter and automatic exposure lock function to acquire a total of 412 high-definition images with RTK positioning accuracy of ±1 cm. Image preprocessing: To address the issues of water surface reflection and low contrast on wet surfaces, a joint enhancement algorithm of Retinex decomposition and adaptive Gamma correction is adopted, and an invalid reflective area is removed using a semantic segmentation mask for the water surface region. Crack identification: The improved U-Net++ model, which was fed into the training set containing interference samples such as "wet concrete", "algae attachment" and "water stains", successfully identified 5 horizontal cracks, one of which was located at the EL.85 elevation (near the normal water level). Geometric quantization: Based on H=3.2 m, f=35 mm, and pixel size d=4.39 µm, GSD=0.40 mm / pixel is calculated; the crack length is 6.8 m, the average width is 0.33 mm, and the maximum width is 0.47 mm obtained by the skeleton method. Safety diagnosis: According to the "Durability Assessment Standard for Hydraulic Concrete Structures" (SL / T 798-2020), cracks with a width >0.3 mm near the water level are classified as "moderate deterioration". Considering that it is located in the principal stress zone, it is judged as "medium to high risk". It is recommended to carry out epoxy grouting treatment before the flood season. Output: The system generates a distribution map of cracks on the dam surface (overlaid with elevation information), a quantitative table, and a maintenance priority list, and pushes them to the reservoir's digital twin platform for annual safety assessment.

[0027] Example 5: Nighttime Diagnosis of Cracks in the Side Walls of Subway Underground Stations Scenario Description: After 8 years of operation, a vertical crack appeared on the side wall of a two-level underground island platform station in a certain city. The inspection needs to be completed during the nighttime shutdown window (0:00–4:00) when the ambient light is extremely weak (<10 lux).

[0028] Implementation steps: Low-light inspection configuration: The system uses a DJI M300 RTK equipped with a Zenmuse L2 lidar and a visible light fusion payload, and is also equipped with a dimmable LED ring light (color temperature 5600K, brightness 0–2000 lm adjustable); the flight path is set parallel to the side wall of the platform, at a height of 2.8 m and a speed of 1.2 m / s. Adaptive fill light control: The ambient illuminance is monitored in real time by an airborne photosensor, and the fill light intensity is dynamically adjusted to ensure that the image signal-to-noise ratio is ≥22 dB; Image acquisition and stitching: The entire 220 m sidewall was scanned within 3 hours, acquiring 286 images; local 3D point clouds were generated using the SfM algorithm to assist in orthorectification to eliminate perspective distortion; Crack segmentation optimization: Due to the high noise in nighttime images, the model adopts multi-scale input (original image + 1.5× super-resolution reconstructed image) and enables the combined loss function described in claim 4 to effectively suppress false positives; Quantitative results: GSD=0.62 mm / pixel, 12 cracks were identified, the longest one is located above the shielding door, with a length of 3.4 m, an average width of 0.29 mm, and a maximum width of 0.51 mm (caused by localized peeling). Diagnosis and Response: Referring to the "Technical Specification for Safety Monitoring of Urban Rail Transit Structures", if the width is >0.5 mm and it is located in the equipment installation area, it is judged as "high risk". The system automatically generates an emergency work order and recommends local support and implantation of strain gauges within 72 hours. Data archiving: All data is encrypted and uploaded to the subway operation and maintenance cloud platform, which supports automatic alignment with the previous year's data. The calculated annual expansion rate of the crack is 0.08 mm / year, showing an accelerating trend.

[0029] Example 6: Intelligent Assessment of Weathering Cracks on the Surface of Ancient Brick and Stone Pagodas Scenario Description: The Ming Dynasty octagonal brick pagoda, a national key cultural relic protection unit, has developed a network of microcracks on its outer wall due to weathering. Traditional contact-based testing methods are prone to damaging the cultural relic, and a non-invasive, high-precision method is urgently needed.

[0030] Implementation steps: Cultural relic protection level inspection: Use a lightweight DJI Mini 4 Pro (takeoff weight <250 g, in compliance with cultural relic protection airspace restrictions), fly at an altitude of 5–15 m, maintain a minimum distance of 3 m from the tower to avoid airflow disturbance; set the camera to RAW format, ISO ≤200 to preserve details; Multi-angle image acquisition: The system spiraled upwards and flew around the tower in eight sectors, hovering and taking pictures at the eaves of each level, acquiring a total of 530 images, covering all eight facades; Complex texture processing: The tower surface has strong interfering textures such as brick joints, moss, and remnants of painted decorations. In the preprocessing stage, frequency domain filtering (wavelet denoising + homomorphic filtering) is used to separate high-frequency crack signals from low-frequency background. Small crack enhancement recognition: For microcracks with a width of <0.1 mm, the model enables the sub-pixel edge refinement module (corresponding to claim 5) and adds synthetic microcrack data enhancement during the training phase; Quantification and evaluation: At H=8 m, GSD=1.8 mm / pixel, and after subpixel optimization, cracks of 0.08 mm can be distinguished; a total of 47 effective cracks were identified, mainly concentrated in the top three layers of the tower, with an average width of 0.12–0.25 mm. Cultural heritage protection diagnostic logic: The system has built-in "Code for Protection and Repair of Ancient Building Timber / Brick and Stone Structures", which classifies cracks into "structural" and "weathering" types. In this case, the cracks are not penetrating and have no misalignment, so they are judged as "weathering micro-cracks" with a "low" risk level. It is recommended to only perform surface water-repellent treatment to avoid excessive intervention. Digital archiving: High-precision orthophotos and crack vector maps are generated and incorporated into the national digital archive of cultural relics as a benchmark for comparing health status over the next 5 years.

[0031] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for intelligent quantitative diagnosis of structural cracks integrating UAV inspection and deep learning, characterized in that, Includes the following steps: S1. Based on the 3D model or BIM information of the structure to be inspected, plan the autonomous flight path of the UAV, control the UAV equipped with a high-resolution camera and RTK positioning module to perform the inspection task, and acquire high-definition image sequences with geographic coordinates. S2. Perform distortion correction, illumination equalization, and multi-scale enhancement processing on the image sequence to generate a preprocessed image; S3. Input the preprocessed image into a pre-trained improved U-Net++ deep learning model and output a crack pixel-level semantic segmentation mask; the improved U-Net++ model includes nested skip connections, channel attention modules, and multi-scale feature fusion units; S4. Calculate the ground sampling distance (GSD) based on the UAV flight altitude, camera focal length and sensor parameters, convert the crack mask into actual physical size, and quantify the length, average width, maximum width and direction of each crack through skeleton extraction and distance transformation algorithm; S5. Compare the quantitative parameters with the preset structural safety assessment standard thresholds, and combine the crack location and historical development data to generate risk levels and maintenance recommendations; S6. Automatically generate a visual diagnostic report and upload it to the structural health monitoring cloud platform.

2. The method as described in claim 1, characterized in that, In step S1, the UAV flight path is dynamically adjusted according to the structure type: for bridges, a straight path parallel to the web or flange is used; for tunnels, a spiral or reciprocating path along the central axis is used; and for high-rise buildings, a segmented circular path is used.

3. The method as described in claim 1, characterized in that, The multi-scale enhancement process in step S2 includes: constructing a Gaussian pyramid for the original image, performing contrast-limited adaptive histogram equalization at different scales, and then fusing the images using a Laplacian pyramid to generate an enhanced image.

4. The method as described in claim 1, characterized in that, The loss function of the improved U-Net++ model uses a combined loss: L=α⋅Dice Loss+β⋅Focal Loss+γ⋅Boundary-aware Loss Where α, β, and γ are weighting coefficients, and Boundary-aware Loss is used to enhance the gradient response of pixels at the crack edge.

5. The method as described in claim 1, characterized in that, In step S4, when the flight altitude is greater than 30 meters, subpixel interpolation and edge fitting algorithms are used to perform super-resolution optimization of the crack width so that the quantization error does not exceed ±0.05 mm.

6. The method as described in claim 1, characterized in that, It also includes step S7: aligning the current diagnostic results with historical inspection data in time and space, calculating the crack development rate, and predicting the expansion trend over the next 6–12 months.

7. The method as described in claim 1, characterized in that, The diagnostic report includes: a spatial distribution map of cracks on the structure, a table of quantitative parameters for each crack, a heat map of risk levels, a ranking of maintenance priorities, and a link to a 3D visualization.

8. The method as described in claim 1, characterized in that, The drone is equipped with an active supplemental lighting device in low-light environments. The supplemental lighting intensity is adjusted in real time according to the ambient light sensor to ensure that the image signal-to-noise ratio is ≥20 dB.

9. The method as described in claim 1, characterized in that, The structural health monitoring cloud platform supports API interfaces, which can push diagnostic results to the city information model system or digital twin platform to achieve full life cycle management of infrastructure.

Images