A Bridge Crack Detection System and Method Based on Deep Learning and UAV Imagery

Abstract

This invention discloses a bridge crack detection system and method based on deep learning and UAV imagery, belonging to the application of computer vision and artificial intelligence technologies in the health monitoring of civil engineering structures. The method includes: UAVs acquiring bridge images (overlap rate ≥ 60%) along a preset flight path; image preprocessing in the cloud; initial screening of crack images using an improved MobileNetV3-Small model; segmentation and classification of cracks using a UNet++ model (four categories: transverse, longitudinal, etc.); calculation of crack dimensions using an improved DeepLabv3+ network combined with camera parameters; and finally, generation of annotated visual images and reports. The system includes sequentially connected modules for image acquisition, cloud processing, initial crack screening, segmentation and classification, width measurement, result generation, and an image user interface, employing a cascading strategy to optimize resources. This invention can improve the level of automation, accuracy, and efficiency of detection, reduce labor costs and safety risks, and address the shortcomings of traditional detection methods and existing technologies.

Description

Technical Field

[0001] This invention relates to the application of computer vision and artificial intelligence technologies in the health monitoring of civil engineering structures. Specifically, it relates to an automatic bridge crack width detection method and system that combines UAV image acquisition, deep learning classification and semantic segmentation algorithms, and integrates a graphical user interface. It is suitable for the automated identification, location and quantitative measurement of surface cracks in concrete structures such as bridge piers. Background Technology

[0002] With the large-scale construction and increasing service life of my country's transportation infrastructure, a large number of bridges are entering the aging stage, posing a serious challenge to their structural safety. Among them, concrete surface cracks are one of the important indicators reflecting structural deterioration, especially vertical or diagonal cracks in the piers, which may indicate a decline in load-bearing capacity or even a potential risk of collapse, requiring regular inspection and evaluation.

[0003] Existing bridge crack detection technologies have the following significant shortcomings:

[0004] (1) Traditional bridge crack detection mainly relies on manual inspection combined with handheld equipment for shooting, which has problems such as low efficiency, strong subjectivity, danger of high-altitude operation and difficulty in quantification.

[0005] (2) In the existing technology, some studies use traditional image processing methods (such as edge detection and threshold segmentation) to identify cracks, but they are sensitive to interference such as complex lighting, shadows, and stains, and have a high false detection rate. Although some deep learning-based methods have improved the recognition accuracy, they still lack a complete end-to-end process design in actual engineering deployment, especially the ability to integrate crack existence, location classification, and width measurement, which limits their application and promotion in engineering sites.

[0006] To address the aforementioned issues, there is an urgent need for a bridge crack detection system that integrates image acquisition, intelligent analysis, and precision measurement, which can ensure detection accuracy while improving engineering practicality and deployment efficiency. Summary of the Invention

[0007] The purpose of this invention is to provide a bridge crack detection method and system based on lightweight deep learning, which solves the problems of high cost, large computing power requirements, complex operation, and redundant accuracy. This aims to significantly improve the automation level, detection accuracy, and work efficiency of bridge crack detection, while reducing labor costs and safety risks.

[0008] To achieve the above objectives, this invention provides an intelligent method for detecting bridge crack width based on the fusion of UAV image acquisition and deep learning model, comprising the following steps:

[0009] S1. Obtain bridge section detection images through drone aerial photography;

[0010] S2. The detected images of the bridge sections are transmitted to the cloud server for preprocessing to obtain preprocessed images;

[0011] S3. Determine the presence of cracks in the preprocessed images. Images with cracks are included in the next analysis stage, while the rest are archived.

[0012] S4. Pixel-level semantic segmentation is achieved based on the UNet++ model. Geometric features of cracks are extracted based on morphological analysis and principal component analysis, including principal orientation angle, aspect ratio and number of branches. Crack types are automatically classified according to preset rules.

[0013] S5. Through the crack width precision measurement module, the maximum width, length and location coordinates of each crack are finally output;

[0014] S6. Overlay the above detection results back onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations; at the same time, automatically generate the result file.

[0015] Preferably, the image acquisition device in step S1 uses a drone equipped with a multi-sensor camera to fly around the bridge pier in a preset flight path to ensure that the image overlap rate is not less than 60% in order to improve the detection integrity. The drone performs multi-angle imaging around key parts such as the bridge pier in the preset flight path to ensure that the crack area is fully exposed and unobstructed.

[0016] Preferably, in step S2, the cloud server focuses on extracting images of the bridge pier area to reduce subsequent computational redundancy, while simultaneously performing image normalization processing, scaling them to a standard size, and converting them into an input format acceptable to the model.

[0017] Preferably, the crack classification model in step S3 is an improved MobileNetV3-Small model.

[0018] The improvements include:

[0019] S31. Introduce the Squeeze-and-Excitation (SE) attention mechanism;

[0020] S32. Perform transfer learning fine-tuning training on the Kaggle and self-built bridge crack datasets to improve feature representation ability and classification accuracy.

[0021] S33. Set the judgment threshold to 0.7. When the probability value is greater than or equal to the threshold, it is judged as a positive sample and enters the next analysis stage; otherwise, it is archived.

[0022] Preferably, in step S4, the UNet++ model is used to achieve pixel-level semantic segmentation, and the geometric features of the crack are extracted based on morphological analysis and principal component analysis. The features include the principal orientation angle, aspect ratio and number of branches.

[0023] Preferably, in step S5, the crack width precision measurement module employs an improved DeepLabv3+ semantic segmentation network, the improvement including:

[0024] S51 is equipped with the Xception backbone network and ASPP dilated convolution module to perform fine segmentation of crack regions and improve edge accuracy;

[0025] S52. Obtain the crack centerline using a skeleton extraction algorithm, and uniformly sample multiple points along the centerline;

[0026] Preferably, in step S6, the result visualization module overlays the above detection results back onto the original image, generates a visualization image with color-coded crack areas, yellow outlines, and width numerical annotations, and generates an exportable file.

[0027] This invention also provides a bridge crack detection system based on deep learning and UAV images, as detailed below:

[0028] It includes an image acquisition module, an image transmission and pan-tilt processing module, a crack initial screening and classification module, a crack area labeling and type classification module, a crack width precise measurement module, and a result visualization and file generation module. These modules are sequentially connected via data transmission links, with the specific connection relationships as follows:

[0029] The output of the image acquisition module is connected to the input of the image transmission and PTZ processing module, and is used to transmit the acquired bridge image data to the cloud server.

[0030] The output of the image transmission and gimbal processing module is connected to the input of the crack primary screening and classification module, and is used to transmit the preprocessed image data to the crack primary screening and classification module.

[0031] The output of the crack initial screening and classification module is connected to the input of the crack region labeling and type classification module, and is used to transmit the crack initial screening and classification results to the crack region labeling and type classification module.

[0032] The output of the crack region labeling and type classification module is connected to the input of the crack width precision measurement module, and is used to transmit the region labeling and type classification results to the crack width precision measurement module.

[0033] The output of the crack width precision measurement module is connected to the input of the result visualization and file generation module. It is used to overlay the detection results back onto the original image to generate a visualized image with color coding and width annotation, and automatically generate PDF reports and CSV tables, which are then transmitted to the result visualization and file generation module.

[0034] The functions of each module are as follows:

[0035] Image acquisition module: used to execute the above method step S1 to acquire surface image data of the bridge section;

[0036] Gimbal Image Processing Module: Used to execute the above method step S2, receive raw image data from the UAV, perform noise reduction, white balance correction, contrast enhancement and region cropping operations, focus on extracting the image of the bridge pier area and complete normalization processing;

[0037] Crack initial screening and classification module: used to execute the above method step S3, using a lightweight convolutional neural network MobileNetV3-Small to build a binary classification model to realize crack inference detection;

[0038] Crack region labeling and type classification module: It is used to execute the above method step S4, generate crack pixel-level masks using the UNet++ model, and extract the principal orientation angle, aspect ratio and number of branches based on principal component analysis to achieve automatic classification of transverse, longitudinal, oblique or mesh cracks;

[0039] Crack width precision measurement module: used to execute the above method step S5, adopts DeepLabv3+ semantic segmentation network, and combines camera intrinsic parameters, sensor size and shooting distance to convert pixel width into real physical width (unit: millimeter), and outputs the maximum width, average width, length and position coordinates of each crack;

[0040] Results visualization and document generation module: Overlays the detection results back onto the original image to generate a visualization image with color coding and width annotation, and automatically generates PDF reports and CSV tables;

[0041] Preferably, the image acquisition module uses a drone equipped with a multi-sensor camera to fly around important parts such as bridge piers along a preset flight path, ensuring that the image overlap rate is not less than 60% to improve the integrity of the detection.

[0042] Preferably, the crack initial screening and classification module uses an improved MobileNetV3-Small neural network to construct a binary classification model, determines whether there is a crack in the image and outputs a probability value of the "cracked" category. When the probability value is greater than or equal to a preset threshold of 0.7, it enters the subsequent analysis process.

[0043] Preferably, the crack region labeling and type classification module uses the UNet++ model to generate pixel-level masks, and uses principal component analysis to extract key information to achieve automatic classification of transverse, longitudinal, oblique, or mesh cracks.

[0044] Preferably, the crack width precision measurement module uses the DeepLabv3+ semantic segmentation network equipped with the Xception backbone network and the ASPP dilated convolution module to perform fine segmentation of the crack region, calculate the distance between intersection points, and combine the camera intrinsic parameters, sensor size and shooting distance to convert the pixel width into the real physical width (unit: millimeters), and output the maximum width, average width, length and position coordinates of each crack.

[0045] Preferably, the graphical user interface is developed based on PyQt5 + Python, supporting project management, image display, result viewing and one-click export functions, enabling staff to operate in one stop.

[0046] Therefore, the intelligent bridge crack width detection method and system based on deep learning using the above-mentioned structure has the following beneficial effects:

[0047] (1) The dual-stage AI processing architecture, with a two-level strategy of "MobileNetV3 initial screening + DeepLabv3 + fine testing", significantly reduces computational overhead and improves system response speed while ensuring detection accuracy;

[0048] (2) Integrate deep learning segmentation results with geometric feature analysis to achieve automated classification of crack types and assist in structural safety assessment;

[0049] (3) By combining camera parameters and spatial mapping models, a precise conversion from pixel width to millimeter-level real size can be achieved;

[0050] (4) Full-process closed-loop system integration: covering image acquisition → transmission → AI analysis → result output → human-computer interaction, forming a complete intelligent detection solution;

[0051] (5) Provides an intuitive operation interface to support engineers to easily browse, review and export results, thereby improving the system's usability and user experience.

[0052] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating a bridge crack detection system and method based on deep learning and UAV images according to the present invention.

[0054] Figure 2This is a block diagram of a bridge crack detection system and method based on deep learning and UAV images according to the present invention.

[0055] Figure 3 This is a visualization interface diagram of the system detection process of a bridge crack detection system and method based on deep learning and UAV images according to the present invention.

[0056] Figure 4 This is a schematic diagram illustrating the storage of the exported results of a bridge crack detection system and method based on deep learning and UAV images according to the present invention. Detailed Implementation

[0057] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0059] Example 1

[0060] like Figure 1 As shown, the present invention provides a bridge crack detection method based on deep learning and UAV images, comprising the following steps:

[0061] S1, Bridge Inspection Image Acquisition

[0062] The drones are used to capture images of the bridge deck, piers and other structural surfaces, and to collect image data of high piers and hard-to-reach areas, which are then transmitted to the cloud server in real time.

[0063] S2: Detection Image Preprocessing

[0064] The image is denoised, white balance corrected, and contrast enhanced. The image of the bridge pier area is extracted. At the same time, the image is normalized and scaled to a standard size to fit the model input.

[0065] S3: Crack Preliminary Screening Classification

[0066] The preprocessed images are fed into an improved MobileNetV3-Small binary classification model to determine whether cracks exist in the preprocessed images. Images with cracks are included in the next analysis stage, while the rest are archived.

[0067] S4: Crack Area Marking and Type Classification

[0068] The UNet++ model is used to achieve pixel-level semantic segmentation, generate binary mask images of cracks, mark the spatial distribution range of cracks, and automatically classify crack types according to preset rules.

[0069] S5: Precise measurement of crack width

[0070] An improved DeepLabv3+ neural network is used to perform fine segmentation of the crack region, improve edge accuracy, calculate the distance between two points as the local width, and then combine the camera intrinsic parameters (focal length, sensor size), shooting distance and scale calibration relationship to convert the pixel width into the real physical width (unit: mm), and finally output the maximum width, length and position coordinates of each crack.

[0071] S6: Results Visualization and Document Generation

[0072] The above detection results are superimposed onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations; at the same time, a result file is automatically generated.

[0073] Example 2

[0074] like Figure 2 As shown, a bridge crack detection system based on deep learning and UAV images includes a UAV image acquisition module, a gimbal image processing module, a crack initial screening and classification module, a crack area labeling and type classification module, a crack width precise measurement module, a result visualization and file generation module, and a graphical user interface module.

[0075] The image acquisition module automatically cruises and photographs the bridge structure surface using a drone to obtain high-resolution visible light images, and transmits the images along with GPS coordinates, shooting time information, and camera parameter information to a cloud server. The drone is equipped with a high-definition camera and performs multi-angle imaging around key parts such as bridge piers along a preset flight path to ensure that the cracked areas are fully exposed and unobstructed.

[0076] The gimbal image processing module extracts images of the bridge pier area based on the raw image data from the UAV, performs normalization processing, scales the images to a standard size, and converts them into an input format acceptable to the model.

[0077] The crack initial screening and classification module introduces the SE attention mechanism based on the improved MobileNetV3-Small neural network to determine whether cracks exist in the preprocessed image. The judgment threshold is set to 0.7. When the probability value is greater than or equal to the threshold, it is judged as a positive sample and enters the next module; otherwise, it is archived.

[0078] The crack region labeling and type classification module accurately labels the spatial distribution range of cracks based on the UNet++ model, extracts crack set features based on morphological analysis and principal component analysis, and automatically classifies crack types (transverse cracks / longitudinal cracks / oblique cracks / network cracks) according to preset rules.

[0079] The crack width precision measurement module performs regional segmentation of the crack area based on the improved DeepLabv3+ semantic segmentation network to improve edge accuracy. It obtains the crack centerline based on the skeleton extraction algorithm and samples multiple points. It calculates the distance between two points as the local width and converts the pixel width into the real distance by combining the camera's intra-camera parameters. It outputs the maximum width, length and position coordinates of each crack.

[0080] like Figure 3 , Figure 4 As shown, the result visualization and file generation module overlays the above detection results back onto the original image (with color-coded crack areas, yellow outlines, and width numerical annotations), and automatically generates a result file.

[0081] The graphical user interface (GUI) module is developed using a desktop application with a front-end based on PyQt5 and Python, providing an interactive operating environment. The interface includes an image display window and a detection information display area. It supports users to import images and start the detection process. It has a one-click export function, which can package and download the detection report. The system also supports multi-user permission management and operation log recording to ensure data security and traceability.

[0082] This invention employs a bridge crack detection system and method based on deep learning and UAV imagery, as described above. The method includes: acquiring image data of the bridge surface using a UAV; performing noise reduction, white balance correction, and contrast enhancement on the acquired images on a cloud server, focusing on extracting images of the pier area, while simultaneously performing image normalization and scaling them to a standard size to adapt to the input model; determining the presence of cracks in the preprocessed images, placing images with cracks into the next analysis stage, and archiving the rest; for images determined to have cracks, defining the spatial range of the cracks, extracting the geometric features of the cracks, and generating data including category labels, confidence levels, bounding boxes, and mask data; outputting the maximum width, length, and location coordinates of each crack in all cracked images; superimposing the detection results back onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations, while automatically generating a result file. The system includes data acquisition, preprocessing, detection, classification, data annotation, and file storage modules connected sequentially by signals. This invention achieves crack screening and classification, reduces resource waste, significantly improves the automation level, detection accuracy, and work efficiency of bridge crack detection, and reduces labor costs and safety risks.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting bridge cracks based on deep learning and UAV images, characterized in that, Includes the following steps: S1. Use a drone equipped with a multi-sensor camera to fly along a preset route to collect images of the bridge surface, ensuring that the image overlap rate is not less than 60%; S2. The image is transmitted to a cloud server for preprocessing, including noise reduction, white balance correction, contrast enhancement and pier area extraction, and the image is normalized to adapt to the model input. S3. An improved MobileNetV3-Small classification model is used to determine the presence of cracks in the preprocessed image. When the output probability value is greater than or equal to a set threshold, the image is determined to contain cracks and proceeds to subsequent processing; otherwise, it is archived and stored. S4. Based on the UNet++ model, pixel-level semantic segmentation is performed on images containing cracks to generate a binary mask of cracks. Then, morphological operations and principal component analysis are combined to extract the geometric features of cracks, including the main orientation angle, aspect ratio and number of branches. Crack types are automatically classified according to preset rules. S5. The DeepLabv3+ network is used to perform fine segmentation on the image containing cracks, extract the skeleton center line and sample and measure the local width. Combined with camera parameters and shooting distance, the pixel width is converted into the actual physical width, and the maximum width, length and position coordinates of the crack are output. S6. Overlay the detection results onto the original image to generate a visualization of the crack area with color-coded markings, yellow outlines, and width values. Automatically generate a PDF file containing statistical reports and a CSV table of structured data.

2. The bridge crack detection method based on deep learning and UAV images according to claim 1, characterized in that: In step S3, the MobileNetV3-Small model is fine-tuned through transfer learning on the Kaggle public dataset and the self-built bridge crack dataset, and the SE attention mechanism is introduced to enhance the feature representation capability. The judgment threshold is set to 0.

7. Images with a probability value ≥ 0.7 are entered into the subsequent analysis process.

3. The bridge crack detection method based on deep learning and UAV images according to claim 1, characterized in that: The crack type classification rules in step S4 specifically include: If the principal direction angle of the crack satisfy These are classified as transverse cracks; If satisfied Then it is classified as a longitudinal crack; If satisfied or If the crack is at °, it is classified as a diagonal crack; If there are three or more intersecting cracks forming a mesh-like structure, they are classified as mesh cracks.

4. The bridge crack detection method based on deep learning and UAV images according to claim 1, characterized in that: The formula for converting the crack width in step S5 is: ； in, This represents the actual crack width, in mm. The width is in pixels. The pixel size of an image sensor is expressed in μm. This refers to the lens focal length, in mm. The distance between the drone and the bridge pier is measured in meters (m).

5. A bridge crack detection system based on deep learning and UAV images for implementing the method of any one of claims 1-4, characterized in that, Includes the following modules connected in sequence by signals: The image acquisition module is used to automatically capture high-resolution images of the bridge surface via drone navigation, and record GPS coordinates, shooting time, and camera parameters; The cloud processing module is used to denoise, correct, enhance, and crop regions of uploaded images, and to complete size normalization and format conversion. The crack screening module uses a MobileNetV3-Small network with an introduced SE attention mechanism to perform binary classification reasoning and screen positive samples. The crack segmentation and classification module uses the UNet++ model to achieve pixel-level crack localization and combines principal component analysis to extract principal orientation angle and aspect ratio features, thereby completing the automatic classification of transverse, longitudinal, oblique and mesh cracks. The width measurement module improves edge segmentation accuracy through the DeepLabv3+ network and the Xception backbone network, and outputs the physical size of the crack based on the skeleton algorithm and camera calibration parameters. The results generation module is used to fuse the detection results with the original image to generate a visualized image and output a PDF report and a table file; The graphical user interface module provides project management, image display, result browsing and export functions, and supports multi-user operation.

6. The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The PDF report output by the result generation module includes basic bridge information, inspection time, heat map of crack distribution, and a statistical summary table of crack quantity, maximum width, and average length. The CSV table records the unique ID, type, length, maximum width, component number, and location coordinates of each crack.

7. The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The system is deployed on a cloud server, uses FlaskRESTfulAPI to implement task scheduling and interface services, encapsulates deep learning models in ONNX format and accelerates inference through TensorRT, and supports multi-task concurrent processing and distributed computing.

8. The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The graphical user interface module is an interactive interface based on PyQt5 and Python, providing functions such as comparing the original image and the labeled image, viewing detailed crack information by hovering the mouse, exporting detection results with one click, and supporting permission management and operation log recording.

9. The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The image acquisition module controls the flight path of the UAV by a preset route, and performs multi-angle imaging around the bridge pier to ensure that the crack area is unobstructed and the image overlap rate is not less than 60%, so as to improve the integrity of the detection.

10. The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The crack screening module and crack segmentation module adopt a cascaded processing strategy, and only the UNet++ and DeepLabv3+ high computational complexity model is launched for cracked images determined by the screening module, so as to realize dynamic optimization allocation of computing resources and improve the overall system efficiency.

Images