Bridge crack detection system and method based on deep learning and unmanned aerial vehicle image

Abstract

The invention discloses a bridge crack detection system and method based on deep learning and unmanned aerial vehicle images, and belongs to the application of computer vision and artificial intelligence technologies in civil engineering structure health monitoring. The method comprises the following steps: an unmanned aerial vehicle collects a bridge image (the overlapping rate is greater than or equal to 60%) according to a preset route, and a cloud preprocesses the image; the method comprises the following steps: carrying out primary screening on crack images by using an improved MobileNetV3-Small model, segmenting cracks by using a UNet + + model and classifying the cracks (transverse and longitudinal classes), calculating crack sizes by using an improved DeepLabv3 + network in combination with camera parameters, and finally generating a visual image and a report with labels. The system comprises an image acquisition module, a cloud processing module, a crack preliminary screening module, a segmentation classification module, a width measurement module, a result generation module and an image user interface module which are in data transmission connection in sequence, and resources are optimized by adopting a cascade strategy. The detection automation level, precision and efficiency can be improved, the labor cost and the safety risk are reduced, and the defects of traditional detection and the prior art are overcome.

Description

Technical Field

[0001] This invention relates to the application of computer vision and artificial intelligence technologies in the health monitoring of civil engineering structures. Specifically, it relates to an automatic bridge crack width detection method and system that combines UAV image acquisition, deep learning classification and semantic segmentation algorithms, and integrates a graphical user interface. It is suitable for the automated identification, location and quantitative measurement of surface cracks in concrete structures such as bridge piers. Background Technology

[0002] With the large-scale construction and increasing service life of my country's transportation infrastructure, a large number of bridges are entering the aging stage, posing a serious challenge to their structural safety. Among them, concrete surface cracks are one of the important indicators reflecting structural deterioration, especially vertical or diagonal cracks in the piers, which may indicate a decline in load-bearing capacity or even a potential risk of collapse, requiring regular inspection and evaluation.

[0003] Existing bridge crack detection technologies have the following significant shortcomings: (1) Traditional bridge crack detection mainly relies on manual inspection combined with handheld equipment for shooting, which has problems such as low efficiency, strong subjectivity, danger of high-altitude operation and difficulty in quantification.

[0004] (2) In the existing technology, some studies use traditional image processing methods (such as edge detection and threshold segmentation) to identify cracks, but they are sensitive to interference such as complex lighting, shadows, and stains, and have a high false detection rate. Although some deep learning-based methods have improved the recognition accuracy, they still lack a complete end-to-end process design in actual engineering deployment, especially the ability to integrate crack existence, location classification, and width measurement, which limits their application and promotion in engineering sites.

[0005] To address the aforementioned issues, there is an urgent need for a bridge crack detection system that integrates image acquisition, intelligent analysis, and precision measurement, which can ensure detection accuracy while improving engineering practicality and deployment efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a bridge crack detection method and system based on lightweight deep learning, which solves the problems of high cost, large computing power requirements, complex operation, and redundant accuracy. This aims to significantly improve the automation level, detection accuracy, and work efficiency of bridge crack detection, while reducing labor costs and safety risks.

[0007] To achieve the above objectives, this invention provides an intelligent method for detecting bridge crack width based on the fusion of UAV image acquisition and deep learning model, comprising the following steps: S1. Obtain bridge section detection images through drone aerial photography; S2. The detected images of the bridge sections are transmitted to the cloud server for preprocessing to obtain preprocessed images; S3. Determine the presence of cracks in the preprocessed images. Images with cracks are included in the next analysis stage, while the rest are archived. S4. Pixel-level semantic segmentation is achieved based on the UNet++ model. Geometric features of cracks are extracted based on morphological analysis and principal component analysis, including principal orientation angle, aspect ratio and number of branches. Crack types are automatically classified according to preset rules. S5. Through the crack width precision measurement module, the maximum width, length and location coordinates of each crack are finally output; S6. Overlay the above detection results back onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations; at the same time, automatically generate the result file.

[0008] Preferably, the image acquisition device in step S1 uses a drone equipped with a multi-sensor camera to fly around the bridge pier in a preset flight path to ensure that the image overlap rate is not less than 60% in order to improve the detection integrity. The drone performs multi-angle imaging around key parts such as the bridge pier in the preset flight path to ensure that the crack area is fully exposed and unobstructed.

[0009] Preferably, in step S2, the cloud server focuses on extracting images of the bridge pier area to reduce subsequent computational redundancy, while simultaneously performing image normalization processing, scaling them to a standard size, and converting them into an input format acceptable to the model.

[0010] Preferably, the crack classification model in step S3 is an improved MobileNetV3-Small model. The improvements include: S31. Introduce the Squeeze-and-Excitation (SE) attention mechanism; S32. Perform transfer learning fine-tuning training on the Kaggle and self-built bridge crack datasets to improve feature representation ability and classification accuracy. S33. Set the judgment threshold to 0.7. When the probability value is greater than or equal to the threshold, it is judged as a positive sample and enters the next analysis stage; otherwise, it is archived.

[0011] Preferably, in step S4, the UNet++ model is used to achieve pixel-level semantic segmentation, and the geometric features of the crack are extracted based on morphological analysis and principal component analysis. The features include the principal orientation angle, aspect ratio and number of branches.

[0012] Preferably, in step S5, the crack width precision measurement module employs an improved DeepLabv3+ semantic segmentation network, the improvement including: S51 is equipped with the Xception backbone network and ASPP dilated convolution module to perform fine segmentation of crack regions and improve edge accuracy; S52. Obtain the crack centerline using a skeleton extraction algorithm, and uniformly sample multiple points along the centerline; Preferably, in step S6, the result visualization module overlays the above detection results back onto the original image, generates a visualization image with color-coded crack areas, yellow outlines, and width numerical annotations, and generates an exportable file.

[0013] This invention also provides a bridge crack detection system based on deep learning and UAV images, as detailed below: It includes an image acquisition module, an image transmission and pan-tilt processing module, a crack initial screening and classification module, a crack area labeling and type classification module, a crack width precise measurement module, and a result visualization and file generation module. These modules are sequentially connected via data transmission links, with the specific connection relationships as follows: The output of the image acquisition module is connected to the input of the image transmission and PTZ processing module, and is used to transmit the acquired bridge image data to the cloud server. The output of the image transmission and gimbal processing module is connected to the input of the crack primary screening and classification module, and is used to transmit the preprocessed image data to the crack primary screening and classification module. The output of the crack initial screening and classification module is connected to the input of the crack region labeling and type classification module, and is used to transmit the crack initial screening and classification results to the crack region labeling and type classification module. The output of the crack region labeling and type classification module is connected to the input of the crack width precision measurement module, and is used to transmit the region labeling and type classification results to the crack width precision measurement module. The output of the crack width precision measurement module is connected to the input of the result visualization and file generation module. It is used to overlay the detection results back onto the original image to generate a visualized image with color coding and width annotation, and automatically generate PDF reports and CSV tables, which are then transmitted to the result visualization and file generation module. The functions of each module are as follows: Image acquisition module: used to execute the above method step S1 to acquire surface image data of the bridge section; Gimbal Image Processing Module: Used to execute the above method step S2, receive raw image data from the UAV, perform noise reduction, white balance correction, contrast enhancement and region cropping operations, focus on extracting the image of the bridge pier area and complete normalization processing; Crack initial screening and classification module: used to execute the above method step S3, using a lightweight convolutional neural network MobileNetV3-Small to build a binary classification model to realize crack inference detection; Crack region labeling and type classification module: It is used to execute the above method step S4, generate crack pixel-level masks using the UNet++ model, and extract the principal orientation angle, aspect ratio and number of branches based on principal component analysis to achieve automatic classification of transverse, longitudinal, oblique or mesh cracks; Crack width precision measurement module: used to execute the above method step S5, adopts DeepLabv3+ semantic segmentation network, and combines camera intrinsic parameters, sensor size and shooting distance to convert pixel width into real physical width (unit: millimeter), and outputs the maximum width, average width, length and position coordinates of each crack; Results visualization and document generation module: Overlays the detection results back onto the original image to generate a visualization image with color coding and width annotation, and automatically generates PDF reports and CSV tables; Preferably, the image acquisition module uses a drone equipped with a multi-sensor camera to fly around important parts such as bridge piers along a preset flight path, ensuring that the image overlap rate is not less than 60% to improve the integrity of the detection.

[0014] Preferably, the crack initial screening and classification module uses an improved MobileNetV3-Small neural network to construct a binary classification model, determines whether there is a crack in the image and outputs a probability value of the "cracked" category. When the probability value is greater than or equal to a preset threshold of 0.7, it enters the subsequent analysis process.

[0015] Preferably, the crack region labeling and type classification module uses the UNet++ model to generate pixel-level masks, and uses principal component analysis to extract key information to achieve automatic classification of transverse, longitudinal, oblique, or mesh cracks.

[0016] Preferably, the crack width precision measurement module uses the DeepLabv3+ semantic segmentation network equipped with the Xception backbone network and the ASPP dilated convolution module to perform fine segmentation of the crack region, calculate the distance between intersection points, and combine the camera intrinsic parameters, sensor size and shooting distance to convert the pixel width into the real physical width (unit: millimeters), and output the maximum width, average width, length and position coordinates of each crack.

[0017] Preferably, the graphical user interface is developed based on PyQt5 + Python, supporting project management, image display, result viewing and one-click export functions, enabling staff to operate in one stop.

[0018] Therefore, the intelligent bridge crack width detection method and system based on deep learning using the above-mentioned structure has the following beneficial effects: (1) The dual-stage AI processing architecture, with a two-level strategy of "MobileNetV3 initial screening + DeepLabv3 + fine testing", significantly reduces computational overhead and improves system response speed while ensuring detection accuracy; (2) Integrate deep learning segmentation results with geometric feature analysis to achieve automated classification of crack types and assist in structural safety assessment; (3) By combining camera parameters and spatial mapping models, a precise conversion from pixel width to millimeter-level real size can be achieved; (4) Full-process closed-loop system integration: covering image acquisition → transmission → AI analysis → result output → human-computer interaction, forming a complete intelligent detection solution; (5) Provides an intuitive operation interface to support engineers to easily browse, review and export results, thereby improving the system's usability and user experience.

[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a bridge crack detection system and method based on deep learning and UAV images according to the present invention. Figure 2 This is a block diagram of a bridge crack detection system and method based on deep learning and UAV images according to the present invention. Figure 3 This is a visualization interface diagram of the system detection process of a bridge crack detection system and method based on deep learning and UAV images according to the present invention. Figure 4 This is a schematic diagram illustrating the storage of the exported results of a bridge crack detection system and method based on deep learning and UAV images according to the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] Example 1 like Figure 1 As shown, the present invention provides a bridge crack detection method based on deep learning and UAV images, comprising the following steps: S1, Bridge Inspection Image Acquisition The drones are used to capture images of the bridge deck, piers and other structural surfaces, and to collect image data of high piers and hard-to-reach areas, which are then transmitted to the cloud server in real time. S2: Detection Image Preprocessing The image is denoised, white balance corrected, and contrast enhanced. The image of the bridge pier area is extracted. At the same time, the image is normalized and scaled to a standard size to fit the model input. S3: Crack Preliminary Screening Classification The preprocessed images are fed into an improved MobileNetV3-Small binary classification model to determine whether cracks exist in the preprocessed images. Images with cracks are included in the next analysis stage, while the rest are archived. S4: Crack Area Marking and Type Classification The UNet++ model is used to achieve pixel-level semantic segmentation, generate binary mask images of cracks, mark the spatial distribution range of cracks, and automatically classify crack types according to preset rules. S5: Precise measurement of crack width An improved DeepLabv3+ neural network is used to perform fine segmentation of the crack region, improve edge accuracy, calculate the distance between two points as the local width, and then combine the camera intrinsic parameters (focal length, sensor size), shooting distance and scale calibration relationship to convert the pixel width into the real physical width (unit: mm), and finally output the maximum width, length and position coordinates of each crack. S6: Results Visualization and Document Generation The above detection results are superimposed onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations; at the same time, a result file is automatically generated.

[0024] Example 2 like Figure 2 As shown, a bridge crack detection system based on deep learning and UAV images includes a UAV image acquisition module, a gimbal image processing module, a crack initial screening and classification module, a crack area labeling and type classification module, a crack width precise measurement module, a result visualization and file generation module, and a graphical user interface module.

[0025] The image acquisition module automatically cruises and photographs the bridge structure surface using a drone to obtain high-resolution visible light images, and transmits the images along with GPS coordinates, shooting time information, and camera parameter information to a cloud server. The drone is equipped with a high-definition camera and performs multi-angle imaging around key parts such as bridge piers along a preset flight path to ensure that the cracked areas are fully exposed and unobstructed.

[0026] The gimbal image processing module extracts images of the bridge pier area based on the raw image data from the UAV, performs normalization processing, scales the images to a standard size, and converts them into an input format acceptable to the model.

[0027] The crack initial screening and classification module introduces the SE attention mechanism based on the improved MobileNetV3-Small neural network to determine whether cracks exist in the preprocessed image. The judgment threshold is set to 0.7. When the probability value is greater than or equal to the threshold, it is judged as a positive sample and enters the next module; otherwise, it is archived.

[0028] The crack region labeling and type classification module accurately labels the spatial distribution range of cracks based on the UNet++ model, extracts crack set features based on morphological analysis and principal component analysis, and automatically classifies crack types (transverse cracks / longitudinal cracks / oblique cracks / network cracks) according to preset rules.

[0029] The crack width precision measurement module performs regional segmentation of the crack area based on the improved DeepLabv3+ semantic segmentation network to improve edge accuracy. It obtains the crack centerline based on the skeleton extraction algorithm and samples multiple points. It calculates the distance between two points as the local width and converts the pixel width into the real distance by combining the camera's intra-camera parameters. It outputs the maximum width, length and position coordinates of each crack.

[0030] like Figure 3 , Figure 4As shown, the result visualization and file generation module overlays the above detection results back onto the original image (with color-coded crack areas, yellow outlines, and width numerical annotations), and automatically generates a result file. The graphical user interface (GUI) module is developed using a desktop application with a front-end based on PyQt5 and Python, providing an interactive operating environment. The interface includes an image display window and a detection information display area. It supports users to import images and start the detection process. It has a one-click export function, which can package and download the detection report. The system also supports multi-user permission management and operation log recording to ensure data security and traceability.

[0031] This invention employs a bridge crack detection system and method based on deep learning and UAV imagery, as described above. The method includes: acquiring image data of the bridge surface using a UAV; performing noise reduction, white balance correction, and contrast enhancement on the acquired images on a cloud server, focusing on extracting images of the pier area, while simultaneously performing image normalization and scaling them to a standard size to adapt to the input model; determining the presence of cracks in the preprocessed images, placing images with cracks into the next analysis stage, and archiving the rest; for images determined to have cracks, defining the spatial range of the cracks, extracting the geometric features of the cracks, and generating data including category labels, confidence levels, bounding boxes, and mask data; outputting the maximum width, length, and location coordinates of each crack in all cracked images; superimposing the detection results back onto the original image to generate a visual image with color-coded crack areas, yellow outlines, and width numerical annotations, while automatically generating a result file. The system includes data acquisition, preprocessing, detection, classification, data annotation, and file storage modules connected sequentially by signals. This invention achieves crack screening and classification, reduces resource waste, significantly improves the automation level, detection accuracy, and work efficiency of bridge crack detection, and reduces labor costs and safety risks.

[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1.A bridge crack detection method based on deep learning and unmanned aerial vehicle image, characterized in that, The method comprises the following steps: S1. Flying along the preset flight path by a UAV carrying a multi-sensor camera, collecting bridge surface images, and ensuring that the image overlap rate is not less than 60%; S2. Transmitting the images to a cloud server for preprocessing, including denoising, white balance correction, contrast enhancement, and bridge pier area extraction, and normalizing the images to adapt to the model input; S3. Using an improved MobileNetV3-Small classification model to judge the existence of cracks in the preprocessed images, and when the output probability value is greater than or equal to the set threshold, it is determined that there are cracks and enters the subsequent processing, otherwise it is archived; S4. Based on the UNet++ model, the image containing the crack is subjected to pixel-level semantic segmentation to generate a crack binary mask, and the crack geometric features including the main direction angle, aspect ratio and branch number are extracted by combining morphological operations and principal component analysis, and the crack type is automatically classified according to the preset rules; S5. Using the DeepLabv3+ network to finely segment the crack area, extracting the skeleton center line and sampling the local width, and converting the pixel width to actual physical width by combining the camera parameters and shooting distance, outputting the maximum width, length and position coordinates of the crack; S6. Superimposing the detection results on the original image to generate a visual result with color-coded crack area, yellow outline label and width value, and automatically generating a PDF file containing a statistical report and a CSV table of structured data. 2.The bridge crack detection method based on deep learning and UAV image according to claim 1, characterized in that: In the step S3, the MobileNetV3-Small model is fine-tuned on the Kaggle public dataset and the self-built bridge crack dataset through transfer learning, and the SE attention mechanism is introduced to enhance the feature expression ability; The determination threshold is set to 0.7, and the images with a probability value greater than or equal to 0.7 enter the subsequent analysis process. 3.The bridge crack detection method based on deep learning and UAV images according to claim 1, characterized in that: The crack type classification rules in step S4 specifically include: If the main direction angle of the crack satisfies , it is classified as a transverse crack; If then it is classified as a longitudinal crack; If the following conditions are met or °, then it is classified as a diagonal fracture; If there are more than three intersecting cracks and a grid-like structure is formed, it is classified as a net-like crack. 4.The bridge crack detection method based on deep learning and UAV images according to claim 1, characterized in that: The crack width conversion formula in step S5 is: ； wherein, is the actual crack width, in mm, is the pixel width, is the image sensor single-pixel size, in pm, is the lens focal length, in mm, is the shooting distance between the UAV and the pier, in m. 5.A bridge crack detection system based on deep learning and unmanned aerial vehicle image for implementing the method of any one of claims 1-4, characterized in that, The following modules are connected in sequence: An image acquisition module for automatically cruising and shooting high-resolution images of the bridge surface by a UAV, and recording GPS coordinates, shooting time and camera parameters; A cloud processing module for denoising, correcting, enhancing and regionally cropping uploaded images, completing size normalization and format conversion; A crack preliminary screening module based on the MobileNetV3-Small network with SE attention mechanism for binary classification reasoning to screen positive samples; A crack segmentation and classification module that uses the UNet++ model to realize pixel-level positioning of cracks and extracts main direction angle and aspect ratio features by combining principal component analysis to automatically classify horizontal, vertical, diagonal and net-like cracks; A width measurement module that improves edge segmentation accuracy through the DeepLabv3+ network and Xception backbone network, and outputs the physical size of the crack based on the skeleton algorithm and camera calibration parameters; A result generation module for fusing detection results and original images to generate visual images, and outputting PDF reports and table files; The graphic user interface module provides project management, image display, result browsing and export functions, and supports multi-user operation. 6.The bridge crack detection system based on deep learning and UAV images according to claim 5, characterized in that: The PDF report output by the result generation module includes bridge basic information, detection time, crack distribution heat map, and statistical summary table of crack number, maximum width and average length. The CSV table records the unique ID, type, length, maximum width, component number and position coordinates of each crack. 7.The bridge crack detection system based on deep learning and UAV images of claim 5, wherein: The system is deployed on a cloud server, uses FlaskRESTful API to realize task scheduling and interface service, encapsulates the deep learning model in ONNX format and accelerates inference through TensorRT, and supports multi-task concurrent processing and distributed computing. 8.The bridge crack detection system based on deep learning and UAV images of claim 5, wherein: The graphic user interface module is an interactive interface based on PyQt5 and Python, which provides functions such as original image and labeled image comparison display, mouse hovering to view crack details, one-key export of detection results, and support for permission management and operation log recording. 9.The bridge crack detection system based on deep learning and UAV images of claim 5, wherein: In the image acquisition module, the flight path of the unmanned aerial vehicle is controlled by a preset route, and multi-angle imaging is performed around parts such as piers to ensure that the crack area is not blocked and the image overlap rate is not less than 60%, thereby improving the detection integrity. 10.The bridge crack detection system based on deep learning and UAV images of claim 5, wherein: The crack preliminary screening module and the crack segmentation module use a cascade processing strategy, only start high-computational-complexity models such as UNet++ and DeepLabv3+ for images with cracks determined by the preliminary screening module, thereby realizing dynamic optimization allocation of computational resources and improving the overall efficiency of the system.

Images