Structure surface crack detection and quantitative analysis system based on deep learning

Abstract

The invention discloses a structure surface crack detection and quantitative analysis system based on deep learning, and relates to the crossing field of computer vision, deep learning and structure health monitoring. The system adopts a multi-stage cascade architecture. The method comprises the following steps: firstly, realizing crack intelligent identification through an improved OfficientNet-B0 model (integrating an ECA attention mechanism and an optimized MBConv structure); then, precise segmentation is completed based on a Mask-RCNN architecture; and finally, combining morphological processing and a camera calibration technology to realize size quantification. The core innovation points comprise: (1) a lightweight identification model of a double attention mechanism; (2) segmentation optimization of CLAHE enhancement and morphological refinement; and (3) a sub-millimeter quantization method based on skeletonization. Experimental results show that the recognition accuracy rate reaches 99.0%, the segmentation IoU is about 0.623, and the width quantization error lt is achieved; the thickness is 0.1 mm.

Description

TECHNICAL FIELD

[0001] The present application belongs to the cross technical field of computer vision and structural health monitoring, and specifically relates to a crack detection system based on deep learning. The system comprehensively applies an improved EfficientNet-B0 deep learning model, a Mask-RCNN segmentation architecture and image processing technology to realize automatic identification, accurate segmentation and quantitative analysis of surface cracks of concrete structures. It is particularly suitable for health monitoring and safety evaluation of engineering structures such as bridges and buildings. BACKGROUND

[0002] 1. Technical background and status quo

[0003] Structural surface cracks, as an important representation of early damage to engineering structures, have important significance for ensuring engineering safety. Traditional crack detection methods mainly include contact and non-contact methods: contact detection methods include crack meters, strain gauges and other sensor monitoring technologies, which have high precision but have problems such as complex installation, high cost, limited coverage, etc. Non-contact detection methods are mainly based on digital image processing technology, which has the advantages of low cost, high efficiency and wide applicability, and is the mainstream direction of current research.

[0004] 2. Limitations of existing technology

[0005] 2.1 Limitations of traditional image processing methods

[0006] Early crack detection mainly relies on traditional image processing algorithms:

[0007] (1) Edge detection method: Gradient operators such as Canny operator and Sobel operator can detect edge information in images, but are sensitive to noise, prone to false edges, and difficult to distinguish cracks from other linear features.

[0008] (2) Threshold segmentation method: Adaptive threshold algorithms such as Otsu method and maximum entropy method can achieve binary image processing, but the segmentation effect is poor in complex backgrounds and is easily affected by light changes and shadows.

[0009] (3) Morphological processing method: Remove noise and connect broken areas through morphological operations such as opening and closing operations, but the selection of structural elements is subjective and has poor adaptability to cracks of different shapes.

[0010] Main technical defects:

[0011] (1) Poor environmental adaptability: Poor robustness to complex environmental factors such as light changes, surface contamination and texture interference.

[0012] (2) Limited feature extraction: Artificially designed features are difficult to fully represent different morphological features of cracks.

[0013] (3) Parameter dependence is strong: manual adjustment of parameters is needed for different scenarios, and the generalization ability is limited.

[0014] (4) Insufficient quantization accuracy: it is difficult to achieve accurate measurement of crack geometric parameters.

[0015] 2.2 Status and problems of deep learning methods

[0016] In recent years, deep learning technology has shown great potential in crack detection:

[0017] Classification network application:

[0018] (1) VGG network: Zhang et al. used VGG-16 to realize crack image classification, with an accuracy of 95.7%.

[0019] (2) ResNet network: Kim et al. used ResNet-50 for crack detection, which performed well in complex backgrounds.

[0020] (3) EfficientNet network: Dais et al. used EfficientNet-B4 to realize lightweight crack recognition.

[0021] Semantic segmentation network application:

[0022] (1) U-Net network: Liu et al. improved the U-Net structure for crack pixel-level segmentation.

[0023] (2) DeepLabv3+: Cao et al. combined dilated convolution to realize multi-scale crack segmentation.

[0024] (3) Mask-RCNN: Ren et al. used instance segmentation technology to realize accurate positioning of cracks.

[0025] Limitations of existing deep learning methods:

[0026] (1) High model complexity: mainstream models have large parameter quantities and high computational resource requirements, making them difficult to deploy on mobile devices.

[0027] (2) Strong data dependence: a large amount of labeled data is needed, and performance decreases significantly in small sample cases.

[0028] (3) Insufficient generalization ability: detection stability under different materials and different environmental conditions needs to be improved.

[0029] (4) Lack of quantization function: most methods only realize detection and segmentation, lacking accurate quantization of geometric parameters.

[0030] (5) System integration is low: lack of complete solutions from data collection to result output.

[0031] 3. Technology development trend

[0032] Current crack detection technology is developing in the following directions:

[0033] (1) Lightweight model design: reduce model parameters and improve inference speed.

[0034] (2) Multi-modal fusion: combine RGB, infrared, laser and other sensor information.

[0035] (3) End-to-end system: build a complete processing flow from detection to quantification.

[0036] (4) Real-time monitoring: realize continuous and automatic structural health monitoring. SUMMARY

[0037] The purpose of the present invention is to provide a lightweight high-precision crack detection system based on cascade optimization. In order to achieve the above purpose, the present invention provides the following technical solutions: a lightweight high-precision crack detection system based on cascade optimization, comprising data preprocessing module, crack recognition module, crack segmentation module and crack geometric quantification module connected in turn; the data preprocessing module receives the input original image signal, and performs standardization and multi-stage enhancement processing; the crack recognition module performs binary classification judgment on the preprocessed image based on the improved EfficientNet-B0 model, and distinguishes crack and non-crack areas; the crack segmentation module performs pixel-level segmentation on the recognized crack area based on the improved Mask R-CNN model; the crack geometric quantification module extracts the crack skeleton based on the segmentation result and performs geometric parameter calculation; further comprising a system visualization module connected with the geometric quantification module, for receiving and displaying detection results, geometric parameter data and generating detection report.

[0038] Preferably, the data preprocessing module includes: a grayscale conversion unit for converting the input RGB image into a grayscale image I_gray = 0.299R + 0.587G + 0.114B; a resolution normalization unit for uniformly adjusting the grayscale image size to 128×128 pixels; a data augmentation unit for sequentially applying geometric transformations (including rotation ±30°, cropping 20%, and horizontal / vertical flipping) and optical transformations (including brightness adjustment ±30%, contrast adjustment ±40%, and Gaussian blur with standard deviation σ = 1.5) to the normalized image; and a contrast enhancement unit for processing the enhanced image using the Constrained Contrast Adaptive Histogram Equalization (CLAHE) algorithm, with parameters set to clip_limit = 2.0 and tile_size = 8×8. (Improvement note: This process significantly improves the robustness of the model under complex lighting, viewing angles, noise, and other interferences through a structured enhancement strategy, providing stable and reliable input for subsequent modules.)

[0039] Preferably, the improved EfficientNet-B0 model used in the crack recognition module includes the following optimizations: First, the original ReLU function is replaced with a Sigmoid-Weighted Linear Unit (SiLU) activation function, SiLU(x) = x·σ(βx), where β is a learnable parameter with a default value of 1.0. (Improvement explanation: SiLU combines the smoothness of Sigmoid and the sparse activation advantage of ReLU, enhancing the model's nonlinear fitting ability and improving the recognition of subtle cracks.) Second, an efficient channel attention (ECA) mechanism is introduced, adaptively calculating the weights w_c = σ(1 / k*Σw_i^T*y_i) for each feature channel and weighting the output feature F_out = w_c·F_in. The convolution kernel size k is adaptively adjusted by the channel dimension C, k = ψ(C) = (log2(C) / γ) + b / γ. (Improvement explanation: The ECA mechanism allows the model to focus on key channel features related to cracks without significantly increasing computation, improving accuracy.) Third, in Mobile Inverted... The Bottleneck (MBConv) block is configured with an expansion factor ε = 4, uses a 3×3 depthwise separable convolution kernel, and integrates a squeeze-activation (SE) module. (Improvement note: The optimized MBConv block enhances feature extraction capability and efficiency, improving recognition accuracy.) The fourth improvement is the adoption of a label-smooth cross-entropy loss function. Where the smoothed label y_i = 1 - α (when the true label is 1) or α / (K-1) (when the true label is 0), the smoothing factor is set to α = 0.1, K = 2; (Improvement note: Label smoothing effectively alleviates model overfitting and enhances its generalization ability to noise and imperfect labels.)

[0040] Preferably, the improved Mask R-CNN model used in the crack segmentation module includes the following innovations: it adopts a dual-backbone input design, simultaneously receiving the enhanced image output by the data preprocessing module and the key features extracted by the crack recognition module (or directly extracted from the enhanced image); after generating the segmentation mask, it performs morphological thinning processing. Where B is a 3×3 structuring element. Indicates corrosion operation. This indicates an opening operation; (Improvement notes: Dual-backbone input provides complementary information (global texture + local edges), morphological thinning effectively removes segmentation noise and accurately approximates the crack skeleton, improving segmentation boundary accuracy.) Otsu's adaptive thresholding method is applied: T* = arg maxσ_b 2 (T) Automatically determines the optimal binarization threshold. (Improvement note: The Otsu method automatically adapts to images with different contrast levels, improving the stability of the binarization results.)

[0041] Preferably, the crack geometry quantization module performs the following step: skeleton extraction, which iteratively extracts the skeleton from the segmented binary mask image I_mask. The process continues until the erosion result is empty. The geometric parameter calculation step uses the extracted skeleton and original mask image to calculate the length L = Σs_i·ρ (s_i is the number of skeleton pixels, ρ is the pixel resolution), average width W_avg = A_total / L (A_total is the total pixel area of ​​the crack region), and maximum width W_max = max(dt(p_i)) (dt is the distance transformation value). (Improvement note: This step achieves accurate conversion from pixel space to physical size.) The camera calibration model application step obtains system parameters through the calibration board and uses the perspective relationship f / d = dx / w = dy / h to calculate the actual physical area of ​​a single pixel S_pixel = w·h = d. 2 / ((f / dx)·(f / dy)) is used to convert pixel-level geometric parameters into actual physical dimensions (millimeters, millimeters). 2 (Etc.). (Improvement note: This calibration model solves the scale calibration problem of pixel-to-physical size conversion and is the core of accurate quantization.)

[0042] Preferably, the system visualization module adopts a layered architecture with a PyQt5-based front-end user interface and a PyTorch / OpenCV-based back-end processing engine. The front-end user interface provides dual-modal display (overlaying the original image with the detection results), a real-time display panel for crack geometric parameters, historical detection record storage and management functions, and the ability to generate txt files containing quantified parameters. The back-end processing engine sequentially executes the image acquisition, preprocessing, crack identification, segmentation, quantification analysis, and result visualization processes, storing the processing results in a database for front-end access. (Improvement note: The visualization system achieves a complete closed loop integrating detection, quantification, analysis, and reporting, greatly enhancing the system's practicality, ease of use, and engineering value.)

[0043] In the above technical solution, the lightweight high-precision crack detection system based on cascade optimization provided by this invention patent has the following beneficial effects: 1. This invention patent, through the "layered processing + cascade optimization" architecture (data preprocessing → recognition → segmentation → quantization) and the improved lightweight model (EfficientNet-B0 and dual-backbone Mask R-CNN incorporating advanced components such as SiLU and ECA), significantly improves the robustness and generalization ability of crack detection in complex environments (light changes, noise, and viewpoint differences) (recognition accuracy of 99.0%, IoU of 0.899, and false alarm rate of <0.5%), while realizing a lightweight design for high-precision recognition and segmentation (the number of model parameters is reduced to 30% of the traditional method, the single frame processing time is 45ms, and the GPU memory usage is reduced by 50%). 2. This invention patent significantly improves the accuracy of segmented edges through an innovative multi-source input segmentation network (original image + edge image) and morphological post-processing (thinning, Otsu adaptive thresholding); and constructs a complete geometric quantization solution by combining skeletonization with a camera calibration model (f / d = dx / w = dy / h, S_pixel calculation), effectively solving the scale calibration problem in the accurate quantization of crack dimensions (width measurement MAE 0.078mm, length relative error <0.5%, area accuracy 96.8%). 3. This invention patent integrates a complete visualization front-end (PyQt5) and automated back-end processing workflow, providing an integrated system including dual-modal display, parameter panel, historical data management, and generation of quantization parameter TXT files, significantly improving practicality and ease of use, and meeting the needs of real-time detection and data analysis in engineering sites. Attached Figure Description

[0044] Figure 1 The accompanying drawings are for the abstract of this invention;

[0045] Figure 2 This is a structural diagram of the improved EfficientNet-B0 model of the present invention;

[0046] Figure 3 This is a flowchart of the Mask-RCNN segmentation network of the present invention;

[0047] Figure 4 This is a schematic diagram of the camera calibration model of the present invention;

[0048] Figure 5 This is a screenshot of the system interface of the present invention. Detailed Implementation

[0049] 1. Data preparation and preprocessing

[0050] 1.1 Data Acquisition and Annotation: The publicly available Concrete Crack Image Classification Dataset (CCIC) was used. This dataset contains 40,000 images of cracked and non-cracked structures, covering various engineering structure types. Semi-automatic annotation was performed using the EISeg semantic annotation tool to generate COCO format annotation files.

[0051] 1.2 Data Preprocessing Flow

[0052] (1) Preprocessing includes the following key steps:

[0053] Grayscale conversion: I gray =0.299R + 0.587G + 0.114B.

[0054] Size standardization: The resolution has been uniformly adjusted from 240×240 to 128×128.

[0055] (2) Data augmentation:

[0056] Geometric transformations: Rotate (±30°), Shear (20%), Horizontal / Vertical Flip.

[0057] Optical transformations: brightness (±30%), contrast (±40%), Gaussian blur (σ=1.5).

[0058] Data partitioning: The data is randomly divided into training and validation sets in an 8:2 ratio.

[0059] 2. Model Training and Optimization

[0060] 2.1 Improvements to the EfficientNet-B0 model

[0061] Four core improvements were made to the original EfficientNet-B0 architecture:

[0062] Improvement 1: Use the SiLU activation function.

[0063]

[0064] Where β is a learnable parameter with a default value of 1.0.

[0065] Improvement 2: Introduce ECA attention mechanism.

[0066]

[0067] Improvement point 3: MBConv block optimization.

[0068]

[0069] 2.2 Optimization of Mask-RCNN Segmentation Model

[0070] The segmentation model uses a cascaded architecture, and the processing flow is as follows:

[0071] (1) RGB to grayscale conversion: I gray =0.299R + 0.587G + 0.114B

[0072] (2) CLAHE contrast enhancement: H′(i,j)=CLAHE(I gray (i,j))

[0073] (3) Otsu adaptive threshold segmentation:

[0074]

[0075] (4) Morphological refinement:

[0076]

[0077] B is a 3×3 structuring element.

[0078] 2.3 Model Training Parameter Configuration

[0079]

[0080] 3. Crack Quantification Method

[0081] 3.1 Skeleton Extraction and Geometric Parameter Calculation

[0082] The crack skeleton was extracted using an iterative morphological method.

[0083]

[0084] Where B is a 3×3 structuring element, iterated until... Geometric parameter calculation:

[0085] length

[0086] Average width

[0087] Maximum width W max =max(dt(p) i ))

[0088] Where ρ is the pixel resolution and dt is the distance transformation.

[0089] 3.2 Camera Calibration Model

[0090] Pixel size quantization based on the principle of similar triangles:

[0091]

[0092] Where f is the focal length, d is the object distance, dx and dy are the pixel size, and w and h are the actual width and height of a single pixel.

[0093] 3.3 Quantization accuracy verification

[0094] The accuracy of artificial cracks in concrete test blocks was verified using a film ruler (accuracy 0.01mm):

[0095]

[0096] Average absolute error: 0.078 mm, average relative error: 7.35%.

[0097] 4. System Deployment and Visualization

[0098] 4.1 System Architecture

[0099] Using a layered architecture of PyQt5 + PyCharm:

[0100] 1. Front-end interface (PyQt5):

[0101] (1) Dual-modal display: original image + detection result superimposed;

[0102] (2) Parameter panel: Real-time display of crack geometry parameters;

[0103] (3) Historical data management: storage and retrieval of detection records;

[0104] (4) Report generation: Automatically generate a txt file containing content parameters.

[0105] 2. Backend processing (PyCharm):

[0106] (1) Image acquisition and preprocessing;

[0107] (2) Crack identification (improved EfficientNet-B0);

[0108] (3) Crack segmentation (improved Mask-RCNN);

[0109] (4) Geometric parameter quantification, result storage and visualization.

[0110] 4.2 Deployment Requirements

[0111]

[0112] 4.3 Operating Procedures

[0113] (1) Camera calibration: Camera parameters were calibrated using a 20×20mm calibration plate;

[0114] (2) Image acquisition: Images of the concrete surface are acquired using a USB 3.0 industrial camera;

[0115] (3) Preprocessing: Automatically perform grayscale conversion, size normalization and enhancement processing;

[0116] (4) Crack detection: Click the "Start Detection" button to perform identification and segmentation;

[0117] (5) Results Viewing: View the quantification parameters and visualization results in the right panel of the interface; (6) Report Export: Generate a txt file containing detailed parameters.

Claims

1. A deep learning based structural surface crack detection and quantification analysis system, characterized in that Comprise: (1) Data preprocessing module, for performing image gray scale conversion, size standardization to 128x128 pixels, CLAHE contrast enhancement and data enhancement operation; (2) Improved EfficientNet-B0 recognition module, integrating ECA attention mechanism and optimized MBConv structure, using SiLU activation function; (3) Segmentation module based on Mask-RCNN, using improved EfficientNet-B0 as backbone network, combined with morphological refinement processing; (4) Crack quantification module, based on skeletonization algorithm, distance transformation and camera calibration model to realize geometric parameter measurement; (5) Visual interactive module, based on PyQt5 framework to realize dual-mode display and parameter report generation.

2. The system of claim 1, wherein The technical features of the improved EfficientNet-B0 model include: (1) SiLU activation function: Where β is a learnable parameter, the default value is 1.0; (2) Weight calculation of ECA attention mechanism: w c = σ(Conv1D k (GAP(X))) Attention(X) = w c ⊙X wherein γ = 2, b = 1 ; (3) Optimized MBConv block structure: MBConv(X) = X + SE(DWConv 3×3 (PWConv ∈ (X))) Where ε is the expansion factor, set to 4; (4) Improved cross-entropy loss function: wherein a = 0.1 is a smoothing parameter.

3. The system of claim 1, wherein The processing flow of the Mask-RCNN segmentation module includes: (1) Image preprocessing: Igray = 0.299R + 0.587G + 0.114B (2) CLAHE contrast enhancement: I enhanced = CLAHE(I gray , clip_limit = 2.0, tile_size = 8 x 8) (3) Otsu adaptive threshold segmentation: (4) Morphological refinement operation: where B is a 3x3 structuring element, denotes an erosion operation, denotes an opening operation.

4. The system of claim 1, wherein The calculation method of the crack quantification module includes: (1) Skeleton extraction algorithm: Iterate until (2) Geometric parameter calculation: • Length: • average width: • maximum width: W max = max(DT(p i )) Where ρ is the pixel resolution, DT is the distance transformation function; (3) Camera calibration model: Where f is the focal length, d is the object distance, dx and dy are the pixel sizes.

5. The system of claim 1, wherein The model training parameter optimization strategy includes: (1) Optimizer: AdamW, parameter configuration is β1=0.9, β2=0.999, weight decay λ=×10 -4 ; (2) Learning rate schedule: cosine annealing strategy, initial learning rate η0= 0.001, minimum learning rate η_min = 1 x 10 -6 ; (3) Batch size: 32 for recognition module, 16 for segmentation module; (4) Regularization: label smoothing (α = 0.1) and Dropout (p = 0.2); (5) Training strategy: phased training, 50 rounds for recognition module, 100 rounds for segmentation module, 30 rounds for transfer learning.

6. The system of claim 1, wherein The evaluation index system includes: (1) Recognition performance index: • Accuracy: • F1 score: (2) Segmentation performance index: • IoU: • Average precision: (3) Quantitative accuracy index: • Mean Absolute Error: • Relative error:

7. The system of claim 1, wherein The functions of the visual interactive module include: (1) Dual-mode display: superimposed display of original image and detection results; (2) Real-time parameter panel: display of crack length L, average width W_avg, maximum width W_max, and area A; (3) Historical data management: storage, query and statistical analysis of detection records; (4) Report generation: automatic generation of txt file containing quantitative parameters; (5) Accuracy guarantee: error rate between displayed value and algorithm output value ≤0.5.

Images