Concrete crack detection method and system based on windmill-shaped convolution improved YOLO

Abstract

The invention relates to a concrete crack detection method and system based on windmill-shaped convolution improved YOLO, and belongs to the technical field of crossing of computer vision and civil engineering. The objective of the invention is to solve the problems of insufficient fine crack capture, poor multi-direction adaptation, insufficient multi-scale fusion and incapability of accurate quantification in the prior art. According to the technical scheme, an exclusive standardized data set is constructed; a backbone network is improved by using a windmill-shaped convolution (PConv) and an APC3k2 module, and a P2 feature fusion branch and a detection head are newly added to form a full-scale framework; an EIoU-L and Focal Dice mixed loss function is designed, and the model is trained and optimized in a staged mode; and integration of crack detection, segmentation and real scale quantification is realized based on unmanned aerial vehicle parameters. The method is high in detection precision and generalization capability, is suitable for real-time accurate detection of concrete cracks in scenes such as buildings, bridges and tunnels, and provides quantitative support for structural safety assessment.

Description

A concrete crack detection method and system based on windmill-shaped convolution improved YOLO Technical Field

[0001] The invention belongs to the interdisciplinary field of computer vision and civil engineering, specifically involving a concrete crack detection and semantic segmentation method that integrates windmill-shaped convolution and the YOLO object detection framework. It is particularly suitable for real-time and accurate detection and pixel-level segmentation of concrete surface cracks in civil engineering scenarios such as buildings, bridges, and tunnels. Background Technology

[0002] Concrete, as the most widely used structural material in civil engineering, is extensively used in core components such as building beams and columns, bridge piers and abutments, and tunnel linings. During construction and service, concrete is susceptible to surface cracks such as fissures, spalling, honeycombing, and exposed reinforcement due to factors such as loads, environmental erosion (e.g., humidity changes, chemical corrosion), and material aging. If fine cracks are not detected in time, they will gradually expand, leading to a decrease in structural durability and, in severe cases, causing safety accidents such as collapse and leakage. Achieving high-precision, real-time detection and segmentation of concrete cracks plays an irreplaceable role in extending the service life of structures, reducing operation and maintenance costs, and ensuring public safety.

[0003] With the development of computer vision and deep learning, automated crack detection technology based on object detection has become a research hotspot. The YOLO series models, due to their exceptional real-time performance and ease of use, are widely used in the field of concrete crack recognition based on computer vision technology. However, they still face the following technical bottlenecks:

[0004] (1) The ability to capture fine crack features is weak. Fine cracks have a low pixel ratio and blurred edges. The standard Conv convolution kernel of traditional YOLO is a fixed receptive field feature extraction. It lacks the ability to capture the directional features of long and thin cracks and is difficult to effectively capture the linear extension features of fine cracks, which easily leads to missed detection.

[0005] (2) Poor adaptability to multi-directional cracks. Concrete cracks often have irregular directions such as transverse, longitudinal, and oblique. The existing YOLO model convolution kernel lacks rotation invariance, resulting in low accuracy in identifying non-transverse cracks.

[0006] (3) Insufficient fusion of multi-scale features. Traditional deep learning models such as the YOLO model Neck network rely only on the fusion of original scale features and do not optimize for the scale difference between concrete cracks "fine cracks (small scale) - spalling (large scale)", resulting in small-scale cracks being easily covered and large-scale cracks having incomplete edge segmentation.

[0007] (4) Insufficient dataset adaptability. Existing public datasets for concrete cracks have problems such as single category and low annotation accuracy. Some high-precision crack segmentation datasets only cover a single crack type, which cannot meet the needs of simultaneous detection of multiple cracks in engineering. Moreover, existing technologies have not optimized the model structure for the characteristics of the dataset. Summary of the Invention

[0008] The purpose of this invention is to overcome the technical defects of existing concrete crack detection technologies, such as insufficient capture of fine cracks, poor multi-directional adaptation, insufficient multi-scale fusion, low detection accuracy due to sample imbalance, and inability to accurately quantify. This invention provides a concrete crack detection method and system based on windmill-shaped convolution improved YOLO, which integrates high-precision crack detection, pixel-level segmentation and real-scale quantization.

[0009] This invention forms a complete technical system of "data support - model adaptation - training reinforcement - detection and segmentation - scale quantization" by constructing an adaptive dataset, optimizing the YOLO model architecture, designing targeted training strategies, and developing scale quantization methods. This system meets the needs of automated and high-precision detection in engineering fields. The specific technical solution is as follows:

[0010] To address the issues of insufficient sample size and limited scene representation in existing datasets, this invention constructs a dedicated concrete crack dataset through a combination of "integration of publicly available datasets and supplementation with self-taken photographs," and completes the standardization process. The specific steps are as follows:

[0011] (1) Data source integration: Collect and publish valid samples of concrete crack dataset. At the same time, use high-definition cameras and drones to capture images of various engineering scenarios such as bridge box girders, bridge piers, and tunnel linings, covering multiple environmental conditions such as normal lighting, strong light and backlight, and cloudy and rainy weather, as well as multiple types of cracks such as horizontal, vertical, mesh, and oblique cracks, and integrate them to form the original dataset.

[0012] (2) Standardization process: Remove invalid samples that are blurry, overexposed, or occluded to obtain valid images. Adjust the pixel size uniformly and use min-max normalization to map the pixel values ​​to the [0,1] interval. Use the COCO annotation format uniformly to complete the bounding box annotation (detection task) and pixel-level mask annotation (segmentation task) of the crack area, and annotate auxiliary information such as crack type and shooting scene.

[0013] (3) Division and augmentation: The training set, validation set and test set are randomly divided in a ratio of 7:2:1. Targeted data augmentation is performed on the training set, including random rotation, horizontal / vertical flipping, adaptive adjustment of brightness / contrast, addition of Gaussian noise, priority random cropping of crack areas, and elastic deformation (simulating crack morphology differences) to improve the model's generalization ability and anti-interference ability and avoid overfitting.

[0014] To address the characteristics of concrete cracks—namely, elongated shape, low pixel ratio, multiple directions, and multiple scales—a three-layer optimization was performed on the YOLO model's backbone network, feature fusion module, and detection head to construct an architecture of "accurate feature extraction - multi-scale fusion - efficient detection and segmentation."

[0015] (1) Backbone network improvement: The first two 3×3 standard convolutional layers (Conv) of the YOLO backbone network are replaced with the existing windmill-shaped convolution (PConv). At the same time, the C3k2 module in the backbone network is replaced with the lightweight improved module APC3k2. PConv expands the receptive field and strengthens the capture of slender crack features through 4 sets of asymmetric filling branches and multi-directional convolutional layers. APC3k2 replaces the original Bottleneck with APBottleneck, retains the residual connection logic, and only optimizes the feature extraction capability of the dimension reduction convolution to ensure that the number of model parameters remains basically unchanged after the replacement, thus ensuring real-time performance.

[0016] (2) Neck feature fusion optimization: Based on the original P3 (8x downsampling), P4 (16x downsampling), and P5 (32x downsampling) feature fusion system of YOLO, a new P2 (4x downsampling) high-resolution feature fusion branch is added to construct a P2-P3-P4-P5 multi-scale feature fusion architecture; after the P2 features are extracted from the backbone network, they are preprocessed by two layers of 3×3 convolution enhancement, batch normalization (BN) and SiLU activation function to improve feature purity and then spliced ​​and fused with P3 features to avoid high-scale feature noise interference and realize full interaction between fine crack detail features and global semantic features;

[0017] (3) Addition and adaptation of detection head: In conjunction with P2 feature fusion, a new P2 detection head is added on the basis of the original P3-P5 detection heads to form a four-scale detection architecture; based on the statistics of fine crack samples in the exclusive dataset, the anchor frame size adapted to the slender shape is generated by clustering; the detection head adopts the "detection-segmentation" integrated design, and outputs crack boundary boxes, confidence and pixel-level segmentation masks simultaneously, taking into account both positioning accuracy and segmentation effect.

[0018] To address issues such as extreme imbalance between positive and negative samples, location errors in slender cracks, and vanishing segmentation gradients in concrete crack detection, a targeted training strategy is designed:

[0019] (1) Construction of hybrid loss function: The combined loss function of “EIoU-L regression loss + FocalDice segmentation loss” is adopted; EIoU-L reconstructs the aspect ratio penalty term to strengthen the bounding box constraint of slender cracks; FocalDice loss solves the problem of missed detection caused by the low pixel ratio of micro-cracks by introducing a weighting factor.

[0020] (2) Optimizer and training parameter adjustment: AdamW optimizer is selected to suppress overfitting through weight decay; cosine annealing learning rate scheduler is adopted to keep the learning rate stable in the early stage of training to achieve fast convergence, and reduce the learning rate for fine optimization in the later stage; gradient clipping mechanism is introduced to avoid gradient explosion; a phased training method is adopted. In the first stage, the non-improved layers of the backbone network are frozen, and only modules such as PConv and APC3k2 are trained to achieve parameter initialization. In the second stage, the entire network is unfrozen and full fine-tuning is performed to ensure that the model performance converges fully.

[0021] Achieve end-to-end automation of the process from image input to true-scale output:

[0022] (1) Detection and segmentation process: After the image to be detected is preprocessed such as size adjustment and normalization, it is input into the improved model for inference and outputs the crack bounding box, confidence and segmentation mask; duplicate detection boxes are filtered by weighted nonmaximum suppression (WNSM), the segmentation mask is optimized by performing "erosion-dilation" operation, and the crack pixel area and pixel length are calculated.

[0023] (2) Real-scale quantification: Based on the GPS and attitude angle sensor built into the DJI basic drone, the vertical distance from the camera to the crack surface is obtained by subtracting the ground elevation of the crack area from the flight line height and combining the pitch angle and roll angle correction; based on the camera focal length, pixel physical size and vertical distance, the pixel-physical scale conversion coefficient is calculated, and finally the pixel area and length are converted into the real-world crack area, length and average width, providing quantitative data for structural safety assessment.

[0024] This application provides an improved concrete identification scheme, which has the following significant advantages compared to existing technologies:

[0025] (1) The dataset has strong adaptability and excellent generalization ability: The dedicated dataset covers concrete surface images of various scenarios and types, such as road surfaces, bridges, and building walls. After standardization and targeted enhancement, it provides sufficient high-quality data support for model training, enabling the model to adapt to complex engineering detection environments.

[0026] (2) High detection accuracy and low false detection rate: Through the feature extraction optimization of PConv and APC3k2, multi-scale feature fusion and adaptive anchor frame design, combined with EIoU-L and FocalDice loss functions, the problems of insufficient capture of fine cracks, deviation of positioning of long and thin cracks and sample imbalance are effectively solved, and the false detection rate is greatly reduced.

[0027] (3) Integrated detection, segmentation and quantification: Simultaneously output crack location results, pixel-level segmentation mask and real physical scale parameters, solve the defect of existing technology that can only locate but not quantify, and provide comprehensive data support for safety assessment;

[0028] (4) High degree of automation, improved detection efficiency and safety: This invention realizes end-to-end automated detection and segmentation of concrete cracks without manual intervention, which greatly reduces the labor intensity of detection personnel and the safety risks in high-altitude and dangerous scenarios, and significantly reduces the cost of engineering detection.

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Attached Figure Description

[0030] Figure 1 is a general flowchart of the concrete crack detection method based on windmill-shaped convolution improved YOLO provided in the embodiments of this application;

[0031] Figure 2. Architecture diagram of concrete crack detection model based on improved YOLO; Detailed Implementation

[0032] This embodiment discloses a concrete crack detection method based on an improved YOLO, aiming to achieve high-precision, real-time detection of concrete cracks through an integrated process of dataset construction, model structure improvement, training optimization, and detection segmentation. Those skilled in the art can reproduce the technical solution of this invention based on the description of this embodiment.

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0034] Figure 1 is a general flowchart of the concrete crack detection method of the present invention, corresponding to the following 6 core steps:

[0035] S1: Dataset construction steps. A concrete crack sample library is built using multi-source data (public datasets + self-taken photos), which is the basic data support for model training.

[0036] S2: Improve the YOLO model construction steps. Based on the YOLO framework, complete the model structure improvement by replacing the backbone network PConv, fusing P2 features in the Neck layer, and adding a P2 detection head.

[0037] S3: Training optimization step, introducing a hybrid loss function to complete the configuration and optimization of the model's training parameters;

[0038] S4: Crack detection step, which improves the model to detect cracks and outputs the detection box and confidence score;

[0039] S5: Crack segmentation step, based on the mask output by the integrated detection and segmentation head, to complete the semantic segmentation and area calculation of the crack;

[0040] S6: Crack quantization step, which combines camera calibration parameters to convert the segmentation results into real-world physical dimensions.

[0041] Furthermore, to address the issue of insufficient adaptability of existing datasets, this embodiment constructs a custom dataset through "integration of public datasets + supplementation by self-shot footage," as detailed below:

[0042] S101: Collect valid samples from publicly available concrete crack datasets; simultaneously, use high-definition cameras and drones to capture images covering normal lighting, strong backlighting, cloudy and rainy weather, dust coverage, and stain pollution in scenarios such as bridge box girders, bridge piers, tunnel linings, building walls, and water conservancy dams, including samples with horizontal, vertical, mesh, diagonal cracks and no cracks, and integrate them to form the original dataset.

[0043] S102: OpenCV tools are used to remove invalid samples that are blurred, overexposed, or occluded, and finally valid images are obtained; the pixels of all images are uniformly adjusted by resizing, and the pixel values ​​are mapped to the [0,1] interval by normalization; the bounding boxes of the detection task are labeled using the LabelImg tool (VOC format), and the pixel-level masks of the segmentation task are labeled using the LabelMe tool (COCO format).

[0044] S103: Divide the dataset into training, validation, and test sets in a 7:2:1 ratio; For the training set, perform data augmentation based on the Albumentations library: random rotation, horizontal / vertical flipping; brightness adjustment, contrast adjustment; Gaussian noise, random cropping; elastic deformation to simulate crack morphology differences.

[0045] Furthermore, this embodiment uses YOLO as the basic framework and builds an improved model based on PyTorch. The model architecture is shown in Figure 2. Figure 2 clearly marks the core improvements, including the replacement of the backbone network PConv, the APC3k2 module, the P2 feature fusion branch, and the addition of the P2 detection head.

[0046] S201: PConv replacement of the backbone network (corresponding to "backbone network module" on the left side of Figure 2), replacing the first two 3×3 standard convolutional Conv layers of the YOLO backbone network with windmill-shaped convolutions (PConv); at the same time, the original C3k2 module is replaced with a lightweight feature fusion module (APC3k2), and the original module and the improved module are flexibly switched through controllable parameters to ensure compatibility.

[0047] Specifically, the first two layers of the original YOLO backbone network are 3×3 standard convolutional layers Conv (with 64 and 128 output channels respectively, and a stride of 2 for both). The subsequent layers contain multiple C3K2 modules (with 3×3 standard convolutions inside, 128 output channels, and a stride of 1).

[0048] S2011: Furthermore, the Conv module is the basic unit for YOLO to achieve feature dimension adjustment and spatial downsampling. It consists of a two-dimensional convolutional layer, a batch normalization layer, and an activation layer connected in series, and its structure satisfies:

[0049]

[0050] It should be noted that:

[0051] S20111: For the input feature tensor, These are the height, width, and input channels of the input feature map, respectively.

[0052] S20112: For a two-dimensional convolution operation with a square kernel, the parameters satisfy: kernel size Step length ,filling Number of groups No bias;

[0053] S20113: For batch normalization operations;

[0054] S20114: This is the activation function.

[0055] S20115: The Conv module outputs a feature tensor with the following dimensions: , ;

[0056] S2012: The further C3k2 module inherits the topology of the C2f module and implements multi-branch feature fusion, with the following structure:

[0057] It should be noted that:

[0058] S20121: For pre-dimension reduction convolution, the number of input channels is... Down to ;

[0059] S20122: for A series of bottleneck modules, with a fixed kernel size of 2×2, where the input and output are:

[0060]

[0061] S20123: Shortcut branch feature (optionally disabled);

[0062] S20124: This is a channel-level splicing operation;

[0063] S20125: For post-convolutional upscaling, the number of channels after concatenation is restored to [previous value]. ;

[0064] S2013: Furthermore, the PConv windmill convolutional module consists of 4 asymmetric filled branches, 2 types of asymmetric convolutional layers, channel splicing operations, and fused convolutional layers. Its forward propagation logic satisfies:

[0065] It should be noted that:

[0066] S20131: Asymmetric Filled Branches: Design of 4 Sets of Differentiated Filling Parameters

[0067] (k is the kernel reference size, consistent with the kernel size of the original Conv module), for the input tensor Perform zero-filling operations respectively ;

[0068] S20132: Asymmetric convolutional layer: It is a 1×k dimension convolutional layer with a certain number of input channels. Number of output channels Step length With the original The modules are identical, with no extra padding. For a k×1 dimension convolutional layer, the number of input channels is... Number of output channels Step length With the original The modules are identical, with no extra padding.

[0069] S20133: Channel splicing: Splices the 4 branch outputs along the channel dimension, resulting in a channel count of [number missing]. ;

[0070] S20134: To merge convolutional layers, the following is adopted: Convolution kernel, stride

[0071] The number of input / output channels is 1. ;

[0072] S2014: Furthermore, APC3k2 adopts the structure of C3k2, as follows:

[0073]

[0074] It should be noted that:

[0075] S20141: The main modification is to Bottleneck. The original Bottleneck is replaced with APBottleneck, retaining the residual connection logic and channel dilation rate of the original Bottleneck, while only reducing the dimensionality of the convolutional layers. Replaced with 4-way asymmetric padding + windmill convolution branch, the APBottleneck forward propagation logic satisfies:

[0076] It should be noted that:

[0077] S201411: Asymmetric Filled Branches: Design of 4 Sets of Differentiated Filling Parameters To meet the size requirements of lightweight 2×2 convolution kernels;

[0078] S201412: Dimensionality Reduction Convolutional Layer : Convolution kernel size and the original Bottleneck Consistent;

[0079] S201413: Upgraded Convolutional Layer : Convolution kernel size and the original Bottleneck Consistent;

[0080] The asymmetric padding of the windmill-shaped convolution allows the effective receptive field of each branch to break through the physical size limitation of the convolution kernel. The design of multiple z-branches allows the effective receptive field to cover the extended regions in different directions, improving the efficiency of feature extraction for small-sized receptive field branches and enhancing pixel-level micro-cracks. Taking the replacement of the first two receptive fields as an example, when k=3, the receptive field is increased by 177%, while the number of parameters is only increased by 111%.

[0081] S202: P2 feature fusion of the Neck layer (corresponding to the "feature fusion layer module" in the middle of Figure 2), introduces the P2-level high-resolution features (4x downsampling) output by the backbone network into the Neck layer.

[0082] S2021: Specifically, YOLO's Neck module (feature fusion layer) adopts a PANet architecture of "upsampling + feature concatenation + downsampling". The original design only fused the features of three layers of the backbone network output: P3 (8x downsampling), P4 (16x downsampling), and P5 (32x downsampling), lacking the fine-scale P2 (4x downsampling). However, the core features of concrete microcracks (small width, low pixel ratio) are concentrated in high-resolution fine-scale feature maps, which are easily missed due to the lack of fine-grained information. The old neck network output topology logic was:

[0083]

[0084] S2022: This embodiment addresses the shortcomings of the original Neck network by first extracting P2 features (56×56×128) from the backbone network. Feature representation is then enhanced through two 3×3 convolutional layers (256 output channels, stride 1, padding=1). Each convolutional layer is followed by a batch normalization (BN) layer and a SiLU activation function to construct a multi-scale feature fusion architecture of "P2-P3-P4-P5". The improved Neck network output topology with added P2 sampling is as follows:

[0085]

[0086] It should be noted that in the backbone network The corresponding logical features are as follows:

[0087] S20221: , The convolution is a windmill shape, and the input and output are based on the formula structure shown in S2013.

[0088] S20222: , The input and output structures are based on the formula structure shown in S2014.

[0089] S20223:

[0090] S20224: ,in and It is the original structural model.

[0091] S203: A new design for the detection head. Based on the existing P3-P5 detection heads, a new detection head adapted to P2 features is added. Based on the bounding boxes of fine crack samples in the training set, clustering is used to obtain anchor box sizes adapted to the fine crack morphology, improving the detection accuracy of small-scale cracks. The input to the P2 detection head is... The inputs corresponding to P3-P4-P5 are respectively: - - The final segmentation output layer is as follows:

[0092]

[0093] S2031: Furthermore, the features output to F6 after P2 scale fusion are as follows:

[0094]

[0095] It should be noted that the original The logical characteristics are as follows:

[0096] S20321:

[0097] S20322:

[0098] S20323:

[0099] The P2 feature detection head is designed based on the highest resolution features (4x downsampling) output by the backbone network. Its core function is to detect small-scale micro-cracks and narrow cracks: targeting fine-grained targets in concrete cracks that are "small in width (pixel percentage ≤ 1%) and fine in texture," it utilizes... The high-resolution detailed information compensates for the microcracks missed by the original YOLO due to the lack of fine-scale features, thus improving the recall rate of cracks in small targets.

[0100] The model structure modification method in this embodiment achieves core improvements by defining the network structure through a model configuration file. This approach offers high operability and reproducibility. Specifically, the hierarchical structure, module types, and parameter relationships of the model are clearly defined using a YAML configuration file. During model loading, the network is automatically constructed based on the configuration file, ensuring the engineering feasibility of the improvement solution.

[0101] The core functionality lies in defining the module composition of the Backbone network through configuration files, enabling seamless integration of PConv with the existing network. The core function of the configuration file is to precisely define the input sources, module repetition counts, module types, and key parameters (such as the number of output channels and stride) for each layer of the model.

[0102] Furthermore, this embodiment designs a hybrid loss function adapted to the crack detection task, and combines it with transfer learning to complete model training, ensuring that the model accurately learns crack features. Specifically, it includes the following sub-steps:

[0103] S301: The original loss function has poor aspect ratio penalty term adaptation in crack image training: the aspect ratio (w / h) or (h / w) of cracks is usually >10 (elongated type), and the original function's penalty for extreme aspect ratios is insufficient, causing the predicted bounding box to fail to fit the elongated shape of the crack; in the crack detection samples, the pixel proportion of micro-cracks is <1% (background pixel proportion is >99%), and the numerator term of BCELoss is easily dominated by the background, causing the model to tend to predict "full background" to reduce loss, resulting in complete missed detection of micro-cracks. The final solution adopts regression loss ( The formula for Focal Dice Loss is:

[0104]

[0105] in: (return), (segmentation);

[0106] S3011: The main issue addressed was the aspect ratio deviation of slender cracks. Based on EIoU-based "width-height independent penalty" reconstruction, it focused on the extreme ratio of the crack's "long side / short side," as shown in the following formula:

[0107]

[0108] It should be noted that:

[0109] S30111: These are the width and height of the smallest bounding box, respectively.

[0110] S30112: Penalty coefficient The lower the value (the greater the regression bias), the higher the penalty coefficient, thus strengthening the constraint on the slender box with large bias.

[0111] S30112: In formula These are "the length of the long side of the actual slender crack frame" and "the length of the long side of the predicted crack frame," respectively. These are respectively "the short side length of the actual crack frame" and "the short side length of the predicted crack frame"—this application replaces the original "general width and height dimensions" of EIoU with "the long / short side dimensions of the slender crack", focusing on the extreme ratio characteristics of the crack's "long side / short side".

[0112] S30113: Compared with the original EIoU's "general width and height independent penalty", the core improvement of EIoU-L is "scenario-specific directional constraint": The original EIoU's constraint on width and height is a general constraint without difference, while EIoU-L strengthens the shape penalty for "extremely fine and long cracks (long side / short side ratio > 1:5)" through "directional splitting of long / short sides", which solves the problem that ordinary EIoU is insufficient in constraining such extreme proportion targets.

[0113] S3012: In DiceLoss, a Focal mechanism is introduced to weight low-probability micro-crack pixels, solving the problem of extreme sample imbalance. The formula is as follows:

[0114]

[0115] Among them, the weighting factor is: ,in Predict the probability that a pixel is a crack. To add weighting factors only to the numerator while retaining the core structure of DiceLoss, the model predicts the probability of micro-crack pixels. At lower levels (i.e., the model tends to ignore these pixels). This will significantly amplify the loss weights of these micro-cracked pixels; while for background pixels, which account for a very large proportion, the predicted probability will be significantly reduced. High, This will reduce the weighting of the background pixels.

[0116] S302: Employs a phased training approach, combining transfer learning to improve training efficiency and model performance. The specific training process is as follows:

[0117] S30201: The first stage is initial training, using pre-trained weights from YOLO-seg on the COCO dataset, which are then loaded into the improved model. During training, the first 10 layers of the Backbone network (excluding the two PConv layers, which form the basic structure) are frozen, and only PConv and subsequent optimization modules are trained. The goal of this stage is to enable the PConv module to quickly adapt to the crack features and achieve initial convergence.

[0118] S30202: The second stage is full fine-tuning, where all network layers of the model are unfrozen, and the overall parameters are optimized. Training parameters are adjusted, and the learning rate and training epochs are increased. This stage focuses on optimizing the convergence effect of the hybrid loss function, guiding the model to focus on fine crack features through scale weights, and finally achieving stable convergence of the validation set loss, with the model performance reaching the expected target.

[0119] Furthermore, inference is performed using the trained optimal model to accurately output the detection bounding box (location information) and pixel-level segmentation mask (region information) of concrete cracks. This includes the following sub-steps:

[0120] S401: Image preprocessing; Receive raw images captured by UAV / camera, supporting single or batch image input; Adjust the input image to match the training sample size using OpenCV's resize function, employing bilinear interpolation; Perform min-max normalization in S103, mapping pixel values ​​to [0,1] and converting them to PyTorch tensor format, adding batch dimension; Move the tensor to the GPU, or use the CPU if no GPU is available.

[0121] S402: Model inference execution, setting confidence thresholds to filter low-confidence prediction boxes, and using IOU thresholds for subsequent NMS filtering; calling the model.predict() function, inputting the preprocessed image tensor, the model extracts features through the Backbone, fuses features through the Neck, and outputs results through the four-scale detection head, obtaining the bounding box (x1, y1, x2, y2), confidence, category (all cracks), and pixel-level segmentation mask of the crack;

[0122] Furthermore, the semantic segmentation optimization of the crack is completed by obtaining the segmentation mask output by S4, and the image pixel area of ​​the crack is calculated at the same time, providing a basis for subsequent real-scale conversion. Specifically, it includes the following sub-steps:

[0123] S501: Weighted Non-Maximum Suppression (WNSM) filters duplicate detection boxes. It calculates the IOU between all detection boxes. When the IOU is ≥ 0.45, it is determined to be a duplicate box. For the set of duplicate boxes, the detection box with the highest confidence is retained. The confidence of the remaining duplicate boxes is weighted and accumulated into the retained boxes (weight = IOU value) to increase the target confidence. Detection boxes with confidence < 0.5 are filtered out, and finally the set of valid detection boxes is retained.

[0124] S502: Segmentation mask optimization. Based on the boundaries of the effective detection boxes, the segmentation mask is cropped, retaining the mask area within the detection boxes and removing external background noise. Using 3×3 rectangular structuring elements, "erosion-dilation" operations (once each) are performed on the cropped mask to eliminate small noise points and fill tiny holes in the mask. The area of ​​the optimized mask is calculated. If the area is smaller than the minimum pixel area of ​​the fine crack, it is judged as a noise mask and is discarded. Crack image area calculation: Based on the optimized effective segmentation mask, the pixel area S_pixel of the crack is calculated using the OpenCV contourArea function. At the same time, the crack contour is extracted, and the pixel length L_pixel is calculated using the arcLength function (parameter closed=False) to provide supplementary parameters for subsequent scale conversion.

[0125] Furthermore, this embodiment is applied to the drone inspection scenario, acquiring flight parameters and camera shooting parameters, and combining the crack image pixel area calculated by S5 to complete the conversion from pixel area to the real-world physical scale. The specific steps include the following:

[0126] S601: Preliminary preparations: First, check the ground elevation information of the area where the crack is located. Use map software (such as Amap or Google Earth) to obtain the ground elevation data of the crack location as a benchmark for subsequent distance calculations. Second, confirm the core camera parameters of the drone being used (such as Mavic 3 or Air 2S). Look up information such as the camera's fixed focal length and sensor pixel physical size from publicly available data from the drone manufacturer to ensure the accuracy of the converted basic parameters. Third, ensure that the drone camera is installed horizontally with the body; no additional modifications are required, and the original configuration of the device can be used.

[0127] S602: Inspection and Parameter Acquisition. The drone flies along the preset inspection route. After discovering a concrete crack, it hovers near the crack area and takes a clear image of the crack. At the same time, the drone's software (such as DJIFly or GSPro) automatically records the following data: drone flight line altitude, i.e., the altitude of the drone relative to the ground at the takeoff point; drone attitude angles, including pitch angle (forward and backward tilt angle) and roll angle (left and right tilt angle); and the time of capture, ensuring that the image and the acquisition time of each parameter are consistent to avoid data misalignment.

[0128] S603: Calculate the vertical height of the UAV relative to the ground where the crack is located. Subtract the previously queried ground elevation of the crack area from the flight line height to obtain the initial vertical height of the UAV relative to the ground where the crack is located. Combine the attitude angle for distance correction. Adjust the initial vertical height using the pitch and roll angles of the UAV to eliminate the influence of slight tilt of the fuselage on the distance calculation, and obtain the final vertical distance from the camera to the crack surface, ensuring that this distance is the effective distance between the camera optical axis and the crack surface in the vertical direction.

[0129] S604: Based on the camera focal length, pixel physical size, and final vertical distance, the conversion factor between pixels and the real physical scale is calculated. This factor represents the real-world length corresponding to one pixel in the image. Combining the crack pixel area, pixel length, and other data obtained from the crack segmentation step above, the real scale conversion is completed through the conversion factor. The real area, real length, and real average width of the crack are calculated respectively, providing quantitative data support for the safety assessment of concrete structures.

[0130] It should be noted that the specific expressions, connecting words and symbols used in this specification are only used to clearly clarify the logical relationship of the technical solution and simplify the expression, and have no limiting meaning.

[0131] The specific terms, symbols, and abbreviations used in this specification have the following meanings, and are only used to clearly describe the technical solutions of this application. They do not represent the sole definition or limitation of general technical terms. Specifically: "PConv" (Pinwheel-shaped Convolution): refers to the existing feature extraction module used in this application, whose structure consists of 4 asymmetric filled branches, 2 types of asymmetric convolutional layers, channel splicing operations, and fusion convolutional layers. It is used to enhance the multi-directional feature capture capability of concrete cracks and does not represent the general name of other convolutional modules; "APC3k2": refers to the lightweight feature fusion module in this application, which is improved based on the C3k2 module. It is obtained by replacing the original Bottleneck with APBottleneck, retaining the residual connection logic, and only optimizing the feature extraction capability of dimensionality reduction convolution. It does not represent other modules in the prior art; "" refers specifically to the reconstructed regression loss function in this application, which adapts to the bounding box regression requirements of slender cracks by optimizing the aspect ratio penalty term, and does not represent a general variant of the EIoU loss function; "Focal Dice Loss" refers specifically to the segmentation loss function that introduces the Focal mechanism in this application, which is used to solve the problem of extreme imbalance between positive and negative samples in concrete crack detection, and does not represent an improved form of other Dice loss functions;

[0132] Symbols involved in the formula (such as) , , , , (etc.): These are parameter definitions specific to the technical solutions of this application. Their numerical ranges and physical meanings have been clearly defined in the specific implementation methods and do not represent the default meanings of general mathematical or engineering symbols.

[0133] In the description of the embodiments of this application, ordinal numbers such as "firstly", "secondly", "first", and "second" are only used to distinguish the order of steps, module classification, or parameter hierarchy of the technical solution. They do not represent the unique order of execution of steps or the priority of modules. Those skilled in the art can reasonably adjust the order of steps or make equivalent substitutions for modules and parameters without departing from the core concept of the invention.

[0134] In the description of the embodiments of this application, conjunctions such as "further", "specific", and "need to be explained" are used to supplement or refine the preceding technical features, aiming to make the technical solution clearer and more reproducible. They do not indicate that the preceding features are necessary technical features while the following features are not. All explicitly recorded technical features are key components for achieving the purpose of the invention.

[0135] In the description of the embodiments of this application, the symbol " / " represents the logical relationship of "or" and is used to simplify the expression of parallel content. For example, "horizontal / vertical flip" means "horizontal flip or vertical flip", and "w / h or h / w" means "w / h or h / w". It does not represent any other special logical relationship.

[0136] The above description is merely a specific embodiment of this application and is not intended to limit the scope of protection of this invention. Equivalent modifications, substitutions, or extensions that can be made by those skilled in the art based on the technical solutions disclosed in this application without inventive effort are all within the scope of protection of this invention. The scope of protection of this application is determined by the claims.

Claims

1. A method and system for detecting concrete cracks based on improved YOLO using windmill-shaped convolution, characterized in that, Includes the following steps: S1: Constructing a concrete crack defect sample library: A concrete crack defect sample library is constructed using multi-source data (public concrete crack dataset + self-captured images); S2: Constructing a multi-scale concrete defect detection network: Based on the YOLO algorithm, the first two convolutional layers of the backbone network and the convolutions in the C3K2 module are replaced with existing windmill-shaped convolutions (PConv), P2-level high-scale features are integrated into the neck network, and a new P2-level detection head is added to obtain a multi-scale defect detection network adapted for concrete crack detection; S3: Introducing a hybrid loss function: In the multi-scale concrete defect detection network, the "EloU-L + FocalDice" hybrid loss function is introduced; S4: Implementing crack detection and segmentation: Through the multi-scale concrete defect detection network, the integrated detection and segmentation head is used to achieve target detection of cracks, generating target detection boxes, confidence scores, and pixel-level crack segmentation masks. S5: Calculate the crack pixel area: Perform semantic segmentation based on the segmentation mask output by the integrated detection and segmentation head to obtain the crack code, and calculate the crack area based on the size of the encoded image. S6: Calculate the true physical size of the crack: Based on the pixel-to-actual size conversion relationship calibrated by the camera, combined with the shooting position parameters, calculate the physical size of the crack area in the real world.

2. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The multi-source data mentioned in step S1 includes publicly available concrete crack datasets and self-taken concrete crack images, and all images are uniformly adjusted to the 448×448 specification.

3. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The improvements to the multi-scale concrete defect detection network described in step S2 include: (1) replacing the first two convolutional layers of the original YOLO algorithm backbone network with windmill-shaped convolutions (PConv); (2) replacing the convolutions in the C3K2 module of the backbone network with windmill-shaped convolutions (PConv); (3) incorporating P2-level high-scale features into the neck network; and (4) adding a detection head adapted to P2-level features on the basis of the original detection head.

4. The method for detecting and quantifying concrete cracks according to claim 3, characterized in that, The windmill-shaped convolution (PConv) is an existing feature extraction module. This invention only applies it to the improvement of the YOLO algorithm in the concrete crack detection scenario.

5. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The "EloU-L+FocalDice" hybrid loss function mentioned in step S3 combines the EloU-L loss with the FocalDice loss to balance the localization accuracy of crack detection with the matching degree of segmented pixels.

6. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The integrated detection and segmentation head described in step S4 simultaneously achieves target detection of cracks (outputting detection box and confidence score) and pixel-level segmentation (outputting segmentation mask).

7. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The crack area in step S5 is calculated by converting the crack pixel area based on the ratio of the number of pixels in the segmentation mask to the image size.

8. The method for detecting and quantifying concrete cracks according to claim 1, characterized in that, The shooting position parameters mentioned in step S6 include the flying line height and attitude angle of the shooting device, and the camera calibration parameters include the camera focal length and pixel physical size.

Images