Cross-scene concrete surface crack unsupervised identification method based on transfer learning

Abstract

The invention relates to a cross-scene concrete surface crack unsupervised identification method based on transfer learning, and the method comprises the steps: collecting concrete surface crack images in different scenes, and constructing a data set; binarizing the image based on an Otsu algorithm, removing noise pixels, and constructing a data set with a label and a data set without a label; embedding a residual structure in the cyclic generative adversarial network, constructing a feature migration model, and inputting a labeled data set and an unlabeled data set for adversarial training to generate a crack image with consistent feature distribution; an SE attention mechanism is embedded in the U-Net, a semantic segmentation model is constructed, the original image and the generated image are input for training, and the trained semantic segmentation model is obtained; and embedding the trained feature migration model and the semantic segmentation model into a sliding window, collecting a to-be-identified concrete surface image, and inputting and outputting crack edge and geometric information. The semantic segmentation precision of the model in different appearance feature differences can be improved.

Description

Technical Field

[0001] This invention relates to the fields of generative artificial neural networks, transfer learning, and identification of surface defects in concrete structures, and particularly to an unsupervised method for identifying cross-scenario cracks in concrete surfaces based on transfer learning. Background Technology

[0002] Cracks are an unavoidable defect in concrete structures during long-term service. If not detected and repaired in time, crack growth can lead to serious safety accidents. Supervised training models require a high degree of matching of appearance feature distributions between training and detection images. Pre-trained deep learning models can achieve high crack detection performance. However, in practical engineering applications, due to the diversity of concrete materials and the complex influence of the external environment, the appearance of crack images acquired for new tasks is often difficult to maintain consistency with the model training set, thus failing to satisfy the independent and identically distributed condition.

[0003] To address the domain discrepancy issue between cracked images, a Cycle GAN is employed for feature transfer. Furthermore, three feature transfer strategies are proposed based on different transfer directions: feature transfer from labeled to unlabeled datasets, feature transfer from unlabeled to labeled datasets, and a method that maps the two domains to a shared auxiliary feature space. Since these different strategies generate multiple datasets, which are classified together with the original dataset, including two labeled datasets and three unlabeled datasets, and these datasets have varying degrees of overlap in the abstract feature space, this approach is crucial.

[0004] Deep learning-based semantic segmentation techniques have shown great potential for pixel-level crack detection by utilizing geometric features such as crack width and direction. However, domain bias, overfitting, and uncontrollable feature mapping between labeled and unlabeled datasets can subtly alter these morphological features, thus affecting the robustness of the model. To address this issue, this invention proposes an unsupervised method for cross-scene concrete surface crack recognition based on transfer learning. Summary of the Invention

[0005] The purpose of this invention is to provide an unsupervised method for identifying cracks on concrete surfaces across different scenarios based on transfer learning, thereby improving the semantic segmentation accuracy of the model under different appearance feature differences and ensuring accurate classification of crack edge pixels at different scales.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] An unsupervised method for cross-scene concrete surface crack identification based on transfer learning includes:

[0008] Collect images of concrete surface cracks in different scenarios and construct a dataset;

[0009] Based on the Otsu algorithm, the images in the dataset are binarized and noisy pixels are removed to construct labeled and unlabeled datasets;

[0010] A residual structure is embedded in a recurrent generative adversarial network to construct a feature transfer model. The model is then trained adversarially using the labeled and unlabeled datasets to generate crack images with consistent feature distributions.

[0011] The SE attention mechanism is embedded in U-Net to build a semantic segmentation model. The original image and the generated image are input for training to obtain the trained semantic segmentation model.

[0012] The trained feature transfer model and semantic segmentation model are embedded in a sliding window, and images of the concrete surface to be identified are collected and input, outputting crack edges and geometric information.

[0013] Optionally, the feature transfer model includes: two sets of convolutional neural network generators based on residual structures, and two sets of discriminators with fully connected structures.

[0014] Optionally, the feature transfer model is trained adversarially by calculating a loss, which includes adversarial loss, cycle consistency loss, and identity recognition loss.

[0015] Optionally, the loss function for the adversarial loss is:

[0016] ;

[0017] ;

[0018] in, and Generators and generator The losses in the fight, The feature distributions of the source domain and the target domain are respectively. These are individual image samples from the source domain and the target domain, respectively. To generate image samples from the source domain , Let be the sample probability distributions of the source and target domains; For the generated image samples that conform to the target domain , For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Recognized as real The average probability of the domain sample For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Recognized as real The average probability of a domain sample.

[0019] Optionally, the loss function for the cycle consistency loss is:

[0020] ;

[0021] in, For cycle consistency loss, From Constraints on a domain looping through itself. From Starting from the domain and returning The process of cyclic transformation of the domain.

[0022] Optionally, the loss function for the identity recognition loss is:

[0023] ;

[0024] in, For loss of identity verification, The goal is to ensure that samples in the target domain maintain their consistency after bidirectional transformation. The source domain samples should maintain their consistency after bidirectional transformation. These are samples from the source domain and the target domain, respectively. These are the feature probability distributions of real samples in the target domain and source domain, respectively.

[0025] Optionally, the loss function for the overall loss of the feature transfer model is:

[0026] ;

[0027] in, For the overall loss, and For the adversarial loss of the two generators, The weights for the cycle consistency loss, Weighting for identity loss.

[0028] Optionally, the semantic segmentation model includes an encoder with an embedded SE attention mechanism and a decoder.

[0029] The beneficial effects of this invention are as follows:

[0030] This invention addresses the differences in the distribution of appearance features in images during cross-scene crack detection of concrete structures. It utilizes a recurrent adversarial neural network to achieve feature transfer of images, constructs a feature consistency environment required for the pre-trained model, and suppresses changes in the size of crack sets caused by transfer by matching various combinations of generated datasets and real datasets. Thus, it provides an optimal unsupervised crack detection method for high-precision crack detection.

[0031] This invention utilizes a strategy combining deep learning, transfer learning, and a sliding window procedure to construct and train an unsupervised method for cross-scene crack recognition of concrete surfaces based on transfer learning. It can be used not only for high-precision detection of small-sized crack images but also for scanning large-sized, high-resolution images using a sliding window procedure. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of the unsupervised method for cross-scene concrete surface crack recognition based on transfer learning, according to an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of the residual structure of the convolutional neural network according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the feature transfer model structure according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of a cyclic generative adversarial network structure according to an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the semantic segmentation model structure according to an embodiment of the present invention;

[0038] Figure 6 The image shows a comparison of the geometric accuracy of the segmentation cracks through sample migration in five groups, which is a verification of an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] This embodiment proposes an unsupervised framework that integrates a hybrid transfer learning strategy with a feature domain alignment mechanism. This framework employs feature-based transfer learning driven by a recurrent generative adversarial network (RGAN) to explore three transfer directions: from labeled to unlabeled datasets, from unlabeled to labeled datasets, and bidirectional mapping to a shared auxiliary feature space. The datasets generated by these strategies have controllable source-target domain overlap, enabling systematic analysis of feature domain alignment. Furthermore, five combinations of sample transfers are designed to evaluate feature alignment between RGAN-generated datasets and real datasets. A key finding is that transferring features from unlabeled to labeled datasets induces negative transfer, amplifying fine cracks and reducing geometric consistency. Conversely, transfer from labeled to unlabeled data preserves crack width and orientation, making it more suitable for morphology-sensitive tasks. Bidirectional mapping to a shared auxiliary space further reinforces the domain-invariant feature distribution. When using a squeeze-excitement (SE) attention-embedded U-Net, this approach improves the detection of irregular and fine cracks by emphasizing channel feature relationships and minimizing hierarchical feature loss.

[0042] Based on the above technical content, this embodiment provides an unsupervised method for cross-scene concrete surface crack identification based on transfer learning, including:

[0043] Collect images of concrete surface cracks in different scenarios and construct a dataset;

[0044] Based on the Otsu algorithm, the images in the dataset are binarized and noisy pixels are removed to construct labeled and unlabeled datasets;

[0045] A residual structure is embedded in a recurrent generative adversarial network to construct a feature transfer model. The model is then trained adversarially using the labeled and unlabeled datasets to generate crack images with consistent feature distributions.

[0046] The SE attention mechanism is embedded in U-Net to build a semantic segmentation model. The original image and the generated image are input for training to obtain the trained semantic segmentation model.

[0047] The trained feature transfer model and semantic segmentation model are embedded in a sliding window, and images of the concrete surface to be identified are collected and input, outputting the geometric and location information of the cracks.

[0048] Specifically, such as Figure 1 As shown, it includes the following steps:

[0049] Step 1: Collect concrete crack datasets under different scenarios, including concrete crack images with different materials, lighting, structures and life cycles, and the geometric features of the cracks also have obvious differences, specifically in the color of the image and the thickness of the crack. Classify and construct datasets for different scenarios.

[0050] Step 2: Using the Otsu algorithm, the maximum variance of inter-class pixel values ​​between foreground and background pixels in the image is statistically analyzed to adaptively determine the optimal threshold, thereby obtaining a binarized image of the crack image. Noise pixels in the binarized image are then manually removed to obtain the labels for the training set.

[0051] The Otsu algorithm and a manual denoising method for creating crack image labels: a rule-based automated algorithm and a small amount of manual image annotation.

[0052] Step 3: Construct a Cycle Generative Adversarial Network (CGN), where the generator embeds residual modules, such as... Figure 2 As shown, this avoids gradient vanishing and improves the model's ability to contain information. Adversarial loss, cycle consistency loss, and identity recognition loss are constructed, and the model is trained using the sum of the three losses. When the geometric dimensions of the cracks in the source domain and the target domain are similar, crack images with similar appearances are generated.

[0053] like Figure 3 As shown, the feature transfer model includes three directions of feature transfer: transfer from labeled datasets to unlabeled datasets, transfer from unlabeled datasets to labeled datasets, and transfer from labeled and unlabeled datasets to a mixed feature domain.

[0054] Sample transfer uses combinations of generated and original images to establish consistency in feature distribution. Five combinations are established: original labeled dataset with generated unlabeled dataset, generated labeled dataset with original unlabeled dataset, generated labeled dataset with generated unlabeled dataset, generated labeled dataset with shared auxiliary feature dataset, and original labeled dataset with shared auxiliary feature dataset. This ensures the geometric and edge accuracy of crack segmentation. Supervised training of the U-Net encoding / decoding structure semantically segments crack images to obtain crack edge and geometric information.

[0055] like Figure 4 As shown, considering the over-transfer problem in the feature transfer process, a recurrent adversarial neural network is constructed to ensure accurate preservation of crack details in the transferred image. This network includes two sets of convolutional neural network generators based on residual structures and two sets of discriminators with fully connected structures. The total loss function for optimizing the network parameters includes three important losses: adversarial loss, recurrent consistency loss, and identity recognition loss.

[0056] The calculated loss includes: constructing a recurrent GAN by connecting the inverse generator network (generator F) and the discriminator D. X To constrain the generator network (generator G) and the discriminator D Y Two sets of generator models are trained simultaneously.

[0057] Specifically, an adversarial loss function is constructed to generate synthetic target instances that conform to the feature distribution of the target domain. The construction of this loss function is as follows:

[0058] ;

[0059] ;

[0060] in, and Generators and generator The losses in the fight, The feature distributions of the source domain and the target domain are respectively. These are individual image samples from the source domain and the target domain, respectively. To generate image samples from the source domain , Let be the sample probability distribution of the source and target domains; For the generated image samples that conform to the target domain . For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Recognized as real The average probability of the domain sample For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Recognized as real The average probability of a domain sample.

[0061] A cycle consistency loss is constructed to ensure that the output image of the inverse generator network F matches the original source image. The loss function is constructed as follows:

[0062] ;

[0063] in, For cycle consistency loss, From Constraints on domain-to-self loops ensure the reversibility of transformations and semantic consistency. From Starting from the domain and returning The cyclic transformation process of the domain ensures Domain samples also maintain semantic consistency and reversibility in cross-domain transformation.

[0064] Constructing an identity recognition loss function ensures color consistency between input and output images and plays a crucial role in controlling feature transfer. The loss function is constructed as follows:

[0065] ;

[0066] in, For loss of identity verification, The goal is to ensure that samples in the target domain maintain their consistency after bidirectional transformation. The source domain samples should maintain their consistency after bidirectional transformation to ensure that the generator retains the geometric information of cracks in the original image when processing source domain samples. For samples from the source domain and the target domain, respectively, These are the feature probability distributions of real samples in the target domain and source domain, respectively.

[0067] The overall recurrent GAN loss combines these three loss functions, and the construction of this loss function is as follows:

[0068] ;

[0069] in, For the overall loss, and For the adversarial loss of the two generators, As the weight for the cycle consistency loss, this method sets it to 10 to ensure that the core semantic information of the crack image is not lost during feature transfer. As the weight for identity loss, this embodiment sets it to 3 to balance the conversion process of target domain features and crack features in the original image, and to avoid the generator focusing too much on the distribution of target domain features and destroying the original content of the crack image.

[0070] Step 4: Build the Squeeze Excitation U-Net (SE U-Net) model to improve and enhance the crack segmentation accuracy of the model. It is designed to capture fine-grained details in crack images and can ensure reliable and robust semantic segmentation results.

[0071] Figure 5To construct the semantic segmentation model structure, the encoder part is equipped with squeezing and excitation attention mechanisms. This structure can improve the information loss caused by pooling layers and fixed-size convolutional kernels in traditional convolutional neural networks. Specifically, the crack edge information after semantic segmentation becomes blurred, and small cracks cannot even be detected in the segmented image. Adding squeezing and excitation attention mechanisms can increase the weight of pixels in the crack region of the model, thereby improving the edge accuracy of crack segmentation.

[0072] Step 5: Embed the pre-trained recurrent adversarial neural network and the SEU-Net model into the sliding window module to identify large image cracks.

[0073] Image segmentation tasks introduce the Intersection of Union (IoU) metric to evaluate the degree of overlap between the segmented region and the target region in the ground truth label. Since model evaluation typically uses multiple images or multi-class classification results, the mean IoU (mIoU) is also a commonly used evaluation metric, as shown in the formula:

[0074] ;

[0075] Where FN represents a real crack pixel that is predicted as a non-crack pixel, FP represents a pixel predicted as a crack region that is predicted as a non-crack pixel, and TP represents a pixel predicted as a crack region pixel.

[0076] Table 1 shows the contribution of individual components to the overall performance in the ablation study using mIoU values ​​(crack images only). Components include: U: the original U-net; SE: Squeeze-and-Excitation attention mechanism; U2L: feature space transfer from unlabeled dataset to labeled dataset; L2U: feature space transfer from labeled dataset to unlabeled dataset; and SFS: feature transfer from labeled and unlabeled datasets to a shared auxiliary feature space. The crack semantic segmentation results for each combination in Table 1 are shown below. Figure 6 As shown, without using transfer learning techniques, SE U-Net is trained on the ABCD dataset and then the pre-trained model is tested on all four datasets. The results show that semantic segmentation performance degrades across different datasets, especially on the CD dataset which contains only images of fine cracks, where the mIoU value is the lowest. Comparing the experimental results in the first and third rows, the combined-1 strategy involving U2L significantly improves the average mIoU. This improvement is particularly pronounced on the fine crack CD dataset, exceeding 27%. However, as... Figure 6As shown, the width of the fine cracks widens significantly after the transfer, indicating that this improvement may lead to negative transfer results. Further comparison of the first and fifth rows shows that the L2U transfer strategy has a limited impact on mIoU. Compared to the combined-1 dataset, the mIoU values ​​of the CD dataset return to a more reasonable level. Figure 6 As shown, the Combination-2 strategy preserves the crack width features in the original image. Therefore, to maintain the geometric consistency of crack width during transfer learning, transferring from the labeled dataset to the unlabeled dataset might be a better choice. Secondly, Combination-3 exhibits significant negative transfer due to the large domain shift between the training and test sets. Both Combination-4 and Combination-5 reconstruct the test set into a shared auxiliary feature space. However, since Combination-5 lacks U2L, differences still exist between the test and training sets, leading to negative transfer. In contrast, Combination-4 employs independent and identically distributed data structures, effectively improving the average mIoU.

[0077] Furthermore, the contribution of SE attention to performance was verified, and it was integrated into various transfer learning strategies. Comparing the first and second rows of Table 1, it can be found that SE attention improves the performance of the pre-trained model and significantly reduces performance differences on all datasets. This indicates that it can improve semantic segmentation and network stability by establishing weight dependencies between adaptive convolutional feature channels. Comparing the third and fourth rows of Combination-1, it was found that SE attention does not effectively alleviate the negative transfer caused by U2L. Notably, in Combination-2, integrating the SE attention mechanism leads to a slight decrease in the performance of CCIC 1. However, this is the only observed negative fluctuation caused by the SE attention mechanism, indicating that unexpected performance degradation may occur under certain conditions. In addition, from Combination-3 to Combination-5, integrating SE with transfer learning does not change the crack geometry and further improves mIoU, especially in Combination-4.

[0078] Combinations-1,-2, and-4 satisfy the independent and identically distributed (IOD) condition, thus improving performance. Conversely, combinations-3 and-5 do not meet this condition, leading to partial negative transfer. SE attention typically improves semantic segmentation performance and stabilizes network training. When combined with transfer learning, it often brings additional improvements. However, in practical applications, negative transfer should be carefully evaluated, and it should be acknowledged that slight performance degradation may occur in rare cases. It is important to note that in cases where cross-dataset segmentation U2L is unavoidable, it is recommended to transfer features from labeled and unlabeled datasets to a shared auxiliary feature space to mitigate the negative transfer caused by U2L.

[0079] Table 1

[0080]

[0081] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for unsupervised recognition of concrete surface cracks across scenes based on transfer learning, characterized in that, The application relates to a method for recognizing concrete surface cracks. Collecting concrete surface crack images in different scenes to construct a data set; Based on the Otsu algorithm, the images in the data set are binarized, and noise pixels are removed to construct a labeled data set and an unlabeled data set; A residual structure is embedded in a cycle generative adversarial network to construct a feature transfer model, and the labeled data set and the unlabeled data set are input into the feature transfer model for adversarial training to generate crack images with consistent feature distribution; An SE attention mechanism is embedded in a U-Net to construct a semantic segmentation model, and the original image and the generated image are input into the semantic segmentation model for training to obtain a trained semantic segmentation model; The trained feature transfer model and the semantic segmentation model are embedded in a sliding window, a concrete surface image to be recognized is collected and input, and crack edges and geometric information are output.

2. The method of claim 1, wherein, The feature transfer model comprises two groups of convolutional neural network generators based on residual structures and two groups of discriminators with full connection structures.

3. The method of claim 2, wherein the method is based on transfer learning for unsupervised identification of concrete surface cracks across scenes. The feature transfer model is subjected to adversarial training by calculating a loss, and the loss comprises an adversarial loss, a cycle consistency loss and an identity recognition loss.

4. The method of claim 3, wherein, The loss function of the adversarial loss is: ； ； in, and Generators and generator The losses in the fight, The feature distributions of the source domain and the target domain are respectively. These are individual image samples from the source domain and the target domain, respectively. To generate image samples from the source domain , Let be the sample probability distributions of the source and target domains; For the generated image samples that conform to the target domain , For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Identified as real The average probability of the domain sample For discriminator To address the loss on real samples, the discriminator's ability to identify samples is quantified. For generator Generating fake image discriminators Identified as real The average probability of a domain sample.

5. The method of claim 4, wherein, The loss function of the cycle consistency loss is: ； wherein, is a cycle-consistency loss, is a constraint from domain to itself cycle, is a cycle transformation process from domain back to domain.

6. The method of claim 5, wherein the method is based on transfer learning for unsupervised identification of concrete surface cracks across scenes. The loss function of the identity recognition loss is: ； wherein, is an identity recognition loss, is a consistency constraint that the target domain sample should keep after being converted by the bidirectional conversion, is a consistency constraint that the source domain sample should keep after being converted by the bidirectional conversion, are respectively samples from the source domain and the target domain, are respectively feature probability distributions of real samples of the target domain and the source domain.

7. The method of claim 6, wherein the method is based on transfer learning for unsupervised identification of concrete surface cracks across scenes. The loss function of the overall loss of the feature transfer model is: ； wherein, is the overall loss, is the weight of the cycle consistency loss, is the weight of the identity loss.

8. The method of claim 1, wherein, The semantic segmentation model comprises an encoder with an embedded SE attention mechanism and a decoder.

Images