A method for detecting apparent defects of lightweight concrete bridges based on improved Segformer

Abstract

The present invention relates to the technical field of concrete bridge apparent defect detection, and in particular to a lightweight concrete bridge apparent defect detection method based on an improved Segformer. The steps are as follows: S1: Constructing a dataset of concrete bridge apparent defects, selecting concrete bridge images, and annotating areas where concrete bridge apparent defects are located; S2: Enhancing the concrete bridge dataset and dividing it into a training set and a validation set in an 8:2 ratio; S3: Establishing an improved Segformer lightweight model, selecting MiTB0 as the model's encoder, and introducing an SFM module and a PSA self-attention mechanism module into the decoder. The present invention provides a lightweight concrete bridge apparent defect detection method based on an improved Segformer, which can exhibit strong robustness and high accuracy in concrete bridge apparent defect detection, and can achieve efficient and rapid detection of bridge apparent defects with a relatively small number of parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of concrete bridge apparent defect detection, and in particular to a lightweight concrete bridge apparent defect detection method based on an improved Segformer. Background Art

[0002] With the continuous development of my country's infrastructure and transportation, the scale of concrete bridge construction has also continued to expand. By the end of 2021, the total number of highway bridges in my country reached 961,100, with a total length of 73.8021 million meters. Among them, there were 7,417 extra-large bridges and 134,500 large bridges. However, over time, concrete bridges may develop a certain degree of damage and defects both externally and internally. These damages may lead to safety accidents such as concrete bridge collapse. Therefore, the inspection and assessment of the health of concrete bridges is of vital importance. This can promptly detect damage and defects in concrete bridges and ensure the safe operation of highway bridges.

[0003] Detecting surface defects in concrete bridges is a crucial step in concrete bridge inspections, including testing for damage, exposed rebar, and water erosion. Damaged concrete bridges can cause further concrete fallout, potentially injuring pedestrians or damaging vehicles. Furthermore, further concrete loss from damaged bridges can lead to atmospheric corrosion of rebar, compromising the structural performance of the bridge. Long-term water erosion can damage the bridge's structural materials, disrupting its bearing capacity and shortening its service life. Therefore, testing for damage, exposed rebar, and water erosion is essential.

[0004] The traditional inspection method for concrete bridges is mainly based on manual inspection, using scaffolding or bridge inspection vehicles to send inspectors into the inspection area. However, this method has disadvantages such as strong subjectivity, dangerous working environment, large workload, and low efficiency, and has gradually failed to meet people's needs. With the development of electronic technology, bridge surface inspection can be achieved through technologies such as drones and wall-climbing robots, but manual damage identification of photos or videos collected by the machine is still required, and the efficiency is still low. In recent years, the rapid development of computer vision technology has brought new solutions to the detection of surface damage of concrete bridges. At present, computer vision-based methods are widely used in surface damage detection, mainly including surface damage classification, surface damage target detection, and surface damage semantic segmentation;

[0005] The surface damage classification method can only determine the damage type in the bridge image, but cannot locate the damage location. The surface damage target detection method can identify and locate bridge damage in bridge images. Compared with the surface damage classification method, its method can not only determine the damage type but also locate the damage location. The surface damage semantic segmentation method can not only determine the damage category and location, but also obtain more detailed information such as area. Although the computer vision-based concrete bridge apparent disease detection method has achieved certain results in detection accuracy, it has problems such as large number of parameters and slow inference speed. In actual apparent disease detection scenarios, it is necessary to quickly detect and analyze the diseased area;

[0006] To this end, we designed a lightweight concrete bridge apparent defect detection method based on improved Segformer to provide another technical solution to the above technical problems. Summary of the Invention

[0007] Based on this, it is necessary to provide a lightweight concrete bridge surface defect detection method based on an improved Segformer to address the above technical problems and solve the technical problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer is proposed. The steps are as follows:

[0010] S1: Construct a dataset of apparent defects of concrete bridges, select concrete bridge images and mark the areas where apparent defects are located;

[0011] S2: The concrete bridge dataset is enhanced and the training set and validation set are divided into 8:2;

[0012] S3: Build an improved Segformer lightweight model, select MiTB0 as the encoder of the model, and introduce the SFM module and PSA self-attention mechanism module in the decoder;

[0013] S4: Train a model based on a dataset of apparent concrete bridge defects to identify apparent concrete bridge defects such as water erosion, exposed reinforcement, and damage.

[0014] As a preferred embodiment of the method for detecting apparent defects of lightweight concrete bridges based on the improved Segformer provided by the present invention, in step S1, a dataset of apparent defects of concrete bridges is constructed, and the steps are as follows:

[0015] Bridge damage images with clear damage areas and high image resolution are manually collected through various methods.

[0016] As a preferred implementation of the method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer provided by the present invention, the multiple methods include no restriction on focal length, no restriction on object distance, no restriction on lighting conditions, no restriction on occlusion and shadows, and no restriction on shooting angles.

[0017] As a preferred implementation of the method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer provided by the present invention, in the step S1, a picture of a concrete bridge is selected and the area where the apparent defects of the concrete bridge are located is marked, and the general marking tool Labelme is used to mark the area where the apparent defects are located.

[0018] As a preferred embodiment of the method for detecting apparent defects of lightweight concrete bridges based on the improved Segformer provided by the present invention, in step S2, the concrete bridge dataset is enhanced as follows:

[0019] The dataset was enhanced by rotating, flipping, and cropping;

[0020] Select images containing exposed ribs and crop and rotate them to enhance them in order to increase the number of exposed rib categories.

[0021] As a preferred embodiment of the method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer provided by the present invention, in step S3, the improved Segformer lightweight model includes Segformer, SFM and PSA;

[0022] The Segformer is formed by combining the Transformer with a lightweight multi-layer perceptron decoder;

[0023] The SFM obtains more accurate and semantically informative feature representation by exploiting the relationship between high-level features and low-level features;

[0024] The PSA contains only channel-wise self-attention and only spatial self-attention.

[0025] As a preferred implementation of the method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer provided by the present invention, in the step S4, Adamw is used as the optimizer of the training model, and FacalLoss and DiceLoss are used as loss functions to test the detection effect on the validation set.

[0026] It can be seen without a doubt that the above-mentioned technical solution of this application can definitely solve the technical problem to be solved by this application.

[0027] At the same time, through the above technical solutions, the present invention has at least the following beneficial effects:

[0028] The present invention provides a lightweight concrete bridge apparent defect detection method based on an improved Segformer, which has strong robustness and high accuracy in the detection of concrete bridge apparent defects, and can achieve efficient and rapid detection of bridge apparent defects with a small number of parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A flowchart of a specific embodiment of the present invention;

[0031] Figure 2 It is a structural diagram of the present invention;

[0032] Figure 3 This is a schematic diagram of the structure of the semantic fusion module of the present invention;

[0033] Figure 4 Schematic diagram of the polarized self-attention mechanism of the present invention;

[0034] Figure 5 Schematic diagram comparing the present invention with mainstream detection methods. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions therein may be combined with each other.

[0038] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0039] Reference Figure 1-Figure 5 , a lightweight concrete bridge apparent disease detection method based on improved Segformer, the steps are as follows:

[0040] S1: Construct a dataset of apparent defects of concrete bridges, select concrete bridge images and mark the areas where apparent defects are located;

[0041] In S1, we first need to manually collect images of bridge defects to construct a dataset. To ensure data diversity and generalization, bridge defect images were collected from different bridge inspectors. There were no restrictions on fixed shooting conditions, such as focal length, object distance, and shooting equipment, nor were there fixed resolutions for the defect images. The principle for selecting images was that the defect area was clear and the image resolution was high. In addition, these images also included different lighting conditions, shadows, and shooting angles, which ensured the model's adaptability to different complex environments. Ultimately, we manually selected 1,348 images containing apparent bridge defects such as damage, exposed reinforcement, and water erosion, and used the general annotation tool Labelme to annotate the areas where the apparent defects were located.

[0042] S2: The concrete bridge dataset is enhanced and the training set and validation set are divided into 8:2;

[0043] To prevent overfitting, we augmented the dataset in S2 by rotating, flipping, and cropping. Because we found that images with exposed tendons differed significantly from other categories, we further augmented them by cropping and rotating them to increase the number of exposed tendons. After preprocessing, the dataset contained 2,712 images. We then split the dataset into training and validation sets in an 8:2 ratio.

[0044] S3: Build an improved Segformer lightweight model, select MiTB0 as the encoder of the model, and introduce the SFM module and PSA self-attention mechanism module in the decoder;

[0045] An improved lightweight Segformer model was built in S3, with MiTB0 selected as the algorithm's encoder. A Semantic Fusion Module (SFM) was introduced to the original Segformer to incorporate more semantic information from high-level features into low-level features, thereby enriching the low-level features with richer semantic information. Furthermore, after the multi-scale features were aligned using the MLP, a Polarized Self-Attention (PSA) module was added to address the issue of insufficient encoder-extracted features for pixel-level regression in semantic segmentation of concrete bridge surface defects.

[0046] The improved Segformer lightweight model includes the following:

[0047] a: Segformer, a simple, efficient, yet powerful semantic segmentation model that combines a Transformer with a lightweight multilayer perceptron (MLP) decoder. Segformer has two key features: first, it includes a novel hierarchical Transformer encoder, MiT (MixTransformer), which eliminates the need for positional encoding, avoiding interpolation of positional codes while outputting multi-scale features; second, it avoids the need for a complex decoder, while the MLP decoder aggregates information from different layers, combining local and global attention for a more powerful representation capability.

[0048] b: SFM, first, high-dimensional features need to be linearly interpolated so that the high-dimensional features and low-dimensional features have the same size in length and width. Next, to prevent the fused features from being overly dependent on high-level features, we use 1×1 convolution to reduce the dimensionality of the high-level features to reduce the number of channels occupied by the high-level features. Then, the high-level features obtained by dimensionality reduction are concatenated with the low-level features to generate a new feature representation that not only contains the spatial detail information of the low-level features but also integrates the semantic information of the high-level features. Finally, we use 3×3 convolution to fuse the concatenated features to further enhance the expressive power of the feature representation. In this way, we can effectively utilize the relationship between high-level and low-level features to obtain a more accurate and semantically informative feature representation, thereby improving the detection accuracy of the concrete bridge surface defect detection algorithm.

[0049] c:PSA, which consists of two parts: channel-only self-attention and spatial-only self-attention. In the channel dimension, PSA maintains half the dimension of the original feature; in the spatial dimension, PSA maintains the full dimension of the original feature. This design can reduce the information loss caused by dimensionality reduction.

[0050] S4: A model is trained based on a dataset of apparent concrete bridge defects to identify apparent concrete bridge defects such as water erosion, exposed reinforcement, and damage.

[0051] In S4, the model is trained based on the concrete bridge apparent disease dataset, Adamw is used as the optimizer for the training model, FacalLoss and DiceLoss are used as loss functions, and the detection effect is tested on the validation set and compared with the mainstream method.

[0052] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer, characterized in that: Here are the steps: S1: Construct a dataset of apparent defects of concrete bridges, select concrete bridge images and mark the areas where apparent defects are located; S2: The concrete bridge dataset is enhanced and the training set and validation set are divided into 8:2; S3: Build an improved Segformer lightweight model, select MiT B0 as the encoder of the model, and introduce a semantic fusion module and a polarized self-attention mechanism module in the decoder; S4: Train a model based on a dataset of apparent concrete bridge defects to identify apparent concrete bridge defects such as water erosion, exposed reinforcement, and damage.

2. The method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer according to claim 1, characterized in that: In step S1, a concrete bridge apparent disease dataset is constructed, and the steps are as follows: Clear bridge damage images of the damaged area are manually collected through various methods.

3. The method for detecting apparent defects of lightweight concrete bridges based on improved Segformer according to claim 2, characterized in that: The multiple modes include no restriction on focal length, no restriction on object distance, no restriction on lighting conditions, no restriction on shadows, and no restriction on shooting angles.

4. The method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer according to claim 1, characterized in that: In the step S1, a concrete bridge picture is selected and the area where the apparent defects of the concrete bridge are located is marked. The general marking tool Labelme is used to mark the area where the apparent defects are located.

5. The method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer according to claim 1, characterized in that: In step S2, the concrete bridge dataset is enhanced as follows: The dataset was enhanced by rotating, flipping, and cropping; Select images containing exposed ribs and crop and rotate them to enhance them in order to increase the number of exposed rib categories.

6. The method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer according to claim 1, characterized in that: In the S3 step, the improved Segformer lightweight model includes Segformer, semantic fusion module and polarized self-attention mechanism module; The Segformer is formed by combining the Transformer with a lightweight multi-layer perceptron decoder; The semantic fusion module obtains more accurate and semantically informative feature representation by utilizing the relationship between high-level features and low-level features; The polarized self-attention mechanism module includes channel-only self-attention and spatial-only self-attention.

7. The method for detecting apparent defects of lightweight concrete bridges based on an improved Segformer according to claim 1, characterized in that: In the S4 step, Adamw is used as the optimizer for the training model, and Facal Loss and Dice Loss are used as loss functions to test the detection effect on the validation set.

Images