Bridge underwater pier disease identification method based on deep learning, medium and equipment thereof
Through deep learning technology, the UPNet model and feature extraction module are used to solve the problems of low efficiency and insufficient accuracy of disease detection of traditional bridge underwater piers, and efficient and accurate disease identification is achieved, reducing costs and ensuring bridge safety.
Patent Information
- Application Number
- CN202410970930.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional bridge underwater pier disease detection methods are inefficient, costly, and have limited accuracy in complex underwater environments, which poses safety risks.
Using a deep learning-based method, the UPNet model is established by acquiring and labeling the pier disease images, and the pre-trained residual module, spatial attention module and spatial depth module are used to extract and identify features and diseases, combining data expansion and hyperparameter optimization. Improve model performance and recognition accuracy.
It realizes efficient and accurate identification of underwater pier diseases, improves detection accuracy and accuracy, reduces detection costs, and ensures the safe operation of the bridge.
Smart Images

Figure CN118968288A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of underwater bridge pier disease identification, and specifically to an underwater bridge pier disease identification method, medium and equipment thereof based on deep learning. Background Art
[0002] With the continuous expansion of urban development and transportation networks, bridges, as one of the important infrastructures, play an important role in connecting cities and promoting economic development. However, as the service life of bridges increases, underwater bridge piers are prone to erosion and damage due to being in the underwater environment for a long time. The problem of underwater bridge pier diseases is becoming increasingly prominent, which not only affects the safety and stability of bridges, but may also cause traffic accidents and economic losses.
[0003] Traditional underwater bridge pier disease detection methods mainly rely on manual inspections or diver inspections, which have problems such as low efficiency, high cost, and high safety risks. Although there are some sensor-based automated detection methods, their performance and accuracy in complex underwater environments are still limited.
[0004] In recent years, the rapid development of deep learning technology has provided a new solution for the identification of underwater bridge pier defects. Deep learning technology has achieved great success in the field of image processing and target detection with its excellent feature learning and pattern recognition capabilities, providing technical support for automated underwater bridge pier defect identification. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies in the above-mentioned background technology and provide a method, medium and equipment for identifying underwater bridge pier defects based on deep learning, so as to improve the efficiency and accuracy of underwater bridge defect identification, reduce detection costs and ensure the safe operation of bridges.
[0006] The technical solution of the present invention is:
[0007] A method for identifying underwater bridge pier defects based on deep learning, comprising the following steps:
[0008] S1: Obtain the damage images of underwater bridge piers to form a preliminary data set of damage images;
[0009] S2: Classify and label the preliminary disease image dataset to obtain the disease image label dataset;
[0010] S3: Divide the disease image label dataset into training set, validation set and test set, and perform data expansion;
[0011] S4: Establish UPNet model based on UNet;
[0012] S5: Set the hyperparameter combination value of the UPNet model, input the training set and the validation set into the UPNet model for training until the validation set loss converges, and obtain the weight file;
[0013] S6: Load the weight file of the UPNet model and test the UPNet model using the test set;
[0014] S7: Identify the damage of underwater bridge piers.
[0015] In step S2, the classification of the preliminary dataset of disease images includes exposed reinforcement and spalling.
[0016] In step S3, data expansion includes: performing image enhancement processing on the images of each set to increase the number of images; the image enhancement includes saturation enhancement, chroma enhancement, 180-degree rotation, grayscale transformation, Rayleigh noise and coupling noise.
[0017] The image enhancement processing includes performing saturation enhancement, chroma enhancement, 180-degree rotation, grayscale conversion, Rayleigh noise and coupling noise on the collected images, thereby increasing the number of images by six times.
[0018] In step S4, the UPNet model includes:
[0019] (1) Extract features using pre-trained residual modules;
[0020] (2) Setting a spatial attention module between the channel concatenation layer and the upsampling layer;
[0021] (3) Setting the spatial depth module to connect the residual module;
[0022] (4) Using F2-score as the evaluation indicator:
[0023]
[0024] Among them, β is the weight adjustment factor, Precision is the indicator established for the prediction result, and Recall is the indicator established for the true value label.
[0025] In step S4, the spatial depth module includes first dividing the tensor of the feature map, then increasing the number of channels by counterclockwise connection, and then performing strideless convolution.
[0026] In step S5, setting the hyperparameter combination values includes: learning rate: using cosine annealing decay model, initial learning rate 0.01, final learning rate 0.0001; optimizer: SGD, momentum 0.937, weight decay value 0.0005; input image size: 640; batch size: 4; number of training rounds: 300; early stopping function: 30 rounds.
[0027] The defect identification in step S7 includes: collecting images of underwater bridge piers and inputting them into the UPNet model to obtain defect identification results of the underwater bridge piers.
[0028] A computer-readable storage medium, characterized in that a computer program is stored on the storage medium; when the computer program is executed by a processor, the method for identifying underwater bridge pier defects based on deep learning as described in any one of claims 1 to 8 can be implemented.
[0029] A deep learning-based underwater bridge pier defect identification device, comprising a memory and a processor; characterized in that: the memory is used to store a computer program; the processor is used to implement the deep learning-based underwater bridge pier defect identification method as described in any one of claims 1 to 8 when executing the computer program.
[0030] The beneficial effects of the present invention are:
[0031] 1. The present invention provides a method for identifying underwater bridge pier defects based on deep learning, which can efficiently and accurately identify underwater pier defects, improve the detection accuracy of underwater pier defects, and is conducive to accurately measuring the location and area of defects, providing accurate defect data for subsequent safety assessment and maintenance of underwater bridge piers;
[0032] 2. The UPNet model of the present invention uses a residual module to replace the original CNN feature extraction network in the UNet model, which avoids gradient vanishing and explosion, effectively increases the network depth, and improves the performance of the model;
[0033] 3. The UPNet model of the present invention adds a spatial attention module between each channel splicing layer and the upsampling layer to better capture the potential relationship between the diseased pixels at different spatial distances in the pier disease image;
[0034] 4. The UPNet model of the present invention uses a spatial depth module to replace the downsampling layer and pooling layer in the UNet model to ensure that the learnable disease feature information is retained during the downsampling process.
[0035] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a flow chart of the method for identifying underwater bridge pier defects of the present invention.
[0037] Figure 2 It is a schematic diagram of image classification and labeling of the present invention.
[0038] Figure 3It is a schematic diagram of the image enhancement processing of the present invention.
[0039] Figure 4 It is a structural diagram of the UPNet model of the present invention.
[0040] Figure 5 It is a structural diagram of the residual module of the present invention.
[0041] Figure 6 It is a structural diagram of the spatial attention module of the present invention.
[0042] Figure 7 It is a structural diagram of the spatial depth module of the present invention.
[0043] Figure 8 This is a diagram showing the identification results of underwater diseases of bridge piers according to the present invention. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0045] See also Figure 1-8 , a method for identifying underwater bridge pier defects based on deep learning, comprising the following steps:
[0046] S1: Data Collection
[0047] The underwater bridge pier inspection reports are collected, and the damage images of the underwater bridge piers are obtained from the reports to form a preliminary data set of damage images.
[0048] The inspection report is derived from the actual inspection record of the bridge. The damage images include different types of damages of the underwater piers of the bridge.
[0049] S2: Image classification and annotation
[0050] The acquired preliminary dataset of defect images was sorted and classified, and professional bridge inspection personnel manually annotated each image to obtain a dataset of defect image labels;
[0051] The classification of the preliminary dataset of disease images includes exposed rebar and spalling.
[0052] S3: Dataset Partitioning and Expansion
[0053] The obtained disease image label dataset is divided into training set, validation set and test set, and data expansion is performed on each set.
[0054] The data expansion includes performing image enhancement processing on the images in the training set, validation set and test set to increase the number of images in the set. The specific method is: saturation enhancement, chroma enhancement, 180-degree rotation, grayscale transformation, Rayleigh noise and coupling noise are performed on each image in the training set, validation set and test set, so that six enhanced versions are generated for each image, and the total number of images is increased to six times the original.
[0055] S4: Model building
[0056] The UPNet model based on UNet (Net for underwaterpier defect identification) was established.
[0057] The improvements of the UPNet model are:
[0058] (1) In the decoder of the UNet model, the pre-trained residual module is used to extract features, replacing the CNN module of the UNet model.
[0059] The residual modules include a first residual module ResNet101-1, a second residual module ResNet101-2, a third residual module ResNet101-3, and a fourth residual module ResNet101-4.
[0060] (2) In the decoder of the UNet model, a spatial attention module is set between the channel concatenation layer and the upsampling layer.
[0061] The channel splicing layer includes a first channel splicing layer C1, a second channel splicing layer C2, a third channel splicing layer C3, and a fourth channel splicing layer C4.
[0062] The upsampling layers include a first upsampling layer Upsample1, a second upsampling layer Upsample2, a third upsampling layer Upsample3, a fourth upsampling layer Upsample4, and a fifth upsampling layer Upsample5.
[0063] The spatial attention module includes a second spatial attention module Spatial Attention2, a third spatial attention module Spatial Attention3, and a fourth spatial attention module Spatial Attention4.
[0064] (3) Set the spatial depth module to connect the residual module to replace the downsampling layer and pooling layer of the UNet model.
[0065] The spatial depth module includes a first spatial depth module SPD1, a second spatial depth module SPD2, a third spatial depth module SPD3, and a fourth spatial depth module SPD4.
[0066] The spatial depth module first divides the tensor of the feature map, then increases the number of channels through counterclockwise connections, and then performs strideless convolution.
[0067] (4) Using F2-score as the evaluation indicator:
[0068]
[0069] in:
[0070] β is the weight adjustment factor, which takes a value of 2;
[0071] Precision is an indicator established for the prediction results, representing the proportion of true positive samples among the predicted positive samples;
[0072] Recall is an indicator established for the true value label, that is, the proportion of true positive samples in the sample that are predicted correctly.
[0073] S5: Model training
[0074] Set the initial value of the hyperparameter combination of the UPNet model, input the training set and validation set into the UPNet model for training until the model validation set loss converges, and obtain the model weight file;
[0075] Setting the initial value of the hyperparameter combination includes:
[0076] Learning rate: The cosine annealing decay model is used, with an initial learning rate of 0.01 and a final learning rate of 0.0001; optimizer: SGD, momentum 0.937, and weight decay value 0.0005; input image size: 640; batch size: 4; number of training rounds: 300, and the early stopping function is set to 30 rounds.
[0077] S6: Model testing
[0078] The UPNet model loads the weight file and tests the UPNet model using the test set.
[0079] S7: Disease identification
[0080] The images of underwater bridge piers are collected and input into the UPNet model to perform damage identification on the underwater bridge piers and obtain the damage identification results.
[0081] UPNet model
[0082] like Figure 4 The specific structure of the UPNet model shown is: including an encoder and a decoder.
[0083] The encoder includes a CBR module, a first spatial depth module SPD1, a first residual module ResNet101-1, a second spatial depth module SPD2, a second residual module ResNet101-2, a third spatial depth module SPD3, a third residual module ResNet101-3, a fourth spatial depth module SPD4, and a fourth residual module ResNet101-4 connected in sequence.
[0084] The decoder includes a first upsampling layer Upsample1, a first channel splicing layer C1, a first continuous convolution module DoubleConv1, a second upsampling layer Upsample2, a second spatial attention module Spatial Attention2, a second channel splicing layer C2, a second continuous convolution module DoubleConv2, a third upsampling layer Upsample3, a third spatial attention module Spatial Attention3, a third channel splicing layer C3, a third continuous convolution module DoubleConv3, a fourth upsampling layer Upsample4, a fourth spatial attention module Spatial Attention4, a channel splicing layer C4, a first convolution layer 3×3Conv, a fifth upsampling layer Upsample5, and a second convolution layer 1×1Conv, which are connected in sequence.
[0085] The CBR module is directly connected to the fourth splicing layer C4 through a skip connection; the first residual module ResNet101-1 is directly connected to the third splicing layer C3 through a skip connection; the second residual module ResNet101-2 is directly connected to the second splicing module layer C2 through a skip connection; the third residual module ResNet101-3 is directly connected to the first splicing module layer C1 through a skip connection; the fourth residual module ResNet101-4 is directly connected to the first upsampling layer Upsample1 through a skip connection.
[0086] The first continuous convolution module DoubleConv1, the second continuous convolution module DoubleConv2, and the third continuous convolution module DoubleConv3 all represent two consecutive convolutions.
[0087] The CBR module includes convolution Conv, batch normalization BN, and activation function processing on the feature map. The activation function is the ReLU function.
[0088] like Figure 5As shown in the figure, the residual module ResNet101 includes 101 residual blocks. In each residual block, the input feature map is sequentially processed by two-dimensional convolution 1×1Conv2d, batch normalization Batch Norma2d, ReLU function, two-dimensional convolution 3×3Conv2d, batch normalization Batch Norma2d, two-dimensional convolution 1×1Conv2d, batch normalization Batch Norma2d, and then channel concatenation C is performed with the input feature map, and then the output is obtained by ReLU function.
[0089] like Figure 6 As shown in the figure, the spatial attention module Spatial Attention includes: using the global maximum pooling MaxPool and the global average pooling AvgxPool to pool the feature map, concatenating the pooled feature map through channels C, and then passing the two-dimensional convolution 1×1Conv2d, and then concatenating the feature map with the input feature map. Figure 1 The spatial attention map is obtained by passing the sigmoid function.
[0090] like Figure 7 As shown, in the spatial depth module: Assume that the size of the feature map is S×S×C 1 , the tensors of four adjacent channels on the feature map are divided, and the number of channels is increased fourfold by counterclockwise connection, while the length and width of the feature map are reduced to half of the original. Finally, a strideless convolution (stride = 1, convolution kernel 3×3) is performed on the feature map, and the output size is (S / 2)×(S / 2)×C 2 of the image.
[0091] Example
[0092] The underwater pier inspection reports of nearly 20 bridges in a certain area were collected, and 309 defect images were extracted. The defects contained in the obtained preliminary data set of defect images were divided into exposed reinforcement and spalling.
[0093] like Figure 2 As shown in the figure, experienced bridge inspection engineers were invited to perform semantic segmentation and annotation on the preliminary dataset of defect images. Labelme was used as the annotation software. After manual annotation, 124 exposed rebar images and 185 peeling images were obtained, and the corresponding label files were generated to obtain the defect image label dataset.
[0094] In order to improve the generalization ability of the model, the above disease image label dataset is expanded. The specific method is as follows:
[0095] like Figure 3As shown in the figure, each image is enhanced by saturation enhancement, chroma enhancement, 180-degree rotation, grayscale transformation, Rayleigh noise and coupling noise, so that each image generates six enhanced versions. After data augmentation, the total number of images in the dataset is expanded from 309 to 1854.
[0096] The specific parameter settings for image enhancement processing are as follows:
[0097] Saturation enhancement: randomly adjust the saturation between 1.2-1.5;
[0098] Chroma enhancement: randomly adjust the chroma between 0.8-1.2
[0099] 180 degree rotation: rotate 180 degrees clockwise;
[0100] Grayscale transformation: convert the image into a grayscale image;
[0101] Rayleigh Noise: Adds randomly distributed Rayleigh noise to the image;
[0102] Coupling noise: Add randomly distributed coupling noise to the image;
[0103] The expanded dataset is divided into training, validation and test sets in a ratio of 7:2:1, containing 1298, 371 and 185 images respectively.
[0104] The initial values of the hyperparameter combination of the UPNet model are set as follows: learning rate: using the cosine annealing decay model, the initial learning rate is 0.01, and the final learning rate is 0.0001; optimizer: SGD, momentum 0.937, weight decay value 0.0005; input image size: 640; batch size: 4; number of training rounds: 300, and the early stopping function is set to 30 rounds.
[0105] The training set and validation set are input into the UPNet model for training. During the training process, the loss of the validation set is calculated in each round. If the loss does not decrease within 30 rounds, the training is stopped. Finally, the loss of the model on the validation set converges and the best model weight file is obtained.
[0106] The UPNet model loads the weight file and uses the test set to test the UPNet model, using F2-score as the evaluation indicator for evaluation.
[0107] The results of the test set show that the F2-score reached 0.85, indicating that the model has a high recognition accuracy. In addition, the recognition accuracy of the model for exposed rebar and spalling defects was 0.87 and 0.83 respectively. The overall performance was excellent and it could effectively identify exposed rebar and spalling defects on the surface of bridge piers.
[0108] like Figure 8As shown in the figure, the UPNet model is applied to the actual bridge inspection. At the inspection site, the camera is aimed at the underwater bridge pier to collect images and identify defects in real time. The recognition results are displayed in real time on the display screen, and the inspectors can perform corresponding maintenance operations based on the recognition results. After many experiments, the recognition accuracy in the actual inspection is consistent with the test set results, proving that it has high practicality and reliability.
[0109] Through the detailed description of the above embodiments, the present invention successfully implements a method for identifying underwater bridge pier defects based on deep learning, which has high recognition accuracy and practical application value, and can effectively improve the efficiency and accuracy of bridge detection.
[0110] In addition, in other embodiments, a computer-readable storage medium may be provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying underwater bridge pier defects based on deep learning as described in S1 to S7 above can be implemented.
[0111] In addition, in other embodiments, a deep learning-based underwater bridge pier disease identification device may be provided, including a memory and a processor;
[0112] The memory is used to store computer programs;
[0113] The processor is used to implement the method for identifying underwater bridge pier defects based on deep learning as described in S1 to S7 above when executing the computer program.
[0114] It should be noted that the above-mentioned memory may include random access memory (RAM) and non-volatile memory (NVM), such as at least one disk storage. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a neural network processor (NPU), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Of course, the device should also have necessary components for realizing program operation, such as power supply, communication bus, etc.
[0115] The preferred embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.
Claims
1. A method for identifying underwater bridge pier defects based on deep learning. The following steps are involved: S1: Obtain the damage images of underwater bridge piers to form a preliminary data set of damage images; S2: Classify and label the preliminary disease image dataset to obtain the disease image label dataset; S3: Divide the disease image label dataset into training set, validation set and test set, and perform data expansion; S4: Establish UPNet model based on UNet; S5: Set the hyperparameter combination value of the UPNet model, input the training set and the validation set into the UPNet model for training until the validation set loss converges, and obtain the weight file; S6: Load the weight file of the UPNet model and test the UPNet model using the test set; S7: Identify the damage of underwater bridge piers.
2. According to the method for identifying underwater bridge pier defects based on deep learning in claim 1, Features: In step S2, the classification of the preliminary dataset of disease images includes exposed reinforcement and spalling.
3. According to the method for identifying underwater bridge pier defects based on deep learning in claim 1, Features: In step S3, data expansion includes: performing image enhancement processing on the images of each set to increase the number of images; the image enhancement includes saturation enhancement, chroma enhancement, 180-degree rotation, grayscale transformation, Rayleigh noise and coupling noise.
4. According to the method for identifying underwater bridge pier defects based on deep learning in claim 3, Features: The image enhancement processing includes performing saturation enhancement, chroma enhancement, 180-degree rotation, grayscale conversion, Rayleigh noise and coupling noise on the collected images, thereby increasing the number of images by six times.
5. According to the method for identifying underwater bridge pier defects based on deep learning in claim 1, Features: In step S4, the UPNet model includes: (1) Extract features using pre-trained residual modules; (2) Setting a spatial attention module between the channel concatenation layer and the upsampling layer; (3) Setting the spatial depth module to connect the residual module; (4) Using F2-score as the evaluation indicator: Among them, β is the weight adjustment factor, Precision is the indicator established for the prediction result, and Recall is the indicator established for the true value label.
6. According to claim 5, a method for identifying underwater bridge pier defects based on deep learning, Features: In step S4, the spatial depth module includes first dividing the tensor of the feature map, then increasing the number of channels by counterclockwise connection, and then performing strideless convolution.
7. According to claim 1, a method for identifying underwater bridge pier defects based on deep learning, Features: In step S5, setting the hyperparameter combination values includes: learning rate: using cosine annealing decay model, initial learning rate 0.01, final learning rate 0.0001; optimizer: SGD, momentum 0.937, weight decay value 0.0005; input image size: 640; batch size: 4; number of training rounds: 300; early stopping function: 30 rounds.
8. According to the method for identifying underwater bridge pier defects based on deep learning in claim 1, Features: The defect identification in step S7 includes: collecting images of underwater bridge piers and inputting them into the UPNet model to obtain defect identification results of the underwater bridge piers.
9. A computer-readable storage medium, It is characterized by: The storage medium stores a computer program; when the computer program is executed by the processor, it can implement the method for identifying underwater bridge pier defects based on deep learning as described in any one of claims 1 to 8.
10. A device for identifying underwater bridge pier defects based on deep learning, comprising a memory and a processor; Features: The memory is used to store computer programs; the processor is used to implement the method for identifying underwater bridge pier defects based on deep learning as described in any one of claims 1 to 8 when executing the computer program.
Citation Information
Cited By
Typical bridge disease characterization method
CN120279344A
