Bridge abutment retaining wall disease detection method

Abstract

The invention discloses a bridge abutment retaining wall disease detection method. The method comprises the following steps: 1, constructing a bridge abutment retaining wall disease image database; 2, constructing a bridge abutment retaining wall disease detection model; 3, training a bridge abutment retaining wall disease detection model; 4, testing the abutment retaining wall disease detection model; 5, setting an anchor frame for frame selection of the abutment retaining wall disease type; 6, acquiring an actual bridge abutment retaining wall disease image by using the bridge abutment retaining wall disease inspection trolley; 7, inputting an actually acquired image into the abutment retaining wall disease detection model, and outputting a classification prediction result with boundary coordinates, sizes and category probabilities; and 8, judging whether the abutment retaining wall disease corresponding to the classification prediction result needs to be repaired or not. According to the invention, the C2fDCNv3 module variable convolution and the PBM parallel multi-scale module are introduced, and the sampling point position of the convolution kernel can be dynamically adjusted according to the shape and the position of the input feature, so that the accuracy and the speed of bridge abutment retaining wall disease detection are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of bridge abutment retaining wall disease identification, and specifically relates to a bridge abutment retaining wall disease detection method. Background Art

[0002] The abutments are located at both ends of the bridge, supporting the superstructure of the bridge and connecting with the embankment. In addition to transferring the load of the superstructure of the bridge to the foundation, they also have the functions of resisting the fill pressure behind the abutment, stabilizing the bridgehead roadbed, and reliably and smoothly connecting the bridgehead line and the bridge line. The abutment retaining wall is usually set at the front end or both sides of the abutment to prevent vehicles from exceeding the limited range of the bridge when driving on the bridge, thereby preventing vehicles from falling off the edge of the bridge and ensuring driving safety. It can also effectively prevent the bridge from tilting or collapsing due to uneven force or external factors (such as wind, water, etc.), enhance the overall stability of the bridge, and also play a role in protecting the embankment and soil. Common structures of abutment retaining walls include gravity abutment retaining walls, cantilever abutment retaining walls, buttress abutment retaining walls, and reinforced abutment soil retaining walls. However, under the influence of natural environment and human factors for a long time, bridge abutment retaining walls will also have various disease problems, such as cracks, voids, damage, settlement, etc. The disease of bridge abutment retaining walls may cause leakage of the abutment back, and the leakage of rainwater will accelerate the corrosion of internal steel bars, further weakening the bearing capacity of the structure; the disease of bridge abutment retaining walls may also have adverse effects on the surrounding environment, such as soil loss, water pollution, etc.; the disease of bridge abutment retaining walls may also cause the abutment to tilt or shift, and this deformation will destroy the overall stability of the bridge, making the bridge more prone to damage when subjected to force, shortening the service life of the bridge and endangering driving safety. Traditional bridge abutment retaining wall disease detection mainly relies on manual inspection and regular inspection. This method is not only time-consuming and labor-intensive, but also the detection effect is limited by the experience and subjective judgment of the inspectors, and it is difficult to ensure the accuracy and comprehensiveness of the detection results. Deep learning has shown good application prospects in the field of target detection. Therefore, applying deep learning to bridge abutment retaining wall disease detection can well solve the drawbacks brought by traditional detection. Therefore, a bridge abutment retaining wall disease detection method is needed to solve the problem of automatic detection of bridge abutment retaining wall diseases. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for detecting defects in abutment retaining walls in response to the above-mentioned deficiencies in the prior art. Based on the relevant theories of deep learning, the sampling point positions of the convolution kernel can be dynamically adjusted according to the shape and position of the input features, thereby improving the accuracy of defect detection. The automatic and intelligent detection has high efficiency and is easy to promote and use.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting a bridge abutment retaining wall disease, characterized in that the method comprises the following steps:

[0005] Step 1: Construct a database of abutment retaining wall defect images: obtain abutment retaining wall defect images containing various defect types, enhance and annotate the abutment retaining wall defect images, construct an image database, randomly classify the image database, and obtain an image training data set and an image test data set;

[0006] Step 2: Construct a bridge abutment retaining wall disease detection model. The bridge abutment retaining wall disease detection includes Backbone network, Neck network and Head network. Backbone network includes Conv module, C2f module, SCDown module, SPPF module and improved PSA module, which are used to extract high, medium and low layer features of the image; Neck network includes Concat module, Upsample module and improved C2f_DCNv3 module, which are used for image processing and enhancement; Head network includes detection head and classification head. The detection head includes convolution layer and deconvolution layer to generate detection results, including the bounding box regression value of the target and the confidence of the target existence. The classification head uses global average pooling to classify each feature map, and outputs the probability distribution of each category by reducing the dimension of the feature map.

[0007] Among them, the improved PSA module replaces the MHSA multi-head self-attention module in the PSA module with the PBM module;

[0008] Step 3: Training the abutment retaining wall disease detection model: sequentially retrieve the abutment retaining wall disease images after image enhancement from the image training data set and input them into the abutment retaining wall disease detection model. Use the precision, recall rate and F1 curve of the training result indicators to analyze the advantages and disadvantages of the abutment retaining wall disease detection model, and complete the abutment retaining wall disease detection model training process;

[0009] Among them, the accuracy The proportion of correct predictions among all the results predicted as Positive; Recall The proportion of correct predictions among all results whose true values ​​are Positive; F1 curve TP is the true positive value, FP is the false positive value, and FN is the false negative value;

[0010] Step 4: Testing the abutment retaining wall disease detection model: Retrieving the abutment retaining wall disease images from the image test data set, testing the abutment retaining wall disease detection model until the test is completed, and obtaining the trained and tested abutment retaining wall disease detection model;

[0011] Step 5: Set the anchor frame to select the type of damage to the abutment retaining wall;

[0012] Step 6: Use the abutment retaining wall disease inspection vehicle to collect actual abutment retaining wall disease images: the abutment retaining wall disease inspection vehicle is equipped with a camera, a laser radar detector and a GPS locator;

[0013] Step 7: Input the actual collected image in step 6 into the bridge abutment retaining wall disease detection model trained in step 4, and output the classification prediction result with boundary coordinates, size and category probability;

[0014] Step 8, determine whether the abutment retaining wall defect corresponding to the classification prediction result needs to be repaired: set the missing size threshold of the abutment retaining wall defect, use the laser radar detector to obtain the missing size of the abutment retaining wall defect, and when the missing size of the abutment retaining wall defect is greater than the threshold, repair the abutment retaining wall defect; otherwise, execute step 6 until the abutment retaining wall defect detection is completed;

[0015] Among them, the process of the laser radar detector obtaining the missing size of the abutment retaining wall disease:

[0016] The three-dimensional information of the abutment retaining wall disease is obtained by using the laser radar detector. That is, when the laser radar detector is used to scan the disease at a uniform speed v, the laser radar detector measures the depth h of the abutment retaining wall disease at a time interval t. Through data fitting, the relationship between the time interval t and h is obtained as a parabolic function h = at 2 +bt+c, a is the fitting coefficient of the quadratic term of the time interval t in the parabola function, b is the fitting coefficient of the linear term of the time interval t in the parabola function, c is the fitting constant term in the parabola function, for the cross section of the disease, the velocity v is integrated in the x and y directions to obtain the cross-sectional area Volume V = S × h, that is Obtain the missing volume V of the abutment retaining wall defect, obtain the location of the abutment retaining wall defect according to the GPS locator, and upload relevant data to the abutment retaining wall defect detection host computer in real time.

[0017] The above-mentioned method for detecting bridge abutment retaining wall defects is characterized in that: in step one, the bridge abutment retaining wall defect image is processed by random cropping, resizing, flipping, rotating, scaling and color gamut changes to increase data diversity and effectively improve the performance and robustness of the model.

[0018] The above-mentioned method for detecting defects in abutment retaining walls is characterized in that the defect types include cracks, cavities, breakages and settlements.

[0019] The above-mentioned method for detecting bridge abutment retaining wall defects is characterized in that the number of bridge abutment retaining wall defect images of the various defect types is 5,000 to 10,000, and the image database is randomly classified according to a ratio of 8:2 to obtain an image training data set and an image test data set.

[0020] The above-mentioned abutment retaining wall disease detection method is characterized in that: in step 3, the true positive value TP is the number of positive class predictions as positive class, that is, the correct prediction, the actual is 0, and the prediction is also 0;

[0021] The false positive value FP is the number of negative classes predicted as positive classes, that is, wrong prediction, the actual value is 1, and the prediction is 0;

[0022] The false negative value FN is the number of positive classes predicted as negative classes, that is, wrong prediction, the actual value is 0, and the prediction is 1.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] 1. The present invention uses the PBM parallel multi-scale module to replace the MHSA attention mechanism in the PAS module in the backbone network, and adopts a more powerful structure for processing short-term and long-term dependencies, which solves the problem of feature loss and gradient disappearance caused by the failure to better utilize context information in intermediate features, improves the model's perception and feature extraction capabilities for objects of different scales, and improves recognition accuracy.

[0025] 2. The present invention uses the C2f_DCNv3 convolution module to replace the C2f convolution module in the neck module, introduces deformable convolution, and deforms the sampling position by introducing an offset. It is no longer limited to a regular grid position, but is freely sampled on the input feature map as needed, which solves the problem of the model's recall rate for target detection in complex environments and under occlusion, and improves the flexibility and accuracy of feature extraction.

[0026] 3. The method of the present invention has simple steps. The trained bridge abutment retaining wall disease detection model, camera, laser radar detector and GPS locator are deployed on the bridge abutment retaining wall disease inspection vehicle to obtain three-dimensional disease information. While detecting the type and number of bridge abutment retaining wall diseases, the laser radar detector is used to scan the disease at a uniform speed v. The laser radar detector measures the depth h of the bridge abutment retaining wall disease at a time interval t. Through data fitting, the relationship between the time interval t and h is obtained as a parabolic function h=at 2 +bt+c, for the damaged cross section, the velocity v is integrated in the x and y directions to obtain the cross-sectional area, and then the missing volume of the abutment retaining wall damage is calculated, providing data information for subsequent repair and maintenance of the abutment retaining wall, which is convenient for promotion and use.

[0027] In summary, the present invention introduces the C2f_DCNv3 module variable convolution and the PBM parallel multi-scale module, which can dynamically adjust the sampling point position of the convolution kernel according to the shape and position of the input feature, thereby improving the accuracy and rate of bridge abutment retaining wall disease detection and facilitating popularization and use.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The figure is a flowchart of the method of the present invention.

[0030] Figure 2 4 is a model diagram of the C2f_DCNv3 module in an embodiment of the present invention.

[0031] Figure 3 It is a model diagram of the improved PSA module in an embodiment of the present invention.

[0032] Figure 4 It is a model diagram of the PBM module in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] like Figure 1 Medium to Figure 4 As shown, a method for detecting a bridge abutment retaining wall defects of the present invention comprises the following steps:

[0034] Step 1: Construct a database of abutment retaining wall defect images: obtain abutment retaining wall defect images containing various defect types, enhance and annotate the abutment retaining wall defect images, construct an image database, randomly classify the image database, and obtain an image training data set and an image test data set;

[0035] Step 2: Construct a bridge abutment retaining wall disease detection model. The bridge abutment retaining wall disease detection includes Backbone network, Neck network and Head network. Backbone network includes Conv module, C2f module, SCDown module, SPPF module and improved PSA module, which are used to extract high, medium and low layer features of the image; Neck network includes Concat module, Upsample module and improved C2f_DCNv3 module, which are used for image processing and enhancement; Head network includes detection head and classification head. The detection head includes convolution layer and deconvolution layer to generate detection results, including the bounding box regression value of the target and the confidence of the target existence. The classification head uses global average pooling to classify each feature map, and outputs the probability distribution of each category by reducing the dimension of the feature map.

[0036] Among them, the improved PSA module replaces the MHSA multi-head self-attention module in the PSA module with the PBM module;

[0037] It should be noted that the multi-head self-attention module MHSA attention module in the PSA layer of the backbone module is replaced by the PBM module. While modeling the relationship of long-range dependencies, the PBM embeds precise location information into the PBM module. Neurons can obtain spatial information of various scales through their wide local receptive fields. The PBM module extracts the attention weight descriptor of the feature map through four different parallel lines. Two parallel paths are on the 1×1 branch, and two parallel paths are on the 3×3 branch. The channels are encoded along the X and Y directions on the 1×1 branch and the 3×3 branch, respectively. Two 1D global average pooling operations are used. In the two branches, the two encoded features are connected along the image height direction. In the 1×1 branch, So that they share the same 1×1 convolution without reducing the dimension of the 1×1 branch. In the 3×3 branch, so that they share the same 3×3 convolution without reducing the dimension of the 3×3 branch. After decomposing the output of the convolution into two vectors, two nonlinear Sigmoid functions are used to fit the two-dimensional binomial distribution after linear convolution; after redistributing the weights, the results are group normalized, average pooled and activated, and finally the feature matrices of the two branches are multiplied to obtain the final result; the fusion of contextual information at different scales enables the network to generate better pixel-level attention to graphs of high-level features, and a multi-scale parallel sub-network is used to establish short-term and long-term dependencies. The parallel use of 3×3 and 1×1 convolutions can utilize more contextual information in intermediate features.

[0038] Among them, the PBM parallel multi-scale module extracts the attention weight descriptor of the group feature map through four parallel lines, and uses two 1D global average pooling operations in the 1×1 branch and the 3×3 branch respectively to connect the two encoded features along the image height direction. After decomposing the outputs of the 1×1 convolution and the 3×3 convolution into two vectors, two nonlinear Sigmoid functions are used to fit the binomial distribution after the linear convolution, and the feature weights are redistributed. The features extracted by the 1×1 convolution and the 3×3 convolution are connected as outputs, which solves the problems of low feature extraction capability and slow running speed, and solves the problems of more sequential processing and higher latency caused by the large layer depth of the convolutional network. The parallel network makes full use of multi-core processors, improves the efficiency of feature extraction, speeds up the training speed of the model, and improves the recognition speed.

[0039] In addition, C2f in the neck module is replaced by C2f_DCNv3 to form a new convolution module. The sampling position of C2f_DCNv3 is deformed by the offset and represented by the enhanced offset. It is no longer limited to regular grid positions and can be freely sampled on the input feature map as needed. It has stronger generalization ability in shape transformation, scale, aspect ratio, rotation, etc., which makes up for the long-distance dependence and adaptive spatial aggregation ability of traditional convolution and has higher computational efficiency.

[0040] Step 3: Training the abutment retaining wall disease detection model: sequentially retrieve the abutment retaining wall disease images after image enhancement from the image training data set and input them into the abutment retaining wall disease detection model. Use the precision, recall rate and F1 curve of the training result indicators to analyze the advantages and disadvantages of the abutment retaining wall disease detection model, and complete the abutment retaining wall disease detection model training process;

[0041] Among them, the accuracy The proportion of correct predictions among all the results predicted as Positive; Recall The proportion of correct predictions among all results whose true values ​​are Positive; F1 curve TP is the true positive value, FP is the false positive value, and FN is the false negative value;

[0042] Step 4: Testing the abutment retaining wall disease detection model: Retrieving the abutment retaining wall disease images from the image test data set, testing the abutment retaining wall disease detection model until the test is completed, and obtaining the trained and tested abutment retaining wall disease detection model;

[0043] Step 5: Set the anchor frame to select the type of damage to the abutment retaining wall;

[0044] Step 6: Use the abutment retaining wall disease inspection vehicle to collect actual abutment retaining wall disease images: the abutment retaining wall disease inspection vehicle is equipped with a camera, a laser radar detector and a GPS locator;

[0045] Step 7: Input the actual collected image in step 6 into the bridge abutment retaining wall disease detection model trained in step 4, and output the classification prediction result with boundary coordinates, size and category probability;

[0046] Step 8, determine whether the abutment retaining wall defect corresponding to the classification prediction result needs to be repaired: set the missing size threshold of the abutment retaining wall defect, use the laser radar detector to obtain the missing size of the abutment retaining wall defect, and when the missing size of the abutment retaining wall defect is greater than the threshold, repair the abutment retaining wall defect; otherwise, execute step 6 until the abutment retaining wall defect detection is completed;

[0047] Among them, the process of the laser radar detector obtaining the missing size of the abutment retaining wall disease:

[0048] The three-dimensional information of the abutment retaining wall disease is obtained by using the laser radar detector. That is, when the laser radar detector is used to scan the disease at a uniform speed v, the laser radar detector measures the depth h of the abutment retaining wall disease at a time interval t. Through data fitting, the relationship between the time interval t and h is obtained as a parabolic function h = at 2+bt+c, a is the fitting coefficient of the quadratic term of the time interval t in the parabola function, b is the fitting coefficient of the linear term of the time interval t in the parabola function, c is the fitting constant term in the parabola function, for the cross section of the disease, the velocity v is integrated in the x and y directions to obtain the cross-sectional area Volume V = S × h, that is Obtain the missing volume V of the abutment retaining wall defect, obtain the location of the abutment retaining wall defect according to the GPS locator, and upload relevant data to the abutment retaining wall defect detection host computer in real time.

[0049] In this embodiment, in step one, the abutment retaining wall disease image is processed by random cropping, resizing, flipping, rotating, scaling, and color gamut change to increase data diversity and effectively improve the performance and robustness of the model.

[0050] In this embodiment, the defect types include cracks, voids, breakages and settlements.

[0051] In this embodiment, the number of abutment retaining wall disease images of the various defect types is 5,000 to 10,000, and the image database is randomly classified according to a ratio of 8:2 to obtain an image training data set and an image test data set.

[0052] In this embodiment, in step 3, the true positive value TP is the number of positive class predictions, that is, the correct prediction, the actual value is 0, and the prediction is also 0;

[0053] The false positive value FP is the number of negative classes predicted as positive classes, that is, wrong prediction, the actual value is 1, and the prediction is 0;

[0054] The false negative value FN is the number of positive classes predicted as negative classes, that is, wrong prediction, the actual value is 0, and the prediction is 1.

[0055] When the present invention is used, the PBM parallel multi-scale module is used to replace the MHSA attention mechanism in the PAS module in the backbone network, and a more powerful structure for processing short-term and long-term dependencies is adopted, which solves the problem of feature loss and gradient disappearance caused by failure to better utilize context information in intermediate features, improves the model's perception and feature extraction capabilities for objects of different scales, and improves recognition accuracy. The C2f_DCNv3 convolution module is used to replace the C2f convolution module in the neck module, and deformable convolution is introduced. The sampling position is deformed by introducing an offset, and is no longer limited to a regular grid position, but is freely sampled on the input feature map as needed, which solves the problem of the model's target detection recall rate in complex environments and under occlusion, and improves the flexibility and accuracy of feature extraction. The trained abutment retaining wall disease detection model, camera, lidar detector and GPS locator are deployed on the abutment retaining wall disease inspection vehicle to obtain three-dimensional disease information. While detecting the type and quantity of abutment retaining wall diseases, the missing volume of abutment retaining wall diseases is calculated, providing data information for subsequent abutment retaining wall repair and maintenance.

[0056] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting bridge abutment retaining wall defects, characterized in that: The method comprises the following steps: Step 1: Construct a database of abutment retaining wall defect images: obtain abutment retaining wall defect images containing various defect types, enhance and annotate the abutment retaining wall defect images, construct an image database, randomly classify the image database, and obtain an image training data set and an image test data set; Step 2: Construct a bridge abutment retaining wall disease detection model. The bridge abutment retaining wall disease detection includes Backbone network, Neck network and Head network. Backbone network includes Conv module, C2f module, SCDown module, SPPF module and improved PSA module, which are used to extract high, medium and low layer features of the image; Neck network includes Concat module, Upsample module and improved C2f_DCNv3 module, which are used for image processing and enhancement; Head network includes detection head and classification head. The detection head includes convolution layer and deconvolution layer to generate detection results, including the bounding box regression value of the target and the confidence of the target existence. The classification head uses global average pooling to classify each feature map, and outputs the probability distribution of each category by reducing the dimension of the feature map. Among them, the improved PSA module replaces the MHSA multi-head self-attention module in the PSA module with the PBM module; Step 3: Training the abutment retaining wall disease detection model: sequentially retrieve the abutment retaining wall disease images after image enhancement from the image training data set and input them into the abutment retaining wall disease detection model. Use the precision, recall rate and F1 curve of the training result indicators to analyze the advantages and disadvantages of the abutment retaining wall disease detection model, and complete the abutment retaining wall disease detection model training process; Among them, the accuracy The proportion of correct predictions among all the results predicted as Positive; Recall The proportion of correct predictions among all results whose true values ​​are Positive; F1 curve TP is the true positive value, FP is the false positive value, and FN is the false negative value; Step 4: Testing the abutment retaining wall disease detection model: Retrieving the abutment retaining wall disease images from the image test data set, testing the abutment retaining wall disease detection model until the test is completed, and obtaining the trained and tested abutment retaining wall disease detection model; Step 5: Set the anchor frame to select the type of damage to the abutment retaining wall; Step 6: Use the abutment retaining wall disease inspection vehicle to collect actual abutment retaining wall disease images: the abutment retaining wall disease inspection vehicle is equipped with a camera, a laser radar detector and a GPS locator; Step 7: Input the actual collected image in step 6 into the bridge abutment retaining wall disease detection model trained in step 4, and output the classification prediction result with boundary coordinates, size and category probability; Step 8, determine whether the abutment retaining wall defect corresponding to the classification prediction result needs to be repaired: set the missing size threshold of the abutment retaining wall defect, use the laser radar detector to obtain the missing size of the abutment retaining wall defect, and when the missing size of the abutment retaining wall defect is greater than the threshold, repair the abutment retaining wall defect; otherwise, execute step 6 until the abutment retaining wall defect detection is completed; Among them, the process of the laser radar detector obtaining the missing size of the abutment retaining wall disease: The three-dimensional information of the abutment retaining wall disease is obtained by using the laser radar detector. That is, when the laser radar detector is used to scan the disease at a uniform speed v, the laser radar detector measures the depth h of the abutment retaining wall disease at a time interval t. Through data fitting, the relationship between the time interval t and h is obtained as a parabolic function h = at 2 +bt+c, a is the fitting coefficient of the quadratic term of the time interval t in the parabola function, b is the fitting coefficient of the linear term of the time interval t in the parabola function, c is the fitting constant term in the parabola function, for the cross section of the disease, the velocity v is integrated in the x and y directions to obtain the cross-sectional area Volume V = S × h, that is Obtain the missing volume V of the abutment retaining wall defect, obtain the location of the abutment retaining wall defect according to the GPS locator, and upload relevant data to the abutment retaining wall defect detection host computer in real time.

2. A method for detecting bridge abutment retaining wall defects according to claim 1, characterized in that: In step one, the abutment retaining wall disease image is processed by random cropping, resizing, flipping, rotating, scaling, and color gamut changes to increase data diversity and effectively improve the performance and robustness of the model.

3. A method for detecting bridge abutment retaining wall defects according to claim 1, characterized in that: The defect types include cracks, voids, breakages and settlements.

4. A method for detecting bridge abutment retaining wall defects according to claim 1, characterized in that: The number of abutment retaining wall disease images of the various defect types is 5,000 to 10,000, and the image database is randomly classified according to a ratio of 8:2 to obtain an image training data set and an image test data set.

5. A method for detecting bridge abutment retaining wall defects according to claim 1, characterized in that: In step 3, the true positive value TP is the number of positive class predictions, that is, the correct prediction, the actual value is 0, and the prediction is also 0; The false positive value FP is the number of negative classes predicted as positive classes, that is, wrong prediction, the actual value is 1, and the prediction is 0; The false negative value FN is the number of positive classes predicted as negative classes, that is, wrong prediction, the actual value is 0, and the prediction is 1.

Images