Bridge defect detection method based on acousto-optic joint detection and identification

Abstract

The invention discloses a bridge defect detection method based on acousto-optic joint detection and identification. The method comprises the steps that a bridge defect detection deep learning network is established and trained; a cloud deck carrying a visible light camera is used for obtaining a bridge concrete surface image and preprocessing the image, then the image is input into the bridge defect detection deep learning network for defect judgment, and the defect of the bridge concrete surface is recognized through a feature engineering method; detecting a defect signal in the bridge concrete by using an ultrasonic flaw detection device, and filtering to obtain a defect filtering signal; and inputting the defect filtering signal into a neural network for processing, wherein the neural network identifies the defect in the bridge concrete. According to the invention, an acousto-optic combined detection and identification technology is adopted, the surface defects of the bridge can be identified, the internal defects of the bridge can be detected, and the detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The invention relates to a bridge defect detection method, in particular to a bridge defect detection method based on acoustic and optical combined detection and identification. Background Art

[0002] Large bridges, represented by cross-sea bridges, are an important part of transportation facilities and one of the key links in the arteries of transportation. Cross-sea bridges are located at sea, with harsh working environments, large volumes and structures, and complex system components, which poses great challenges to the operation and maintenance of bridges. Existing technical equipment is not only complex to operate, has high work intensity, and high safety risks, but also has a large number of dead zones and blind spots, which make it impossible to effectively detect and maintain them, thus bringing serious safety hazards to the long-term safe and stable operation of bridges. It is urgent to establish an effective bridge defect detection method.

[0003] The current method of detecting bridge defects based on infrared thermometer temperature data requires the configuration of a heating / cooling unit to heat the bridge. The heating / cooling unit includes a water source, a steam engine, a steam booster pump, a gas tank, a valve, a pipeline, etc. It is relatively bulky and cannot be operated lightly. There are also a large number of dead zones and blind spots in the operation, which makes it impossible to effectively detect and maintain the bridge. The current image detection method based on multi-camera processes the detected image through image stitching technology, but can only detect surface defects of the bridge.

[0004] With the development of sensor technology, the methods of bridge defect detection are becoming more and more diverse, so many technologies that combine multiple sensor information for bridge defect detection have emerged. For example, a bridge defect detection method that combines surface defect acquisition sensor and internal defect acquisition sensor information is currently provided. The internal defect acquisition unit (such as an ultrasonic probe) and a control unit are installed on the side of the bridge, and the surface defect acquisition unit (such as a rebound tester probe) is installed on the upper and lower surfaces of the bridge to detect bridge defects. However, its defects are quite obvious: the sensor installation position is fixed and cannot be moved flexibly. If defect detection in multiple areas is to be achieved, the sensor needs to be repeatedly installed, which is costly. This method uses the architecture of acquisition unit-server-intelligent terminal to construct the acquisition unit, which is troublesome to install and maintain.

[0005] The bridge defect detection technology in the prior art has at least the following defects: 1. The detection device is complex to operate, has high work intensity, high safety risks, and there are a large number of dead zones and blind spots, which make it impossible to effectively detect and maintain the bridge; 2. The bridge defect detection method is single, and it is impossible to combine the advantages of various detection sensors to achieve multi-dimensional and multi-means bridge defect detection; 3. The sensor is fixed and cannot be moved flexibly. If large-area monitoring is required, it will take a lot of time and the sensor will have to be reinstalled repeatedly. Summary of the invention

[0006] In order to solve the problems existing in the background technology, the present invention provides a bridge defect detection method based on combined acoustic and optical detection and identification.

[0007] The technical solution adopted by the present invention is:

[0008] The bridge defect detection method based on acoustic and optical joint detection and identification of the present invention comprises:

[0009] Step 1: Preprocess several bridge concrete surface images with and without defects and construct them into a training set; establish a deep learning network for bridge defect detection, input the training set into the deep learning network for bridge defect detection for training, and obtain a trained deep learning network for bridge defect detection.

[0010] Step 2: Use a gimbal equipped with a visible light camera to obtain and preprocess the bridge concrete surface image, and then input it into the trained bridge defect detection deep learning network for processing, so as to determine whether the current bridge concrete surface image contains defects, and identify the defect parameters in the defective bridge concrete surface image through feature engineering methods, and identify the defects on the bridge concrete surface.

[0011] Step three, use an ultrasonic flaw detection device to detect the internal defect signal of the bridge concrete and obtain a defect filter signal after filtering.

[0012] Step 4: Input the defect filter signal into the neural network for processing, and the neural network identifies the defects inside the bridge concrete.

[0013] Step 5: Combine the defect parameters in the bridge concrete surface image identified in step 2 and the defects inside the bridge concrete identified in step 4 to achieve overall detection of bridge defects.

[0014] In the step 1, the preprocessing specifically includes sequentially performing image enhancement, noise reduction, size unification and grayscale processing.

[0015] In the step 1, the deep learning network for bridge defect detection includes a segmentation network and a classification network connected in sequence. The bridge concrete surface image is first input into the segmentation network for pixel-level segmentation, and each pixel in the bridge concrete surface image is used as a training sample for training and its cross entropy loss value is calculated, and then back-propagated to the segmentation network for adjusting the network weights until the cross entropy loss value converges to obtain a trained segmentation network, and the weights of the trained segmentation network are frozen; the processing results of the segmentation network are then input into the classification network for training and the cross entropy loss value is calculated, and then back-propagated to the classification network for adjusting the network weights until the cross entropy loss value converges to obtain a trained classification network.

[0016] The two networks need to be learned separately. First, only the segmentation network is trained. After the training is completed, the weights of the segmentation network are frozen and then the classification network is trained. The loss is calculated through the cross entropy loss function, so that the weights are adjusted to minimize the loss value.

[0017] The output result of the segmentation network of the bridge defect detection deep learning network is the probability that each pixel block in the bridge concrete surface image belongs to a defective part, and the output result of the classification network is the probability that the bridge concrete surface image belongs to a defective image. When the output probability of the classification network is greater than the preset probability threshold, the current bridge concrete surface image is judged to be a defective image.

[0018] In the step 2, the feature engineering method is used to first extract the defect features in the surface image of the bridge concrete containing defects by a threshold segmentation method, and then obtain the defect parameters by a skeleton extraction method.

[0019] In the step three, the internal defect signal of the bridge concrete is decomposed by wavelet packet to select three layers of wavelet packets for filtering. In the specific implementation, the db15 wavelet basis is used as the basis function of the wavelet packet decomposition.

[0020] In the step 4, the neural network is specifically a three-layer feedforward BP (Back Propagation) neural network, and the BP neural network includes seven input nodes, fifteen hidden nodes and four output nodes, wherein the hidden layer function adopts the hyperbolic tangent tansig function, and the output layer adopts the linear transfer purelin function.

[0021] The beneficial effects of the present invention are:

[0022] 1. The present invention adopts a collaborative integrated detection method combining sound and light, fully utilizing the advantages of ultrasonic detection of internal structural defects and visual image detection of surface defects, and uses cross-domain multi-dimensional sensor information to perceive bridge defects in multiple dimensions and by multiple means.

[0023] 2. The present invention has a wide monitoring range. By using a camera to take photos, a large number of photos can be taken, and basically all defects of the bridge can be monitored.

[0024] 3. The present invention is easy to use and does not require repeated installation.

[0025] The present invention adopts the acoustic and optical combined detection and identification technology, which can not only identify the surface defects of the bridge, but also detect the internal defects of the bridge, thereby improving the detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail in conjunction with the accompanying drawings. This description introduces specific embodiments consistent with the principles of the present invention by way of example and not limitation, and the description of these embodiments is sufficiently detailed to enable those skilled in the art to practice the present invention, and other embodiments may be used and the structure of each element may be changed and / or replaced without departing from the scope and spirit of the present invention. Therefore, the following detailed description should not be understood in a restrictive sense.

[0028] The bridge defect detection method based on acoustic and optical joint detection and identification of the present invention is specifically as follows:

[0029] Step 1: Preprocess several images of bridge concrete surfaces with and without defects and construct them into a training set; establish a deep learning network for bridge defect detection, input the training set into the deep learning network for bridge defect detection for training, and obtain a trained deep learning network for bridge defect detection. The preprocessing specifically includes image enhancement, noise reduction, uniform size, and grayscale processing in sequence.

[0030] The deep learning network for bridge defect detection includes a segmentation network and a classification network connected in sequence. The bridge concrete surface image is first input into the segmentation network for pixel-level segmentation. Each pixel in the bridge concrete surface image is used as a training sample for training and its cross entropy loss value is calculated. Then, it is back-propagated to the segmentation network for adjusting the network weights until the cross entropy loss value converges to obtain the trained segmentation network, and the weights of the trained segmentation network are frozen. The processing results of the segmentation network are then input into the classification network for training and the cross entropy loss value is calculated. Then, it is back-propagated to the classification network for adjusting the network weights until the cross entropy loss value converges to obtain the trained classification network. The two networks need to be learned separately. First, only the segmentation network is trained. After the training is completed, the weights of the segmentation network are frozen and then the classification network is trained. The loss selection is calculated through the cross entropy loss function, so that the weights are adjusted to minimize the loss value.

[0031] In the specific implementation, the input image of the deep learning network is a grayscale image of 1280×512 pixels; the feature extraction layer of the deep learning network is mainly composed of 11 convolutional layers and 3 maximum pooling layers. In order to improve the convergence speed of the model, a batch normalization layer and a rectified linear unit are added after each convolutional layer.

[0032] The output result of the segmentation network of the deep learning network for bridge defect detection is the probability that each pixel block in the bridge concrete surface image belongs to a defective part, and the output result of the classification network is the probability that the bridge concrete surface image belongs to a defective image. When the output probability of the classification network is greater than the preset probability threshold, the current bridge concrete surface image is judged to be a defective image.

[0033] Step 2: Use a gimbal equipped with a visible light camera to obtain and preprocess the bridge concrete surface image, and then input it into the trained bridge defect detection deep learning network for processing, so as to determine whether the current bridge concrete surface image contains defects, and identify the defect parameters in the bridge concrete surface image containing defects through feature engineering methods, and identify the defects on the bridge concrete surface. The feature engineering method is to first extract the defect features in the bridge concrete surface image containing defects through the threshold segmentation method, and then obtain the defect parameters through the skeleton extraction method. The defect parameters include defect categories such as cracking, breakage, cracks, subsidence, and cracking, as well as the location and quantity of defects.

[0034] Step 3: Use ultrasonic flaw detection device to detect the internal defect signal of bridge concrete and obtain defect filter signal after filtering. The internal defect signal of bridge concrete is filtered by wavelet packet decomposition and three layers of wavelet packets are selected for filtering. In the specific implementation, db15 wavelet basis is used as the basis function of wavelet packet decomposition.

[0035] Step 4: Input the defect filter signal into the neural network for processing, and the neural network identifies the defects inside the bridge concrete.

[0036] The neural network is specifically a three-layer feedforward BP neural network, which includes seven input nodes, fifteen hidden nodes and four output nodes, wherein the hidden layer function adopts the hyperbolic tangent tansig function, and the output layer adopts the linear transfer purelin function.

[0037] Step 5: Combine the defect parameters in the bridge concrete surface image identified in step 2 and the defects inside the bridge concrete identified in step 4 to achieve overall detection of bridge defects.

[0038] The specific embodiments of the present invention are as follows:

[0039] The method of the present invention can be applied or combined with a variety of bridge defect detection and maintenance equipment, such as being applied to a bridge detection flying adsorption robot.

[0040] In step 1, the training samples of the deep learning network are collected from actual cross-sea bridges or similar concrete structures and annotated using LabelMe software. The image enhancement method of the concrete surface image is implemented using the Retinex theory. After experimental verification, the Retinex scale in this preferred example is selected as three. The denoising algorithm of the concrete surface image adopts the bilateral filtering method.

[0041] In step 2, the visible light camera is mounted on a gimbal, and images of the bridge surface at different positions, angles, and lighting conditions can be acquired by rotating the gimbal.

[0042] The specific operation of the method for identifying concrete structure defect parameters based on feature engineering is as follows:

[0043] The first step is to extract defect features by threshold segmentation. In this preferred embodiment, the Kittler algorithm is used for threshold segmentation. Then, the pseudo noise generated in the threshold segmentation result is removed, that is, the smaller connected domain in the background image. The search method for the connected domain adopts the seed filling method. The specific method is: traverse the image in sequence. When encountering an unlabeled foreground pixel (gray value is 255), it is used as a seed point to determine the pixel values ​​of other points in its eight neighborhoods. If there are other pixels, they are classified as pixels in the same connected domain as the seed point. After obtaining the connected domain labeled image, a statistical method based on morphological area and contour features is used to distinguish the pseudo noise in the background from small cracks. After removing the noise, the crack image is thinned by the skeleton extraction method to make it a single pixel line segment. The Zhang Suen thinning method is selected as the image thinning method. In actual tests, this method has the least crack burrs and the best crack connectivity in the crack image thinning results.

[0044] In the second step, the parameters of defects such as cracks are estimated by using the skeleton extraction method. For the extracted skeleton image, starting from the corresponding starting point on the skeleton image, the crack boundaries are searched for non-skeleton points on both sides, and the sum of the search distances on both sides of each point is calculated, which is the crack width of each point. Further indicators such as the average crack width and maximum crack width can be obtained.

[0045] In step three, the ultrasonic flaw detection device uses the flat measurement method to detect the internal defects of the bridge, that is, the transmitting and receiving transducers are placed on the same side of the concrete to be tested, and the concrete defects are detected based on the reflected waves. The ultrasonic flaw detection device is equipped with a lifting mechanism, which is used to drive the ultrasonic probe to move upward, so that when the drone performs ultrasonic internal defect detection on the surface to be tested at the bottom of the bridge, the probe is first attached to the concrete wall.

[0046] The wavelet packet decomposition method is used to filter out high-frequency noise signals in concrete defect data, and extract multiple eigenvalues ​​related to concrete void defects in the signal. The db15 wavelet basis is selected as the basis function of wavelet packet decomposition, and 3 layers of wavelet packets are selected to analyze the concrete defect signal. After wavelet packet decomposition, the initial signal is decomposed into 8 different frequency band signals, thereby realizing the analysis and feature extraction of signals in different frequency bands.

[0047] In step 4, the BP neural network algorithm is used to identify the internal defects of concrete. A 3-layer BP neural network is selected, with 7 input nodes, 15 hidden nodes, and 4 output nodes.

[0048] Finally, the present invention conducts a comprehensive inspection of bridge defects through the combined detection and identification of sound and light and performs comprehensive data analysis.

Claims

1. A bridge defect detection method based on combined acoustic and optical detection and identification, characterized in that: include: Step 1: Preprocess several bridge concrete surface images with and without defects and construct them into a training set; Establish a deep learning network for bridge defect detection, input the training set into the deep learning network for bridge defect detection for training, and obtain a trained deep learning network for bridge defect detection; Step 2: Use a gimbal equipped with a visible light camera to obtain and pre-process the bridge concrete surface image, and then input it into the trained bridge defect detection deep learning network for processing, so as to determine whether the current bridge concrete surface image contains defects, and identify the defect parameters in the bridge concrete surface image containing defects through feature engineering methods, and identify the defects on the bridge concrete surface; Step 3: Use an ultrasonic flaw detection device to detect the internal defect signal of the bridge concrete and obtain a defect filter signal after filtering; Step 4: Input the defect filter signal into the neural network for processing, and the neural network identifies the defects inside the bridge concrete; Step 5: Combine the defect parameters in the bridge concrete surface image identified in step 2 and the defects inside the bridge concrete identified in step 4 to achieve overall detection of bridge defects.

2. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 1 is characterized by: In the step 1, the preprocessing specifically includes sequentially performing image enhancement, noise reduction, size unification and grayscale processing.

3. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 1 is characterized by: In the step 1, the deep learning network for bridge defect detection includes a segmentation network and a classification network connected in sequence. The bridge concrete surface image is first input into the segmentation network for pixel-level segmentation, and each pixel in the bridge concrete surface image is used as a training sample for training and its cross entropy loss value is calculated, and then back-propagated to the segmentation network for adjusting the network weights until the cross entropy loss value converges to obtain a trained segmentation network, and the weights of the trained segmentation network are frozen; the processing results of the segmentation network are then input into the classification network for training and the cross entropy loss value is calculated, and then back-propagated to the classification network for adjusting the network weights until the cross entropy loss value converges to obtain a trained classification network.

4. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 3 is characterized by: The output result of the segmentation network of the bridge defect detection deep learning network is the probability that each pixel block in the bridge concrete surface image belongs to a defective part, and the output result of the classification network is the probability that the bridge concrete surface image belongs to a defective image. When the output probability of the classification network is greater than the preset probability threshold, the current bridge concrete surface image is judged to be a defective image.

5. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 1 is characterized in that: In the step 2, the feature engineering method is used to first extract the defect features in the surface image of the bridge concrete containing defects by a threshold segmentation method, and then obtain the defect parameters by a skeleton extraction method.

6. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 1 is characterized in that: In the step three, the internal defect signal of the bridge concrete is decomposed by wavelet packet and three layers of wavelet packets are selected for filtering.

7. The bridge defect detection method based on acoustic and optical joint detection and identification according to claim 1 is characterized by: In the step 4, the neural network is specifically a three-layer feedforward BP neural network, which includes seven input nodes, fifteen hidden nodes and four output nodes, wherein the hidden layer function adopts the hyperbolic tangent tansig function and the output layer adopts the linear transfer purelin function.

Images