Bridge underwater disease detection method based on unmanned ship and image enhancement

Abstract

The invention discloses a bridge underwater disease detection method based on an unmanned ship and image enhancement, and the method achieves the accurate recognition of bridge underwater structure diseases. The specific implementation process comprises the following steps: A, acquiring a bridge underwater structure image by using equipment such as an unmanned ship; b, improving the quality of the bridge underwater structure image by adopting an underwater image enhancement algorithm; c, enhancing the structure disease data set image in the land environment to simulate the underwater environment; and D, training and positioning bridge structure diseases by using the disease detection model. According to the method disclosed by the invention, through an image enhancement algorithm, the brightness and definition of the bridge underwater structure image shot by the unmanned ship equipment are improved. A plurality of data set enhancement methods are adopted to solve the difficulty of insufficient bridge underwater disease data sets. And improving a YOLOv11 target detection model by using an attention mechanism, and accurately identifying underwater hidden diseases of the bridge structure. The method can provide better method support for bridge underwater disease detection, and is a basis for bridge underwater structure hidden disease identification and bridge underwater structure service state evaluation.

Description

Technical Field

[0001] The present invention relates to a method for detecting underwater diseases of bridges based on unmanned vessels and image enhancement. An unmanned vessel is used to load an underwater camera to capture images of the underwater structure of a bridge. The quality of the images of the underwater structure of the bridge is improved through an underwater image enhancement algorithm. The training data set is enhanced to simulate the underwater environment, and a trained model is used to detect the diseases of the underwater structure of the bridge with high precision, belonging to the field of structural disease detection. Background Art

[0002] Bridges often span waters such as rivers and lakes. Due to the complex underwater working environment, the service state of the underwater structure of a bridge is vulnerable to various factors such as corrosion, erosion, and scouring, resulting in a decrease in the structural bearing capacity. The detection of hidden diseases of the underwater structure of a bridge is particularly important to ensure the service safety of the bridge structure. Conventional methods for detecting underwater diseases of bridges rely on manual detection by divers, which have difficulties such as low detection efficiency, limited visibility, and high safety risks.

[0003] The detection of underwater diseases of bridges can be divided into contact and non-contact methods. The contact method requires the sensor to be in direct contact with the structure to detect changes in physical parameters and accurately locate the damage position. The contact sensor needs to be in direct contact with the structure, which has limitations such as difficult installation, difficult maintenance, and limited monitoring range. The non-contact method has advantages such as convenient equipment installation and maintenance and a wide monitoring range. In the non-contact method, computer vision technology uses a photographic device to capture images of the underwater structure of a bridge, and then automatically analyzes these images to detect potential diseases of the underwater structure of the bridge. With the rapid development of computer technology, deep learning is applied to the detection of underwater diseases of bridges based on computer vision. The deep learning technology adopts a data-driven training mechanism, can self-learn useful information from a large amount of data, and does not require manual design of detection rules, and can obtain more accurate detection results for underwater diseases of bridges.

[0004] Deep learning models for detecting underwater diseases of bridges require a large amount of high-quality data for training. Due to the turbid and dim underwater environment, the images captured by computer vision devices in the underwater environment are often blurred, have color distortion, and have a large amount of noise, which will reduce the accuracy of disease detection. In order to overcome these difficulties that are not conducive to computer vision underwater disease detection, it is necessary to improve the quality of image data to enhance the performance of the deep learning model. In addition, it is not easy to obtain underwater disease images of bridges, resulting in a scarcity of available training data. It is necessary to enhance the data set of relatively easily obtained land structure disease images to simulate the underwater diseases of bridges, expand the number of pictures in the data set, and be used for accurately detecting underwater diseases of bridges. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention proposes a method for detecting underwater diseases of bridges based on unmanned boats and image enhancement, which is used to accurately identify the diseases of the underwater parts of bridges. The specific content includes:

[0006] A method for detecting underwater diseases of bridges based on unmanned boats and image enhancement, comprising the following steps:

[0007] A. Use equipment such as unmanned boats to collect images of the underwater structure of the bridge;

[0008] B. Adopt an underwater image enhancement algorithm to improve the quality of the underwater structure image of the bridge;

[0009] C. Enhance the images in the structural disease dataset in the land environment to simulate the underwater environment;

[0010] D. Train and use a disease detection model to locate the structural diseases of the bridge.

[0011] Furthermore, step A specifically includes:

[0012] A1. Install an underwater camera on the towing device of the unmanned boat to ensure that the camera can stably capture the underwater structure of the bridge. The underwater camera has the ability to capture high-definition images. The depth of the camera can be adjusted through the towing device.

[0013] A2. Deploy an unmanned boat in the water area where the underwater structure of the target bridge is located. The unmanned boat can be remotely controlled or autonomously controlled and has the ability to freely navigate on the water surface.

[0014] A3. The unmanned boat sails along the surrounding area of the underwater structure of the bridge according to a predetermined route and speed, towing the underwater camera and continuously adjusting the angle and depth of the underwater camera to ensure that the underwater camera can fully cover the underwater structure of the bridge.

[0015] A4. The underwater camera captures the surface of the underwater structure of the bridge and transmits the images in real time. The captured image data is transmitted back to the device terminal through a wireless signal for subsequent detection and analysis.

[0016] Furthermore, step B specifically includes:

[0017] B1. Input the captured underwater structure image of the bridge into the feature extraction module of the PUIE-Net model to initially obtain the image feature f:

[0018] f∈R B×C×H×W (1)

[0019] where B, C, H, and W represent the batch size, number of channels, height, and width respectively.

[0020] B2. The prior / posterior (Pr / Po) module calculates the mean vector Mean and standard deviation vector Std for each channel of f. These two vectors are used to generate Gaussian distributions N of the mean and standard deviation after 1×1 convolution. m and N s . Random samples a from the mean distribution and random samples b from the standard deviation distribution are input into the AdaIN module for feature statistical transformation to ensure that the style of the original image is consistent with the ground truth. The operation of the AdaIN module can be expressed as

[0021]

[0022] where x and y represent the original image and the ground truth, and μ(x) and σ(x) represent the mean and standard deviation of the original image. The features output by the AdaIN module are fed into the output module to generate the final enhanced image.

[0023] B3. The above PUIE-Net model is used to enhance the images of bridge underwater structure diseases multiple times, and the enhanced image with the highest probability close to the waterless environment is selected as the result. This process can be expressed as

[0024] p(y|x) = p(y|z max ,x), z max ~p(z|x) (3)

[0025] where p represents the uncertainty distribution and z represents the uncertainty between the image enhancement result and the ground truth.

[0026] Furthermore, step C specifically includes:

[0027] C1. Collect images of structural diseases in the terrestrial environment, and label the disease locations in the images as the initial dataset.

[0028] C2. Adjust the brightness of the images to weaken the brightness and simulate the dim underwater lighting environment.

[0029] C3. Add different types of noise (such as Gaussian noise, salt-and-pepper noise, etc.) to the images to simulate the image blurring phenomenon caused by factors such as water flow and suspended matter in underwater images.

[0030] C4. Change the image color by adjusting the image saturation and contrast to simulate the colors of different water qualities in the underwater environment.

[0031] C5. Randomly rotate the images to simulate the scenes under different shooting angles.

[0032] C6. Repeat the above dataset enhancement method multiple times, and add the new images obtained from the above steps to the dataset.

[0033] Furthermore, step D specifically includes:

[0034] D1. Improve the YOLOv11 model by adopting the global attention mechanism (GAM), and insert the GAM module between the C3k2 module and the SPPF module. Train the bridge underwater disease detection model using the enhanced disease image dataset.

[0035] D2. Input the underwater structure image of the bridge enhanced by the PUIE-Net model into the trained model to obtain the disease detection result.

[0036] The present invention has the following advantages:

[0037] (1) The image enhancement model is adopted to improve the brightness and clarity of the underwater image, making the enhanced underwater image brighter and clearer, which is beneficial to subsequent detection of underwater diseases of the bridge;

[0038] (2) The YOLOv11 model is improved by adding the global attention mechanism, and the disease detection performance is improved. It can effectively cope with the changes in the camera shooting angle, small-scale diseases, camera shaking, etc. in the underwater environment, and shows strong disease detection robustness;

[0039] (3) A variety of dataset enhancement methods are adopted to transform the relatively easily obtained land environment structure disease images to achieve dataset enhancement, improve the performance of the deep learning model in the underwater environment, and solve the difficulty of insufficient bridge underwater disease datasets. Description of the Drawings

[0040] Figure 1 is the flowchart of the method of the present invention;

[0041] Figure 2a and Figure 2b is the schematic diagram of the bridge underwater disease detection scenario example of the present invention;

[0042] Figure 3 is the schematic diagram of the model structure when the trained PUIE-Net model of the present invention is used for underwater image enhancement;

[0043] Figures 4(a) - 4(e) is the schematic diagram of the dataset enhancement effect of the present invention. Among them, Fig. 4(a) is the original image, Fig. 4(b) is the schematic diagram of light change, Fig. 4(c) is the schematic diagram of adding noise, Fig. 4(d) is the schematic diagram of changing color, and Fig. 4(e) is the schematic diagram of rotating the image;

[0044] Figures 5(a) - 5(e)It is a diagram showing the detection results of structural diseases of the present invention under different underwater conditions (the solid line is the detection result of the method model of the present invention, and the dotted line is the true value of the marked disease position); among them, Fig. 5(a) is the detection result diagram during normal shooting, Fig. 5(b) and Fig. 5(c) are the detection result diagrams of the same damage taken from different angles, Fig. 5(d) is the detection result diagram of minor diseases, and Fig. 5(e) is the detection result diagram when the image is blurred due to camera jitter.

[0045] Figures 6(a) - 6(d) It is a comparison diagram of the results of disease detection after enhancing the underwater structure image of the bridge in the present invention and the results of direct detection without enhancement; among them, Fig. 6(a) is the original image, Fig. 6(b) is the direct detection result diagram, Fig. 6(c) is the underwater structure image enhanced by the present invention, and Fig. 6(d) is the detection result diagram of the enhanced image. Detailed implementation manners

[0046] The following further elaborates on a method for detecting underwater diseases of bridges based on an unmanned ship and image enhancement in the present invention with reference to the accompanying drawings. The implementation method of the present invention is as Figure 1 shown, and specifically includes the following steps:

[0047] A. Use equipment such as an unmanned ship to collect images of the underwater structure of the bridge; specifically including:

[0048] A1. The underwater camera is connected to the unmanned ship by a cable, and a lead weight is suspended below the camera to ensure that the camera can stably photograph the underwater structure of the bridge. The resolution of the underwater camera is 1920*1080. The depth of the camera can be adjusted by extending and shortening the cable, and the cable length can reach 20m.

[0049] A2. Detect the underwater part of the pier of an actual concrete bridge. The test scenarios are as Figure 2a and Figure 2b shown. Deploy an unmanned ship in the water area where the underwater structure of the target bridge is located. The unmanned ship is equipped with an antenna for receiving remote control signals and can freely navigate on the water surface according to the remote control signals.

[0050] A3. The unmanned ship sails around the pier according to a predetermined route and speed, towing the underwater camera and continuously adjusting the angle and depth of the underwater camera to ensure that the underwater camera can fully cover the underwater part of the pier.

[0051] A4. The underwater camera photographs the surface of the underwater structure of the bridge, and transmits the captured image data back to the device terminal through wireless signals for subsequent detection and analysis.

[0052] B. Adopt an underwater image enhancement algorithm to improve the quality of the underwater structure image of the bridge; specifically including:

[0053] B1. Input the captured underwater structure image of the bridge into the feature extraction module of the trained PUIE-Net model (the model structure is as shown in Figure 3 ) to initially obtain the image feature f.

[0054] B2. The prior / posterior (Pr / Po) module calculates the mean vector Mean and the standard deviation vector Std for each channel of f. These two vectors are used to generate the Gaussian distributions N m and N s after 1×1 convolution. The random samples from the mean distribution and the random samples from the standard deviation distribution are input into the AdaIN module for feature statistical transformation to ensure that the style of the original image is consistent with the ground truth. The features output by the AdaIN module are sent to the output module to generate the final enhanced image.

[0055] B3. Use the above PUIE-Net model to enhance the underwater disease image of the bridge multiple times, and select the enhanced image with the maximum probability close to the waterless environment as the result. Compare the enhancement results of this method and the traditional white balance algorithm for an underwater structure image of a bridge, and use the UCIQE and UIQM metrics to evaluate the enhancement effect of the underwater image. The higher the metric, the better the image enhancement effect. For the underwater structure image of the bridge enhanced by the white balance method, the UCIQE and UIQM are 0.1160 and 0.2902 respectively. The UCIQE and UIQM of the image enhanced by the PUIE-Net model are 0.2524 and 0.9678 respectively, which are significantly better than the white balance algorithm. The PUIE-Net model achieves a clearer and brighter enhancement effect than the white balance algorithm.

[0056] C. Enhance the images in the structural disease dataset in the terrestrial environment to simulate the underwater environment; specifically including:

[0057] C1. Use 1183 concrete surface disease images captured in the terrestrial environment as the initial dataset, mainly including typical diseases of concrete structures such as pitting, abrasion, and cracks, and label the disease locations in these images.

[0058] C2. Adjust the brightness of the image to weaken the brightness of the image and simulate the dim underwater lighting environment.

[0059] C3. Add Gaussian noise and salt-and-pepper noise to the image to simulate the image blurring phenomenon caused by factors such as water flow and suspended matter in the underwater image.

[0060] C4. Change the image color by adjusting the image saturation and contrast to simulate the colors of different water qualities in the underwater environment.

[0061] C5. Randomly rotate the image to simulate the scenes under different shooting angles.

[0062] C6. Some of the images obtained by the above dataset augmentation method are as follows Figures 4(a) - 4(e) As shown, repeat the above steps multiple times, and add the obtained new images to the dataset. 5915 images are generated through dataset augmentation, and finally 7098 images are obtained by adding them to the original dataset images. Among them, 5880 images are used as the training set and 1680 images are used as the validation set.

[0063] D. Train and use a disease detection model to locate bridge structure diseases; specifically including:

[0064] D1. Improve the YOLOv11 model using the global attention mechanism (GAM), and insert the GAM module between the C3k2 module and the SPPF module. Use the augmented dataset in step C to train the improved YOLOv11 model for underwater bridge disease detection. The training results are shown in Table 1. All indicators of the improved YOLOv11 model are improved compared with the original model, and higher accuracy than the original model is achieved.

[0065] Table 1 Performance comparison between the original model and the improved model

[0066]

[0067] D2. Input the underwater bridge structure images enhanced by the PUIE-Net model in step B into the trained model to obtain disease detection results. Figure 5 shows some of the disease detection results of the underwater bridge structure. Among them, Figure 5(a) shows the disease detection results of the normally photographed underwater bridge structure, Figures 5(b) and 5(c) show the detection results of the same disease from two different shooting angles on the left and right, Figure 5(d) shows the detection results of minor diseases of the underwater structure, and Figure 5(e) shows the disease detection results of the underwater structure photographed when the camera shakes. The model can accurately detect the diseases of the underwater bridge structure in these cases. Figures 6(a) - 6(d) The comparison of the results of directly performing disease detection and performing disease detection after enhancing the images by this method is shown. Direct detection cannot detect the underwater diseases of the bridge on the image. In contrast, after enhancing the image using the PUIE-Net model of this method, the underwater diseases of the bridge can be fully detected. The method for underwater bridge disease detection based on an unmanned ship and image enhancement proposed by the present invention obtains disease detection results with relatively high accuracy on the actual underwater structure of the bridge.

[0068] The content described in the embodiments of this specification is only an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms stated in the embodiments. The protection scope of the present invention also extends to equivalent technical means that those skilled in the art can think of according to the inventive concept of the present invention.

Claims

1. A method for detecting underwater diseases of bridges based on unmanned boats and image enhancement, comprising the following steps: A. Use equipment such as unmanned boats to collect images of the underwater structure of the bridge; B. Adopt an underwater image enhancement algorithm to improve the quality of the underwater structure image of the bridge; C. Enhance the images in the structural disease dataset in the terrestrial environment to simulate the underwater environment; D. Train and use a disease detection model to locate the structural diseases of the bridge.

2. The method for detecting underwater diseases of a bridge based on an unmanned ship and image enhancement according to claim 1, characterized in that, Step A specifically includes: A1. Install the underwater camera on the towing device of the unmanned boat to ensure that the camera can stably capture the underwater structure of the bridge. The underwater camera has the ability to capture high-definition images. The depth of the camera can be adjusted by the towing device. A2. Deploy the unmanned boat in the water area where the underwater structure of the target bridge is located. The unmanned boat can be remotely controlled or autonomously controlled and has the ability to freely navigate on the water surface. A3. The unmanned boat sails along the surrounding area of the underwater structure of the bridge according to a predetermined route and speed, towing the underwater camera and continuously adjusting the angle and depth of the underwater camera to ensure that the underwater camera can fully cover the underwater structure of the bridge. A4. The underwater camera captures the surface of the underwater structure of the bridge and transmits the images in real time. The captured image data is transmitted back to the device terminal through wireless signals for subsequent detection and analysis.

3. The underwater disease detection method for bridges based on unmanned ships and image enhancement according to claim 1, wherein, Step B specifically includes: B1. Input the captured underwater structure image of the bridge into the feature extraction module of the PUIE-Net model to initially obtain the image feature f: f ∈ R B×C×H×W (1) Among them, B, C, H, and W represent the batch size, number of channels, height, and width respectively. B2. The prior / posterior (Pr / Po) module calculates the mean vector Mean and the standard deviation vector Std for each channel of f. These two vectors are used to generate Gaussian distributions N of the mean and the standard deviation after 1×1 convolution m and N s . The random sample a from the mean distribution and the random sample b from the standard deviation distribution are input into the AdaIN module for feature statistical transformation to ensure that the style of the original image is consistent with the ground truth. The operation of the AdaIN module can be expressed as Where x and y represent the original image and the ground truth, and μ(x) and σ(x) represent the mean and standard deviation of the original image. The features output by the AdaIN module are sent to the output module to generate the final enhanced image. B3. Use the above PUIE-Net model to enhance the underwater structure image of the bridge multiple times, and select the enhanced image with the maximum probability close to the waterless environment as the result. This process can be expressed as p(y|x) = p(y|z max ,x), z max ~p(z|x) (3) Where p represents the uncertainty distribution and z represents the uncertainty between the image enhancement result and the ground truth.

4. The underwater disease detection method for bridges based on unmanned boats and image enhancement according to claim 1, characterized in that, Step C specifically includes: C1. Collect images of structural diseases in the terrestrial environment, mark the disease locations in the images as the initial dataset. C2. Adjust the brightness of the images to weaken the brightness of the images to simulate the dim underwater lighting environment. C3. Add different types of noise (such as Gaussian noise, salt-and-pepper noise, etc.) to the images to simulate the image blurring phenomenon caused by factors such as water flow and suspended matter in underwater images. C4. Change the image color by adjusting the image saturation and contrast to simulate the colors of different water qualities in the underwater environment. C5. Randomly rotate the images to simulate the scenes under different shooting angles. C6. Repeat the above dataset enhancement method multiple times, and add the new images obtained in the above steps to the dataset.

5. The underwater disease detection method for bridges based on unmanned ships and image enhancement according to claim 1, characterized in that, Step D specifically includes: D1. Improve the YOLOv11 model using the global attention mechanism (GAM), and insert the GAM module between the C3k2 module and the SPPF module. Use the enhanced disease image dataset to train the underwater disease detection model of the bridge. D2. Input the underwater structure image of the bridge enhanced by the PUIE-Net model into the trained model to obtain the disease detection result.

Images