Underwater Pier Apparent Disease Detection Method Based on Image Fusion and Deep Learning

Abstract

The present invention relates to an underwater pier apparent disease detection method based on image fusion and deep learning, comprising the following steps: Step S1: Obtain the original image of the underwater pier apparent disease; Step S2: Perform enhancement processing on the original image of the underwater pier apparent disease respectively based on the CLAHE algorithm and the ACE algorithm to obtain the fused image 1 and the fused image 2; Step S3: According to the fused image 1 and the fused image 2, perform weighted fusion through the point sharpness weight and perform sharpening processing using the USM algorithm to obtain the final fused image; Step S4: Input the fused image into the deep learning algorithm model for training, and output an image with disease type and location information.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater structure surface detection, and specifically relates to a method for detecting apparent diseases of underwater bridge piers based on image fusion and deep learning. Background Art

[0002] During the operation of the pile pier structure of a bridge, it is in a complex hydrological environment, and the frequency and degree of diseases are more serious. Its aging process is often accompanied by the occurrence of apparent diseases such as cracks, defects, exposed steel bars, and bulges. The lack of detection of bridge underwater structure diseases and the neglect of monitoring the underwater structure of in-service bridges are the main reasons for the frequent occurrence of bridge collapse accidents. Due to the influence of light and water quality, the images obtained by optical devices are not only relatively blurred, with color distortion, but also full of various noises. Coupled with the large number of disease images obtained, if the traditional method is still used and manual interpretation is relied on, not only is the efficiency low, but also it will be affected by the subjective factors of the detection personnel, resulting in poor recognition accuracy and unable to meet the actual engineering requirements. In order to achieve high-efficiency identification and positioning of diseases of underwater pile pier structures and ensure the safety of bridge underwater structures, it is urgent to study an automatic and high-precision identification and classification method. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method for detecting apparent diseases of underwater bridge piers based on image fusion and deep learning, so as to solve the problems that the detection of apparent diseases of underwater bridge piers is greatly affected by image quality and the recognition efficiency is slow.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A method for detecting apparent diseases of underwater bridge piers based on image fusion and deep learning includes the following steps:

[0006] Step S1: Obtain the original image of the apparent disease of the underwater pier column;

[0007] Step S2: Perform enhancement processing on the original image of the apparent disease of the underwater pier column based on the CLAHE algorithm and the ACE algorithm respectively to obtain the fused image 1 and the fused image 2;

[0008] Step S3: According to the fused image 1 and the fused image 2, perform weighted fusion through point sharpness weights and perform sharpening processing using the USM algorithm to obtain the final fused image

[0009] Step S4: Input the fused image into the trained deep learning algorithm model, and output the image with disease types and location information.

[0010] Further, the enhancement processing based on the CLAHE algorithm is specifically as follows:

[0011] (1) Divide the original image of the underwater pier column's apparent diseases into several sub-region images according to the size of the original image.

[0012] (2) Establish the histogram H(x) of each sub-region and calculate its clipping amplitude T.

[0013]

[0014] Where: c is the acquisition coefficient; H and W are the number of pixels in the height and width directions of the sub-region image; M is the number of gray levels.

[0015] (3) Fill the part of the histogram above the threshold T to the bottom of the histogram to obtain a new histogram H’(x).

[0016] (4) Perform bilinear interpolation on different sub-region images to obtain the fused image 1.

[0017] Furthermore, the enhancement process based on the ACE algorithm is as follows:

[0018] (1) Adjust the original image of the underwater pier column's apparent diseases in the image domain through the formula to obtain the intermediate image Rz.

[0019]

[0020] Where: R z (k) is the brightness value of the pixel point k in the intermediate image; I z (k) - I z (q) is the lateral inhibition mechanism; d(k,q) is the distance function; r(x,y) represents the brightness function of each pixel.

[0021] (3) Perform dynamic range adjustment on the obtained intermediate image Rz, substitute it into the formula to calculate O z , that is, obtain the fused image 2;

[0022] O z (k) = round[127.5 + s z R z (k)]

[0023] Where round() is the rounding function; s z is the slope of the line segment [(m z ,0),(M z ,255)], m z and M z are calculated as follows:

[0024]

[0025]

[0026] Further, step S3 is specifically as follows:

[0027] Step S31: Calculate the corresponding point sharpness value E and weight coefficient W for the obtained fused image 1 and fused image 2 G and weight coefficient W A 、W C ;

[0028]

[0029]

[0030]

[0031] where m and n are the image sizes; dG / dx represents the gray level change rate; E is the calculated point sharpness value; E(G) G and E(G) A and E(G) C are the image point sharpness values after enhancement by the ACE algorithm and the CLAHE algorithm respectively; W A and W C are the corresponding image weight coefficients;

[0032] Step S32: Decompose the obtained fused image 1 and fused image 2 into three RGB single-channel images respectively, and perform weighted pixel fusion on the three RGB single-channel images using the calculated weight coefficients;

[0033] Step S33: Recombine the fused 3 RGB single-channel images to obtain the fused image 3;

[0034] Step S34: Adopt the USM algorithm, first perform Gaussian blur on the fused image 3, then subtract the image after Gaussian blur multiplied by the weight coefficient from the fused image 3, and finally obtain the final fused image through denoising filtering and convolution operations.

[0035] Further, the specific calculation formula of step S34 is as follows:

[0036]

[0037] where f(a,b) represents the input image; h(a,b) is the high-frequency component; ω is the sharpening coefficient, g σ and represent denoising filtering and convolution operations respectively.

[0038] Further, the deep learning algorithm model adopts the YOLOv3 algorithm model.

[0039] The present invention has the following beneficial effects compared with the prior art:

[0040] The present invention effectively improves the accuracy and efficiency of identifying underwater diseases. Description of the Drawings

[0041] Figure 1 is a flowchart of the image processing method of the present invention;

[0042] Figure 2 is a flowchart of the identification method of the present invention;

[0043] Figure 3 is the underwater disease image fusion algorithm process in an embodiment of the present invention;

[0044] Figure 4 is the YOLOv3 network model structure in an embodiment of the present invention;

[0045] Figure 5 is the original image captured by the optical camera in an embodiment of the present invention;

[0046] Figure 6 is the image enhanced by the CLAHE algorithm in an embodiment of the present invention;

[0047] Figure 7 is the image enhanced by the ACE algorithm in an embodiment of the present invention;

[0048] Figure 8 is the final fused image in an embodiment of the present invention;

[0049] Figure 9 is the result image output by the YOLOv3 deep learning model in an embodiment of the present invention. Detailed Embodiment

[0050] The present invention will be further described below with reference to the drawings and embodiments.

[0051] Please refer to Figures 1-9 , the present invention provides an underwater bridge pier apparent disease detection method based on image fusion and deep learning, including the following steps:

[0052] Step S1: Obtain the original image of the underwater pier column apparent disease;

[0053] Step S2: Perform enhancement processing on the original image of the underwater pier column apparent disease based on the CLAHE algorithm and the ACE algorithm respectively to obtain the fused image 1 and the fused image 2;

[0054] Step S3: According to the fused image 1 and the fused image 2, perform weighted fusion through the point sharpness weight and perform sharpening processing using the USM algorithm to obtain the final fused image

[0055] Step S4: Input the fused image into the trained deep learning algorithm model, and output an image with disease type and location information.

[0056] In this embodiment, an underwater pier apparent disease image is obtained through an underwater optical camera. After determining the location of the underwater pier disease, the device equipped with the underwater camera is controlled to make the camera face the disease of the pier for optical image shooting to obtain the original underwater disease image.

[0057] In this embodiment, enhancement processing is performed based on the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm, specifically as follows:

[0058] (1) According to the size of the original underwater pier apparent disease image, it is segmented into several sub-region images;

[0059] (2) Establish the histogram H(x) of each sub-region and calculate its clipping amplitude T;

[0060]

[0061] Where: c is the acquisition coefficient; H and W are the number of pixels in the height and width directions of the sub-region image; M is the number of gray levels;

[0062] (3) Fill the part of the histogram higher than the threshold T to the bottom of the histogram to obtain a new histogram H'(x);

[0063] (4) Perform bilinear interpolation operation on different sub-region images to obtain the fused image 1.

[0064] In this embodiment, enhancement processing is performed based on the Automatic Color Equalization (ACE) algorithm, specifically as follows:

[0065] (1) Adjust the original underwater pier apparent disease image in the image domain through the formula to obtain the intermediate image Rz;

[0066]

[0067] Where: R z (k) is the brightness value of the pixel point k of the intermediate image; I z (k) - I z (q) is the lateral inhibition mechanism; d(k,q) is the distance function; r(x,y) represents the brightness function of each pixel;

[0068] (3) Perform dynamic range adjustment on the obtained intermediate image R z and substitute it into the formula to calculate O z , that is, obtain the fused image 2;

[0069] O z (k) = round[127.5 + sz R z (k)]

[0070] where round() is the rounding function; s z is the slope of the line segment [(m z ,0),(M z ,255)], and m z and M z are calculated as follows:

[0071]

[0072]

[0073] In this embodiment, step S3 is specifically as follows:

[0074] Step S31: Calculate the corresponding point sharpness values E G and weight coefficients W A and W C respectively according to the obtained fused image 1 and fused image 2;

[0075]

[0076]

[0077]

[0078] where m and n are the image sizes; dG / dx represents the gray level change rate; E G is the calculated point sharpness value; E(G) A and E(G) C are the image point sharpness values after enhancement by the ACE algorithm and the CLAHE algorithm respectively; W A and W C are the corresponding image weight coefficients;

[0079] Step S32: Decompose the obtained fused image 1 and fused image 2 into three RGB single-channel images respectively, and perform weighted pixel fusion on the three RGB single-channel images using the calculated weight coefficients;

[0080] Step S33: Recombine the fused 3 RGB single-channel images to obtain the fused image 3;

[0081] Step S34: Adopt the USM algorithm. First, perform Gaussian blur on the fused image 3, then subtract the image after Gaussian blur multiplied by the weight coefficient from the fused image 3, and finally, through denoising filtering and convolution operations, obtain the final fused image. The specific calculation formula is as follows:

[0082]

[0083] Among them, f(a, b) represents the input image; h(a, b) is the high-frequency component; ω is the sharpening coefficient, and g σ and respectively represent denoising filtering and convolution operations.

[0084] In this embodiment, the deep learning algorithm model adopts the YOLOv3 algorithm model. The YOLOv3 deep learning algorithm model includes an input module, a feature extraction module, a neck module, and a prediction module;

[0085] A large number of original images are passed through steps S1 - S3 to obtain a fused image training set, which is input into the input module of the YOLOv3 algorithm model for model training, and the trained model and parameters are saved. As shown in the appendix Figure 3 shown, use the bounding box loss function L location , the classification loss function L class and the improved confidence loss function L object for training, and output an image with disease location information and type information, and save the trained model and parameters.

[0086] The above are only the preferred embodiments of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims

1. An underwater bridge pier apparent disease detection method based on image fusion and deep learning, characterized in that, It includes the following steps: Step S1: Obtain the original image of the apparent diseases of the underwater pier column; Step S2: Perform enhancement processing on the original image of the apparent diseases of the underwater pier column respectively based on the CLAHE algorithm and the ACE algorithm to obtain the fused image 1 and the fused image 2; Step S3: According to the fused image 1 and the fused image 2, perform weighted fusion through the point sharpness weight and adopt the USM algorithm for sharpening processing to obtain the final fused image; Step S4: Input the fused image into the trained deep learning algorithm model and output the image with disease type and location information; The specific content of the said step S3 is as follows: Step S31: Calculate the corresponding point sharpness value E for the obtained fused image 1 and fused image 2 respectively G and the weight coefficient W A 、W C ; where m and n are the image sizes; dG / dx represents the gray-scale change rate; E G is the calculated point sharpness value; E(G) A and E(G) C are the image point sharpness values after enhancement by the ACE algorithm and the CLAHE algorithm respectively; W A and W C are the corresponding image weight coefficients; Step S32: Decompose the obtained fused image 1 and fused image 2 into three RGB single-channel images respectively, and perform weighted pixel fusion on the three RGB single-channel images by using the calculated weight coefficients; Step S33: Recombine the 3 fused RGB single-channel images to obtain the fused image 3; Step S34: Adopt the USM algorithm. First, perform Gaussian blur on the fused image 3, then subtract the image after Gaussian blur multiplied by the weight coefficient from the fused image 3, and finally, through denoising filtering and convolution operations, obtain the final fused image; The specific calculation formula of the said step S34 is as follows: Among them, f(a, b) represents the input image; h(a, b) is the high-frequency component; ω is the sharpening coefficient, and g σ and represent denoising filtering and convolution operations respectively; The deep learning algorithm model adopts the YOLOv3 algorithm model.

2. The underwater pier apparent disease detection method based on image fusion and deep learning according to claim 1, wherein The enhancement processing based on the CLAHE algorithm is specifically as follows: (1) According to the size of the original image of the apparent diseases of the underwater pier column, divide it into several sub-region images; (2) Establish the histogram H(x) of each sub-region and calculate the clipping amplitude T of the histogram H(x) as the threshold; Where: c is the acquisition coefficient; H and W are the number of pixels in the height and width directions of the sub-region image; M is the number of gray levels; (3) Fill the part of the histogram higher than the threshold T to the bottom of the histogram to obtain the new histogram H'(x); (4) Perform bilinear interpolation operation on different sub-region images to obtain the fused image 1.

3. The underwater pier apparent disease detection method based on image fusion and deep learning according to claim 1, wherein The enhancement processing based on the ACE algorithm is specifically as follows: (1) The original image of the apparent diseases of the underwater pier column is adjusted in the image domain through a formula to obtain an intermediate image R z ; Where: R z (k) is the luminance value of the intermediate image pixel point k; I z (k) - I z (q) is the lateral inhibition mechanism; d(k, q) is the distance function; r() represents the luminance function; (2) Dynamically adjust the dynamic range of the obtained intermediate image Rz, and substitute it into the formula to calculate O z , that is, the fused image 2 is obtained; O z (k) = round[127.5 + s z R z (k)] where round() is the rounding function; s z is the slope of the line segment [(m z ,0), (M z ,255)], and m z , M z are calculated as follows:

Images