A combined optical and acoustic method for typical defect detection in water tunnels by AUV

Abstract

The present invention discloses an optical-acoustic combination method for AUV to detect typical defects in a water tunnel, including: step 1: use a camera and a light to shoot an image of the water tunnel, and perform image denoising and image enhancement preprocessing on the obtained optical image; step 2: detect typical defects in the tunnel, and obtain the pixel size of the typical defects, and the typical defects include: cracks and falling blocks; step 3: calculate the actual size of the typical defects according to the pixel size combined with the distance of the ranging sonar; step 4: compare the actual size with the engineering requirements, and mark it if it is greater than the engineering requirements. The advantages of the present invention are: it can calculate the size of the defects in the water tunnel, the result obtained is accurate, the measurement time is short, the engineering amount is small, the efficiency is high, the safety requirements for regular inspection of the water tunnel are low, and it will not cause safety hazards in the water tunnel, and it can meet the needs of regular inspection of the water tunnel.

Description

Technical Field

[0001] The present invention relates to the technical field of water conveyance tunnel detection, and in particular to an optical-acoustic combination method for detecting typical defects in a water conveyance tunnel using an AUV (Autonomous Underwater Vehicle). Background Art

[0002] With the development of the economy, the demand for water resources is increasing. At the same time, the distribution of resources in my country is uneven. Therefore, many cross-regional water diversion projects have been built in various regions. Among them, water diversion tunnels are widely used as an efficient and simple means of transportation. A water diversion tunnel refers to a tunnel-type water diversion building used to transport water for power generation, irrigation, industry, and life. Tunnels are usually buried deep underground. Since the terrain through which the tunnel passes may be relatively curved, the shape of the tunnel is also curved. At the same time, after a long period of water diversion, cracks and other problems will occur in the tunnel. If it is not repaired, it may cause wall ruptures or even tunnel collapse, which will have a serious impact on safety and life. Therefore, AUV is used to detect typical defects such as cracks and falling blocks in water diversion tunnels and describe the size of the defects.

[0003] Due to the complex environment of water tunnels, the typical method for detecting cracks and falling blocks in tunnels is usually to use the emptying method. The measurement of the size of cracks and landslides is time-consuming and labor-intensive, and there are problems of high workload and low efficiency. Under such complex conditions, the safety requirements for regular inspection of water tunnels are high. The lack of a good defect detection method may lead to safety hazards in water tunnels, making it difficult to meet the needs of regular inspections of water tunnels. The size of AUV cracks and falling blocks in water tunnels is the premise and guarantee for carrying out crack and falling block repair tasks. How to accurately identify the size of cracks, landslides and other defects by optical and acoustic combination has always been an urgent problem to be solved. Summary of the invention

[0004] In view of the defects of the prior art, the present invention provides an optical-acoustic combined method for AUV to detect typical defects in a water conveyance tunnel.

[0005] In order to achieve the above invention object, the technical solution adopted by the present invention is as follows:

[0006] An optical-acoustic combined method for AUV to detect typical defects in a water conveyance tunnel comprises the following steps:

[0007] Step 1: Use a camera and a light to capture the image of the water conveyance tunnel, and perform image denoising and image enhancement preprocessing on the obtained optical image;

[0008] Step 2: Detect typical defects in the tunnel and obtain the pixel size of the typical defects, which include cracks and chipping;

[0009] Step 3: Calculate the actual size of typical defects based on the pixel size combined with the distance of the ranging sonar;

[0010] Step 4: Compare the actual size with the engineering requirements, and mark it if it is larger than the engineering requirements.

[0011] Preferably, the step 1 is specifically as follows:

[0012] Step 1.1: Use an optical camera combined with underwater lighting to take pictures of the water tunnel, and use mean filtering to remove noise;

[0013] Select a pixel point F(x,y) in the image except the first row, first column, last row and last column, whose pixel value is f(x,y), select a window with F(x,y) as the center point, the window size is an odd number, and calculate the average value of all pixel values ​​in the window to replace f(x,y). The calculation formula is:

[0014] g(x,y)=∑f(x,y) / n 2

[0015] Where g(x,y) represents the pixel value after mean processing, and n is the processing window size.

[0016] Step 1.2: Use the histogram equalization image enhancement method to evenly distribute the more concentrated pixels in the image within the gray value range to enhance the image contrast;

[0017] Furthermore, the histogram equalization image enhancement method described in step 1.2 is as follows:

[0018] Step 1.2.1: Calculate the grayscale histogram n of the original image k ,k=0,1,...,255;

[0019] Step 1.2.2: N is the total number of pixels, through n k Find the grayscale distribution probability P(k) of the original image:

[0020]

[0021] Step 1.2.3: Calculate the grayscale cumulative distribution probability P of the original image s :

[0022]

[0023] Step 1.2.4: Calculate the pixel value f(i,j) after histogram equalization:

[0024] f(i,j)=N·P s (k)

[0025] Where i, j is the pixel in the i-th row and j-th column in the image.

[0026] Preferably, the step 2 is specifically as follows:

[0027] Step 2.1: Collect pictures of typical defects in the water tunnel, use the adversarial network to transfer the style of the simulated typical defect pictures of the water tunnel, generate a large number of typical defect images for semantic segmentation, and annotate them.

[0028] Step 2.2: Use the semantic segmentation algorithm to segment each typical defect image and train the segmentation model corresponding to each typical defect;

[0029] Step 2.3: In the computer processing center, the image is segmented using the corresponding segmentation model to distinguish the area of ​​each typical defective pixel and perform pixel size statistics;

[0030] The step 3 is specifically as follows:

[0031] Step 3.1: Perform Kalman filtering on the distance value extracted by the ranging sonar and select a stable distance. The stable distance means that after the underwater robot determines the distance, the ranging sonar will have some erroneous values, which are filtered out using filtering.

[0032] Step 3.2: Based on the distance obtained by the ranging sonar, add the distance between the ranging sonar and the camera to get the distance between the camera and the water tunnel wall, and calculate the actual size of the typical defect according to the conversion formula, as follows:

[0033] distance = a + b

[0034] actual area=(pixel area / image area)*((k*distance)^2)

[0035] Where a is the distance obtained by the ranging sonar, b is the vertical distance from the camera to the ranging sonar, and distance is the distance from the camera to the tunnel wall. k is the proportional coefficient, which refers to the coefficient to be multiplied for pixel conversion at the current distance. It needs to be determined based on the distance, the pixel size captured at the current distance, the ratio of the actual size, and the difference in cameras. Pixel area is the pixels occupied by typical defects, and image area is all pixels in the photo.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] The present invention can shoot typical defects such as cracks and falling blocks in the tunnel through the AUV equipped with a camera, and can segment the typical defects such as cracks and falling blocks, and can calculate the size of the defects in the water transfer tunnel in combination with the distance from the tunnel wall measured by the ranging sonar. The result obtained is accurate, which plays an auxiliary role in the subsequent operation of the AUV and the repair of the water transfer tunnel. The present invention is based on the camera of the AUV to shoot in the tunnel, segment the photographed pictures to determine the size of the pixels, and combine the distance from the water transfer tunnel wall measured by the ranging sonar to calculate the actual area size of the typical cracks and falling blocks in the water transfer tunnel. The measurement takes a short time and has a small amount of work, high efficiency, low safety requirements for regular inspection of the water transfer tunnel, no safety hazards of the water transfer tunnel, and can meet the needs of regular inspection of the water transfer tunnel. It can provide technical support for the subsequent operation of the AUV and the repair of the water transfer tunnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a block diagram of optical acoustic defect detection in a water conveyance tunnel by an AUV according to an embodiment of the present invention;

[0039] Figure 2 This is a flow chart of optical acoustic defect detection in a water conveyance tunnel by an AUV according to an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of conversion between pixel size and actual area in an embodiment of the present invention;

[0041] Figure 4 is a flow chart of the image processing process of an embodiment of the present invention;

[0042] Figure 5 Schematic diagram of the ranging sonar before and after filtering at a fixed distance of 1.5 m according to an embodiment of the present invention;

[0043] Figure 6 It is a flowchart of training a pixel size model according to an embodiment of the present invention.

[0044] Figure 7 1 is a comparison diagram of underwater crack histogram equalization before and after the embodiment of the present invention, (a) before equalization, (b) after equalization;

[0045] Figure 8 1 is a histogram before and after equalization of an underwater crack histogram according to an embodiment of the present invention, (a) before equalization, (b) after equalization. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0047] like Figure 1 , 2 As shown, an optical-acoustic combined method for AUV to detect typical defects in a water conveyance tunnel includes the following steps:

[0048] Step 1: Use a camera and a light to capture the water tunnel image, and perform image denoising, image enhancement and other preprocessing on the obtained optical image;

[0049] Step 2: Detect typical defects such as cracks and falling blocks in the tunnel, and obtain the pixel sizes of the typical defects such as cracks and falling blocks;

[0050] Step 3: Calculate the actual size of typical defects such as cracks and falling blocks based on the pixel size combined with the distance of the ranging sonar;

[0051] Step 4: Compare the calculated defect size with the engineering requirements, and mark it if it is larger than the engineering requirements.

[0052] The step 1 is specifically as follows:

[0053] Step 1.1: Use an optical camera combined with underwater lighting to take pictures of the water tunnel, and use mean filtering to remove noise;

[0054] The specific implementation method is to select a pixel point F(x,y) in the image except the first row, first column, last row and last column, whose pixel value is f(x,y), select a window with F(x,y) as the center point, the window size is generally an odd number, and calculate the average value of all pixel values ​​in the window to replace f(x,y). The calculation formula is:

[0055] g(x,y)=∑f(x,y) / n 2

[0056] Step 1.2: Use the histogram equalization image enhancement method to evenly distribute the more concentrated pixels in the image within the grayscale value range to enhance the image contrast; the histogram equalization algorithm process is as follows:

[0057] (1) Calculate the grayscale histogram n of the original image k ,k=0,1,...,255;

[0058] (2) N is the total number of pixels, through n k Find the grayscale distribution probability P of the original image k :

[0059]

[0060] (3) Calculate the grayscale cumulative distribution probability P of the original image s :

[0061]

[0062] (4) Calculate the pixel value f(i,j) after histogram equalization:

[0063] f(i,j)=N·P s (k)

[0064] The step 2 is specifically as follows:

[0065] Step 2.1: First, collect typical pictures of cracks and falling blocks in water transfer tunnels, use the adversarial network to transfer the style of typical defects such as cracks and falling blocks in simulated water transfer tunnels, generate a large number of simulated underwater crack images of water transfer tunnels for semantic segmentation, and annotate them.

[0066] Step 2.2: Use semantic segmentation algorithm to train segmentation models corresponding to cracks, chipping, etc. for these typical defects;

[0067] Step 2.3: In the computer processing center, the image is segmented using the corresponding model to distinguish cracks, landslides and other pixel areas, and the pixel size is counted ( Figure 4 );

[0068] The step 3 is specifically as follows:

[0069] Step 3.1: Perform a Kalman filter operation on the distance value extracted by the ranging sonar and select a distance with relatively stable ranging;

[0070] Step 3.2: Based on the distance obtained by the ranging sonar, add the distance between the ranging sonar and the camera to get the distance between the camera and the water tunnel wall, and calculate the actual size of cracks, blocks, etc. according to the conversion formula, as follows:

[0071] distance = a + b

[0072] actual area=(pixel area / image area)*((k*distance)^2)

[0073] Where a is the distance obtained by the ranging sonar, b is the vertical distance from the camera to the ranging sonar, and distance is the distance from the camera to the tunnel wall. k is the proportional coefficient, which refers to the coefficient to be multiplied for pixel conversion at the current distance. It needs to be determined in combination with the distance, the ratio of the pixel size and the actual size taken at the current distance, and the difference in cameras. For example, the pixel size and the actual size taken at a distance of 1.5m need to be determined according to different cameras. Pixel area is the pixels occupied by cracks or broken blocks, and image area is all pixels in the photo.

[0074] like Figure 3 As shown, the conversion between pixel size and actual area includes the following steps:

[0075] Step 1: The distance from the tunnel wall A is obtained by using the ranging sonar;

[0076] Step 2: The specific values ​​of the ranging sonar B and the camera C in the underwater robot are fixed values ​​b;

[0077] Step 3: Add a and b to get the distance between the camera and the wall;

[0078] Step 4: Calculate the pixel size based on the crack and block loss algorithm. Figure 3 The middle elliptical shape simulates the falling block;

[0079] Step 5: Use the formula combined with the segmentation algorithm to calculate the actual size of the cracks and broken pieces. The formula is as follows:

[0080] actual area=(pixel area / image area)*((k*distance)^2)

[0081] Pixel area refers to the pixels occupied by cracks or broken pieces, image area refers to all pixels of the photo, and k represents the proportional coefficient, which refers to the coefficient to be multiplied for pixel conversion at the current distance. It needs to be determined in combination with the distance, the ratio of the pixel size and the actual size taken at the current distance, and the difference in cameras. For example, the pixel size and the actual size taken at a distance of 1.5m need to be determined according to different cameras.

[0082] like Figure 4 As shown, the image processing process includes the following steps:

[0083] Step 1: Use the underwater robot to move at a fixed distance in the water transfer tunnel;

[0084] Step 2: Turn on the light and camera to collect images of the water tunnel wall;

[0085] Step 3: Perform histogram equalization processing on the collected water conveyance tunnel wall image;

[0086] Step 4: Use the trained model to perform pixel-level detection of cracks, chipping, etc.

[0087] Step 5: Count the sizes of cracks and missing pixels;

[0088] like Figure 5As shown in the figure, there are some outliers that are obviously inconsistent with the fixed distance before filtering (that is, the serious deviation from most of the data caused by the measurement of the equipment itself). The outliers are eliminated through filtering.

[0089] like Figure 6 As shown, the embodiment trains the pixel size model training, comprising the following steps:

[0090] Step 1: Use the underwater robot to move at different distances in the water transfer tunnel;

[0091] Step 2: Turn on the light and camera to collect tunnel wall images under different conditions;

[0092] Step 3: For the collected data set, perform pixel-level annotation of cracks and chipping marks;

[0093] Step 4: Use adversarial networks to generate excessive data sets and annotate them.

[0094] Step 5: Use the segmentation algorithm to train the model of cracks and missing blocks;

[0095] Step 6: After training, the trained model is obtained.

[0096] By performing real-time image processing on a computer using the video captured by the camera, the image of the tunnel in which the AUV is currently located and the data of the ranging sonar under the current conditions can be obtained. By performing pixel differentiation on typical defects of the water conveyance tunnel in the image and combining it with the distance obtained by the ranging sonar, the size of the defects in the water conveyance tunnel can be detected.

[0097] like Figure 7 and 8 As shown in the figure, the comparison of underwater crack histogram before and after equalization. It can be seen that the image contrast is significantly improved after equalization, and the pixel value distribution is more uniform.

[0098] Secondly, the traditional measurement method requires the water transfer tunnel to be drained and requires manual inspection, which is labor-intensive and time-consuming. The present invention combines deep learning and ranging sonar to achieve real-time detection and identification of cracks in the corresponding water transfer tunnel, with high efficiency, low workload, and no need to drain the water in the tunnel.

[0099] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the implementation methods of the present invention, and should be understood that the protection scope of the present invention is not limited to such special statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A combined optical and acoustic method for AUV detection of typical defects in water tunnels. Features: The following steps are involved: Step 1: Use a camera and a light to capture the image of the water conveyance tunnel, and perform image denoising and image enhancement preprocessing on the obtained optical image; Step 2: Detect typical defects in the tunnel and obtain the pixel size of the typical defects. The typical defects include: cracks and chipping, specifically: Step 2.1: Collect pictures of typical defects in the water tunnel, use the adversarial network to transfer the style of the simulated typical defect pictures of the water tunnel, generate a large number of typical defect images for semantic segmentation, and annotate them; Step 2.2: Use the semantic segmentation algorithm to segment each typical defect image and train the segmentation model corresponding to each typical defect; Step 2.3: In the computer processing center, the image is segmented using the corresponding segmentation model to distinguish the area of ​​each typical defective pixel and perform pixel size statistics; Step 3: Calculate the actual size of typical defects based on the pixel size combined with the distance of the ranging sonar, specifically: Step 3.1: Perform a Kalman filter operation on the distance value extracted by the ranging sonar and select a stable distance; Step 3.2: Based on the distance obtained by the ranging sonar, add the distance between the ranging sonar and the camera to get the distance between the camera and the water tunnel wall, and calculate the actual size of the typical defect according to the conversion formula, as follows: distance = a + b actual area=(pixel area / image area)*((k*distance)^2) Where a is the distance obtained by the ranging sonar, b is the vertical distance from the camera to the ranging sonar, and distance is the distance from the camera to the tunnel wall. k is the proportional coefficient, which refers to the coefficient to be multiplied for pixel conversion at the current distance. Pixelarea is the pixels occupied by a typical defect, and image area is all the pixels in the photo. Step 4: Compare the actual size with the engineering requirements, and mark it if it is larger than the engineering requirements.

2. According to claim 1, an optical-acoustic combination method for detecting typical defects in a water tunnel by an AUV, Features: The step 1 is specifically as follows: Step 1.1: Use an optical camera combined with underwater lighting to take pictures of the water tunnel, and use mean filtering to remove noise; Select a pixel point F(x,y) in the image except the first row, first column, last row and last column, whose pixel value is f(x,y), select a window with F(x,y) as the center point, the window size is an odd number, and calculate the average value of all pixel values ​​in the window to replace f(x,y); the calculation formula is: g(x,y)=∑f(x,y) / n 2 In the formula, g(x,y) represents the pixel value after mean processing, and n is the processing window size; Step 1.2: Use the histogram equalization image enhancement method to evenly distribute the more concentrated pixels in the image within the grayscale value range to enhance the image contrast.

3. According to claim 2, the optical-acoustic combined method for detecting typical defects in a water tunnel by an AUV, Features: The process of the histogram equalization image enhancement method described in step 1.2 is as follows: Step 1.2.1: Calculate the grayscale histogram n of the original image k ,k=0,1,...,255; Step 1.2.2: N is the total number of pixels, through n k Find the grayscale distribution probability P(k) of the original image: Step 1.2.3: Calculate the grayscale cumulative distribution probability P of the original image s : Step 1.2.4: Calculate the pixel value f(i,j) after histogram equalization: f(i,j)=N·P s (k) Where i, j is the pixel in the i-th row and j-th column in the image.

Images