Submarine pipeline detection method based on imaging sonar and optical camera

Abstract

The invention relates to the technical field of underwater vehicle vision, in particular to a submarine pipeline detection method based on an imaging sonar and an optical camera, and the method comprises the steps: carrying out the remote detection through the imaging sonar, recognizing a pipeline, and obtaining the initial position of the pipeline in a geodetic coordinate system; controlling the underwater vehicle to approach the pipeline according to the initial position of the pipeline in the geodetic coordinate system, and switching the optical camera to acquire the position of the pipeline in the image coordinate system when the optical camera can identify the pipeline; converting the position of the pipeline in the image coordinate system to obtain the longitude and latitude of the pipeline in the geodetic coordinate system; and controlling the underwater vehicle to sail along the pipeline according to the longitude and latitude of the pipeline in the geodetic coordinate system, and detecting the submarine pipeline. On the basis of the optical camera, the imaging sonar is introduced, so that a higher success rate can be realized in different seabed environments.

Description

Technical Field

[0001] The present invention relates to the field of underwater vehicle vision technology, and in particular to a submarine pipeline detection method and system based on imaging sonar and optical camera. Background Art

[0002] With the combined advantages of low cost, high efficiency and high reliability, the construction of submarine pipelines has made rapid progress. The purpose of submarine pipeline inspection is to find various defects and problems that actually exist in submarine pipelines, so as to evaluate and take corresponding preventive measures based on the inspection results. At present, the inspection of submarine pipelines is mainly achieved by optical cameras carried by underwater vehicles.

[0003] The images obtained by the optical camera contain various details and features of the scene, including the edges of objects. The edge of the image contains most of the information in the image features. After research by scholars at home and abroad, many methods have been applied to image edge detection. Canny proposed the Canny edge detection operator from the signal perspective. Later, scholars analyzed it from the perspective of mathematical methods. Wavelet transform edge detection, multi-scale wavelet transform image edge detection, wavelet packet edge detection, mathematical morphology edge detection, neural network edge detection, etc. have appeared one after another. Many scholars have applied edge detection to the detection and positioning of submarine pipelines. Combined with appropriate straight line segment extraction methods, it can be applied to the inspection of submarine pipelines.

[0004] The above edge detection method can identify pipelines, but there are problems such as complex calculations, long detection time, and low accuracy. It is necessary to combine the actual project to reduce the complexity of calculations and improve the efficiency of pipeline detection. In addition, the disadvantage of the small range of optical cameras is particularly prominent in underwater applications. Optical cameras alone are difficult to meet the autonomous detection tasks of submarine pipelines. Summary of the invention

[0005] In order to solve the above technical problems, the present invention mainly solves the problems of single method, complex calculation and low precision in the existing pipeline detection technology, and proposes a submarine pipeline detection method based on imaging sonar and optical camera, improves the capture classification network, and improves the ability of network evaluation. The present invention provides a submarine pipeline detection method based on imaging sonar and optical camera, which relies on imaging sonar to locate the pipeline at a long distance, controls the vehicle to approach the pipeline, switches the camera to detect the pipeline line after the camera can identify the pipeline, and extracts the longitude and latitude of the pipeline in the geodetic coordinates, controls the vehicle to navigate along the pipeline and detect the pipeline. Compared with the traditional method that relies only on optical cameras, the introduction of imaging sonar makes the detection process more reliable and can achieve a higher success rate in different seabed environments. The EDLine algorithm is used as a method for detecting pipelines, which can effectively suppress noise and interference and improve the accuracy and efficiency of pipeline detection.

[0006] The first object of the present invention is to provide a method for detecting submarine pipelines based on imaging sonar and optical camera, wherein the submarine pipelines are detected by imaging sonar and optical camera carried on an underwater vehicle, comprising: The pipeline is identified through long-distance detection by imaging sonar, and the initial position of the pipeline in the geodetic coordinate system is obtained; According to the initial position of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to approach the pipeline. When the optical camera can identify the pipeline, the optical camera is switched to collect optical images, and the position of the pipeline in the image coordinate system is obtained based on the EDLine algorithm. The position of the pipeline in the image coordinate system is converted to obtain the longitude and latitude of the pipeline in the geodetic coordinate system; According to the longitude and latitude of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to navigate along the pipeline to inspect the submarine pipeline.

[0007] Preferably, the pipeline is detected and identified from a distance by imaging sonar, including: Acquire sonar images; The sonar image is preprocessed, and then the ellipse fitting method is used to obtain the major and minor axes of the ellipse, the intersection of the major and minor axes, and the rotation angle of the ellipse; The center of the smallest circumscribed ellipse is taken as the center of the pipeline, the major axis is taken as the direction of the pipeline, and the direction of rotation of the ellipse is taken as the direction of the pipeline.

[0008] Preferably, the initial position of the pipeline in the geodetic coordinate system is obtained by calculation based on the center of the pipeline.

[0009] Preferably, the sonar image is preprocessed, including grayscale and smoothing processing of the sonar image.

[0010] Preferably, the position of the pipeline in the image coordinate system includes: Acquire an optical image according to an optical camera; The EDLine edge detection algorithm is used to perform edge detection and line segment extraction on the pipeline line in the optical image to obtain the pipeline line; Get the position of the pipeline in the image coordinate system based on the pipeline line.

[0011] Preferably, converting the position of the pipeline in the image coordinate system to obtain the longitude and latitude of the pipeline in the geodetic coordinate system includes: Convert the position of the pipeline in the image coordinate system to the position in the carrier coordinate system; Convert the position of the pipeline in the carrier coordinate system to the position in the earth coordinate system; The longitude and latitude of the pipeline in the geodetic coordinate system are calculated according to the position of the pipeline in the geodetic coordinate system.

[0012] Preferably, the position of the pipeline in the image coordinate system refers to the position of the center point on the central axis of the pipeline.

[0013] Preferably, the position of the pipeline in the image coordinate system is converted to the position in the carrier coordinate system, and the conversion relationship is:

[0014] In the formula, Represents the coordinates of the pipeline center in the image coordinate system; Indicates the coordinates of the pipeline center in the carrier coordinate system; dx, dy are the physical sizes of a single pixel in the OX and OY directions of the camera coordinate system respectively; , are the equivalent focal lengths in OX and OY directions respectively; Indicates the vertical distance from the downward-looking camera to the pipe.

[0015] Preferably, the longitude and latitude of the pipeline in the geodetic coordinate system are calculated according to the position of the pipeline in the geodetic coordinate system, and the conversion relationship is:

[0016] In the formula, , The longitude and latitude information obtained by the underwater vehicle through dead reckoning; H is the height of the underwater vehicle at the bottom of the water, , is the longitude and latitude of the pipeline.

[0017] The present invention has at least the following beneficial effects: Compared with the existing submarine pipeline detection methods, the submarine pipeline detection method based on imaging sonar and optical camera provided by the present invention has the following beneficial effects: (1) Fast detection speed Commonly used Canny operators, Laplace operators, etc. all obtain edge information of images by calculating the relationship between pixels and judging whether there is a sudden boundary. The information obtained in this way is mostly discontinuous, and it is difficult to have intersecting pixels. The present invention is based on the EDLine algorithm and can not only generate a complete and continuous chain of edge pixels, but also because it is connected through pixels, the generated boundary is continuous. From the perspective of processing speed, the EDLine algorithm has a faster response time than other processing algorithms.

[0018] (2) Wide detection range As the main carrier of underwater information transmission, acoustic signals have a higher detection range. Pipelines can be located at a long distance by using imaging sonar, and the vehicle can be controlled to approach the pipeline. After the camera can identify the pipeline, the camera is switched to detect the pipeline line, which increases the detection range of submarine pipelines.

[0019] (3) High detection accuracy The EDLine algorithm generates the minimum initial line segment according to the least squares method, and then extends the straight line segment by adding pixels, which can effectively suppress noise and interference and improve the accuracy of straight line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Imaging geometry model; Figure 2 Schematic diagram of ellipse fitting; Figure 3 Imaging sonar processing results; Figure 4 Line feature extraction process; Figure 5 Gradient threshold calculation; Figure 6 Subsea pipeline inspection results; Figure 7 Pixel coordinates and angle of the pipeline center point; Figure 8 Pipeline image acquisition effect. DETAILED DESCRIPTION

[0021] In order to illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description in conjunction with embodiments.

[0022] The present invention is mainly used to control underwater vehicles (AUVs) to navigate underwater. A hybrid image processing method is designed by utilizing the advantages of imaging sonar, which has a wide detection range and is less affected by light sources, and optical cameras, which have high close-range target recognition and high resolution. The method relies on sonar positioning at long distances and optical cameras for detection at close ranges to identify submarine pipelines and extract information.

[0023] Example 1 A method for detecting submarine pipelines based on imaging sonar and optical camera, which detects submarine pipelines by using imaging sonar and optical camera carried on an underwater vehicle, comprising: S1. Identify the pipeline by long-distance detection through imaging sonar and obtain the initial position of the pipeline in the geodetic coordinate system; Long-range detection using imaging sonar identifies pipelines, including: Acquire sonar images; The sonar image is preprocessed, and then the ellipse fitting method is used to obtain the major and minor axes of the ellipse, the intersection of the major and minor axes, and the rotation angle of the ellipse; The center of the smallest circumscribed ellipse is taken as the center of the pipeline, the major axis is taken as the direction of the pipeline, and the direction of rotation of the ellipse is taken as the direction of the pipeline.

[0024] The initial position of the pipeline in the geodetic coordinate system is obtained by calculation based on the center of the pipeline.

[0025] Preprocess the sonar image, including grayscale and smoothing processing of the sonar image.

[0026] S2. According to the initial position of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to approach the pipeline. When the optical camera can identify the pipeline, the optical camera is switched to collect an optical image, and the position of the pipeline in the image coordinate system is obtained based on the EDLine algorithm. The position of the pipeline in the image coordinate system includes: The optical image is acquired by the optical camera; the EDLine edge detection algorithm is used to perform edge detection and line segment extraction on the pipeline line in the optical image to obtain the pipeline line; the position of the pipeline in the image coordinate system is obtained according to the pipeline line. It should be noted that before performing edge detection and line segment extraction on the pipeline line in the optical image, the optical image needs to be grayscaled and smoothed.

[0027] S3, converting the position of the pipeline in the image coordinate system to obtain the longitude and latitude of the pipeline in the geodetic coordinate system; The position of the pipeline in the image coordinate system is converted to obtain the longitude and latitude of the pipeline in the geodetic coordinate system, including: Convert the position of the pipeline in the image coordinate system to the position in the carrier coordinate system; Convert the position of the pipeline in the carrier coordinate system to the position in the earth coordinate system; The longitude and latitude of the pipeline in the geodetic coordinate system are calculated according to the position of the pipeline in the geodetic coordinate system.

[0028] The position of the pipeline in the image coordinate system refers to the position of the center point on the central axis of the pipeline.

[0029] Among them, the position of the pipeline in the image coordinate system is converted to the position in the carrier coordinate system, and the conversion relationship is:

[0030] In the formula, Represents the coordinates of the pipeline center in the image coordinate system; Indicates the coordinates of the pipeline center in the carrier coordinate system; dx, dy are the physical sizes of a single pixel in the OX and OY directions of the camera coordinate system respectively; , are the equivalent focal lengths in OX and OY directions respectively; Indicates the vertical distance from the downward-looking camera to the pipe.

[0031] Specifically, the longitude and latitude of the pipeline in the geodetic coordinate system are calculated according to the position of the pipeline in the geodetic coordinate system. The conversion relationship is:

[0032] In the formula, , The longitude and latitude information obtained by the underwater vehicle through dead reckoning; H is the height of the underwater vehicle at the bottom of the water, , is the longitude and latitude of the pipeline.

[0033] S4. According to the longitude and latitude of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to navigate along the pipeline to inspect the submarine pipeline.

[0034] Example 2 A method for detecting submarine pipelines based on imaging sonar and optical camera, comprising: In underwater pipeline inspection systems, the camera is usually located below the AUV, with the lens pointing toward the bottom of the AUV and fixed to the AUV. Figure 1 The imaging geometry model of the camera is shown in the figure. The OXYZ coordinate system is the carrier coordinate system, which refers to the coordinate system where the AUV is located. The coordinate system is the image coordinate system, with the origin is the intersection of the lens optical axis and the imaging plane. The coordinate system is the pixel coordinate system. uv Represents the rows and columns of pixels respectively. The pixel coordinates obtained by the camera are in In the coordinate system. The coordinate system is the camera coordinate system. Combined with the camera internal parameters and other information, the coordinate points of the pixel coordinate system can be converted to the carrier coordinate system. The conversion matrix between the carrier coordinate system and the pixel coordinate system can be derived as:

[0035] Where, the coordinates of the pipeline center in the pixel coordinate system are , dx , dy Respectively, a single pixel point in the camera coordinate system OX , OY The physical size of the direction. The vertical distance from the downward camera to the pipe , that is, the height from the vehicle to the seabed. , is the equivalent focal length in two directions, is the rotation matrix from the camera coordinate system to the carrier coordinate system, and the coordinates of the pipeline center in the carrier coordinate system are .

[0036] In order to reduce the amount of calculation in the subsequent image processing part, it is necessary to grayscale the original image to obtain the grayscale image of the pipeline. The grayscale transformation formula is as follows:

[0037] Gray represents the gray value at a certain pixel, and R, G, and B represent the red, green, and blue components of the pixel in the color space, respectively. The image is then processed by mean filtering to reduce the noise of the image, and the preprocessed image is obtained to prepare for the subsequent pipeline information extraction. The original image includes sonar images and optical images.

[0038] This embodiment considers pipeline line extraction based on imaging sonar and vision respectively. First, consider pipeline detection based on imaging sonar at a long distance. An approximate pipeline outline can be obtained through preprocessing and region growing methods, and the ellipse fitting method is used to realize the detection of ellipse features. First, calculate the covariance matrix of the edge points of the image:

[0039] Among them, S_xx, S_xy and S_yy represent the sum of the squares of the x-coordinates, the sum of the products of the x-coordinates and the y-coordinates, and the average of the sum of the squares of the y-coordinates of the edge points, respectively.

[0040] Then solve the eigenvalues ​​and eigenvectors of the covariance matrix. Select the eigenvector with the largest eigenvalue as the direction of the major and minor axes of the ellipse. The length of the major axis:

[0041] Minor axis length:

[0042] The rotation angle of the ellipse:

[0043] Center coordinates:

[0044] in, a is the length of the major axis of the ellipse, b is the minor axis length, is the rotation angle of the ellipse, are the coordinates of the center of the ellipse, and are the eigenvalues, is the eigenvector corresponding to the larger eigenvalue, express of y Coordinate components, express of x Coordinate components, represents the average value of the x-coordinates of the edge points, Indicates the average value of the y coordinates of the edge points. Figure 2 As shown in the figure, the center of the smallest circumscribed ellipse is taken as the center of the pipeline, the major axis is taken as the direction of the pipeline, and the direction of the ellipse rotation is taken as the direction of the pipeline. Then, the position of the pipeline in the geodetic coordinate system is obtained by calculation, and the aircraft is controlled to approach the pipeline to prepare for the subsequent pipeline detection based on the optical camera. Using the ellipse feature detection method proposed in the present invention, the ellipse feature information of the pipeline is extracted in the Gazebo simulation environment. The processing results are shown in FIG. Figure 3 shown.

[0045] Next, consider pipeline line extraction based on optical cameras, using an edge detection method (Edge DetectionLine, EDLine) to detect submarine pipelines. The EDLine algorithm requires edge detection and line segment extraction. The detection process is as follows: Figure 4 Edge detection requires setting some internal parameters, including gradient calculation, gradient threshold, and anchor point connection.

[0046] The first step is to calculate the gradient magnitude and gradient direction. The horizontal line angle at , using the edge detection algorithm, calculates the projection (i.e., component) of the gradient in the x-axis and y-axis directions as follows:

[0047]

[0048] in, The input image at pixel The strength of and Pixels According to the above formula, the gradient amplitude of the pixel point can be calculated. The horizontal angle at for:

[0049] The pixel point can be obtained by calculating the Euclidean distance between the x-axis and the y-axis. The gradient amplitude formula .

[0050]

[0051] The obtained gradient and horizontal line angle can be used to encode the input image and realize the conversion of the image edge.

[0052] Next is the calculation of the gradient threshold: the anchor point is a pixel with a very high probability among edge pixels, corresponding to the pixel where the gradient operator produces the maximum value, that is, the peak of the gradient map. The gradient amplitude of the current pixel is calculated and compared with the given threshold. For comparison, less than The pixels that are not edge pixels will be ignored, which can eliminate those pixels that definitely do not contain edge pixels. The threshold is set to the median of the image gradient. This embodiment divides the image gradient into different groups in units of 10. After obtaining the pixel gradient, the pixel gradient value is divided by 10, and the integer part is taken as a group. A median is taken for each group of data, which is the gradient threshold of this group of data, and then the anchor point is obtained. Figure 5 shown.

[0053] The last step is to connect the anchor points. The anchor points obtained in the previous step are connected by straight lines to generate a set of edge segments, which are chains of pixels corresponding to the boundaries of the object.

[0054] The EDLine algorithm can not only realize the functions of other line segment detectors, but also generate a set of edge segments, which are chains of pixels at the boundaries of objects. The following figure shows the pipeline image captured by the camera, and the EDLine algorithm is used for recognition and detection. The processing results are as follows: Figure 6 As shown in Figure 1, all possible pixels are extracted using the edge drawing algorithm in (a). By connecting the anchor points and extracting the line segments in (b) and (c), most interference is eliminated and the pipeline information is identified. Figure 1 shows the recognition results under different parameters.

[0055] After the above steps, a set of continuous pixel chains can be obtained, and some processing is needed to segment them to obtain separate straight line segments. The least squares method is used to track the pixel points, and the error is used to determine whether it exceeds the threshold. If it exceeds the threshold, a new straight line segment is generated.

[0056] The extracted line segments are optimized using the basic information of the line segments, such as the line segment length, slope, and starting point position. By extracting the region of interest, the computational cost can be saved. Only the lower half of the image can be selected as the image to be extracted, ignoring some irrelevant factors. The central axis of the pipeline line is selected as a substitute for the pipeline, and the center of the camera field of view is used as the target. The pixel coordinates of the center point on the central axis are selected. As the pixel coordinate of the center point of the pipeline, take the angle between the central axis of the pipeline and the vertical line of the field of view As the direction of the pipeline, such as Figure 7 shown.

[0057] The position of the pipeline in the image coordinate system is converted to the carrier coordinate system, and the longitude and latitude information of the pipeline is estimated by using the premise of maintaining constant altitude during the navigation of the AUV. The origin of the camera coordinate system is kept in the same horizontal plane as the gravity and buoyancy center of the AUV. The camera is located in the longitudinal midsection of the AUV and points to the direction of the bow of the AUV. The distance from the gravity and buoyancy center is a. The conversion relationship between the pixel coordinates of the center of the pipeline and the carrier coordinate system is:

[0058] The coordinates of the pipeline center in the pixel coordinate system are , dx , dy Respectively, a single pixel point in the camera coordinate system OX , OY Since the vehicle moves at a constant altitude during pipeline tracking, the vertical distance from the downward camera to the pipeline is It is known that the height from the vehicle to the seabed. , is the equivalent focal length in two directions, and the coordinates of the pipeline center in the carrier coordinate system can be obtained .

[0059] According to the above formula, the position information from the pixel coordinate system to the carrier coordinate system can be obtained:

[0060] Convert to the geodetic coordinate system:

[0061] in, is the pitch angle of the AUV, , It is the position coordinate of the pipeline center in the geodetic coordinate system.

[0062] Combined with the longitude and latitude information of the AUV's own dead reckoning, the longitude and latitude of the pipeline center in the geodetic coordinate system can be calculated as:

[0063] in, , is the longitude and latitude information obtained by AUV through dead reckoning, H is the height from AUV to the seabed, , It is the estimated longitude and latitude information of the pipeline.

[0064] The collected image data is processed using the algorithm proposed in this invention, and the position of the camera result is solved using the EDLine algorithm. The processing result is as follows: Figure 8 As shown in the figure, it can be seen that the algorithm proposed in this paper accurately identifies the pipeline line. The right side is the pipeline image collected by the imaging sonar.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting submarine pipelines based on imaging sonar and optical camera, wherein the submarine pipelines are detected by imaging sonar and optical camera carried on an underwater vehicle, characterized in that: include: The pipeline is identified through long-distance detection by imaging sonar, and the initial position of the pipeline in the geodetic coordinate system is obtained; According to the initial position of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to approach the pipeline. When the optical camera can identify the pipeline, the optical camera is switched to collect optical images, and the position of the pipeline in the image coordinate system is obtained based on the EDLine algorithm. The position of the pipeline in the image coordinate system is converted to obtain the longitude and latitude of the pipeline in the geodetic coordinate system; According to the longitude and latitude of the pipeline in the geodetic coordinate system, the underwater vehicle is controlled to navigate along the pipeline to inspect the submarine pipeline.

2. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 1, characterized in that: Long-range detection using imaging sonar identifies pipelines, including: Acquire sonar images; The sonar image is preprocessed, and then the ellipse fitting method is used to obtain the major and minor axes of the ellipse, the intersection of the major and minor axes, and the rotation angle of the ellipse; The center of the smallest circumscribed ellipse is taken as the center of the pipeline, the major axis is taken as the direction of the pipeline, and the direction of rotation of the ellipse is taken as the direction of the pipeline.

3. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 2, characterized in that: The initial position of the pipeline in the geodetic coordinate system is obtained by calculation based on the center of the pipeline.

4. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 2, characterized in that: Preprocess the sonar image, including grayscale and smoothing processing of the sonar image.

5. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 1, characterized in that: The position of the pipeline in the image coordinate system includes: Acquire an optical image according to an optical camera; The EDLine edge detection algorithm is used to perform edge detection and line segment extraction on the pipeline line in the optical image to obtain the pipeline line; Get the position of the pipeline in the image coordinate system based on the pipeline line.

6. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 1, characterized in that: The position of the pipeline in the image coordinate system is converted to obtain the longitude and latitude of the pipeline in the geodetic coordinate system, including: Convert the position of the pipeline in the image coordinate system to the position in the carrier coordinate system; Convert the position of the pipeline in the carrier coordinate system to the position in the earth coordinate system; The longitude and latitude of the pipeline in the geodetic coordinate system are calculated according to the position of the pipeline in the geodetic coordinate system.

7. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 6, characterized in that: The position of the pipeline in the image coordinate system refers to the position of the center point on the central axis of the pipeline.

8. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 7, characterized in that: The position of the pipeline in the image coordinate system is converted to the position in the carrier coordinate system. The conversion relationship is: In the formula, Represents the coordinates of the pipeline center in the image coordinate system; Indicates the coordinates of the pipeline center in the carrier coordinate system; dx, dy are the physical sizes of a single pixel in the OX and OY directions of the camera coordinate system respectively; , are the equivalent focal lengths in OX and OY directions respectively; Indicates the vertical distance from the downward-looking camera to the pipe.

9. The method for detecting submarine pipelines based on imaging sonar and optical camera according to claim 7, characterized in that: The longitude and latitude of the pipeline in the geodetic coordinate system are calculated based on the position of the pipeline in the geodetic coordinate system. The conversion relationship is: In the formula, , The longitude and latitude information obtained by the underwater vehicle through dead reckoning; H is the height of the underwater vehicle at the bottom of the water, , is the longitude and latitude of the pipeline.

Images