Method for measuring position from pipeline defect to pipe orifice based on binocular vision

Abstract

The invention relates to the field of pipeline defect measurement, in particular to a method for measuring the position from a pipeline defect to a pipe orifice based on binocular vision. Internal and external parameter calibration of the binocular camera is completed through the calibration module, the binocular vision acquisition module enters a pipeline to obtain a binocular image sequence containing pipe orifices and defects, and precise solving of internal parameters and external parameters of the camera is achieved by adopting a checkerboard and multi-attitude calibration set and combining a Zhang Zhengyou calibration method. The image is preprocessed, the image quality is improved, dust noise in a large pipe diameter is removed, and edge features are enhanced. On the basis of Canny edge detection, a least square method and a YOLOv8 algorithm are matched to complete pipe orifice reference and defect feature collaborative identification, and efficient stereo matching and three-dimensional reconstruction of defect points are achieved in combination with SGBM stereo matching and triangulation. Finally, the problems that traditional pipeline defect positioning depends on manual work and is low in precision and poor in adaptability are solved, and automatic and high-precision measurement from the defects in the pipeline to the pipe opening position is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pipeline defect measurement, in particular to a pipeline defect to pipe opening position measurement method based on binocular vision. BACKGROUND

[0002] Pipeline defects refer to various damages or incomplete parts of the pipeline generated during manufacturing, installation, use or maintenance, which may affect the structural strength, sealing performance or service life of the pipeline. Common pipeline defects include cracks, scratches and dents, which may extend along the length of the pipeline and easily cause stress concentration, leading to leakage or rupture. In order to avoid the impact of pipeline defects on the service life of the pipeline, it is necessary to locate the pipeline defects. However, the existing pipeline defect measurement methods have low automation level and poor environmental adaptability, and the image quality obtained is easily affected by dust and noise in the pipeline, resulting in low measurement accuracy. SUMMARY

[0003] The technical solution adopted by the present application to solve its technical problems is to provide a pipeline defect to pipe opening position measurement method based on binocular vision, comprising the following steps:

[0004] Step S1: system calibration:

[0005] Place the checkerboard calibration plate in the pipeline and collect multiple sets of binocular images at different positions and angles, and calculate the camera internal and external parameters through the calibration module;

[0006] Step S2: send the detection device carrying the binocular vision acquisition module and the light source module along the pipeline axis, synchronously acquire binocular images containing the pipe opening edge and the defect area, and transmit the image data to the data processing module through wired or wireless mode;

[0007] Step S3: pre-process the image to improve image quality, reduce data volume, remove dust noise in large pipe diameter and enhance edge features;

[0008] Step S4: reference positioning and defect recognition of the pre-processed image:

[0009] The reference positioning and defect recognition of the pre-processed image comprises:

[0010] Step S41, extract the circular arc features of the pipe opening edge using Canny edge detection algorithm;

[0011] Step S42, select no less than 10 edge points and fit the pipeline axis and the pipe opening reference surface through least squares method;

[0012] Step S43, recognize the defect target using YOLOv8 algorithm and output the two-dimensional coordinates of the defect in the image;

[0013] Step S5: Perform stereo matching and 3D reconstruction on the preprocessed image:

[0014] Step S6: Distance Calculation: Calculate the straight-line distance from the defect point to the reference plane of the pipe opening based on the formula for the distance from a spatial point to a plane, and output the measurement results;

[0015] Step S7: Error Correction: Use the marker point with known distance as the standard reference to perform multiple measurements, establish an error compensation model, correct the measurement results, and ensure that the measurement error is less than or equal to ±0.5mm.

[0016] Furthermore, the extraction of the arc features of the pipe opening edge using the Canny edge detection algorithm includes the following steps:

[0017] Step S411: Input the preprocessed image;

[0018] Step S412: Calculate the gradient magnitude and gradient direction for each pixel.

[0019] Step S413: Traverse each pixel. If the gradient magnitude of the pixel is the maximum value of its neighboring pixels in the gradient direction, then retain it; if the gradient magnitude of the pixel is not the maximum value of its neighboring pixels in the gradient direction, then set it to 0.

[0020] Step S414: Use dual threshold filtering to retain strong edges, connect weak edges with strong edges, and remove isolated weak edges;

[0021] Step S415: Output a binary image of the pipe opening edge, and extract 12-15 consecutive edge feature points within the binary image of the pipe opening edge.

[0022] Furthermore, selecting no fewer than 10 edge points and fitting them using the least squares method to obtain the pipe axis and the pipe nozzle reference plane includes the following steps:

[0023] Step S421: Convert the pipe opening edge feature points obtained by the Canny edge detection algorithm into three-dimensional coordinates by combining them with the camera intrinsic parameters;

[0024] Step S422: Since the pipe axis is along the x-axis and the reference plane is perpendicular to the x-axis, determine the form of the plane equation;

[0025] Step S423: Substitute the coordinates of the extracted continuous edge feature points to calculate the parameters to be determined for the plane equation.

[0026] Furthermore, the YOLOv8 algorithm is used to identify defect targets, and the two-dimensional coordinates of the defects in the image are output through the following steps:

[0027] Step S431: Load the pre-trained YOLOv8 model;

[0028] Step S432: Input the preprocessed image, use the prediction result output by the YOLOv8 model inference, and convert the prediction result output by the YOLOv8 model inference into the original coordinates of the image;

[0029] Step S433: Confidence screening, retaining prediction boxes with a confidence level greater than or equal to 0.5, and excluding false positives;

[0030] Step S434: NMS processing, remove overlapping predicted boxes, and retain the best predicted box;

[0031] Step S435, coordinate transformation: convert the normalized coordinates of the center of the optimal prediction box into the original image coordinates, which are used as the two-dimensional coordinates of the defect center;

[0032] Step S436, output the result.

[0033] Furthermore, stereo matching and 3D reconstruction are performed on the preprocessed image. Stereo matching and 3D reconstruction include the following steps:

[0034] Step S51: Perform epipolar correction on the preprocessed binocular image to make corresponding points lie on the same horizontal line;

[0035] Step S52: The disparity map is calculated using the SGBM semi-global stereo matching algorithm;

[0036] Step S53: Using the calibration parameters and the principle of triangulation, obtain the three-dimensional equation of the nozzle reference surface and the three-dimensional coordinates of the defect point.

[0037] Furthermore, the disparity map is calculated using the SGBM semi-global stereo matching algorithm, including the following steps:

[0038] Step S521: Input the grayscale image corrected by the binocular camera;

[0039] Step S522: For each pixel in the grayscale image, traverse the disparity range and calculate the matching cost through Census transform;

[0040] Step S523: Aggregate the matching costs along the 8 paths to obtain the aggregated cost;

[0041] Step S524: Using the WTA criterion, select the disparity with the minimum aggregation cost as the optimal disparity for that pixel.

[0042] Step S525: Post-process the disparity map;

[0043] Step S526: Output the final disparity map and obtain the disparity value corresponding to the defect center through the final disparity map.

[0044] Furthermore, by combining the calibration parameters and using the principle of triangulation, the three-dimensional equation of the nozzle reference plane and the three-dimensional coordinates of the defect points are obtained, including the following steps:

[0045] Step S531: Obtain the coordinates of the defect center pixel from the left image of the binocular camera;

[0046] Step S532: Read the disparity corresponding to the pixel from the final disparity map;

[0047] Step S533: Substitute the values ​​into the triangulation formula to calculate the three-dimensional coordinates of the defect point.

[0048] Furthermore, using markers with known distances as standard references, multiple measurements are performed to establish an error compensation model and correct the measurement results, including:

[0049] Step S71: Substitute the data from multiple measurements into the model formula based on least squares fitting to obtain the linear fitting coefficients of the model formula.

[0050] Step S72: Obtain the corrected formula for the final measurement result based on the model formula.

[0051] Furthermore, the image preprocessing includes:

[0052] Step S31: Convert the binocular images to grayscale sequentially;

[0053] Step S32: Perform 7×7 Gaussian filtering on the grayscale binocular image to remove noise;

[0054] Step S33: Perform histogram equalization on the denoised image.

[0055] Furthermore, the binocular vision acquisition module uses two 5-megapixel industrial cameras, the calibration module incorporates Zhang Zhengyou's calibration algorithm, the light source module includes LED fill lights, and the data processing module uses an embedded processor.

[0056] The beneficial effects of this invention are as follows: This invention completes the calibration of the intrinsic and extrinsic parameters of the binocular camera through a calibration module. It utilizes a binocular vision acquisition module to enter the pipeline and acquire a sequence of binocular images including the pipe opening and defects. A checkerboard and multi-pose calibration set are used, combined with the Zhang Zhengyou calibration method to acquire 20-22 sets of multi-pose images, achieving accurate solutions for the camera's intrinsic and extrinsic parameters. Based on Canny edge detection, least squares method, and YOLOv8 algorithm, collaborative identification of pipe opening reference and defect features is achieved. Combined with SGBM stereo matching and triangulation, efficient stereo matching and 3D reconstruction of defect points are realized. Based on pre-set known distance markers within the pipeline, an error compensation model adapted to long-distance measurement scenarios is established, effectively eliminating system errors, environmental dust, and equipment vibration interference, controlling the measurement error within ±0.5mm. Furthermore, the use of LED supplementary lighting makes it suitable for complex industrial scenarios such as darkness and high dust levels. Ultimately, this invention solves the problems of traditional pipeline defect location relying on manual labor, low accuracy, and poor adaptability, realizing automated and high-precision measurement of the location of defects within the pipeline from the pipe opening. Attached Figure Description

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] In the picture: Figure 1 The flowchart of a method for measuring the location of pipe defects to pipe openings based on binocular vision provided by the present invention is shown. Detailed Implementation

[0059] To make the technical problem to be solved, the technical solution, and the beneficial effects of this invention clearer, the invention will now be described in detail with reference to the accompanying drawings. This drawing is a simplified schematic diagram, illustrating only the basic aspects of the invention, and therefore only shows the components relevant to the invention. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0060] Please refer to Figure 1 The technical solution adopted by this invention to solve its technical problem is: to provide a method for measuring the location of pipe defects to pipe openings based on binocular vision, including the following steps:

[0061] Step S1: System calibration: Place the 14×11 checkerboard calibration plate inside a pipe with a diameter of 152.4 mm or larger, acquire multiple sets of binocular images at different positions and angles, calculate the camera's internal and external parameters through the calibration module, and establish a binocular vision 3D measurement model adapted to the pipe diameter range.

[0062] Specifically, in this embodiment, 20 sets of binocular images are acquired.

[0063] The calibration module incorporates the Zhang Zhengyou calibration algorithm and uses a 14×11 calibration plate (i.e., 15mm grid size), which is suitable for calibration spaces within large-diameter pipes. The Zhang Zhengyou calibration algorithm is used to calibrate the camera's intrinsic parameters (focal length f, principal point coordinates (u0, v0), distortion coefficients k1, k2, p1, p2) and extrinsic parameters (rotation matrix R, translation vector T), achieving accurate solutions for both intrinsic and extrinsic parameters and providing a fundamental guarantee for 3D reconstruction.

[0064] Step S2: The detection device equipped with a binocular vision acquisition module and a light source module is sent along the pipeline axis to simultaneously acquire binocular images including the pipe opening edge and defect area. The image data is transmitted to the data processing module via wired or wireless means.

[0065] The binocular vision acquisition module employs two 5-megapixel industrial cameras with lens focal lengths of 12-16mm. The baseline length of the industrial cameras can be adjusted via a bracket. This binocular vision acquisition module can adapt to the field of view requirements of large-diameter pipes and accurately accommodate pipe diameters of 6 inches and above, solving the problem of field of view coverage in image acquisition within large-diameter, long-distance pipelines.

[0066] The light source module includes an LED fill light with a spectrum of 450-650nm and an adaptive light intensity of 2000-15000 lux. The ring layout ensures that the edges and defect areas of the pipe openings in large-diameter pipes can obtain uniform illumination, avoiding reflection and shadow interference, and ensuring that the edges and defect areas of the pipe openings can be clearly imaged in dark and dusty environments inside the pipe.

[0067] The data module uses an embedded processor and integrates algorithms such as image preprocessing, feature extraction, stereo matching, and coordinate calculation. The data processing module processes image data in real time and outputs measurement results.

[0068] Step S3: Preprocess the image to improve image quality. This includes reducing data volume, removing dust and noise from large-diameter pipes, and enhancing edge features.

[0069] The image preprocessing includes:

[0070] Step S31: Convert the binocular images to grayscale sequentially;

[0071] The grayscale conversion of a binocular image includes the following steps:

[0072] Step S311: Read the color image captured by the binocular camera;

[0073] Step S312: Traverse each pixel in the image. Extract the corresponding R, G, and B channel values;

[0074] Step S313: Calculate the grayscale value of the pixel. The formula is as follows:

[0075]

[0076] in These are the two-dimensional coordinates of the image pixels; , , Color images in The pixel values ​​of the red, green, and blue channels at that location; After grayscale The grayscale value at that location.

[0077] Step S314, based on the calculated grayscale value Generate a single-channel grayscale image, which is then output for subsequent filtering processing.

[0078] Step S32: Perform 7×7 Gaussian filtering on the grayscale binocular image to remove noise;

[0079] The noise reduction process for the grayscale binocular image using a 7×7 Gaussian filter includes the following steps:

[0080] Step S321, take =1.5 Calculate the Gaussian kernel in The weight values ​​at each point are calculated, and the results are arranged to form the original weight matrix, which is the original weight matrix of the 7×7 Gaussian kernel, as shown in the following formula;

[0081] Two-dimensional Gaussian function:

[0082] in The relative coordinates within the Gaussian kernel (relative to the center point (the pixel in the 3rd row and 3rd column is the center point (0,0))). and The range of values ​​for all of them is {-3, -2, -1, 0, 1, 2, 3}; The standard deviation of the Gaussian function; For Gaussian kernel in The weight value at that location.

[0083] Step S322: Normalize the original weight matrix to obtain the normalized Gaussian kernel, as shown in the following formula:

[0084]

[0085] Step S323: Traverse each pixel in the grayscale image. ;

[0086] Step S324, with Centered on the target, take 7×7 neighboring pixels, multiply them element-wise with the final weight matrix obtained by the normalized Gaussian kernel, and then sum them up. The formula is as follows:

[0087]

[0088] in After filtering Pixel value at; Grayscale images in The original grayscale value at that location, The calculation is based on step S313, which converts the binocular image to grayscale.

[0089] Step S325: Use the summation result as the filtered pixel value. This generates a denoised image.

[0090] Step S33: Perform histogram equalization on the denoised image.

[0091] Histogram equalization of the denoised image includes the following steps:

[0092] Step S331: Calculate the gray-level histogram of the denoised image. The formula is as follows;

[0093]

[0094] in Grayscale This represents the number of pixels with a grayscale value of k.

[0095] Step S332, calculate the cumulative distribution function. and The formula is as follows;

[0096]

[0097] Step S333: Establish a mapping table between the original gray level k and the equalized gray level k', as shown in the following formula;

[0098]

[0099] in The gray levels after equalization; This represents the minimum value of the cumulative distribution function; This represents the total number of pixels in the image. This is the rounding function.

[0100] Step S334: Traverse each pixel of the image and replace the original grayscale value according to the mapping table;

[0101] Step S335: Generate a contrast-enhanced image and output it for edge detection.

[0102] In summary, the binocular images are first converted to grayscale, so that the color images (RGB three channels) obtained by the binocular camera are converted into single-channel grayscale images, which reduces the amount of data while preserving the key edge features of the image.

[0103] The grayscale binocular image is then subjected to 7×7 Gaussian filtering for noise reduction. A 7×7 Gaussian kernel is convolved with the grayscale image to suppress high-frequency interference such as dust inside the pipe and image sensor noise, while preserving low-frequency features of the pipe opening edge and defects, thus avoiding noise affecting the accuracy of subsequent feature extraction.

[0104] Finally, histogram equalization is performed on the denoised image. By adjusting the distribution of gray values ​​in the image, the gray range of low-contrast areas is stretched, which increases the gray difference between the pipe edge and defect area and the inner wall of the pipe, thereby improving the feature recognition.

[0105] Step S4: Perform baseline localization and defect identification on the preprocessed image:

[0106] The process of benchmark localization and defect identification of the preprocessed image includes:

[0107] Step S41: Use the Canny edge detection algorithm to extract the arc features of the pipe opening edge;

[0108] Extracting the arc features of the pipe opening edge using the Canny edge detection algorithm includes the following steps:

[0109] Step S411: Input the preprocessed image;

[0110] Step S412: Calculate the gradient magnitude G and gradient direction for each pixel, using the following formula;

[0111] Horizontal gradient

[0112]

[0113] Vertical gradient

[0114]

[0115] gradient magnitude

[0116]

[0117] Gradient Angle

[0118]

[0119] in This is the image after histogram equalization.

[0120] Step S413: Traverse each pixel. If the gradient magnitude of the pixel is the maximum value of its neighboring pixels in the gradient direction, then retain it; if the gradient magnitude of the pixel is not the maximum value of its neighboring pixels in the gradient direction, then set it to 0.

[0121] Step S414: Use dual threshold filtering to retain strong edges, connect weak edges with strong edges, and remove isolated weak edges, as shown in the following formula;

[0122] High threshold ;

[0123] low threshold ;

[0124] If the pixel's G> If the pixel's G matches, then the pixel is a strong edge; <G If a pixel is connected to a strong edge, then that pixel is a weak edge; the rest are pseudo edges.

[0125] in , This is the arithmetic mean function. Connecting weak edges and strong edges is analyzed using the 8-neighborhood connectivity criterion or the 4-neighborhood connectivity criterion. The neighboring pixels of the pixels identified as weak edges in the above steps are judged. If the neighboring pixels of the pixel are strong edge pixels, the weak edge is retained; if the neighboring pixels of the pixel are not strong edge pixels, the pixel is set to 0.

[0126] Specifically, when analyzing using the 8-neighborhood connectivity criterion, we examine the pixel identified as a weak edge and its 8 neighboring pixels. If any one of these pixels is a strong edge pixel, then that pixel can be retained as a true edge.

[0127] When analyzing using the 4-neighborhood connectivity criterion, examine the pixel identified as a weak edge and its 8 neighboring pixels. If any one of these pixels is a strong edge pixel, then that pixel can be retained as a true edge.

[0128] Step S415: Output a binary image of the pipe opening edge, and extract 12-15 consecutive edge feature points within the binary image of the pipe opening edge.

[0129] Step S42: Select no less than 10 edge points and use the least squares method to fit the pipe axis and the pipe nozzle reference plane;

[0130] The process of selecting no fewer than 10 edge points and fitting them using the least squares method to obtain the pipe axis and the pipe nozzle reference plane includes the following steps:

[0131] Step S421: Convert the pipe opening edge feature points obtained by the Canny edge detection algorithm into three-dimensional coordinates by combining them with the camera intrinsic parameters;

[0132] Step S422: Since the pipe axis is along the x-axis and the datum plane is perpendicular to the x-axis, the plane equation is determined to be in the form of... ;

[0133] Specifically, the calculation process for determining the plane equation is as follows:

[0134] Plane equation form

[0135]

[0136] Constraints: The pipe axis is along the x-axis, therefore the datum plane is perpendicular to the x-axis, which simplifies to... (A=1, B=0, C=0), only solution D is required.

[0137] objective function

[0138]

[0139] Here are the three-dimensional coordinates of the feature points at the pipe edge, and n is the number of feature points. The parameters to be determined for the plane equation

[0140] Solve for D

[0141] Take the derivative of the objective function and set the derivative to 0:

[0142]

[0143] Solving

[0144]

[0145] Step S423, substitute the extracted multiple continuous edge feature points Coordinates, calculate the parameters to be determined for the plane equation. The formula is as follows:

[0146]

[0147] Specifically, in this embodiment, 12 consecutive feature points are selected.

[0148] Step S43: Use the YOLOv8 algorithm to identify the defect target and output the two-dimensional coordinates of the defect in the image.

[0149] The YOLOv8 algorithm is used to identify defect targets (the training set includes common defects such as cracks, holes, and corrosion). The output of the two-dimensional coordinates of the defects in the image includes the following steps:

[0150] Specifically, the training set used for pre-training the YOLOv8 model in the YOLOv8 algorithm includes common defects such as cracks, holes, and corrosion.

[0151] Step S431: Load the pre-trained YOLOv8 model;

[0152] Step S432: Input the preprocessed image, use the prediction result output by the YOLOv8 model inference, and convert the prediction result output by the YOLOv8 model inference into the original coordinates of the image;

[0153] Specifically, the predicted bounding box output by the YOLOv8 model inference is a normalized coordinate system. The formula for converting coordinates to the original coordinates of the image is as follows:

[0154]

[0155]

[0156]

[0157]

[0158] in Normalized coordinates for the center of the prediction box. These are the normalized width and height of the prediction box, respectively. , These are the width and height of the original image, respectively. : The two-dimensional coordinates of the defect center in the original image.

[0159] Step S433: Confidence screening. Retain prediction boxes with a confidence level greater than or equal to 0.5 and exclude false positives. The formula is as follows:

[0160] Confidence threshold: conf=0.5;

[0161] Step S434: NMS processing, remove overlapping predicted boxes, and retain the best predicted box;

[0162] Specifically, the IOU threshold is iou=0.45, and duplicate prediction boxes with an overlap rate exceeding 45% are removed.

[0163] Step S435, coordinate transformation: convert the normalized coordinates of the center of the optimal prediction box into the original image coordinates, which are used as the two-dimensional coordinates of the defect center;

[0164] Step S436, output the result.

[0165] Through the above processing, a unified benchmark is provided for defect distance measurement. The pre-trained YOLOv8 model can quickly and accurately identify pipeline defects and output the two-dimensional coordinates of the defects in the image, providing target points for three-dimensional reconstruction, thereby ensuring the accurate positioning of both the benchmark and the target of pipeline defects.

[0166] Step S5: Perform stereo matching and 3D reconstruction on the preprocessed image:

[0167] The stereo matching and 3D reconstruction of the preprocessed image includes:

[0168] Step S51: Perform epipolar correction on the preprocessed binocular image to make corresponding points lie on the same horizontal line;

[0169] Epipolar correction of the preprocessed binocular images includes the following steps:

[0170] Step S511: Based on the extrinsic parameters of the binocular camera, the left and right images are projected onto the same epipolar plane using a stereo correction algorithm, so that the y coordinates of corresponding points are consistent.

[0171] Step S52: The disparity map is calculated using the SGBM semi-global stereo matching algorithm;

[0172] The calculation of disparity maps using the SGBM semi-global stereo matching algorithm includes the following steps:

[0173] Step S521: Input the grayscale image corrected by the binocular camera;

[0174] Step S522, for each pixel in the grayscale image Traverse the disparity range and calculate the matching cost using the Census transform, as shown in the following formula;

[0175]

[0176] in For parallax Matching cost at that time; A 3x3 matching window; , These are the images after left and right correction; This represents the disparity value. This is an XOR operation.

[0177] Step S523: Aggregate the matching costs along the 8 paths to obtain the aggregated cost. The formula is as follows;

[0178]

[0179] The cost after aggregation; There are 8 aggregation paths (horizontal, vertical, 45°, 135°); Penalty for adjacent parallax; Non-adjacent parallax penalty; This represents the minimum aggregation cost for all disparities in that row.

[0180] Step S524: Using the WTA (Winner-Take-All) criterion, select the parallax with the lowest aggregation cost. The optimal disparity for that pixel is given by the following formula;

[0181]

[0182] in This is the optimal disparity value.

[0183] Step S525: Post-process the disparity map;

[0184] Specifically, the post-processing of the disparity map is performed using threshold filtering algorithms, Gaussian filtering, disparity filling algorithms, connected component labeling algorithms, etc.

[0185] Step S526: Output the final disparity map, and obtain the disparity value corresponding to the defect center through the final disparity map. The formula is as follows:

[0186]

[0187] Step S53: Using the calibration parameters and the principle of triangulation, obtain the three-dimensional equation of the nozzle reference surface and the three-dimensional coordinates of the defect point.

[0188] By combining calibration parameters and using the principle of triangulation, the three-dimensional equation of the nozzle reference plane and the three-dimensional coordinates of the defect points are obtained, including the following steps:

[0189] Step S531: Obtain the coordinates of the defect center pixel from the left image of the binocular camera. ;

[0190] Step S532: Read the disparity d corresponding to the pixel from the final disparity map;

[0191] Step S533: Substitute the values ​​into the triangulation formula to calculate the three-dimensional coordinates (X, Y, Z) of the defect point.

[0192] The formulas involved are as follows;

[0193] (1) Camera intrinsic parameter matrix:

[0194]

[0195] in For camera focal length, The coordinates of the main point.

[0196] (2) External parameters of binocular camera

[0197] The left camera's coordinate system is the world coordinate system. The extrinsic parameters of the right camera relative to the left camera are: rotation matrix R and translation vector. (B is the baseline length)

[0198] (3) Calculation of three-dimensional coordinates

[0199] Let the coordinates of the defect point in the left image be... The coordinates in the right image are parallax Then the three-dimensional coordinates The calculation formula is as follows:

[0200]

[0201]

[0202]

[0203] in The distance from the defect point to the left camera. These are the horizontal and vertical coordinates of the defect point in the world coordinate system.

[0204] In summary, by employing the SGBM semi-global stereo matching algorithm, pixel-level correspondences are established on the epipolar-corrected binocular images by calculating the disparity between the left and right images, providing disparity data for 3D reconstruction. Simultaneously, using the triangulation principle, the 3D coordinates of defect points are calculated through trigonometric relationships using the calibration parameters of the binocular camera and the disparity obtained from SGBM, achieving the restoration from 2D images to 3D space. This enables accurate reconstruction of the 3D coordinates of defect points, meeting the 3D positioning requirements within long-distance pipelines.

[0205] Step S6: Distance Calculation: Based on the formula for the distance from a spatial point to a plane, calculate the straight-line distance from the defect point to the reference plane of the pipe opening, and output the measurement result. The unit of the measurement result is mm.

[0206] Step S7: Error Correction: Use the marker point with known distance as the standard reference to perform multiple measurements, establish an error compensation model, correct the measurement results, and ensure that the measurement error is less than or equal to ±0.5mm.

[0207] Specifically, the process involves using markers with known distances as standard references for multiple measurements, establishing an error compensation model, and correcting the measurement results, including the following steps:

[0208] Step S71: Substitute the data from multiple measurements into the model formula based on least squares fitting. The formula is as follows:

[0209] in

[0210]

[0211]

[0212] in For the measurement deviation of a single set of standard points, The distance measured from the standard point. The true distance to the standard point. .

[0213] Step S72: Obtain the correction formula for the final measurement result based on the model formula. The correction formula is as follows:

[0214]

[0215] in, This is the original measured distance from the defect to the pipe opening. The final measured distance after error correction (the error must be less than or equal to ±0.5 mm).

[0216] In summary, based on the known distance markers pre-set inside the pipeline, an error compensation model adapted to long-distance measurement scenarios is established, which effectively eliminates system errors, environmental dust and equipment vibration interference, and controls the measurement error within ±0.5mm.

[0217] This invention provides a method for measuring the location of pipe defects to pipe openings based on binocular vision. The method uses a calibration module to calibrate the intrinsic and extrinsic parameters of a binocular camera. A binocular vision acquisition module enters the pipe to acquire a sequence of binocular images containing both the pipe opening and the defect. A 14×11 checkerboard pattern and a multi-pose calibration set are used, combined with the Zhang Zhengyou calibration method to acquire 20-22 sets of multi-pose images, achieving accurate solutions for the camera's intrinsic and extrinsic parameters. Image preprocessing improves image quality, reduces data volume, removes dust noise from large-diameter pipes, and enhances edge features. Canny edge detection, least squares method, and YOLOv8 algorithm are used to collaboratively identify the pipe opening reference and defect features. Combined with SGBM stereo matching and triangulation, efficient stereo matching and 3D reconstruction of defect points are achieved. Based on pre-set known distance markers within the pipe, an error compensation model adapted to long-distance measurement scenarios is established, effectively eliminating system errors, environmental dust, and equipment vibration interference, controlling the measurement error within ±0.5mm. Furthermore, the use of LED supplementary lighting makes it suitable for complex industrial scenarios such as darkness and high dust levels. Ultimately, it solved the problems of traditional pipeline defect location relying on manual labor, low accuracy, and poor adaptability, and realized automated and high-precision measurement of the location of defects inside the pipeline to the pipe opening.

Claims

1. A method for measuring the location of a pipe defect to its opening based on binocular vision, characterized in that, Includes the following steps: Step S1: System Calibration: The checkerboard calibration plate is placed inside the pipe, and multiple sets of binocular images are acquired at different positions and angles. The camera's internal and external parameters are calculated through the calibration module. Step S2: The detection device equipped with a binocular vision acquisition module and a light source module is sent along the pipeline axis to simultaneously acquire binocular images including the pipe opening edge and defect area. The image data is transmitted to the data processing module via wired or wireless means. Step S3: Preprocess the image to improve image quality, reduce data volume, remove dust noise in large-diameter pipes, and enhance edge features; Step S4: Perform baseline localization and defect identification on the preprocessed image: The process of benchmark localization and defect identification of the preprocessed image includes: Step S41: Use the Canny edge detection algorithm to extract the arc features of the pipe opening edge; Step S42: Select no less than 10 edge points and use the least squares method to fit the pipe axis and the pipe nozzle reference plane; Step S43: Use the YOLOv8 algorithm to identify the defect target and output the two-dimensional coordinates of the defect in the image; Step S5: Perform stereo matching and 3D reconstruction on the preprocessed image: Step S6: Distance Calculation: Calculate the straight-line distance from the defect point to the reference plane of the pipe opening based on the formula for the distance from a spatial point to a plane, and output the measurement results; Step S7: Error Correction: Use the marker point with known distance as the standard reference to perform multiple measurements, establish an error compensation model, correct the measurement results, and ensure that the measurement error is less than or equal to ±0.5mm.

2. The method for measuring the location of a pipe defect to its opening based on binocular vision according to claim 1, characterized in that: Extracting the arc features of the pipe opening edge using the Canny edge detection algorithm includes the following steps: Step S411: Input the preprocessed image; Step S412: Calculate the gradient magnitude and gradient direction for each pixel. Step S413: Traverse each pixel. If the gradient magnitude of the pixel is the maximum value of its neighboring pixels in the gradient direction, then retain it; if the gradient magnitude of the pixel is not the maximum value of its neighboring pixels in the gradient direction, then set it to 0. Step S414: Use dual threshold filtering to retain strong edges, connect weak edges with strong edges, and remove isolated weak edges; Step S415: Output a binary image of the pipe opening edge, and extract 12-15 consecutive edge feature points within the binary image of the pipe opening edge.

3. The method for measuring the location of pipe defects to pipe openings based on binocular vision according to claim 1, characterized in that: The process of selecting no fewer than 10 edge points and fitting them using the least squares method to obtain the pipe axis and the pipe nozzle reference plane includes the following steps: Step S421: Convert the pipe opening edge feature points obtained by the Canny edge detection algorithm into three-dimensional coordinates by combining them with the camera intrinsic parameters; Step S422: Since the pipe axis is along the x-axis and the reference plane is perpendicular to the x-axis, determine the form of the plane equation; Step S423: Substitute the coordinates of the extracted continuous edge feature points to calculate the parameters to be determined for the plane equation.

4. The method for measuring the location of a pipe defect to its opening based on binocular vision according to claim 1, characterized in that: The process of identifying defects using the YOLOv8 algorithm and outputting the two-dimensional coordinates of the defects in the image includes the following steps: Step S431: Load the pre-trained YOLOv8 model; Step S432: Input the preprocessed image, use the prediction result output by the YOLOv8 model inference, and convert the prediction result output by the YOLOv8 model inference into the original coordinates of the image; Step S433: Confidence screening, retaining prediction boxes with a confidence level greater than or equal to 0.5, and excluding false positives; Step S434: NMS processing, remove overlapping predicted boxes, and retain the best predicted box; Step S435, coordinate transformation: convert the normalized coordinates of the center of the optimal prediction box into the original image coordinates, which are used as the two-dimensional coordinates of the defect center; Step S436, output the result.

5. The method for measuring the location of pipe defects to pipe openings based on binocular vision according to claim 1, characterized in that: Stereo matching and 3D reconstruction of the preprocessed image includes the following steps: Step S51: Perform epipolar correction on the preprocessed binocular image to make corresponding points lie on the same horizontal line; Step S52: The disparity map is calculated using the SGBM semi-global stereo matching algorithm; Step S53: Using the calibration parameters and the principle of triangulation, obtain the three-dimensional equation of the nozzle reference surface and the three-dimensional coordinates of the defect point.

6. The method for measuring the location of pipe defects to pipe openings based on binocular vision according to claim 5, characterized in that: The calculation of disparity maps using the SGBM semi-global stereo matching algorithm includes the following steps: Step S521: Input the grayscale image corrected by the binocular camera; Step S522: For each pixel in the grayscale image, traverse the disparity range and calculate the matching cost through Census transform; Step S523: Aggregate the matching costs along the 8 paths to obtain the aggregated cost; Step S524: Using the WTA criterion, select the disparity with the minimum aggregation cost as the optimal disparity for that pixel. Step S525: Post-process the disparity map; Step S526: Output the final disparity map and obtain the disparity value corresponding to the defect center through the final disparity map.

7. The method for measuring the location of pipe defects to pipe openings based on binocular vision according to claim 5, characterized in that: By combining calibration parameters and using the principle of triangulation, the three-dimensional equation of the nozzle reference plane and the three-dimensional coordinates of the defect points are obtained, including the following steps: Step S531: Obtain the coordinates of the defect center pixel from the left image of the binocular camera; Step S532: Read the disparity corresponding to the pixel from the final disparity map; Step S533: Substitute the values ​​into the triangulation formula to calculate the three-dimensional coordinates of the defect point.

8. The method for measuring the location of a pipe defect to its opening based on binocular vision according to claim 1, characterized in that: Using markers at known distances as standard references, multiple measurements are performed. An error compensation model is established to correct the measurement results, including: Step S71: Substitute the data from multiple measurements into the model formula based on least squares fitting to obtain the linear fitting coefficients of the model formula. Step S72: Obtain the corrected formula for the final measurement result based on the model formula.

9. The method for measuring the location of a pipe defect to its opening based on binocular vision according to claim 1, characterized in that: The image preprocessing includes: Step S31: Convert the binocular images to grayscale sequentially; Step S32: Perform 7×7 Gaussian filtering on the grayscale binocular image to remove noise; Step S33: Perform histogram equalization on the denoised image.

10. The method for measuring the location of a pipe defect to its opening based on binocular vision according to claim 1, characterized in that: The binocular vision acquisition module uses two 5-megapixel industrial cameras, the calibration module incorporates Zhang Zhengyou's calibration algorithm, the light source module includes LED fill lights, and the data processing module uses an embedded processor.

Images