A 3D reconstruction method for underground pipelines based on binocular laser scanning

Abstract

The present invention discloses a method for 3D reconstruction of underground pipelines based on binocular laser scanning, comprising the following steps: pre-calibrating the binocular camera on the robot, recording the internal and external calibration parameters of the camera; during the robot's movement, turning on the annular laser projector, projecting an annular laser pattern on the inner wall of the pipeline, using the calibrated binocular camera to collect the binocular image sequence inside the pipeline and perform correction processing; using the improved Steger algorithm to extract the center line of the annular laser pattern in the binocular image respectively; performing stereo matching on the laser center line extracted from the binocular image; reconstructing the 3D ring of the laser center line in the current binocular image frame according to the binocular camera calibration parameters; using the inertial measurement unit and the wheel odometer to deduce the robot's motion trajectory in the pipeline, splicing the 3D rings reconstructed in different image sequence frames, and completing the overall 3D reconstruction of the underground pipeline. The present invention can solve the problem of 3D reconstruction in various narrow underground pipelines.

Description

Technical Field

[0001] The invention relates to a three-dimensional reconstruction method of an underground pipeline based on binocular laser scanning, and belongs to the technical field of three-dimensional reconstruction and computer vision. Background Art

[0002] With the rapid development of urban infrastructure modernization, underground pipeline construction is becoming more and more intensive, and it is an important part of the national infrastructure network. However, during the use of underground pipelines, abnormal problems often occur inside the pipelines, which will seriously affect the safety and function of the facilities and need to be inspected regularly. Due to the concealment and difficulty of underground pipeline construction, it is difficult for workers to directly handle it using conventional methods, so it is a very challenging task to inspect its interior.

[0003] At present, in the field of pipeline inspection, some commonly used methods still have certain defects. For example, in "A robot for urban underground pipeline inspection", a pipeline inspection robot equipped with closed-circuit television is used to manually judge the collected pipeline video, which is time-consuming and labor-intensive and has low accuracy. The two-dimensional picture lacks a lot of spatial information and cannot accurately show the real internal structure. It is also impossible to visualize the internal data of the pipeline and lacks intuitive observation effects. In addition, there are also "A supported drainage pipeline laser inspection robot and method", which uses a three-dimensional modeling method based on laser scanning, but its robot cannot locate its position in the pipeline, cannot establish a connection between the collected data and the pipeline position, and cannot determine the position of the pipeline abnormality, thereby reducing the applicability and limiting the role of the robot. In recent years, computer vision technology for three-dimensional reconstruction has developed rapidly and has also been used in the field of detection. Therefore, how to use three-dimensional reconstruction technology and combine inertial navigation positioning to perform three-dimensional reconstruction of underground pipelines has become a technical problem that needs to be studied. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a three-dimensional reconstruction method for underground pipelines based on binocular laser scanning, which solves the problems of poor three-dimensional reconstruction effect and lack of positioning of existing underground pipelines.

[0005] A method for 3D reconstruction of underground pipelines based on binocular laser scanning comprises the following steps:

[0006] Step S1: using a pipeline robot equipped with a binocular camera, a ring laser projector, an inertial measurement unit and a wheel odometer as an information collection device, and pre-calibrating the binocular camera of the pipeline robot;

[0007] Step S2: Turn on the annular laser projector carried by the pipeline robot, project the annular laser pattern onto the inner wall of the pipeline, and control the pipeline robot to move forward in the pipeline, obtain and record the inertial measurement unit data, the wheel odometer data, and the binocular image of the inner wall of the pipeline containing the annular laser pattern, and calibrate the binocular image;

[0008] Step S3: using the improved Steger algorithm to extract the center lines of the annular laser patterns in the binocular image respectively;

[0009] Step S4: performing stereo matching on the laser center line extracted from the binocular image;

[0010] Step S5: reconstructing the three-dimensional ring of the laser center line in the current binocular image frame according to the binocular camera calibration parameters;

[0011] Step S6: Repeat steps S2-S5 to calculate and reconstruct the three-dimensional ring in each frame of the binocular image during the robot's motion;

[0012] Step S7: using the inertial measurement unit and wheel odometer data to derive the motion trajectory of the robot in the pipeline;

[0013] Step S8: splicing the reconstructed rings according to the movement trajectory and posture data to complete the three-dimensional point cloud reconstruction of the entire target pipeline.

[0014] Specifically, step S1 includes:

[0015] Step S1-1: Paste the printed checkerboard paper on a flat plate to make a calibration plate, and use the binocular camera on the pipeline robot to take pictures of the calibration plate at different angles to obtain more coordinate information;

[0016] Step S1-2: Use Zhang's calibration method to pre-calibrate the binocular camera to obtain the camera's internal and external parameters.

[0017] Specifically, step S2 includes:

[0018] Step S2-1: The pipeline robot waits for a few minutes at the entrance of the underground pipeline, and the Beidou positioning system measures the position coordinates of the entrance of the pipeline (x start ,y start ), then turn on the ring laser projector carried by the pipeline robot to project a ring laser pattern onto the inner wall of the pipeline;

[0019] Step S2-2: Control the robot to move forward in the pipeline, obtain and record the inertial measurement unit data and the wheel odometer data through the built-in inertial measurement unit and the wheel odometer of the pipeline robot, take a binocular image of the inner wall of the pipeline containing the ring laser pattern, and calibrate the binocular image;

[0020] Step S2-3: The pipeline robot stops at the end point, and the Beidou positioning system measures the exit position (x end ,y end ) to complete the navigation and positioning of the pipeline.

[0021] Specifically, step S3 includes:

[0022] Step S3-1: grayscale the acquired image of the pipeline ring laser, and select the grayscale image with the highest contrast among the three channels;

[0023] Step S3-2: performing an image denoising operation based on Gaussian filtering and median filtering on the grayscale image;

[0024] Step S3-3: In order to reduce the similar parts of the images and highlight the changes in the images, the image difference operation is continued, which is specifically expressed as follows:

[0025] G(d)=G(f)-G(b) (1)

[0026] Wherein, G(f) represents the gray value of the image including the laser stripes, G(b) represents the gray value of the image without the laser stripes, and G(d) represents the gray value of the difference result;

[0027] Step S3-4: extract the edge of the laser using Canny edge detection, and then segment to obtain the region of interest;

[0028] Step S3-5: smoothing the laser edge by joint bilateral filtering to remove the burrs and improve the accuracy of the Steger algorithm;

[0029] Step S3-6: Use the Steger algorithm to extract the center lines of the ring lasers in the binocular images.

[0030] Specifically, in step S3-5, it is proposed to use bilateral filtering to smooth the edge. This method combines the pixel value similarity and spatial proximity in the image field, introduces the value domain weight and the spatial domain weight, which are specifically expressed as:

[0031]

[0032]

[0033] Among them, G r , G s denote the pixel domain kernel function and the spatial domain kernel function respectively, σ r , σ s Represent the standard deviation of the range and space domain respectively, S P represents the output image, represents the normalization term of joint filtering, I represents the input image, and Ip ,I q represents the pixel value of the input image at p and q, where p and q represent the pixel coordinates (x p ,y p )、(x q ,y q ), represents the guide image, which is the image after Gaussian smoothing of the input original image. Represents the pixel value of the guidance image at p and q.

[0034] Specifically, in step S3-6, the Steger algorithm calculates the Hessian matrix of the pixel point in the laser stripe, finds the direction of the normal line, performs Taylor expansion on the grayscale distribution of the stripe along this direction, and finds the maximum function value, the first-order zero point and the second-order minimum value, which is the laser center point. The Hessian matrix of a pixel point (x, y) in the image is specifically expressed as:

[0035]

[0036] Among them, d xx d xy d yy They represent the partial derivatives of the convolution image d(x,y) through the discrete two-dimensional Gaussian partial derivative kernel, and the specific calculation is as follows:

[0037]

[0038]

[0039]

[0040] G(x,y) is a two-dimensional Gaussian convolution kernel, which is specifically expressed as:

[0041]

[0042] Where σ represents the standard deviation, d(x,y) is the image matrix centered at point (x,y), and is equal to the size of the Gaussian kernel G(x,y).

[0043] Then, the pixel point (x 0 ,y 0 ) is determined by the eigenvector (n x ,n y ) is determined. Pixel point (x 0 ,y 0 ) is expanded by the second-order Taylor formula, and the specific calculation is:

[0044]

[0045] The grayscale extreme point in the normal direction is taken as the center point, and d(x 0 +tn x ,y 0 +tn y ) is zero to get this point, so the sub-pixel center is (x 0 +tn x ,y 0 +tn y ), t is specifically expressed as:

[0046]

[0047] Among them, d x d y Respectively expressed as:

[0048]

[0049]

[0050] Specifically, step S4 includes:

[0051] Step S4-1: Divide the center lines of the laser rings extracted by the left eye and the right eye into two half circles from the middle, respectively;

[0052] Step S4-2: Count the coordinates of the center lines of the four semicircular lasers respectively, and record them as a coordinate set of two-dimensional pixel points;

[0053] Step S4-3: Scan the left semicircle of the left eye image and the left semicircle of the right eye image from the top to the bottom, and count the pixel coordinates (x l ,y l )、(x r ,y r );

[0054] Step S4-4: Since image correction has been performed before, the same points are in the same row in the left and right images, that is, the vertical coordinates are the same. According to the pixel distribution information of the laser center line in the binocular image, a stereo matching loss function is constructed, and the same-name points on the center line of the laser pattern in the binocular image are searched and determined one by one based on the principle of minimizing the loss function. Parallax optimization is performed through left and right consistency detection. If the parallax value of the corresponding pixel point exceeds the set threshold, it is an erroneous matching point and needs to be deleted to improve the matching accuracy, thereby completing the matching of the two left semicircles, which is specifically expressed as:

[0055]

[0056] Among them, d represents the disparity of the corresponding points, d threshold Indicates the set threshold, d falseParallax that indicates error;

[0057] Step S4-5: Repeat the above steps for the right semicircle of the left eye image and the right semicircle of the right eye image to complete the matching of the two right semicircles;

[0058] Step S4-6: The left semicircle and the right semicircle are matched and merged to complete the stereo matching of the center line of the entire laser ring.

[0059] Specifically, step S7 includes:

[0060] The pipeline robot uses a micro-inertial measurement unit of a micro-electromechanical system sensor. When the robot system starts working, it is necessary to quickly align the pipeline robot to obtain the initial posture matrix. The accelerometer takes the average value of the collected acceleration in a few minutes of stillness. Specifically expressed as:

[0061]

[0062] Where N is the amount of output data, t is the sampling time, T is the sampling time interval, and a(k) is the acceleration value at time k.

[0063] The initial pitch angle θ of the robot can be obtained based on the acceleration data 0 and roll angle φ 0 , specifically expressed as:

[0064]

[0065]

[0066] Among them, a x 、a y 、a z They respectively represent the acceleration values ​​of the three coordinate axes when the pipeline robot is stationary, and g represents the acceleration due to gravity.

[0067] The initial heading angle of the approximately straight pipeline can be calculated based on the starting point coordinates of the pipeline using Beidou positioning. Specifically expressed as:

[0068]

[0069] According to the above-mentioned attitude angle, it can be converted into the initial attitude matrix The update formula of the posture matrix when the robot moves forward is specifically expressed as:

[0070]

[0071] in, represents the posture matrix at each sampling, k represents the number of sampling times, I represents the unit matrix, Δt represents the sampling time, Ω k represents the angular velocity ω k The antisymmetric matrix of .

[0072] The angular velocity formula measured by the gyroscope is specifically expressed as:

[0073]

[0074] Among them, b represents the right front upward coordinate system, ω b represents the angular velocity of the system, is the angular velocity of the Earth, represents the angular velocity of the robot, ω error Represents the measurement error of the gyroscope.

[0075] The speed of the pipeline robot is calculated by the installed wheel odometer. Since the odometer is installed on the wheel, it is inconsistent with the center of mass position of the micro-inertial measurement unit and needs to be converted. The calculation formula is specifically expressed as:

[0076]

[0077] Among them, v (k) represents the final transformed center of mass velocity, v l(k) 、v r(k) They represent the speeds of the left and right wheels respectively, and R represents the distance from the cross-sectional wheel to the center of mass of the micro inertial measurement unit.

[0078] Based on the above, the moving position of the robot in the pipeline can be calculated. The calculation formula is specifically expressed as:

[0079]

[0080] Among them, l (k) =[x (k) ,y (k) ,z (k) ], x (k) ,y (k) 、z (k) They represent the position coordinates of the pipeline robot along the east, north and up directions at each sampling, Indicates the moving speed along the right front upper coordinate system at time k. The default lateral speed and vertical speed are zero.

[0081] The data from the inertial measurement unit and wheel odometer are used to obtain the position and posture data of the pipeline robot, and the motion trajectory of the robot in the pipeline is derived.

[0082] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention proposes a three-dimensional reconstruction system for underground pipelines based on binocular laser scanning. Binocular laser scanning can capture images containing strong features of laser stripes inside the pipeline, effectively achieve matching of feature points, quickly obtain depth information, reduce interference, and achieve better three-dimensional reconstruction of a single ring. Adding an inertial measurement unit and a wheeled odometer to the pipeline robot can achieve real-time positioning of the robot in the pipeline, splice the reconstructed rings through the moving trajectory and posture data, complete accurate positioning and three-dimensional reconstruction of underground pipelines, quickly locate abnormal positions, and have a wider range of applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A flowchart of a method for 3D reconstruction of underground pipelines based on binocular laser scanning proposed by the present invention;

[0084] Figure 2 This is a working schematic diagram of the underground pipeline robot of the present invention;

[0085] Figure 3 Schematic diagrams of stereo matching of the left semicircle and the right semicircle of the present invention;

[0086] Figure 4 This is a flowchart of the underground pipeline robot positioning of the present invention. DETAILED DESCRIPTION

[0087] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] The present invention adopts the following technical scheme, combined with Figure 1 In order to complete the three-dimensional reconstruction task, the present invention proposes a three-dimensional reconstruction system for underground pipelines based on binocular laser scanning, which includes the following steps:

[0089] Step S1: using a pipeline robot equipped with a binocular camera, a ring laser projector, an inertial measurement unit and a wheel odometer as an information collection device, and pre-calibrating the binocular camera of the pipeline robot;

[0090] Step S2: Turn on the annular laser projector carried by the pipeline robot, project the annular laser pattern onto the inner wall of the pipeline, and control the pipeline robot to move forward in the pipeline, obtain and record the inertial measurement unit data, the wheel odometer data, and the binocular image of the inner wall of the pipeline containing the annular laser pattern, and calibrate the binocular image;

[0091] Step S3: using the improved Steger algorithm to extract the center lines of the annular laser patterns in the binocular image respectively;

[0092] Step S4: performing stereo matching on the laser center line extracted from the binocular image;

[0093] Step S5: reconstructing the three-dimensional ring of the laser center line in the current binocular image frame according to the binocular camera calibration parameters;

[0094] Step S6: Repeat steps S2-S5 to calculate and reconstruct the three-dimensional ring in each frame of the binocular image during the robot's motion;

[0095] Step S7: using the inertial measurement unit and wheel odometer data to derive the motion trajectory of the robot in the pipeline;

[0096] Step S8: splicing the reconstructed rings according to the movement trajectory and posture data to complete the three-dimensional point cloud reconstruction of the entire target pipeline.

[0097] Preferably, step S1 includes:

[0098] Step S1-1: Paste the printed checkerboard paper on a flat plate to make a calibration plate, and use the binocular camera on the pipeline robot to take pictures of the calibration plate at different angles to obtain more coordinate information;

[0099] Step S1-2: Use Zhang's calibration method to pre-calibrate the binocular camera to obtain the camera's internal and external parameters.

[0100] Combination Figure 2 Preferably, step S2 includes:

[0101] Step S2-1: The pipeline robot waits for a few minutes at the entrance of the underground pipeline, and the Beidou positioning system measures the position coordinates of the entrance of the pipeline (x start ,y start ), then turn on the ring laser projector carried by the pipeline robot to project a ring laser pattern onto the inner wall of the pipeline;

[0102] Step S2-2: Control the robot to move forward in the pipeline, obtain and record the inertial measurement unit data and the wheel odometer data through the built-in inertial measurement unit and the wheel odometer of the pipeline robot, take a binocular image of the inner wall of the pipeline containing the ring laser pattern, and calibrate the binocular image;

[0103] Step S2-3: The pipeline robot stops at the end point, and the Beidou positioning system measures the exit position (x end ,y end ) to complete the navigation and positioning of the pipeline.

[0104] Preferably, step S3 includes:

[0105] Step S3-1: grayscale the acquired image of the pipeline ring laser, and select the grayscale image with the highest contrast among the three channels;

[0106] Step S3-2: performing an image denoising operation based on Gaussian filtering and median filtering on the grayscale image;

[0107] Step S3-3: In order to reduce the similar parts of the images and highlight the changes in the images, the image difference operation is continued, which is specifically expressed as follows:

[0108] G(d)=G(f)-G(b) (1)

[0109] Wherein, G(f) represents the gray value of the image including the laser stripes, G(b) represents the gray value of the image without the laser stripes, and G(d) represents the gray value of the difference result;

[0110] Step S3-4: extract the edge of the laser using Canny edge detection, and then segment to obtain the region of interest;

[0111] Step S3-5: smoothing the laser edge by joint bilateral filtering to remove the burrs and improve the accuracy of the Steger algorithm;

[0112] Step S3-6: Use the Steger algorithm to extract the center lines of the ring lasers in the binocular images.

[0113] Preferably, in step S3-5, it is proposed to use bilateral filtering to smooth the edge. This method combines the pixel value similarity and spatial proximity in the image field, introduces value domain weight and spatial domain weight, which are specifically expressed as:

[0114]

[0115]

[0116] Among them, G r , G s denote the pixel domain kernel function and the spatial domain kernel function respectively, σ r , σ s Represent the standard deviation of the range and space domain respectively, S P represents the output image, represents the normalization term of joint filtering, I represents the input image, and I p ,I q represents the pixel value of the input image at p and q, where p and q represent the pixel coordinates (x p ,y p )、(x q ,yq ), represents the guide image, which is the image after Gaussian smoothing of the input original image. Represents the pixel value of the guidance image at p and q.

[0117] Preferably, in step S3-6, the Steger algorithm calculates the Hessian matrix of the pixel points in the laser stripe, finds the direction of the normal line, performs Taylor expansion on the grayscale distribution of the stripe along this direction, and finds the maximum function value, the first-order zero point and the second-order minimum value, which is the laser center point. The Hessian matrix of a pixel point (x, y) in the image is specifically expressed as:

[0118]

[0119] Among them, d xx d xy d yy They represent the partial derivatives of the convolution image d(x,y) through the discrete two-dimensional Gaussian partial derivative kernel, and the specific calculation is as follows:

[0120]

[0121]

[0122]

[0123] G(x,y) is a two-dimensional Gaussian convolution kernel, which is specifically expressed as:

[0124]

[0125] Where σ represents the standard deviation, d(x,y) is the image matrix centered at point (x,y), and is equal to the size of the Gaussian kernel G(x,y).

[0126] Then, the pixel point (x 0 ,y 0 ) is determined by the eigenvector (n x ,n y ) is determined. Pixel point (x 0 ,y 0 ) is expanded by the second-order Taylor formula, and the specific calculation is:

[0127]

[0128] The grayscale extreme point in the normal direction is taken as the center point, and d(x 0 +tn x ,y 0 +tn y) is zero to get this point, so the sub-pixel center is (x 0 +tn x ,y 0 +tn y ), t is specifically expressed as:

[0129]

[0130] Among them, d x d y Respectively expressed as:

[0131]

[0132]

[0133] Combination Figure 3 Preferably, step S4 includes:

[0134] Step S4-1: Divide the center lines of the laser rings extracted by the left eye and the right eye into two half circles from the middle, respectively;

[0135] Step S4-2: Count the coordinates of the center lines of the four semicircular lasers respectively, and record them as a coordinate set of two-dimensional pixel points;

[0136] Step S4-3: Scan the left semicircle of the left eye image and the left semicircle of the right eye image from the top to the bottom, and count the pixel coordinates (x l ,y l )、(x r ,y r );

[0137] Step S4-4: Since image correction has been performed before, the same points are in the same row in the left and right images, that is, the vertical coordinates are the same. According to the pixel distribution information of the laser center line in the binocular image, a stereo matching loss function is constructed, and the same-name points on the center line of the laser pattern in the binocular image are searched and determined one by one based on the principle of minimizing the loss function. Parallax optimization is performed through left and right consistency detection. If the parallax value of the corresponding pixel point exceeds the set threshold, it is an erroneous matching point and needs to be deleted to improve the matching accuracy, thereby completing the matching of the two left semicircles, which is specifically expressed as:

[0138]

[0139] Among them, d represents the disparity of the corresponding points, d threshold Indicates the set threshold, d false Parallax that indicates error;

[0140] Step S4-5: Repeat the above steps for the right semicircle of the left eye image and the right semicircle of the right eye image to complete the matching of the two right semicircles;

[0141] Step S4-6: The left semicircle and the right semicircle are matched and merged to complete the stereo matching of the center line of the entire laser ring.

[0142] Combination Figure 4 Preferably, step S7 includes:

[0143] The pipeline robot uses a micro-inertial measurement unit of a micro-electromechanical system sensor. When the robot system starts working, it is necessary to quickly align the pipeline robot to obtain the initial posture matrix. The accelerometer takes the average value of the collected acceleration in a few minutes of stillness. Specifically expressed as:

[0144]

[0145] Where N is the amount of output data, t is the sampling time, T is the sampling time interval, and a(k) is the acceleration value at time k.

[0146] The initial pitch angle θ of the robot can be obtained based on the acceleration data 0 and roll angle φ 0 , specifically expressed as:

[0147]

[0148]

[0149] Among them, a x 、a y 、a z They respectively represent the acceleration values ​​of the three coordinate axes when the pipeline robot is stationary, and g represents the acceleration due to gravity.

[0150] The initial heading angle of the approximately straight pipeline can be calculated based on the starting point coordinates of the pipeline using Beidou positioning. Specifically expressed as:

[0151]

[0152] According to the above-mentioned attitude angle, it can be converted into the initial attitude matrix The update formula of the posture matrix when the robot moves forward is specifically expressed as:

[0153]

[0154] in, represents the posture matrix at each sampling, k represents the number of sampling times, I represents the unit matrix, Δt represents the sampling time, Ω krepresents the angular velocity ω k The antisymmetric matrix of .

[0155] The angular velocity formula measured by the gyroscope is specifically expressed as:

[0156]

[0157] Among them, b represents the right front upward coordinate system, ω b represents the angular velocity of the system, is the angular velocity of the Earth, represents the angular velocity of the robot, ω error Represents the measurement error of the gyroscope.

[0158] The speed of the pipeline robot is calculated by the installed wheel odometer. Since the odometer is installed on the wheel, it is inconsistent with the center of mass position of the micro-inertial measurement unit and needs to be converted. The calculation formula is specifically expressed as:

[0159]

[0160] Among them, v (k) represents the final transformed center of mass velocity, v l(k) 、v r(k) They represent the speeds of the left and right wheels respectively, and R represents the distance from the cross-sectional wheel to the center of mass of the micro inertial measurement unit.

[0161] Based on the above, the moving position of the robot in the pipeline can be calculated. The calculation formula is specifically expressed as:

[0162]

[0163] Among them, l (k) =[x (k) ,y (k) ,z (k) ], x (k) ,y (k) 、z (k) They represent the position coordinates of the pipeline robot along the east, north and up directions at each sampling, Indicates the moving speed along the right front upper coordinate system at time k. The default lateral speed and vertical speed are zero.

[0164] The data from the inertial measurement unit and wheel odometer are used to obtain the position and posture data of the pipeline robot, and the motion trajectory of the robot in the pipeline is derived.

[0165] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for 3D reconstruction of underground pipelines based on binocular laser scanning, characterized in that: The following steps are involved: Step S1: using a pipeline robot equipped with a binocular camera, a ring laser projector, an inertial measurement unit and a wheel odometer as an information collection device, and pre-calibrating the binocular camera of the pipeline robot; Step S2: Turn on the annular laser projector carried by the pipeline robot, project the annular laser pattern onto the inner wall of the pipeline, and control the pipeline robot to move forward in the pipeline, obtain and record the inertial measurement unit data, the wheel odometer data, and the binocular image of the inner wall of the pipeline containing the annular laser pattern, and calibrate the binocular image; Step S3: using the improved Steger algorithm to extract the center lines of the annular laser patterns in the binocular image respectively; Step S4: performing stereo matching on the laser center line extracted from the binocular image; Step S5: reconstructing the three-dimensional ring of the laser center line in the current binocular image frame according to the binocular camera calibration parameters; Step S6: Repeat steps S2-S5 to calculate and reconstruct the three-dimensional ring in each frame of the binocular image during the robot's motion; Step S7: using the data from the inertial measurement unit and the wheel odometer to derive the motion trajectory of the robot in the pipeline; Step S8: splicing the reconstructed rings according to the movement trajectory and posture data to complete the three-dimensional point cloud reconstruction of the entire target pipeline; Step S3 includes: Step S3-1: grayscale the acquired image of the pipeline ring laser, and select the grayscale image with the highest contrast among the three channels; Step S3-2: performing an image denoising operation based on Gaussian filtering and median filtering on the grayscale image; Step S3-3: In order to reduce the similar parts of the images and highlight the changes in the images, the image difference operation is continued, which is specifically expressed as follows: G(d)=G(f)-G(b) Wherein, G(f) represents the gray value of the image including the laser stripes, G(b) represents the gray value of the image without the laser stripes, and G(d) represents the gray value of the difference result; Step S3-4: extract the edge of the laser using Canny edge detection, and then segment to obtain the region of interest; Step S3-5: smoothing the laser edge by joint bilateral filtering to remove the burrs and improve the accuracy of the Steger algorithm; Step S3-6: Use the Steger algorithm to extract the center lines of the binocular image ring lasers respectively; In step S7, the pipeline robot uses a micro-inertial measurement unit of a micro-electromechanical system sensor. When the robot system starts working, it is necessary to quickly align the pipeline robot to obtain an initial posture matrix. The accelerometer averages the collected accelerations for a few minutes of static time. Specifically expressed as: Where N is the output data volume, t is the sampling time, T is the sampling time interval, and a(k) is the acceleration value at time k; According to the acceleration data, the initial pitch angle θ0 and roll angle φ0 of the robot can be obtained, which can be specifically expressed as: Among them, a x 、a y 、a z They represent the acceleration values ​​of the three coordinate axes when the pipeline robot is stationary, and g represents the acceleration due to gravity; According to the Beidou positioning pipeline starting point coordinates (x start ,y start ) and the end point coordinates (x end ,y end ) can calculate the initial heading angle of the approximately straight pipeline Specifically expressed as: According to the above-mentioned attitude angle, it can be converted into the initial attitude matrix The update formula of the posture matrix when the robot moves forward is specifically expressed as: in, represents the posture matrix at each sampling, k represents the number of sampling times, I represents the unit matrix, Δt represents the sampling time, Ω k represents the angular velocity ω k The antisymmetric matrix of ; The angular velocity formula measured by the gyroscope is specifically expressed as: Among them, b represents the right front upward coordinate system, ω b represents the angular velocity of the system, is the angular velocity of the Earth, represents the angular velocity of the robot, ω error Represents the measurement error of the gyroscope; The speed of the pipeline robot is calculated by the installed wheel odometer. Since the odometer is installed on the wheel, it is inconsistent with the center of mass position of the micro-inertial measurement unit and needs to be converted. The calculation formula is specifically expressed as: Among them, v (k) represents the final transformed center of mass velocity, v l(k) 、v r(k) denote the speeds of the left and right wheels, respectively, and R denotes the distance from the cross-sectional wheel to the center of mass of the micro-inertial measurement unit; Based on the above, the moving position of the robot in the pipeline can be calculated, and the calculation formula is specifically expressed as: Among them, l (k) =[x (k) ,y (k) ,z (k) ], x (k) ,y (k) 、z (k) They represent the position coordinates of the pipeline robot along the east, north and up directions at each sampling, Indicates the moving speed along the right front upper coordinate system at time k, and the default lateral speed and vertical speed are zero; The data from the inertial measurement unit and wheel odometer are used to obtain the position and posture data of the pipeline robot, and the motion trajectory of the robot in the pipeline is derived.

2. The method for 3D reconstruction of underground pipelines based on binocular laser scanning according to claim 1, characterized in that: Step S1 includes: Step S1-1: Paste the printed checkerboard paper on a flat plate to make a calibration plate, and use the binocular camera on the pipeline robot to take pictures of the calibration plate at different angles to obtain more coordinate information; Step S1-2: Use Zhang's calibration method to pre-calibrate the binocular camera to obtain the camera's internal and external parameters.

3. The method for 3D reconstruction of underground pipelines based on binocular laser scanning according to claim 1 is characterized in that: Step S2 includes: Step S2-1: The pipeline robot waits for a few minutes at the entrance of the underground pipeline, and the Beidou positioning system measures the position coordinates of the entrance of the pipeline (x start ,y start ), then turn on the ring laser projector carried by the pipeline robot to project a ring laser pattern onto the inner wall of the pipeline; Step S2-2: Control the robot to move forward in the pipeline, obtain and record the inertial measurement unit data and the wheel odometer data through the built-in inertial measurement unit and the wheel odometer of the pipeline robot, take a binocular image of the inner wall of the pipeline containing the ring laser pattern, and calibrate the binocular image; Step S2-3: The pipeline robot stops at the end point, and the Beidou positioning system measures the exit position (x end ,y end ) to complete the navigation and positioning of the pipeline.

4. The method for 3D reconstruction of underground pipelines based on binocular laser scanning according to claim 1, characterized in that: In step S3-5, bilateral filtering is used to smooth the laser edge, combining the pixel value similarity and spatial proximity in the image domain, and introducing the value domain weight and spatial domain weight, which are specifically expressed as: Among them, G r , G s denote the pixel domain kernel function and the spatial domain kernel function respectively, σ r , σ s Represent the standard deviation of the range and space domain respectively, S P represents the output image, represents the normalization term of joint filtering, I represents the input image, and I p ,I q represents the pixel value of the input image at p and q, where p and q represent the pixel coordinates (x p ,y p )、(x q ,y q ), represents the guide image, which is the image after Gaussian smoothing of the input original image. Represents the pixel value of the guidance image at p and q.

5. The method for 3D reconstruction of underground pipelines based on binocular laser scanning according to claim 1, characterized in that: In step S3-6, the Steger algorithm calculates the Hessian matrix of the pixel points in the laser stripe, finds the direction of the normal line, performs Taylor expansion on the grayscale distribution of the stripe along this direction, and finds the maximum function value, the first-order zero point and the second-order minimum point, which is the laser center point; the Hessian matrix of a pixel point (x, y) in the image is specifically expressed as: Among them, d xx ,d xy ,d yy They represent the partial derivatives of the convolution image d(x,y) through the discrete two-dimensional Gaussian partial derivative kernel, and the specific calculation is as follows: G(x,y) is a two-dimensional Gaussian convolution kernel, which is specifically expressed as: Where σ represents the standard deviation, d(x,y) is the image matrix centered at point (x,y), and is equal to the size of the Gaussian kernel G(x,y); Then, the normal direction of the pixel point (x0, y0) is determined by the eigenvector (n x ,n y ) is determined; the grayscale distribution function in the normal direction of the pixel point (x0, y0) is expanded by the second-order Taylor formula, and the specific calculation is: The grayscale extreme point in the normal direction is taken as the center point, and d(x0+tn x ,y0+tn y ) is zero to get this point, so the sub-pixel center is (x0+tn x ,y0+tn y ), t is specifically expressed as: Among them, d x ,d y Respectively expressed as: 。 6. The method for 3D reconstruction of underground pipelines based on binocular laser scanning according to claim 1, characterized in that: Step S4 includes: Step S4-1: Divide the center lines of the laser rings extracted by the left eye and the right eye into two half circles from the middle, respectively; Step S4-2: Count the coordinates of the center lines of the four semicircular lasers respectively, and record them as a coordinate set of two-dimensional pixel points; Step S4-3: Scan the left semicircle of the left eye image and the left semicircle of the right eye image from the top to the bottom, and count the pixel coordinates (x l ,y l )、(x r ,y r ); Step S4-4: Since image correction has been performed in step S2, the same points are in the same row in the left and right images, that is, the vertical coordinates are the same; according to the pixel distribution information of the laser center line in the binocular image, a stereo matching loss function is constructed, and the same-name points on the center line of the laser pattern in the binocular image are searched and determined one by one based on the principle of minimizing the loss function; parallax optimization is performed through left and right consistency detection. If the parallax value of the corresponding pixel point exceeds the set threshold, it is an erroneous matching point and needs to be deleted to improve the matching accuracy, thereby completing the matching of the two left semicircles, which is specifically expressed as: Among them, d represents the disparity of the corresponding points, d threshold Indicates the set threshold, d false Parallax that indicates error; Step S4-5: Repeat the above steps for the right semicircle of the left eye image and the right semicircle of the right eye image to complete the matching of the two right semicircles; Step S4-6: The left semicircle and the right semicircle are matched and merged to complete the stereo matching of the center line of the entire laser ring.

Images