Apparatus for measuring relative displacement of joint of jacking pipe and detecting water leakage, and method

Abstract

An apparatus for measuring the relative displacement of a joint of a jacking pipe and detecting water leakage. The apparatus comprises a mobile measurement vehicle (20), wherein a binocular image processing module (2), a data processing module (4) and a PID control module (12) are arranged on the mobile measurement vehicle (20). A telescopic rod (9) is arranged at the front end of the mobile measurement vehicle (20), and a plurality of laser range finders (11) are annularly arranged at the end of the telescopic rod (9). The mobile measurement vehicle (20) is provided with a sliding groove (18) and two cameras (3) are arranged on the sliding groove (18). An odometer (17) is arranged on the mobile measurement vehicle (20). Further provided is a method for measuring the relative displacement of a joint of a jacking pipe and detecting water leakage, and the method comprises: acquiring initial information; by means of each laser range finder (11), measuring the distance from an inner surface of a jacking pipe, and controlling the position of a running trajectory of a mobile measurement vehicle (20); by means of an odometer (17), the mobile measurement vehicle (20) stopping moving forward every two meters of travel, and each camera (3) capturing an image of the inner surface; realizing correction of a point cloud; converting the captured image into a black-and-white grayscale level chart; by means of extracting point cloud coordinates at a front position and a rear position of a joint, obtaining the relative displacement between a front joint and a rear joint; and by means of a grayscale level change in the image, positioning a water seepage area. The present invention achieves automatically measuring a relative displacement at a joint of a pipe segment of a jacking pipe and automatically detecting a water leakage area inside the pipe.

Description

Device and method for detecting relative displacement and water leakage of jacking pipe interface Technical Field

[0001] The invention relates to the technical field of trenchless engineering pipe jacking, in particular to a device and method for detecting relative displacement and water leakage of a pipe jacking interface. Background Art

[0002] Pipe jacking is a commonly used pipeline structure in underground projects. Since pipe jacking construction does not require the assembly of tunnel linings and does not require ground excavation, it can pass through roads, railways, rivers, ground buildings, underground structures and various underground pipelines, etc., so it will not hinder traffic and has little impact on the surrounding environment. Therefore, pipe jacking is widely used in urban municipal fields. It is currently widely used in urban water supply and drainage, gas pipelines, power tunnels, communication cables and other infrastructure construction, as well as in the construction of transportation such as roads, railways, and tunnels. Technical issues

[0003] During pipe jacking construction, factors such as pipe segment correction, the complex spatial curve axis required to avoid existing underground structures, uneven foundation settlement, and the slightly larger outer notch than the inner notch of the pipe segment can cause relative displacement at the pipe segment interfaces and cracks within the pipe segment. This can affect key pipe characteristics such as stress distribution within the pipe, stress transfer at the pipe segment interfaces, and pipe sealing performance. However, due to the difficulty of long-term measurement of relative displacement at pipe segment interfaces and water seepage areas within pipes, the impact of relative displacement at pipe segment interfaces on stress transfer in pipe segments is relatively poorly studied, and the influencing mechanism is unclear. Furthermore, during pipe jacking construction, it is difficult to detect cracks, leaks, and water seepage within pipe segments in advance. Preemptive monitoring of crack development within pipes, timely detection, and appropriate treatment measures can reduce the impact on the subsequent use and maintenance of underground pipelines. Technical Solutions

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a device and method for detecting relative displacement and water leakage of a jacking pipe interface, so as to solve the above-mentioned problems.

[0005] The objective of the present invention is achieved through the following technical solutions: a device for detecting relative displacement and water leakage of a jacking pipe interface, comprising a mobile measuring vehicle, a steering wheel being provided below the mobile measuring vehicle, a binocular image processing module, a data processing module and a PID control module being provided on the mobile measuring vehicle, a telescopic rod being provided at the front end thereof, a plurality of laser rangefinders being provided in a ring shape at the end of the telescopic rod, a slide being provided on the mobile measuring vehicle, two cameras being provided on the slide, a main unit, a mobile motor and a lithium battery pack being provided inside the mobile measuring vehicle, and an odometer being provided on the mobile measuring vehicle.

[0006] The mobile measuring vehicle is provided with a warning light.

[0007] The mobile measuring vehicle is provided with a heat dissipation plate.

[0008] The mobile measuring vehicle is provided with a main lighting lamp.

[0009] The mobile measuring vehicle is provided with an emergency button.

[0010] The mobile measuring vehicle is provided with a cooling fan.

[0011] An operation display screen is provided in the mobile measurement vehicle.

[0012] A detection method based on a device for detecting relative displacement and water leakage of a jacking pipe interface comprises the following steps:

[0013] S1. Use a checkerboard to calibrate the binocular image processing module;

[0014] S2. Place the mobile measuring vehicle in the jacking pipe, using the center point of the telescopic rod as the origin of the spatial coordinate system. The right side of the origin is the positive direction of the x-axis, the front is the positive direction of the y-axis, and the top is the positive direction of the z-axis. Based on the design drawings of the mobile measuring vehicle, the relative positions of the multiple laser rangefinders and the center point of the telescopic rod, as well as the relative positions of the center point of the telescopic rod and the left and right cameras, can be obtained. By calculating the relative distances between the laser rangefinders, left and right cameras, and the center point, their initial positions in space can be obtained.

[0015] S3. Calculate the difference between the left and right distances and set it as Deviation 1, which is used to control the left and right rotation of the mobile measuring vehicle so that the mobile measuring vehicle travels along the central axis of the top pipe. Calculate the average value of the total upper, lower, left and right distances and set it as Deviation 2, which is used to control the braking and movement of the mobile measuring vehicle so that the camera can shoot stably.

[0016] S4. Substitute the two deviations into the output control signal of the PID control module respectively. Output control signal = P × deviation + I × integral (deviation) + D × differential (deviation), where P is the proportional coefficient, I is the integral parameter, integral (deviation) is the cumulative sum of the deviations, used to eliminate the steady-state error of the system, D is the differential parameter, and differential (deviation) is the rate of change of the deviation, used to suppress the oscillation of the system. The P, I, and D parameters are all obtained through empirical adjustment or automatic parameter adjustment;

[0017] S5. The control signal obtained from the left and right side distance difference is mapped to an appropriate steering angle range, and the mapped steering angle is then applied to the vehicle's steering system; wherein, when the left and right side distance difference is negative, it indicates that the vehicle is closer to the left side, and the measuring vehicle turns right at this time; when the difference is positive, the measuring vehicle turns left; the deviation 2 measured by the position of the laser rangefinder inside the pipe joint interface is greater than the deviation 2 outside the interface, at this time the measuring vehicle stops moving forward, and the telescopic rod is controlled to extend and retract forward and backward so that the laser rangefinder is at the front and rear sections of the pipe joint interface. At this time, the difference between the average value of the total upper, lower, left and right side distances measured by the laser rangefinder and the average value outside the interface is within a certain threshold, and the measuring vehicle continues to move after the left and right cameras complete shooting; at the same time, in order to shoot the inner surface image of the pipe joint, the measuring vehicle stops and shoots every two meters it moves forward based on the distance information collected by the odometer;

[0018] S6. Output the intrinsic parameter matrices K (internal parameters include focal lengths fx and fy, in pixels) of the two cameras, and their relative R and T, where R represents the rotation matrix and T represents the translation matrix. The overall conversion relationship is as follows:

[0019] Where (X w Y w Z w 1) T Indicates the world coordinates of the object, f is the focal length of the camera, and They are the rotation matrix and translation matrix of the camera coordinate system relative to the world coordinate system, dx and dy indicate how many length units a pixel occupies in the x-direction and y-direction respectively, u0 and v0 indicate the horizontal and vertical pixel differences between the center pixel coordinate of the image and the pixel coordinate of the image origin, γ is the distortion factor, which is generally 0, and Zc is the Z-axis coordinate of the camera. The products of the last two matrices in the formula represent the transformation from the world coordinate system to the camera coordinate system, and the products of the last three matrices represent the transformation from the camera coordinate system to the image physical coordinate system. The image is distorted in the process and is corrected by the focal length diagonal matrix and the distortion coefficient. The products of the four matrices represent the transformation from the image physical coordinate system to the image pixel coordinate system.

[0020] S7. Calculate the implicit correction factors without any physical meaning: λ1, λ2, λ3, λ4. The specific steps are as follows:

[0021] Let (u', v') be the ideal coordinates of the image pixel, and (u, v) be the measured coordinates of the image pixel with radial distortion and tangential distortion. The relationship between the two is as follows:

[0022] Where k u 、k vare the horizontal pixel unit and vertical pixel unit of the image respectively; k1, k2 are radial distortion coefficients; p1, p2 are tangential distortion coefficients;

[0023] Substitute the ideal coordinates (u', v') and measured coordinates (u, v) of the calibration data points into the above formula and use the least squares method to obtain k1, k2, p1, and p2;

[0024] If a total of N calibration points are extracted during camera calibration, the measured values ​​of the image coordinates of the calibration points are (u i , v i )(1≤i≤N), the ideal value is (u′ i , v′ i )(1≤i≤N), given 3 vectors U i 、V i , T is

[0025] T=[U1 V1...U i V i ...U N V N ] T

[0026] r i is the arithmetic square root of the sum of the squares of horizontal and vertical pixels, and the vector p consisting of four implicit correction factors is [λ1 λ2 λ3 λ4] T , let vector e=[A1-u′1 B1-v′1...A N -u′ N B N -v′ N ] T

[0027] Among them, A i =u i +u i (k1r 2 i +k2r 4 i )+2p1u i v i +p2(r 2 i +2u 2 i )

[0028] B i =v i +v i (k1r 2 i +k2r 4 i )+2p2ui v i +p1(r 2 i +2u 2 i )

[0029] According to the formula e=Tp, use the least square method to find the vector p=(T T T) -1 T T e, and the vector p is obtained, which means we can get the four implicit correction factors λ1, λ2, λ3, and λ4. Substituting these parameters into the following explicit-implicit distortion correction model can complete the stereo correction:

[0030] S8. Calibrate and rotate the left and right cameras to the ideal position. The corresponding image coordinates before and after rotation have the following relationship: m n =ARA -1 m0, where m0 refers to the image coordinate before rotation; m n The image coordinates after rotation; the internal parameters of the A camera; the position change matrix of the R camera before and after rotation;

[0031] Let R0 be the rotation matrix of the camera at the real position, R n is the rotation matrix of the camera at the ideal position, then R0, R, R n The following relationship is satisfied between the three: R = R -1 o R n ; R0 can be obtained through camera calibration; R n Decomposed into 3 vectors, namely: R n =[r1 T r2 T r3 T ] T , r1, r2, r3 represent the x, y, z axes in the world coordinate system respectively, c1 is the optical center of the left camera, b is the baseline, r1, r2, r3 are calculated by the following steps:

[0032] (1) In the ideal position, the new axis must be parallel to the baseline b.

[0033] (2) In the ideal position, the new y axis is perpendicular to the new x axis and perpendicular to the plane formed by the new axis and the original z axis, so r2 = k∧r1, k is the unit vector in the direction of the original z axis, and "∧" represents the vector product;

[0034] (3) In the ideal position, the new z-axis is perpendicular to the plane formed by the new x-axis and the new y-axis, so r3 = r1∧r2;

[0035] Get Rn Then, according to R=R o -1 R n Find R, according to mn ＝ ARA -1 m0 completes the polar line calibration;

[0036] S9. Perform edge detection on the corrected image to obtain a relatively consistent edge detection result. Overlap the left and right image contours and perform a correlation operation on the overlapping portion of the two images according to the following formula: When the two contours overlap to the greatest extent, the correlation will reach a maximum, and the movement distance at this time is the average parallax of the left and right images;

[0037] Among them, A represents the left window; B represents the right window; represents the average value of A; represents the average value of B;

[0038] Obtaining three-dimensional depth information of the image;

[0039] S10. Calculate the coordinate mean of all three-dimensional point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the x-axis mean is: Where n is the total number of point clouds; secondly, calculate the standard deviation of all 3D point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the standard deviation on the x-axis is: Then calculate the standard deviation of each point cloud, The calculation formulas for the y and z coordinate points are similar. Finally, a suitable threshold u is selected to determine the outlier. If the standard deviation of the point on each coordinate axis is greater than the threshold u, it is considered an outlier.

[0040] S11. Each time, three non-collinear matching points are selected as a set of data to establish a spatial coordinate system. The calibration points corresponding to the three matching points are used as another set of data to establish a spatial coordinate system. The coordinate transformation matrix H is calculated based on the relative position relationship between the two coordinate systems. All point clouds are corrected using the transformation matrix. The transformation matrix H is specifically:

[0041] Among them, α, β, and γ represent the rotation angles of one spatial coordinate system relative to another spatial coordinate system along the x, y, and z axes, respectively, and t x , t y , t z Represents the translation along the x, y, and z axes respectively;

[0042] S12, filtering out the three-dimensional point cloud located at the pipe joint interface position according to the y-axis coordinate of the calibration point;

[0043] S13, separating the color image into three channels: red, blue, and green; then gray-scaling the image using the formula: grayscale value = 0.299 × red + 0.587 × green + 0.114 × blue; using the Sobel operator to detect the gradient change of the color level in the image to determine the edge of the target area; by setting a color level threshold, marking pixels with a color level change greater than the threshold as the seepage area, and pixels with a color level change less than the threshold as the background of the inner surface of the pipe segment, and visually displaying the marked area on the original color image;

[0044] S14. Output target information: Extract the three-dimensional point cloud of the pipe joint interface location: Filter the three-dimensional point cloud located at the pipe joint interface location based on the y-axis coordinate of the calibration point; Use the odometer to locate the darker area caused by water seepage;

[0045] S15. Repeat S3-S14 until the pipe segment excavation surface is reached.

[0046] In step S4, the calculated control signal is subjected to rate control to ensure the stability and smoothness of the control of the measuring vehicle; finally, the limited control signal is used in the vehicle's execution system.

[0047] In step S9, for matching objects with rich textures, a small window matching is adopted; for matching objects with single textures, a large window matching is adopted;

[0048] When matching an object with a single texture, the left and right images are first sampled and transformed with different resolutions to obtain a multi-resolution pyramid-shaped image series. The original image has the highest resolution and is located at the bottom of the pyramid. The resolution of each layer decreases in sequence. Matching starts from the image with the lowest resolution to the image with the highest resolution, that is, from the highest layer to the lowest layer.

[0049] The left image window is searched on the same epipolar line of the right image to obtain a series of points to be matched, which is called forward search. A window of the same size centered on the point to be matched is searched on the same epipolar line of the left image, which is called reverse search. When the matching point obtained by the reverse search and the forward search is consistent, it is determined to be the point to be matched; if not, there is no matching point for that point. After the matching is completed, the global relaxation method is used for optimization. The specific steps are as follows:

[0050] (1) Arrange the points to be matched into a three-dimensional array format. At point c, point c and its surrounding adjacent points form a window. The value of the neighborhood remains unchanged. Change the different points to be matched at point c and calculate the variance of the window. The point with the smallest variance is the appropriate point.

[0051] (2) Calculate point by point and perform iterations. The number of iterations can be selected, usually 5 to 10 times.

[0052] The beneficial effects of the present invention are:

[0053] (1) The present invention uses tools such as PID controller, laser rangefinder, binocular imaging, etc. to realize the automatic measurement of the relative displacement at the interface of the top pipe section and the water seepage area inside the pipe.

[0054] (2) The remote control movement of the car is achieved by setting up a power motor and an integrated sensor. The odometer is located inside the auxiliary wheel at the rear of the car to record the real-time moving position of the vehicle and assign it to the captured image to locate the location of the water leakage.

[0055] (3) By locating the three-dimensional coordinates of the three-dimensional point cloud and the position of the seepage area, a three-dimensional model of the entire jacking pipe can be obtained, and the direction of the pipe section in the soil layer and the deformation of the pipe section can be intuitively seen. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] FIG1 is a schematic diagram of the external structure of the present invention;

[0057] FIG2 is a schematic diagram of the internal structure of the present invention;

[0058] FIG3 is a schematic diagram of the internal system of the present invention;

[0059] FIG4 is a flow chart of the detection method of the present invention;

[0060] In the figure, 1- warning light, 2- binocular image processing module, 3- camera, 4- data processing module, 5- main lighting, 6- operation display, 7- front lighting, 8- steering wheel, 9- telescopic rod, 10- heat sink, 11- laser rangefinder, 12- PID control module, 13- power motor, 14- host, 15- lithium battery pack, 16- emergency button, 17- odometer, 18- slide, 19- cooling fan, 20- mobile measuring vehicle, DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0062] It is to be noted that the directions of "left", "right", "up", "down", "front", "back", "inside" and "outside" in the following schemes are all relative directions and are not listed here one by one.

[0063] A device for detecting relative displacement and water leakage of a jacking pipe interface, referring to Figures 1 and 2, includes a mobile measuring vehicle 20. A chute 18 is provided on the mobile measuring vehicle 20. Two cameras 3 are provided on the chute 18. A main lighting lamp 5 is provided between the two cameras 3. A heat sink 10, a front lighting lamp 7 and a telescopic rod 9 are provided at the front end of the mobile measuring vehicle 20. Four laser rangefinders 11 are provided on the telescopic rod 9. The four laser rangefinders are distributed on the top, bottom, left and right. Two transformation wheels 8 are provided on the lower side of the mobile measuring vehicle 20. An emergency button 16, an odometer 17, a warning light 1, a binocular imaging processing module 2 and a data processing module 4 are also provided on the upper side of the mobile measuring vehicle 20.

[0064] The mobile measuring vehicle 20 is provided with a PID control module 12 , a cooling fan 19 , a host 14 , a lithium battery pack 15 , a power motor 13 and a storage module.

[0065] Referring to Figure 3, when the camera 3 needs to take a picture, the telescopic rod 9 is extended to ensure that there is no obstruction in front of the camera 3 when shooting. The laser rangefinder 11 is used to obtain the distance between the front center of the mobile measuring vehicle 20 and the inner surface of the pipe segment, and the three-dimensional coordinates of the calibration point can be obtained through the odometer and the initial position information. The PID control module 12 controls the movement of the mobile measuring vehicle 20 according to the distance information obtained by the laser rangefinder 11, providing stable shooting conditions for the camera 3.

[0066] The camera 3 can achieve 360° rotation through the rotating platform, and the left and right cameras 3 can adjust their relative positions by sliding left and right on the slide 18; the initial position and the optimal shooting angle are obtained through initial adjustment. After entering the hole, the position of the camera 3 is not adjusted unless there are special circumstances; the main lighting 5 and the front lighting 7 use DC stabilized LED light sources. The main lighting 5 and the front lighting 7 are composed of a ring-shaped light-emitting unit composed of 64 LEDs to ensure that light is concentrated to achieve the required brightness. The color temperature of 5000K ensures accurate color reproduction, the maximum illumination is 4200lx, and the average illumination is 3400lx; the storage hard disk is used to store images taken by the camera 3 and three-dimensional point cloud data; the binocular image processing module 2 generates three-dimensional point cloud data of the inner surface of the top pipe according to the stereo matching algorithm.

[0067] The 3D point cloud data is corrected to extract feature points corresponding to calibration points from the 3D point cloud. The calibration points are the coordinates of the laser points projected by the laser rangefinder 11 at the pipe joint interface. The corresponding relationship between the two coordinate systems is found, and the data processing module 4 corrects the 3D point cloud using a coordinate transformation matrix. The black and white scale image is used to identify water seepage areas within the pipe and locate the water seepage areas using the odometer 17. The water seepage areas are located by color image separation and setting the color scale threshold.

[0068] The target information output module includes the output of the seepage area position and the relative position output of the front and rear interfaces of the pipe joint. It is used to output the data after target position correction and the darker areas in the black and white scale image caused by seepage.

[0069] Referring to FIG4 , a detection method based on a device for detecting relative displacement and water leakage of a jacking pipe interface is characterized by comprising the following steps:

[0070] S1, use the checkerboard to calibrate the binocular image processing module 2;

[0071] S2. Place the mobile measuring vehicle 20 into the jacking pipe, using the center point of the telescopic rod 9 as the origin of the spatial coordinate system. The right side of the origin is the positive x-axis direction, the front is the positive y-axis direction, and the top is the positive z-axis direction. Based on the design drawings of the mobile measuring vehicle 20, the relative positions of the four laser rangefinders 11 and the center point of the telescopic rod 9, as well as the relative positions of the center point of the telescopic rod 9 and the left and right cameras 3, can be determined. By calculating the relative distances between the laser rangefinders 11 and the left and right cameras 3 and the center points, their initial positions in space can be determined.

[0072] S3. Calculate the difference between the left and right distances and set it as Deviation 1, which is used to control the left and right rotation of the mobile measuring vehicle 20 so that the mobile measuring vehicle 20 travels along the central axis of the top pipe; calculate the average value of the total upper, lower, left and right distances and set it as Deviation 2, which is used to control the braking and movement of the mobile measuring vehicle 20 so that the camera 3 can shoot stably;

[0073] S4. Substitute the two deviations into the output control signal of the PID control module 12 respectively, and the output control signal = P×deviation + I×integral (deviation) + D×differential (deviation), where P is the proportional coefficient, I is the integral parameter, the integral (deviation) is the cumulative sum of the deviations, which is used to eliminate the steady-state error of the system, D is the differential parameter, and the differential (deviation) is the rate of change of the deviation, which is used to suppress the oscillation of the system, where the P, I, and D parameters are all obtained through empirical adjustment or automatic parameter adjustment; at the same time, in order to ensure the stability and smoothness of the control of the mobile measuring vehicle 20, a differential operation is used to limit the rate of change of the control signal. The specific process includes: differentiating the current control signal from the previous control signal to calculate the change amount of the control signal; comparing the obtained change amount with the set maximum slope limit; if the change amount exceeds the maximum slope limit, it is limited to the maximum slope limit range by truncation or scaling;

[0074] S5. Map the control signal obtained from the left and right side distance difference to an appropriate steering angle range, and then apply the mapped steering angle to the vehicle's steering system; wherein, when the left and right side distance difference is negative, it means that the vehicle is closer to the left side, and the mobile measuring vehicle 20 turns right at this time; when the difference is positive, the mobile measuring vehicle 20 turns left; the deviation 2 measured by the position of the laser rangefinder 11 inside the pipe joint interface is greater than the deviation 2 outside the interface. At this time, the mobile measuring vehicle 20 stops moving forward, and controls the telescopic rod 9 to extend and retract forward and backward so that the laser rangefinder 11 is at the front and rear sections of the pipe joint interface. At this time, the difference between the average value of the total upper and lower left and right side distances measured by the laser rangefinder 11 and the average value outside the interface is within a certain threshold. When the left and right cameras 3 complete shooting, the measuring vehicle continues to move; at the same time, in order to shoot the inner surface image of the pipe joint, the mobile measuring vehicle 20 stops and shoots every two meters it moves forward based on the distance information collected by the odometer 17;

[0075] S6. Output the intrinsic parameter matrices K (internal parameters include focal lengths fx and fy, in pixels) of the two cameras, and their relative R and T, where R represents the rotation matrix and T represents the translation matrix. The overall conversion relationship is as follows:

[0076] Where (X w Y w Z w 1) T Indicates the world coordinates of the object, f is the focal length of the camera, and They are the rotation matrix and translation matrix of the camera coordinate system relative to the world coordinate system, dx and dy indicate how many length units a pixel occupies in the x-direction and y-direction respectively, u0 and v0 indicate the horizontal and vertical pixel differences between the center pixel coordinate of the image and the pixel coordinate of the image origin, γ is the distortion factor, which is generally 0, and Zc is the Z-axis coordinate of the camera. The products of the last two matrices in the formula represent the transformation from the world coordinate system to the camera coordinate system, and the products of the last three matrices represent the transformation from the camera coordinate system to the image physical coordinate system. The image is distorted in the process and is corrected by the focal length diagonal matrix and the distortion coefficient. The products of the four matrices represent the transformation from the image physical coordinate system to the image pixel coordinate system.

[0077] S7. After obtaining the pixel coordinates of the image with radial and tangential distortion through calibration technology, the image distortion can be corrected through explicit-implicit correction. Calculate the implicit correction factors without any physical meaning: λ1, λ2, λ3, λ4. The specific steps are as follows:

[0078] Let (u', v') be the ideal coordinates of the image pixel, and (u, v) be the measured coordinates of the image pixel with radial distortion and tangential distortion. The relationship between the two is as follows:

[0079] Where k u 、k v are the horizontal pixel unit and vertical pixel unit of the image respectively; k1, k2 are radial distortion coefficients; p1, p2 are tangential distortion coefficients;

[0080] Substitute the ideal coordinates (u', v') and measured coordinates (u, v) of the calibration data points into the above formula and use the least squares method to obtain k1, k2, p1, and p2;

[0081] If a total of N calibration points are extracted during camera calibration, the measured values ​​of the image coordinates of the calibration points are (u i , v i )(1≤i≤N), the ideal value is (u′ i , v′ i )(1≤i≤N), given 3 vectors U i 、V i , T is

[0082] T=[U1 V1...U i V i ...U N V N ] T

[0083] r i is the arithmetic square root of the sum of the squares of horizontal and vertical pixels, and the vector p consisting of four implicit correction factors is [λ1 λ2 λ3 λ4] T , let vector e=[A1-u′1 B1-v′1...A N -u′ N B N -v′ N ] T Among them, A i =u i +u i (k1r 2 i +k 2 r 4 i )+2p1u i v i +p2(r 2 i +2u 2 i )

[0084] B i =v i +vi ( k1r 2 i +k2r 4i )+2p2u i v i +p1(r 2 i +2u 2 i )

[0085] According to the formula e=Tp, use the least square method to find the vector p=(T T T) -1 T T e, and the vector p is obtained, which means we can get the four implicit correction factors λ1, λ2, λ3, and λ4. Substituting these parameters into the following explicit-implicit distortion correction model can complete the stereo correction:

[0086] S8. Stereo epipolar pair calibration is to calibrate and rotate the left and right cameras 3 to the ideal position, so that their epipolar lines can meet the epipolar line constraint. The rotation of a camera 3 around the optical center will inevitably lead to the rotation of the image space. Therefore, the corresponding image coordinates before and after the rotation have the following relationship: m n =ARA -1 m0, where m0 refers to the image coordinate before rotation; m n The image coordinates after rotation; the internal parameters of the A camera; the position change matrix of the R camera before and after rotation;

[0087] Let R0 be the rotation matrix of the camera at the real position, R n is the rotation matrix of the camera at the ideal position, then R0, R, R n The following relationship is satisfied between the three: R = R -1 o R n ; R0 can be obtained through camera calibration; R n Decomposed into 3 vectors, namely: R n =[r1 T r2 T r3 T ] T , r1, r2, r3 represent the x, y, z axes in the world coordinate system respectively, c1 is the optical center of the left camera, b is the baseline, r1, r2, r3 are calculated by the following steps:

[0088] (1) In the ideal position, the new axis must be parallel to the baseline b.

[0089] (2) In the ideal position, the new y axis is perpendicular to the new x axis and perpendicular to the plane formed by the new axis and the original z axis, so r2 = k∧r1, k is the unit vector in the direction of the original z axis, and "∧" represents the vector product;

[0090] (3) In the ideal position, the new z-axis is perpendicular to the plane formed by the new x-axis and the new y-axis, so r3 = r1∧r2;

[0091] Get R n Then, according to R=R o -1 R n Find R, according to m n =ARA -1 m0 completes the polar line calibration;

[0092] S9. Estimating the average disparity and search range: Perform edge detection on the corrected image to obtain a relatively consistent edge detection result. Overlap the left and right image contours, and perform a correlation operation on the overlapping portion of the two images according to the following formula: When the two contours overlap to the greatest extent, the correlation will reach a maximum. The movement distance at this time is the average disparity of the left and right images;

[0093] Among them, A represents the left window; B represents the right window; represents the average value of A; represents the average value of B;

[0094] For matching objects with rich textures, a smaller window such as 7×7 pixels or 9×9 pixels can contain sufficient matching information; for matching objects with a single texture, a larger window such as 15×15 pixels or 29×29 pixels can contain sufficient matching information. Therefore, the window size should be selected according to the actual situation on site.

[0095] Single textures can easily lead to mismatches. A coarse-to-fine matching strategy can address this issue to a certain extent. First, the left and right images are subsampled to different resolutions to produce a multi-resolution pyramid-like image series. The original image has the highest resolution and is located at the bottom of the pyramid. The resolution decreases with each layer upwards. Matching begins with the lowest resolution image and ends with the highest resolution image, that is, from the highest layer to the lowest.

[0096] The left image window searches on the same epipolar line of the right image to obtain a series of points to be matched, which is called "forward search." Using a window of the same size centered on the point to be matched, the left image epipolar line is searched, which is called "reverse search." If the matching points obtained by the reverse search and the forward search are consistent, they are officially determined to be the points to be matched. If they are inconsistent, the point has no matching point. After the matching is completed, each point in the left image has one or more points to be matched in the right image. At this time, the global relaxation method is used for optimization. The specific steps are as follows:

[0097] (1) Arrange the points to be matched into a three-dimensional array format. Assume that at point c, point c and its surrounding adjacent points form a window, and the value of the neighborhood remains unchanged. Change the different points to be matched at point c, calculate the variance of the window, and the point with the smallest variance is the appropriate point.

[0098] (2) Calculate point by point and perform iterations. The number of iterations can be selected, usually 5 to 10 times.

[0099] According to the stereo matching method, the three-dimensional depth information of the image can be obtained.

[0100] S10 is used to extract feature points corresponding to calibration points from the three-dimensional point cloud, where the calibration points are the coordinates of the laser points projected by the laser rangefinder 11 at the pipe joint interface; perform feature matching on the coordinates of the three-dimensional calibration points collected by the laser rangefinder 11 and the image feature points collected by the camera 3 to find the corresponding relationship between the coordinate systems of the two; and perform deviation correction on the three-dimensional point cloud using a coordinate transformation matrix;

[0101] Among them, the feature points corresponding to each calibration point are extracted from the three-dimensional point cloud generated by the image. The specific process includes:

[0102] First, calculate the coordinate mean of all 3D point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the x-axis mean is: Where n is the total number of point clouds; secondly, calculate the standard deviation of all 3D point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the standard deviation on the x-axis is: Then calculate the standard deviation of each point cloud, The calculation formulas for the y and z coordinate points are similar. Finally, a suitable threshold u is selected to determine the outlier. If the standard deviation of the point on each coordinate axis is greater than the threshold u, it is considered an outlier.

[0103] S11. Each time, three non-collinear matching points are selected as a set of data to establish a spatial coordinate system. The calibration points corresponding to the three matching points are used as another set of data to establish a spatial coordinate system. The coordinate transformation matrix H is calculated based on the relative position relationship between the two coordinate systems. All point clouds are corrected using the transformation matrix. The transformation matrix H is specifically:

[0104] Among them, α, β, and γ represent the rotation angles of one spatial coordinate system relative to another spatial coordinate system along the x, y, and z axes, respectively. x , t y , t z Represents the translation along the x, y, and z axes respectively;

[0105] S12, filtering out the three-dimensional point cloud located at the pipe joint interface position according to the y-axis coordinate of the calibration point;

[0106] S13, black and white gradation diagram, is used to identify the water seepage area in the pipe joint due to water seepage, which becomes darker in color. The water seepage area in the pipe is identified by the color change, and the location of the water seepage area is located by the odometer.

[0107] First, the color image is separated into three channels: red, blue, and green. The image is then grayscaled using the formula: grayscale value = 0.299 × red + 0.587 × green + 0.114 × blue. The Sobel operator is used to detect the gradient change of color levels in the image to determine the edge of the target area. By setting a color level threshold, pixels with a color level change greater than the threshold are marked as water seepage areas, while pixels with a color level change less than the threshold are set as the background of the inner surface of the pipe segment. The marked areas are then visualized on the original color image.

[0108] S14. Output target information: Extract the three-dimensional point cloud of the pipe joint interface position: According to the y-axis coordinate of the calibration point, filter out the three-dimensional point cloud located at the pipe joint interface position; locate the area with darker color caused by water seepage through the odometer; the target information output module includes the output of the position of the water seepage area on the inner surface of the pipe joint and the relative position output of the front and rear interfaces of the pipe joint, which is used to output the data after the target position correction, as well as the area with darker color caused by water seepage in the black and white scale image.

[0109] S15. Repeat S3-S14 until the pipe segment excavation surface is reached.

[0110] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein by utilizing the above description or the techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A device for detecting relative displacement and water leakage of a jacking pipe interface, characterized in that: It includes a mobile measuring vehicle, a steering wheel is provided at the bottom of the mobile measuring vehicle, a binocular image processing module, a data processing module and a PID control module are provided on the mobile measuring vehicle, a telescopic rod is provided at the front end of the mobile measuring vehicle, a plurality of laser rangefinders are provided in a ring at the end of the telescopic rod, a slide is provided on the mobile measuring vehicle, two cameras are provided on the slide, a host, a mobile motor and a lithium battery pack are provided inside the mobile measuring vehicle, and an odometer is provided on the mobile measuring vehicle.

2. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: The mobile measuring vehicle is provided with a warning light.

3. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: The mobile measuring vehicle is provided with a heat dissipation plate.

4. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: The mobile measuring vehicle is provided with a main lighting lamp.

5. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: The mobile measuring vehicle is provided with an emergency button.

6. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: The mobile measuring vehicle is provided with a cooling fan.

7. The device for detecting relative displacement and water leakage of a jacking pipe joint according to claim 1, characterized in that: An operation display screen is provided in the mobile measurement vehicle.

8. A method for detecting relative displacement and water leakage of a jacking pipe interface according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Use a checkerboard to calibrate the binocular image processing module; S2. Place the mobile measuring vehicle in the jacking pipe, using the center point of the telescopic rod as the origin of the spatial coordinate system. The right side of the origin is the positive direction of the x-axis, the front is the positive direction of the y-axis, and the top is the positive direction of the z-axis. Based on the design drawings of the mobile measuring vehicle, the relative positions of the multiple laser rangefinders and the center point of the telescopic rod, as well as the relative positions of the center point of the telescopic rod and the left and right cameras, can be obtained. By calculating the relative distances between the laser rangefinders, left and right cameras, and the center point, their initial positions in space can be obtained. S3, by calculating the difference between the left and right side distances, setting it as deviation 1, which is used to control the left and right rotation of the mobile measuring vehicle so that the mobile measuring vehicle travels along the central axis inside the jacking pipe; By calculating the average value of the total distance between the upper, lower, left and right sides, let it be deviation 2, which is used to control the braking and movement of the mobile measurement vehicle to ensure stable shooting of the camera; S4. Substitute the two deviations into the output control signal of the PID control module respectively. Output control signal = P × deviation + I × integral (deviation) + D × differential (deviation), where P is the proportional coefficient, I is the integral parameter, integral (deviation) is the cumulative sum of the deviations, used to eliminate the steady-state error of the system, D is the differential parameter, and differential (deviation) is the rate of change of the deviation, used to suppress the oscillation of the system. The P, I, and D parameters are all obtained through empirical adjustment or automatic parameter adjustment; S5. The control signal obtained from the left and right side distance difference is mapped to an appropriate steering angle range, and the mapped steering angle is then applied to the vehicle's steering system; wherein, when the left and right side distance difference is negative, it indicates that the vehicle is closer to the left side, and the measuring vehicle turns right at this time; when the difference is positive, the measuring vehicle turns left; the deviation 2 measured by the position of the laser rangefinder inside the pipe joint interface is greater than the deviation 2 outside the interface, at this time the measuring vehicle stops moving forward, and the telescopic rod is controlled to extend and retract forward and backward so that the laser rangefinder is at the front and rear sections of the pipe joint interface. At this time, the difference between the average value of the total upper, lower, left and right side distances measured by the laser rangefinder and the average value outside the interface is within a certain threshold, and the measuring vehicle continues to move after the left and right cameras complete shooting; at the same time, in order to shoot the inner surface image of the pipe joint, the measuring vehicle stops and shoots every two meters it moves forward based on the distance information collected by the odometer; S6. Output the intrinsic parameter matrices K (internal parameters include focal lengths fx and fy, in pixels) of the two cameras, and their relative R and T, where R represents the rotation matrix and T represents the translation matrix. The overall conversion relationship is as follows: Where (X w Y w Z w 1) T Indicates the world coordinates of the object, f is the focal length of the camera, and They are the rotation matrix and translation matrix of the camera coordinate system relative to the world coordinate system, dx and dy indicate how many length units a pixel occupies in the x-direction and y-direction respectively, u0 and v0 indicate the horizontal and vertical pixel differences between the center pixel coordinate of the image and the pixel coordinate of the image origin, γ is the distortion factor, which is generally 0, and Zc is the Z-axis coordinate of the camera. The products of the last two matrices in the formula represent the transformation from the world coordinate system to the camera coordinate system, and the products of the last three matrices represent the transformation from the camera coordinate system to the image physical coordinate system. The image is distorted in the process and is corrected by the focal length diagonal matrix and the distortion coefficient. The products of the four matrices represent the transformation from the image physical coordinate system to the image pixel coordinate system. S7. Calculate the implicit correction factors without any physical meaning: λ1, λ2, λ3, λ4. The specific steps are as follows: Let (u', v') be the ideal coordinates of the image pixel, and (u, v) be the measured coordinates of the image pixel with radial distortion and tangential distortion. The relationship between the two is as follows: Where k u 、k v are the horizontal pixel unit and vertical pixel unit of the image respectively; k1, k2 are radial distortion coefficients; p1, p2 are tangential distortion coefficients; Substitute the ideal coordinates (u', v') and measured coordinates (u, v) of the calibration data points into the above formula and use the least squares method to obtain k1, k2, p1, and p2; If a total of N calibration points are extracted during camera calibration, the measured values of the image coordinates of the calibration points are (u i , v i )(1≤i≤N), the ideal value is (u′ i , v′ i )(1≤i≤N), given 3 vectors U i 、V i , T is T=[U1 V1 ... U i V i ... U N V N ] T r i is the arithmetic square root of the sum of the squares of horizontal and vertical pixels, and the vector p consisting of four implicit correction factors is [λ1 λ2 λ3 λ4] T , let vector e=[A1-u′1 B1-v′1 ... A N -u′ N B N -v′ N ] T Among them, A i =u i +u i (k1r 2 i +k2r 4 i )+2p1u i v i +p2(r 2 i +2u 2 i )B i =v i +v i (k1r 2 i +k2r 4 i )+2p2u i v i +p1(r 2 i +2u 2 i ) According to the formula e=Tp, use the least square method to find the vector p=(T T T) -1 T T e, and the vector p is obtained, which means we can get the four implicit correction factors λ1, λ2, λ3, and λ4. Substituting these parameters into the following explicit-implicit distortion correction model can complete the stereo correction: S8. Calibrate and rotate the left and right cameras to the ideal position. The corresponding image coordinates before and after rotation have the following relationship: m n =ARA -1 m0, where m0 refers to the image coordinate before rotation; m n The image coordinates after rotation; the internal parameters of the A camera; the position change matrix of the R camera before and after rotation; Let R0 be the rotation matrix of the camera at the real position, R n is the rotation matrix of the camera at the ideal position, then R0, R, R n The following relationship is satisfied between the three: R = R -1 o R n ; R0 can be obtained through camera calibration; R n Decomposed into 3 vectors, namely: R n =[r1 T r2 T r3 T ] T , r1, r2, r3 represent the x, y, z axes in the world coordinate system respectively, c1 is the optical center of the left camera, b is the baseline, r1, r2, r3 are calculated by the following steps: (1) In the ideal position, the new axis must be parallel to the baseline b, which can be taken (2) In the ideal position, the new y axis is perpendicular to the new x axis and perpendicular to the plane formed by the new axis and the original z axis, so r2 = k∧r1, k is the unit vector in the direction of the original z axis, and "∧" represents the vector product; (3) In the ideal position, the new z-axis is perpendicular to the plane formed by the new x-axis and the new y-axis, so r3 = r1∧r2; Get R n Then, according to R=R o -1 R n Find R, according to m n =ARA -1 m0 completes the polar line calibration; S9. Perform edge detection on the corrected image to obtain a relatively consistent edge detection result. Overlap the left and right image contours and perform a correlation operation on the overlapping portion of the two images according to the following formula: When the two contours overlap to the greatest extent, the correlation will reach a maximum, and the movement distance at this time is the average parallax of the left and right images; Among them, A represents the left window; B represents the right window; represents the average value of A; represents the average value of B; Obtaining three-dimensional depth information of the image; S10. Calculate the coordinate mean of all three-dimensional point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the x-axis mean is: Where n is the total number of point clouds; secondly, calculate the standard deviation of all 3D point clouds on the x-axis, y-axis, and z-axis. The formula for calculating the standard deviation on the x-axis is: Then calculate the standard deviation of each point cloud, The calculation formulas for the y and z coordinate points are similar. Finally, a suitable threshold u is selected to determine the outlier. If the standard deviation of the point on each coordinate axis is greater than the threshold u, it is considered an outlier. S11. Each time, three non-collinear matching points are selected as a set of data to establish a spatial coordinate system. The calibration points corresponding to the three matching points are used as another set of data to establish a spatial coordinate system. The coordinate transformation matrix H is calculated based on the relative position relationship between the two coordinate systems. All point clouds are corrected using the transformation matrix. The transformation matrix H is specifically: Among them, α, β, and γ represent the rotation angles of one spatial coordinate system relative to another spatial coordinate system along the x, y, and z axes, respectively. x , t y , t z Represents the translation along the x, y, and z axes respectively; S12, filtering out the three-dimensional point cloud located at the pipe joint interface position according to the y-axis coordinate of the calibration point; S13, separating the color image into three channels: red, blue, and green; then gray-scaling the image using the formula: grayscale value = 0.299 × red + 0.587 × green + 0.114 × blue; using the Sobel operator to detect the gradient change of the color level in the image to determine the edge of the target area; by setting a color level threshold, marking pixels with a color level change greater than the threshold as the seepage area, and pixels with a color level change less than the threshold as the background of the inner surface of the pipe segment, and visually displaying the marked area on the original color image; S14. Output target information: Extract the three-dimensional point cloud of the pipe joint interface location: Filter the three-dimensional point cloud located at the pipe joint interface location based on the y-axis coordinate of the calibration point; Use the odometer to locate the darker area caused by water seepage; S15. Repeat S3-S14 until the pipe segment excavation surface is reached.

9. A detection method according to claim 8, characterized in that: In step S4, the calculated control signal is subjected to rate control to ensure the stability and smoothness of the control of the measuring vehicle; finally, the limited control signal is used in the vehicle's execution system.

10. A detection method according to claim 8, characterized in that: In step S9, for matching objects with rich textures, a small window matching is adopted; for matching objects with single textures, a large window matching is adopted; When matching an object with a single texture, the left and right images are first sampled and transformed with different resolutions to obtain a multi-resolution pyramid-shaped image series. The original image has the highest resolution and is located at the bottom of the pyramid. The resolution of each layer decreases in sequence. Matching starts from the image with the lowest resolution to the image with the highest resolution, that is, from the highest layer to the lowest layer. The left image window is searched on the same epipolar line of the right image to obtain a series of points to be matched, which is called forward search. A window of the same size centered on the point to be matched is searched on the same epipolar line of the left image, which is called reverse search. When the matching point obtained by the reverse search and the forward search is consistent, it is determined to be the point to be matched; if not, there is no matching point for that point. After the matching is completed, the global relaxation method is used for optimization. The specific steps are as follows: (1) Arrange the points to be matched into a three-dimensional array format. At point c, point c and its surrounding adjacent points form a window. The value of the neighborhood remains unchanged. Change the different points to be matched at point c and calculate the variance of the window. The point with the smallest variance is the appropriate point. (2) Calculate point by point and perform iterations. The number of iterations can be selected, usually 5 to 10 times.

Images