Edge contour-based straight pipeline space pose visual tracking device and method

Abstract

The invention belongs to the technical field of pipeline pose measurement, and particularly discloses a straight pipeline spatial pose visual tracking device and method based on an edge contour, and the method comprises the steps: collecting a plurality of plane target images at different angles through a camera, and carrying out the calibration of the camera, and obtaining an internal reference matrix and a distortion coefficient of the camera; collecting an equal-diameter straight pipeline image with a known radius by using a camera, and carrying out edge contour feature extraction to obtain contour feature points; distortion correction is carried out on the feature points; linear fitting is conducted on the corrected distortionless contour feature points to obtain two contour lines of the equal-diameter straight pipeline, and the axis space pose is calculated according to the constraint relation between the two contour lines and the axis of the equal-diameter straight pipeline; the method can obtain the axis pose of the equal-diameter straight pipeline through linear calculation without distortion, has the advantages of being simple in hardware structure, convenient to operate, low in cost and the like, can quickly determine the pose of the equal-diameter straight pipeline, and achieves pose tracking of the equal-diameter straight pipeline.

Description

Technical Field

[0001] The invention relates to the technical field of pipeline posture measurement, and in particular to a device and method for visually tracking the spatial posture of a straight pipeline based on edge contours. Background Art

[0002] As one of the indispensable components in the pipeline system, straight pipes of equal diameter are widely used in the transmission system of liquids or gases in many industrial fields such as petroleum, natural gas, and chemical industry. They are important infrastructure to ensure the continuity of industrial production and the stability of the system. The precise docking of straight pipes of equal diameter during assembly and laying will improve the safety and reliability of the pipeline system, avoid safety accidents such as leakage at the joints, increased corrosion, and pipeline rupture caused by inaccurate pipe docking, and provide a strong guarantee for the safe operation of the pipeline system.

[0003] At present, the assembly and laying of equal-diameter straight pipes can be carried out by using the positioning tooling method or the posture visual tracking method. However, the assembly of equal-diameter straight pipes by the positioning tooling method requires the use of complex tooling equipment such as clamps, and requires manual adjustment and monitoring, which may cause large positioning errors. Posture visual tracking can provide high-precision alignment information to ensure that the pipes are accurately docked during the installation process and reduce errors. It also monitors the alignment of the pipes in real time, corrects deviations in a timely manner, and improves work efficiency.

[0004] The invention patent "A pipeline posture measurement device and method" with application number CN202210182068.0 proposes a method for detecting pipeline posture. This method first rotates the laser circumferentially by an arbitrary angle and then projects a laser point on the inner wall of the pipeline. Then, an ellipse is fitted on the laser point to determine the axis vector of the ellipse. Finally, the coordinate system established at the center of the ellipse is rotated to obtain the pipeline axis coordinate system, thereby completing the measurement of the pipeline posture.

[0005] The invention patent "A method and device for measuring the posture of a cylinder" with application number CN202110217140.4 proposes a method for calculating the posture of a cylinder. This method first obtains the relevant parameters of the cylinder through a line laser displacement sensor arranged around the cylinder, and then uses the cylinder posture measurement model to calculate the center position of the two height measurement sections of the cylinder. The line connecting the center positions is the posture direction of the cylinder, and the ratio of its projection deviation to the vertical movement height of the line laser displacement sensor is the inclination of the cylinder posture, so that the posture of the cylinder can be determined.

[0006] The invention patent "A pipeline centerline identification method, system, computer device and storage medium" with application number CN202410433127.6 proposes a method for determining the centerline of a pipeline. The method first uses laser sensors such as laser radar to directly collect the three-dimensional point cloud data of the pipeline, and then extracts the centerline through the three-dimensional point cloud data to obtain the trend information of the pipeline.

[0007] However, the above three solutions all require the use of multiple sensor devices, which makes the device or system structure more complicated, the posture solution process more cumbersome, and the cost is relatively high; and the speed of determining the posture of straight pipes with equal diameters is also slow. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides a device and method for visually tracking the spatial posture of a straight pipe based on edge contours.

[0009] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0010] The first object of the present invention is to provide a straight pipe spatial posture visual tracking method based on edge contour, comprising the following steps:

[0011] S1. Lay the two-dimensional plane target horizontally on the operating platform, and fix a camera obliquely above the two-dimensional plane target, place the two-dimensional plane target within the camera field of view and adjust the position of the two-dimensional plane target, use the camera to collect multiple two-dimensional plane target images at different positions, and use Zhang's calibration method to calibrate the camera for the multiple two-dimensional plane target images collected by the camera to obtain the camera's intrinsic parameter matrix and distortion coefficient;

[0012] S2. Place a straight pipe with a known radius horizontally within the camera field of view using a bracket, and use the camera to capture an image of the straight pipe with a known radius; extract edge contour features from the image of the straight pipe with a known radius to obtain contour feature points with distortion;

[0013] S3, combining the camera's internal parameter matrix and distortion coefficient to perform distortion correction on the distorted contour feature points to obtain undistorted contour feature points;

[0014] S4, performing straight line fitting on the undistorted contour feature points to obtain two contour lines of straight pipes with equal diameters;

[0015] S5. Combining the two contour lines of the straight pipe with equal diameter, the radius of the straight pipe with equal diameter and the intrinsic parameter matrix of the camera, the spatial posture of the straight pipe with equal diameter is solved according to the perspective projection model of the straight pipe with equal diameter.

[0016] Furthermore, in step S2, the canny edge detection algorithm is used to extract edge contour features from the image of straight pipes with equal diameters.

[0017] Furthermore, in step S3, the calculation formula for distortion correction is as shown in formula (1):

[0018]

[0019] in: and (u,v) represent the pixel coordinates of the contour feature points with and without distortion, respectively; (x,y) represent the normalized image coordinates without distortion; (k1,k2) represent the radial distortion coefficients of the camera; (u0,v0) represent the principal point coordinates in the intrinsic parameter matrix of the camera.

[0020] Furthermore, the step S4 comprises:

[0021] S41, use the RANSAC algorithm to screen the undistorted contour feature points to obtain the feature point set {(u′ g ,v′ g )|g∈1,...,G} and {(u′ m ′,v′ m ′)|m∈1,...,M}, where: (u′ g ,v′ g ) and (u′ m ′,v′ m ′) represent the pixel coordinates of the feature points on the two contours respectively;

[0022] S42. Based on the least squares method, the feature point set is fitted with straight lines to obtain two contour lines l1 and l2 of the straight pipe with equal diameter. The equation of the fitted line is shown in formula (2):

[0023]

[0024] Among them: a′, b′, c′ and a″, b″, c″ are the equation coefficients of the two fitting lines respectively; the solution process of a′, b′, c′ and a″, b″, c″ is as follows:

[0025] The equations formed by the homogeneous coordinates of the feature points on the two contours as coefficient matrices are shown in formula (3):

[0026]

[0027] For equation (3), the eigenvector corresponding to the minimum eigenvalue of the coefficient matrix is ​​the least squares solution of the equation system; or the coefficient matrix is ​​decomposed by SVD, and then the vector corresponding to the minimum singular value in the right singular matrix is ​​taken as the least squares solution;

[0028] Substituting the obtained least square solution into equation (2) we can obtain two contour lines l1 and l2 of straight pipes with equal diameters.

[0029] Furthermore, the step S5 comprises:

[0030] S51. Combining the camera's intrinsic parameter matrix and the two contour lines l1 and l2 of the straight pipe with equal diameter, the back-projection planes corresponding to the two contour lines are:

[0031]

[0032] Where: K represents the camera intrinsic parameter matrix, π1 and π2 represent the back-projection planes of the contour lines corresponding to l1 and l2 respectively;

[0033] S52, according to the obtained back-projection planes π1 and π2, obtain their normal vectors n1 and n2, and calculate the bisector π of the back-projection planes π1 and π2 m The normal vector is:

[0034]

[0035] Where: n m is the normal vector of the bisector of the two back-projection planes, ||n|| represents the modulus of the vector, and the angle and intersection vector of the back-projection planes π1 and π2 are calculated as follows:

[0036]

[0037] Where: α is the angle between the two back-projection planes, v L represents the intersection vector of the back-projection planes π1 and π2;

[0038] S53, according to the obtained intersection vector v L and the plane normal vector n m , calculate the center P of the cross-section circle from the origin O to the support plane passing through the origin C The direction vector is:

[0039]

[0040] in: means OP C The direction vector of the equal-diameter straight pipe is combined with the radius r of the two back-projection planes to calculate the origin O and P C The distance between points is:

[0041]

[0042] in: Indicates point O to P C The distance of the points, according to OP C Direction vector and point O and P C Distance of points Get P C The 3D coordinates of a point:

[0043]

[0044] Since the radius of the straight pipe with equal diameter is known, the spatial position of the straight pipe with equal diameter is given by P C The three-dimensional coordinates of the point and the direction vector v of the axis L Sure.

[0045] The second object of the present invention is to provide a straight pipe spatial posture visual tracking device based on edge contour, comprising:

[0046] An operating platform for laying two-dimensional plane targets;

[0047] Two-dimensional plane target, which can be moved and laid on the operating platform to adjust multiple different positions within the camera field of view;

[0048] The camera is fixed on the upper side of the two-dimensional plane target by a frame, and the two-dimensional plane target is always within the camera field of view during the camera calibration process;

[0049] A support is arranged on an operating platform on both sides of the two-dimensional plane target;

[0050] A straight pipe of equal diameter is arranged on a bracket and is located within the field of view of the camera;

[0051] The processing system is connected to the camera and executes the above-mentioned straight pipe spatial posture visual tracking method based on edge contour.

[0052] Preferably, the two-dimensional planar target is a two-dimensional checkerboard.

[0053] Compared with the prior art, the present invention first uses a camera to collect multiple plane target images at different angles, and calibrates the camera to obtain the camera's internal parameter matrix and distortion coefficient; secondly, uses the camera to collect an image of an equal-diameter straight pipe with a known radius to extract edge contour features to obtain contour feature points; due to the existence of lens distortion, the feature points need to be distorted; then, the corrected distortion-free contour feature points are respectively linearly fitted to obtain two contour lines of the equal-diameter straight pipe; finally, the spatial position and posture of the axis is solved according to the constraint relationship between the two contour lines and the axis of the equal-diameter straight pipe; the present invention can obtain the position and posture of the axis of the equal-diameter straight pipe through linear calculation without distortion, has the advantages of simple hardware structure, convenient operation, low cost, etc., can quickly determine the position and posture of the equal-diameter straight pipe, and realize the position and posture tracking of the equal-diameter straight pipe; in actual assembly and laying work, the docking error of the equal-diameter straight pipe is reduced, and the efficiency of assembly and laying of the pipeline system is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a structural layout diagram of the camera calibration process of the edge contour-based straight pipe spatial posture visual tracking device of the present invention.

[0055] Figure 2 It is a schematic structural diagram of a two-dimensional planar target of the present invention.

[0056] Figure 3 The present invention is a structural layout diagram of the process of collecting images of straight pipes of equal diameters by the straight pipe spatial posture visual tracking device based on edge contours of the present invention.

[0057] Figure 4 The present invention is a flow chart of the method for visually tracking the spatial posture of a straight pipe based on edge contours.

[0058] Figure 5 It is a perspective projection model of a straight pipe with equal diameter.

[0059] Figure 6 These are the two contour lines of the straight pipe with equal diameter obtained by fitting.

[0060] Figure 7 is the position of the straight pipe of equal diameter and its axis in the three-dimensional space under the camera coordinate system. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0062] Example 1

[0063] like Figure 4 As shown, this embodiment exemplarily shows a straight pipe spatial posture visual tracking method based on edge contour, including the following steps:

[0064] S1. Reference Figure 1 , the two-dimensional plane target 2 is horizontally laid on the operating platform 1. The two-dimensional plane target used in this embodiment is as follows Figure 2 The two-dimensional plane is a chessboard as shown, and a camera 6 is fixedly arranged by a frame 5 obliquely above the two-dimensional plane target 2. The two-dimensional plane target 2 is placed in the field of view of the camera 6 and the position of the two-dimensional plane target 2 is adjusted. The camera coordinate system O-XYZ is established with the optical center O of the camera as the origin, the X-axis points to the right side of the camera, the Y-axis points downward, and the Z-axis points forward; the camera 6 is used to collect multiple two-dimensional plane target images at different positions, and the camera is calibrated using the "Camera Calibration Toolbox for Matlab" toolbox developed by Jean-Yves Bouguet based on Zhang's calibration method for the multiple two-dimensional plane target images collected by the camera to obtain the camera's intrinsic parameter matrix and distortion coefficients (k1, k2), where (f x ,f y ) represent the normalized focal length of the camera in the horizontal and vertical directions, (u0, v0) represent the coordinates of the principal point, k1 and k2 represent the first-order and second-order radial distortion coefficients, respectively;

[0065] S2, reference Figure 3 , a straight pipe 4 with a radius (outer diameter of the straight pipe) of 8 mm and a length of 300 mm is horizontally placed on the two-dimensional plane target 2 through a bracket 3, and then the chessboard of the two-dimensional plane is removed, and a straight pipe image with the same diameter is collected by a camera 6; the edge contour feature of the straight pipe image with the same diameter is extracted by using a canny edge detection algorithm (which is a conventional operation method in the field and will not be described in detail in this embodiment), and the contour feature points with distortion are obtained;

[0066] S3. Combining the camera's internal parameter matrix and distortion coefficient, the distorted contour feature points are corrected to obtain undistorted contour feature points. The calculation formula for distortion correction is shown in formula (1):

[0067]

[0068] in: and (u,v) represent the pixel coordinates of the contour feature points with and without distortion, respectively; (x,y) represent the normalized image coordinates without distortion; (k1,k2) represent the radial distortion coefficients of the camera; (u0,v0) represent the principal point coordinates in the intrinsic parameter matrix of the camera.

[0069] S4, performing straight line fitting on the undistorted contour feature points to obtain two contour lines of straight pipes with equal diameters;

[0070] S41, use the RANSAC algorithm to screen the undistorted contour feature points to obtain the feature point set {(u′ g ,v′ g )|g∈1,…,G} and {(u′ m ′,v′ m ′)|m∈1,…,M}, where: (u′ g ,v′ g ) and (u′ m ′,v′ m ′) represent the pixel coordinates of the feature points on the two contours respectively;

[0071] S42. Based on the least squares method, the feature point set is fitted with straight lines to obtain two contour lines l1 and l2 of the straight pipe with equal diameter. The equation of the fitted line is shown in formula (2):

[0072]

[0073] Among them: a′, b′, c′ and a″, b″, c″ are the equation coefficients of the two fitting lines respectively; the solution process of a′, b′, c′ and a″, b″, c″ is as follows:

[0074] The equations formed by the homogeneous coordinates of the feature points on the two contours as coefficient matrices are shown in formula (3):

[0075]

[0076] For equation (3), the eigenvector corresponding to the minimum eigenvalue of the coefficient matrix is ​​the least squares solution of the equation system; or the coefficient matrix is ​​decomposed by SVD, and then the vector corresponding to the minimum singular value in the right singular matrix is ​​taken as the least squares solution;

[0077] Substituting the obtained least square solution into equation (2), we can obtain two contour lines l1 and l2 of straight pipes with equal diameters as follows: Figure 6 shown.

[0078] S5. Combining the two contour lines of the straight pipe with equal diameter, the radius of the straight pipe with equal diameter and the intrinsic parameter matrix of the camera, according to the perspective projection model of the straight pipe with equal diameter (see Figure 5 ) Perform spatial pose calculation on straight pipes with equal diameters;

[0079] S51. Combining the camera's intrinsic parameter matrix and the two contour lines l1 and l2 of the straight pipe with equal diameter, the back-projection planes corresponding to the two contour lines are:

[0080]

[0081] Where: K represents the camera intrinsic parameter matrix, π1 and π2 represent the back-projection planes of the contour lines corresponding to l1 and l2 respectively;

[0082] S52, according to the obtained back-projection planes π1 and π2, obtain their normal vectors n1 and n2, and calculate the bisector π of the back-projection planes π1 and π2 m The normal vector is:

[0083]

[0084] Where: n m is the normal vector of the bisector of the two back-projection planes, ||n|| represents the modulus of the vector, and the angle and intersection vector of the back-projection planes π1 and π2 are calculated as follows:

[0085]

[0086] Where: α is the angle between the two back-projection planes, v Lrepresents the intersection vector of the back-projection planes π1 and π2;

[0087] S53, according to the obtained intersection vector v L and the plane normal vector n m , calculate the center P of the cross-section circle from the origin O to the support plane passing through the origin C The direction vector is:

[0088]

[0089] in: means OP C The direction vector of the equal-diameter straight pipe is combined with the radius r of the two back-projection planes to calculate the origin O and P C The distance between points is:

[0090]

[0091] in: Indicates point O to P C The distance of the points, according to OP C Direction vector and point O and P C Distance of points Get P C The 3D coordinates of a point:

[0092]

[0093] Since the radius of the straight pipe with equal diameter is known, the spatial position of the straight pipe with equal diameter is given by P C The three-dimensional coordinates of the point and the direction vector v of the axis L OK, as follows Figure 7 shown.

[0094] Example 2

[0095] like Figure 1 , Figure 2 , Figure 3 As shown, this embodiment exemplarily shows a straight pipe spatial posture visual tracking device based on edge contour, including:

[0096] An operating platform 1, used for laying a two-dimensional plane target 2;

[0097] The two-dimensional plane target 2 can be movably laid on the operating platform 1 to adjust multiple different positions within the camera field of view; the two-dimensional plane target used in this embodiment is as follows Figure 2 The two-dimensional plane of the chessboard shown;

[0098] The camera 6 is fixed obliquely above the two-dimensional plane target 2 through the frame 5, and the two-dimensional plane target 2 is always located within the camera field of view during the camera calibration process;

[0099] The support 3 is arranged on the operating platform 1 on both sides of the two-dimensional plane target;

[0100] A straight pipe 4 of equal diameter is placed on the bracket 3 and is within the field of view of the camera;

[0101] The processing system (not shown in the figure) is connected to the camera 6 and executes the above-mentioned straight pipe spatial posture visual tracking method based on edge contour. Usually, the processing system is stored in a computer.

[0102] By using the device of this embodiment in combination with the straight pipe spatial posture visual tracking method based on edge contours in Example 1, the posture of straight pipes of equal diameter with known radius can be quickly determined, thereby realizing posture tracking of straight pipes of equal diameter.

[0103] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A straight pipe spatial posture visual tracking method based on edge contour, characterized in that: The following steps are involved: S1. Lay the two-dimensional plane target horizontally on the operating platform, and fix a camera obliquely above the two-dimensional plane target, place the two-dimensional plane target within the camera field of view and adjust the position of the two-dimensional plane target, use the camera to collect multiple two-dimensional plane target images at different positions, and use Zhang's calibration method to calibrate the camera for the multiple two-dimensional plane target images collected by the camera to obtain the camera's intrinsic parameter matrix and distortion coefficient; S2. Place a straight pipe with a known radius horizontally within the camera field of view using a bracket, and use the camera to capture an image of the straight pipe with a known radius; extract edge contour features from the image of the straight pipe with a known radius to obtain contour feature points with distortion; S3, combining the camera's internal parameter matrix and distortion coefficient to perform distortion correction on the distorted contour feature points to obtain undistorted contour feature points; S4, performing straight line fitting on the undistorted contour feature points to obtain two contour lines of straight pipes with equal diameters; S5. Combining the two contour lines of the straight pipe with equal diameter, the radius of the straight pipe with equal diameter and the intrinsic parameter matrix of the camera, the spatial posture of the straight pipe with equal diameter is solved according to the perspective projection model of the straight pipe with equal diameter.

2. The method for visually tracking the spatial position of a straight pipe based on edge contour according to claim 1 is characterized in that: In step S2, the edge contour features of the image of straight pipes with equal diameters are extracted using the canny edge detection algorithm.

3. The straight pipe spatial posture visual tracking method based on edge contour according to claim 2 is characterized in that: In step S3, the calculation formula for distortion correction is shown in formula (1): in: and (u,v) represent the pixel coordinates of the contour feature points with and without distortion, respectively; (x,y) represent the normalized image coordinates without distortion; (k1,k2) represent the radial distortion coefficients of the camera; (u0,v0) represent the principal point coordinates in the intrinsic parameter matrix of the camera.

4. The method for visually tracking the spatial position of a straight pipe based on edge contour according to claim 3 is characterized in that: The step S4 comprises: S41, use the RANSAC algorithm to screen the undistorted contour feature points to obtain the feature point set {(u′ g ,v′ g )|g∈1,...,G} and {(u″ m ,v″ m )|m∈1,...,M}, where: (u′ g ,v′ g ) and (u′ m ′,v′ m ′) represent the pixel coordinates of the feature points on the two contours respectively; S42. Based on the least squares method, the feature point set is fitted with straight lines to obtain two contour lines l1 and l2 of the straight pipe with equal diameter. The equation of the fitted line is shown in formula (2): Among them: a′, b′, c′ and a″, b″, c″ are the equation coefficients of the two fitting lines respectively; the solution process of a′, b′, c′ and a″, b″, c″ is as follows: The equations formed by the homogeneous coordinates of the feature points on the two contours as coefficient matrices are shown in formula (3): For formula (3), the eigenvector corresponding to the minimum eigenvalue of the coefficient matrix is ​​the least squares solution of the equation system; or the coefficient matrix is ​​decomposed by SVD, and then the vector corresponding to the minimum singular value in the right singular matrix is ​​taken as the least squares solution; Substituting the obtained least square solution into equation (2) we can obtain two contour lines l1 and l2 of straight pipes with equal diameters.

5. The method for visually tracking the spatial position of a straight pipe based on edge contour according to claim 4 is characterized in that: The step S5 comprises: S51. Combining the camera's intrinsic parameter matrix and the two contour lines l1 and l2 of the straight pipe with equal diameter, the back-projection planes corresponding to the two contour lines are: Where: K represents the camera intrinsic parameter matrix, π1 and π2 represent the back-projection planes of the contour lines corresponding to l1 and l2 respectively; S52, according to the obtained back-projection planes π1 and π2, obtain their normal vectors n1 and n2, and calculate the bisector π of the back-projection planes π1 and π2 m The normal vector is: Where: n m is the normal vector of the bisector of the two back-projection planes, ||n|| represents the modulus of the vector, and the angle and intersection vector of the back-projection planes π1 and π2 are calculated as follows: Where: α is the angle between the two back-projection planes, v L represents the intersection vector of the back-projection planes π1 and π2; S53, according to the obtained intersection vector v L and the plane normal vector n m , calculate the center P of the cross-section circle from the origin O to the support plane passing through the origin C The direction vector is: in: means OP C The direction vector of the equal-diameter straight pipe is combined with the radius r of the two back-projection planes to calculate the origin O and P C The distance between points is: in: Indicates point O to P C The distance of the points, according to OP C Direction vector and point O and P C Distance of points Get P C The 3D coordinates of a point: Since the radius of the straight pipe with equal diameter is known, the spatial position of the straight pipe with equal diameter is given by P C The three-dimensional coordinates of the point and the direction vector v of the axis L Sure.

6. A straight pipe spatial posture visual tracking device based on edge contour, characterized in that: include: An operating platform for laying two-dimensional plane targets; Two-dimensional plane target, which can be moved and laid on the operating platform to adjust multiple different positions within the camera field of view; The camera is fixed on the upper side of the two-dimensional plane target by a frame, and the two-dimensional plane target is always within the camera field of view during the camera calibration process; A support is arranged on an operating platform on both sides of the two-dimensional plane target; A straight pipe of equal diameter is arranged on a bracket and is located within the field of view of the camera; A processing system is connected to the camera and executes the edge contour-based straight pipe spatial posture visual tracking method as described in any one of claims 1-5.

7. The straight pipe spatial posture visual tracking device based on edge contour according to claim 6 is characterized in that: The two-dimensional plane target is a two-dimensional plane checkerboard.

Images