A pipe thread positioning and measuring method and system based on three-dimensional vision

Abstract

The application provides a pipe thread positioning and measuring method and system based on three-dimensional vision, and belongs to the fields of industrial automation and three-dimensional vision technology; the method comprises the following steps: building a measuring system composed of a robot measuring system and a three-dimensional visual positioning system, and completing hand-eye calibration; acquiring a three-dimensional point cloud model of the end of a pipe thread workpiece, and through pretreatment, cylinder fitting, near-end surface point cloud extraction and end surface circle fitting, extracting an end surface normal vector, a tangent vector and a circle center coordinate to establish a measuring reference coordinate system; based on the hand-eye calibration result and the end pose of the robot, converting the reference coordinate system to a robot base coordinate system, generating a robot running track, guiding the robot to position to a measuring position, and completing the preparation before measuring; the application realizes automatic high-precision extraction of a pipe thread measuring reference, solves the problems of non-uniform reference, large error and low efficiency in traditional measurement, is suitable for various pipe thread workpieces, and provides a reliable foundation for subsequent high-precision measurement.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial automation and three-dimensional vision, and particularly relates to a pipe thread positioning and measuring method and system based on three-dimensional vision. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Pipe threads are widely used in the connection of pipelines in the fields of petroleum, chemical industry, and mechanical manufacturing, and the machining precision thereof directly affects the sealing performance and connection reliability of the pipelines. In order to ensure the quality of thread connection, it is usually necessary to perform high-precision measurement on the internal thread geometric parameters (such as pitch, thread angle, taper, etc.). However, internal thread measurement often faces two key challenges: firstly, the measurement probe or vision device needs to be accurately aligned with the thread axis; and secondly, the coordinate system of the measurement data must be consistent with the actual geometric position of the thread, otherwise error accumulation will occur, affecting the reliability of the measurement results.

[0004] Therefore, before performing internal thread measurement, it is necessary to accurately obtain the normal vector and center coordinates of the pipe thread end face, and to establish a coordinate system with the thread axis as the reference, which is a prerequisite for realizing high-precision and high-stability internal thread measurement. This reference coordinate is not only used for the posture adjustment and path planning of the probe, but also serves as the core reference for subsequent measurement data analysis and thread parameter calculation. However, due to installation errors such as eccentricity and inclination of the pipe material during machining clamping or transportation, if thread measurement is directly performed without coordinate calibration, it is easy to cause the measurement reference to be inconsistent, resulting in inaccurate parameter extraction. Especially when using a three-dimensional vision or robot measurement system, the coordinate system of the measurement system often does not coincide with the actual geometry of the workpiece, which further increases the measurement error. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides a pipe thread positioning and measuring method and system based on three-dimensional vision, which extracts the pipe thread end face normal vector, tangent vector, and center coordinates based on three-dimensional point cloud data, and establishes a measurement reference coordinate system by combining spatial fitting algorithm and coordinate transformation technology, thereby providing key technical support for subsequent high-precision automatic measurement of thread parameters, and effectively solving the problems of difficult reference acquisition, low registration accuracy, and large measurement error in the existing measurement process.

[0006] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions:

[0007] The present application provides a pipe thread positioning and measuring method based on three-dimensional vision in a first aspect;

[0008] A pipe thread positioning and measuring method based on three-dimensional vision, comprising:

[0009] A pipe thread measurement system based on three-dimensional vision and robot is built, and hand-eye calibration is performed;

[0010] The pipe thread end is scanned at multiple angles by the built pipe thread measurement system to obtain complete three-dimensional point cloud data, and the position information of each joint of the robot is recorded in real time during the scanning process;

[0011] The obtained point cloud data is preprocessed, and the preprocessed point cloud is fitted with a cylindrical model; according to the cylindrical fitting result, the proximal end surface point cloud region is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitted axis, and two-dimensional circle fitting is performed to obtain three key geometric parameters of the center coordinates, normal vector and tangent vector;

[0012] According to the parameters of the hand-eye calibration, combined with the real-time motion pose of the robot and the three-dimensional point cloud data, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system to obtain the pose of the target point in the robot base coordinate system;

[0013] The obtained pose of the target point in the robot base coordinate system is converted into a robot running track and sent to the controller system, and through robot positioning, it moves to the pipe thread measurement position for measurement work.

[0014] As a further technical solution, the pipe thread measurement system based on three-dimensional vision and robot includes a robot measurement system and a three-dimensional vision positioning system;

[0015] The robot measurement system includes a robot arm, a robot arm controller, and a measurement equipment;

[0016] The three-dimensional vision positioning system includes a three-dimensional vision scanning device and a display interaction system; the three-dimensional vision scanning device is used to obtain workpiece point cloud; the display interaction system is used for image display, image processing and man-machine interaction.

[0017] As a further technical solution, the process of hand-eye calibration is:

[0018] The calibration board is fixed in the robot workspace to ensure that it is within the effective observation range of the three-dimensional vision scanning device; the camera is fixed to the robot end effector, and the eye-in-hand mounting method is adopted;

[0019] Control the robot to drive the end effector to move the camera at different poses, so that the camera collects images of the calibration board from multiple angles;

[0020] For each collection point, the pose parameters of the calibration board in the rubber coordinate system and the pose parameters of the end effector in the robot base coordinate system are recorded synchronously, which are used for solving the hand-eye transformation relationship subsequently.

[0021] As a further technical solution, the obtained point cloud data is preprocessed, including:

[0022] The point cloud data is reduced in quantity by voxel grid downsampling, and the overall shape feature is retained;

[0023] The outliers and noise points are removed by statistical filtering, and the quality of the point cloud is improved;

[0024] The effective area of the main thread end face is extracted by using the Euclidean clustering algorithm.

[0025] As a further technical solution, the point cloud after preprocessing is fitted with a cylindrical model, including:

[0026] The RANSAC algorithm is used to iteratively fit the cylindrical model, and the preliminarily estimated cylindrical axis, radius and fitting inlier set are obtained.

[0027] As a further technical solution, according to the cylindrical fitting result, the near-end face point cloud area is extracted; the extracted end face point cloud is projected onto a two-dimensional plane perpendicular to the fitted axis, and two-dimensional circle fitting is performed to obtain three key geometric parameters of the center coordinates, normal vector and tangent vector, including:

[0028] On the basis of the cylindrical fitting result, a projection plane perpendicular to the direction of the cylindrical axis is set, and a small section of point cloud located at the end part is selected according to the projection distance of the point cloud in the axial direction, to obtain the near-end face annular point cloud;

[0029] The intercepted near-end face annular point cloud is projected onto a plane perpendicular to the main shaft, and two-dimensional circle fitting is performed by using the least squares method to obtain three key geometric parameters of the center coordinates, normal vector and tangent vector.

[0030] As a further technical solution, according to the parameters of hand-eye calibration, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system in combination with the real-time motion pose of the robot and the three-dimensional point cloud data, to obtain the pose of the target point in the robot base coordinate system, including:

[0031] The fixed transformation relationship between the camera coordinate system and the robot end coordinate system is obtained by the Tsai-Lenz calibration algorithm, which is recorded as the first homogeneous transformation matrix; when the three-dimensional visual scanning device collects the pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded synchronously, and the second homogeneous transformation matrix is constructed;

[0032] In combination with the first homogeneous transformation matrix and the second homogeneous transformation matrix, the obtained center coordinates, normal vector and tangent vector are converted from the camera coordinate system to the robot end coordinate system; the direction vector Z axis and X axis are converted to the robot base coordinate system, the Y axis vector is constructed by cross multiplication, and the SVD decomposition is used for orthogonalization processing to obtain the final rotation matrix;

[0033] ZYX Euler angle decomposition is performed on the final rotation matrix, rotation angles around Z, Y and X axes in the base coordinate system are extracted, and a six-dimensional pose of the target point in the robot base coordinate system is formed by combination with the position vector.

[0034] The second aspect of the application provides a pipe thread positioning and measuring system based on three-dimensional vision.

[0035] A pipe thread positioning and measuring system based on three-dimensional vision comprises:

[0036] The measuring system building module is configured to build a pipe thread measuring system based on three-dimensional vision and a robot, and perform hand-eye calibration.

[0037] The point cloud data acquisition module is configured to perform multi-angle scanning on the pipe thread end through the built pipe thread measuring system to acquire complete three-dimensional point cloud data, wherein the position information of each joint of the robot is recorded in real time during the scanning process.

[0038] The point cloud data processing module is configured to pre-process the acquired point cloud data, and perform cylindrical model fitting on the pre-processed point cloud; according to the cylindrical fitting result, extract the near-end surface point cloud area; project the extracted end surface point cloud to a two-dimensional plane perpendicular to the fitting axis, perform two-dimensional circle fitting, and obtain three key geometric parameters of the center coordinates, normal vector and tangent vector.

[0039] The coordinate conversion module is configured to convert the pipe thread measuring reference coordinate system to the robot base coordinate system according to the parameters of the hand-eye calibration, combine the real-time motion pose of the robot with the three-dimensional point cloud data, and obtain the pose of the target point in the robot base coordinate system.

[0040] The path planning module is configured to convert the obtained pose of the target point in the robot base coordinate system into a robot running track and send it to the controller system, and through robot positioning, move to the pipe thread measuring position to perform measuring work.

[0041] The third aspect of the application provides a computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps of the pipe thread positioning and measuring method based on three-dimensional vision according to the first aspect of the application.

[0042] The fourth aspect of the application provides an electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the pipe thread positioning and measuring method based on three-dimensional vision according to the first aspect of the application.

[0043] The above one or more technical solutions have the following beneficial effects:

[0044] (1) The application combines three-dimensional vision system and industrial robot technology, accurately extracts the measurement reference information through the processing and circle fitting of the pipe thread end face point cloud, and automatically completes the pose conversion from the camera coordinate system to the robot base coordinate system, so that the spatial pose of the target can be obtained without manual intervention, and the automation and high efficiency of the pipe thread detection task are realized.

[0045] (2) Through the high-precision three-dimensional vision system, the robust end face circle fitting algorithm and the accurate robot hand-eye calibration method, the application can accurately obtain the pipe thread end face center and normal, realize the real-time conversion of the measurement reference coordinate system in the robot base coordinate system, ensure the high precision and high repeatability of the detection positioning, and provide a reliable foundation for the subsequent automatic processing or positioning operation.

[0046] (3) The application has good universality and robustness, and is suitable for pipe thread workpieces of different forms, sizes and installation modes, especially suitable for flexible detection needs of small and medium batch and multiple varieties, and reduces the dependence of the traditional detection system on the consistency of the workpiece.

[0047] The advantages of the additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which form a part of the present description, are included to provide a further understanding of the application, and are incorporated herein for explanation by illustrating a preferred embodiment in which the application is applied, and do not constitute an improper limitation to the application.

[0049] Figure 1 The method flowchart of the first embodiment.

[0050] Figure 2 The system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0051] It should be noted that the following detailed description is exemplary, and is intended to provide further explanation of the application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the application belongs.

[0052] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the exemplary embodiments according to the application.

[0053] In the case of no conflict, the embodiments in the application and the features in the embodiments can be combined with each other.

[0054] Embodiment one

[0055] The embodiment discloses a pipe thread positioning and measuring method based on three-dimensional vision.

[0056] As shown in the figure, a pipe thread positioning and measuring method based on three-dimensional vision comprises the following steps: Figure 1

[0057] Step S1, a pipe thread measuring system based on three-dimensional vision and a robot is built, and hand-eye calibration is performed.

[0058] Step S2, the end of the pipe thread is scanned at multiple angles by the built pipe thread measuring system, and complete three-dimensional point cloud data is acquired, wherein the position information of each joint of the robot is recorded in real time during the scanning process.

[0059] Step S3, the acquired point cloud data is preprocessed, and the preprocessed point cloud is fitted to a cylindrical model; according to the cylindrical fitting result, a proximal end surface point cloud region is extracted; the extracted end surface point cloud is projected to a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is performed, and three key geometric parameters of the center coordinates, the normal vector and the tangent vector are obtained.

[0060] Step S4, according to the parameters of the hand-eye calibration, the real-time motion pose of the robot and the three-dimensional point cloud data are combined, the pipe thread measuring reference coordinate system is converted to the robot base coordinate system, and the pose of the target point in the robot base coordinate system is obtained.

[0061] Step S5, the obtained pose of the target point in the robot base coordinate system is converted to form a robot running track and is sent to a controller system, the robot is positioned and moved to the pipe thread measuring position to perform the measuring work.

[0062] Specifically, the above steps further comprise the following contents:

[0063] Step S1, a pipe thread measuring system based on three-dimensional vision and a robot is built, and hand-eye calibration is performed.

[0064] In step S1, the built pipe thread measuring system based on three-dimensional vision and a robot comprises a robot measuring system and a three-dimensional vision positioning system.

[0065] The robot measuring system comprises a robot arm, a robot arm controller and a measuring equipment. In the embodiment, the robot measuring system composition device is not limited to use multiple brands of robot arms and measuring equipment with similar principles and functions according to different measured workpieces.

[0066] ​The three-dimensional visual positioning system comprises two parts of a three-dimensional visual scanning device and a display interaction system. The three-dimensional visual scanning device needs to have the function of obtaining a workpiece point cloud, the measurement accuracy is higher than 1 mm, the depth map frame rate is greater than 1 frame per second, and a high-precision 3D surface structure light camera can be used, but it is not limited to using three-dimensional visual scanning devices of multiple brands, multiple structures, having similar principles and the same function,

[0067] The display interaction system needs to have the functions of image display, image processing and human-computer interaction. The image display device includes but is not limited to a touch display, a projector and the like. The image processing device includes but is not limited to a computer with image processing function, a tablet computer and the like. The human-computer interaction device includes but is not limited to a mouse, a touch screen, voice recognition and the like.

[0068] Further, in the process of hand-eye calibration of the measurement system, the combination mode of the robot arm and the three-dimensional visual scanning device includes that the three-dimensional visual scanning device is installed on the robot arm (eye on hand) and the three-dimensional visual scanning device is installed outside the robot arm (eye outside hand). According to the requirements of pipe thread measurement and high precision, the mode of eye on hand is selected and hand-eye calibration is performed in the embodiment. The specific process of hand-eye calibration is as follows:

[0069] By fixing a calibration board, it is ensured that it is visible in the robot workspace, and the camera is installed at the end of the robot, i.e. eye on hand.

[0070] The robot is controlled to move at different poses, so that the camera observes the calibration board from multiple angles, a total of 20 point positions are shot. When shooting each time, the pose of the calibration board under the camera coordinate system is recorded , and the pose of the end effector under the robot base coordinate system .

[0071] In step S2, the pipe thread end is scanned at multiple angles by the pipe thread measurement system built, and complete three-dimensional point cloud data is obtained, wherein the joint position information of the robot is recorded in real time during the scanning process.

[0072] In step S2, the three-dimensional model of the pipe thread workpiece end is obtained by using the three-dimensional visual scanning device, the pipe thread end surface point cloud data is obtained by using the three-dimensional model of the pipe thread workpiece end, and the relationship between the workpiece coordinate system and the robot coordinate system is obtained by recording the six-dimensional pose of the robot end in the robot base coordinate system in real time during the process of obtaining the point cloud data. During the process of obtaining the point cloud data, if the three-dimensional point cloud of the workpiece end cannot be obtained by one scanning, the three-dimensional visual scanning device can be used for multiple scanning, and the complete three-dimensional point cloud model of the workpiece end can be obtained by using point cloud splicing, registration and reverse engineering.

[0073] In addition, in the acquisition of the three-dimensional model of the pipe thread workpiece end, if there is a CAD model, it can also be used as a reference template for registration and correction with the actual scanning data; or through the historical acquisition model data and the current posture transformation relationship, the current end surface point cloud data can be calculated and obtained.

[0074] Step S3, the acquired point cloud data is preprocessed, and the preprocessed point cloud is fitted with a cylindrical model; according to the cylindrical fitting result, the proximal end surface point cloud region is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is performed, and three key geometric parameters of the center coordinates, normal vector and tangent vector are obtained.

[0075] Step S31, the acquired pipe thread end surface point cloud data is preprocessed to improve the subsequent fitting accuracy and robustness. Specifically, first, the voxel grid sampling method is used to downsample the point cloud, the basic principle of which is to divide the point cloud space into regular cubic grids, and to represent each voxel with the geometric center or average point of the points in the voxel, thereby effectively reducing the amount of point cloud data while maintaining the overall geometric shape; then, a statistical filtering method is used to remove noise, which identifies and removes outliers by calculating the average distance of points and their neighbors and setting a threshold to eliminate isolated points and random noise; further, a clustering segmentation method based on Euclidean distance is used, which divides the point cloud into several sub-clusters according to the spatial distance relationship between points, thereby accurately extracting the main region of the thread end surface and removing irrelevant point clouds. After the above multi-level preprocessing, the obtained point cloud not only retains the key geometric features of the thread end surface, but also significantly improves the data quality and compactness, laying a reliable foundation for subsequent cylindrical model fitting and main shaft extraction.

[0076] Through voxel grid down-sampling, the amount of point cloud data is reduced, and the overall shape feature is preserved; statistical filtering is used to remove outliers and noise points to improve the quality of the point cloud; Euclidean clustering is used to extract the main region, retain the main thread end surface region, and remove scattered small clusters.

[0077] Step S32, the preprocessed point cloud is fitted with a cylindrical model to preliminarily obtain the main shaft direction and cylindrical parameters of the pipe body. In this embodiment, the Random Sample Consensus (RANSAC) method is used for cylindrical model fitting. The process is as follows:

[0078] First, a cylindrical model is constructed, and the axis of the cylinder is a straight line:

[0079]

[0080] wherein, is a point on the axis, is a unit directional vector, is the radius.

[0081] Randomly sample a small number of points from the point cloud, construct the candidate cylindrical axis parameters and radius, and calculate the distance of other arbitrary points to the cylindrical axis:

[0082]

[0083] If the difference between the distance and the radius is less than a preset threshold, it is determined as an inner point.

[0084] By repeating the iterative sampling, the number of inner points is counted, and the model with the most inner points is selected as the optimal cylinder. The inner point set of the model is used for parameter optimization to obtain the axis direction, center position and radius of the cylinder and other geometric parameters. Through the above method, the spatial position and size information of the cylinder can be obtained robustly in the presence of noise and outliers.

[0085] Step S33, according to the cylindrical fitting result, extract the pipe thread end surface point cloud close to the camera or the specified measurement direction as the subsequent circular fitting area.

[0086] Specifically, a projection plane perpendicular to the cylindrical axis direction is set; according to the projection distance of the point cloud in the axial direction, a small section of point cloud located at the end (such as ±5mm range) is selected, and then a near-end surface annular point cloud is obtained.

[0087] Step S34, project the extracted end surface point cloud to the two-dimensional plane perpendicular to the fitting axis, and perform two-dimensional circular fitting using the least squares method to obtain the center coordinates and radius of the two-dimensional circle in the projection plane.

[0088] Specifically, after projecting the annular point cloud to the vertical plane of the main shaft, the least squares method is used for circular fitting. The mathematical model is:

[0089]

[0090] For the projection point set , an error function is established by the least squares method:

[0091]

[0092] The above problem is converted into a matrix form:

[0093]

[0094] The optimal solution parameter ( ) can be obtained by the following formula:

[0095]

[0096] And then the center ( ) and radius​ :

[0097]

[0098]

[0099] On this basis: the normal vector is taken as the normal of the projection plane, that is, the direction corresponding to the cylinder principal axis; the tangent vector is obtained by rotating the direction of the line connecting the center and a point on the circumference by ninety degrees. This method can efficiently obtain the three key geometric parameters of the center, normal vector and tangent vector.

[0100] Further, a local coordinate system is established in the vertical plane of the cylinder axis, and the two-dimensional center coordinates are mapped back to the three-dimensional space through the linear combination of the basis vectors of the coordinate system, thereby deducing the center coordinates, end face normal vector and tangent vector in the three-dimensional space as the key geometric parameters of the pipe thread end face. Among them, the end face normal vector is taken as the normal of the projection plane, that is, the direction corresponding to the cylinder principal axis; the tangent vector is obtained by rotating the direction of the line connecting the center and a point on the circumference by ninety degrees.

[0101] Step S4, according to the parameters of the hand-eye calibration, combining the real-time motion pose of the robot and the three-dimensional point cloud data, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system, and the pose of the target point in the robot base coordinate system is obtained.

[0102] Step S41, the fixed transformation relationship between the camera coordinate system and the robot end coordinate system is obtained by Tsai-Lenz calibration method, denoted as the first homogeneous transformation matrix .

[0103] When the three-dimensional vision scanning device collects the pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded synchronously, including three-dimensional position coordinates and ZYX Euler angles, and the second homogeneous transformation matrix .

[0104] The core of Tsai-Lenz algorithm is to solve the following equation:

[0105]

[0106] Wherein is the fixed transformation matrix between the camera coordinate system and the end effector coordinate system. Then, the homogeneous transformation matrix can be split into a rotation matrix and a translation vector:

[0107]

[0108] Wherein, represents the rotation relationship from the camera coordinate system to the end effector coordinate system, denotes the translation relationship between the two coordinate systems. Substituting it into the above formula, we can get:

[0109] ,

[0110] where is the rotation matrix part of , which represents the rotation relationship of the camera coordinate system relative to the calibration board, is the corresponding translation vector part. is the rotation matrix of the pose change of the robot base coordinate system to the end effector, is the corresponding translation vector part.

[0111] Step S42, the center coordinates of the circle are taken as the target point position, and its normal vector (defined as the target Z axis) and tangent vector (defined as the target X axis) are determined to jointly construct the target measurement reference coordinate system (right-handed system XYZ).

[0112] First, the target point and its normal vector and tangent vector are transformed from the camera coordinate system to the end coordinate system:

[0113]

[0114] where, is the end coordinate system; is the camera coordinate system;

[0115] Then, combined with the current transformation matrix of the robot end , it is further converted to the robot base coordinate system:

[0116]

[0117] where, is the robot base coordinate system.

[0118] Step S43, the direction vector (Z axis, X axis) is converted to the robot base coordinate system according to the above transformation method, the Y axis vector is constructed by cross multiplication, and the orthogonal right-handed system rotation matrix is formed. In order to ensure the orthogonality of the rotation matrix, SVD decomposition is used for orthogonalization processing to obtain the final rotation matrix .

[0119] Specifically, the process of using SVD decomposition for orthogonalization processing is as follows:

[0120] The base vector matrix is constructed:

[0121] ;

[0122] The singular value decomposition is performed on the base vector matrix .

[0123]

[0124] Construct the rotation matrix:

[0125]

[0126] If , adjust the sign of to ensure that the final rotation matrix satisfies:

[0127]

[0128] Through this process, the orthogonality and direction consistency of the rotation matrix can be guaranteed, thus achieving the accuracy of coordinate transformation.

[0129] Step S44, perform ZYX Euler angle decomposition on the rotation matrix , and extract its rotation angles around the Z, Y, and X axes in the base coordinate system, which are the Euler angles of the target pose. Combine them with the position vector to form the complete six-dimensional pose of the target point in the robot base coordinate system.

[0130] Specifically, define the obtained final rotation matrix as:

[0131]

[0132] where is the element of the rotation matrix in the th row and the th column.

[0133] According to the ZYX Euler angle definition, we have:

[0134]

[0135] If (i.e., a singular pose occurs), the backup formula is used to handle the singular situation. That is:

[0136]

[0137] Calculate the rotation angle around the Z axis and the rotation angle around the X axis. When , calculate according to the following formula:

[0138]

[0139] If (i.e., a singular pose occurs), according to the rotation matrix relationship, you can choose one of the angles to be zero (or keep the original value), and then calculate the unique solution from the other angle.

[0140] If :

[0141]

[0142] If :

[0143]

[0144] The above method can eliminate the uncertainty caused by the singular pose, ensure the stability and continuity of the rotation angle calculation.

[0145] Finally, by combining the position vector in the base coordinate system With the ZYX Euler angle , the six-dimensional pose of the target point in the robot base coordinate system can be obtained:

[0146]

[0147] Step S5, the obtained pose of the target point in the robot base coordinate system is converted to form a robot running trajectory and sent to a controller system, and the robot is positioned and moved to the pipe thread measurement position for measurement work by using the controller system.

[0148] Embodiment two

[0149] The embodiment discloses a pipe thread positioning and measuring system based on three-dimensional vision.

[0150] As Figure 2 shown, a pipe thread positioning and measuring system based on three-dimensional vision includes:

[0151] The measuring system building module is configured to build a pipe thread measuring system based on three-dimensional vision and a robot, and to perform hand-eye calibration.

[0152] The point cloud data acquisition module is configured to perform multi-angle scanning on the pipe thread end through the built pipe thread measuring system, and to acquire complete three-dimensional point cloud data, wherein the joint position information of the robot is recorded in real time during the scanning process.

[0153] The point cloud data processing module is configured to preprocess the acquired point cloud data, and to perform cylindrical model fitting on the preprocessed point cloud; according to the cylindrical fitting result, to extract the near-end surface point cloud area; to project the extracted end surface point cloud to a two-dimensional plane perpendicular to the fitting axis, to perform two-dimensional circle fitting, and to obtain three key geometric parameters of the center coordinates, the normal vector and the tangent vector.

[0154] The coordinate conversion module is configured to convert the pipe thread measurement reference coordinate system to the robot base coordinate system according to the parameters of the hand-eye calibration, in combination with the real-time motion pose of the robot and the three-dimensional point cloud data, to obtain the pose of the target point in the robot base coordinate system.

[0155] The path planning module is configured to convert the obtained pose of the target point in the robot base coordinate system into a robot running track and send it to the controller system, and move to the pipe thread measurement position for measurement work through robot positioning.

[0156] Embodiment three

[0157] The purpose of the present embodiment is to provide a computer-readable storage medium.

[0158] A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the three-dimensional vision-based pipe thread positioning and measurement method of embodiment one.

[0159] Embodiment four

[0160] The purpose of the present embodiment is to provide an electronic device.

[0161] An electronic device includes a memory, a processor, and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps of the three-dimensional vision-based pipe thread positioning and measurement method of embodiment one.

[0162] The steps and methods involved in the above embodiments two, three and four correspond to embodiment one, and the specific embodiments can be referred to the relevant description part of embodiment one. The term "computer-readable storage medium" should be understood to include a single medium or multiple media of one or more instruction sets; it should also be understood to include any medium capable of storing, encoding or carrying instruction sets for execution by a processor and causing the processor to perform any of the methods of the present application.

[0163] Those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by a general computer device, alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device for execution by a computing device, or they can be made into individual integrated circuit modules, or a plurality of modules or steps among them can be made into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0164] The above describes the specific embodiments of the present application in combination with the drawings, but is not a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for locating and measuring pipe threads based on three-dimensional vision, characterized in that, include: Build a pipe thread measurement system based on 3D vision and robotics, and perform hand-eye calibration; The pipe thread measurement system was built to perform multi-angle scanning of the pipe thread end to obtain complete three-dimensional point cloud data. During the scanning process, the position information of each joint of the robot was recorded in real time. The acquired point cloud data is preprocessed, and a cylindrical model is fitted to the preprocessed point cloud. Based on the cylindrical fitting results, the near-end face point cloud region is extracted. The extracted near-end face point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, and a two-dimensional circle is fitted to obtain three key geometric parameters: the center coordinates, the normal vector, and the tangent vector. The center coordinates are used as the target point, and its normal vector and tangent vector are determined. The normal vector is defined as the target Z-axis, and the tangent vector is defined as the target X-axis, which together construct the measurement reference coordinate system. Based on the parameters calibrated by hand and eye, and combined with the robot's real-time motion pose and 3D point cloud data, the reference coordinate system for pipe thread measurement is transformed to the robot's base coordinate system to obtain the pose of the target point in the robot's base coordinate system. Specifically, the fixed transformation relationship between the camera coordinate system and the robot end effector coordinate system is obtained through the Tsai-Lenz calibration algorithm and denoted as the first homogeneous transformation matrix; when the 3D vision scanning device collects pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded simultaneously, and the second homogeneous transformation matrix is ​​constructed. Combining the first and second homogeneous transformation matrices, the obtained center coordinates, normal vector, and tangent vector are transformed from the camera coordinate system to the robot end effector coordinate system; the Z-axis and X-axis direction vectors are transformed to the robot base coordinate system, and the Y-axis vector is constructed by cross product to form an orthogonal right-handed rotation matrix. The orthogonalization process is then performed using SVD decomposition to obtain the final rotation matrix. The final rotation matrix is ​​decomposed into ZYX Euler angles to extract the rotation angles around the Z, Y, and X axes in the base coordinate system, which are the Euler angles of the target pose. By combining them with the position vector in the base coordinate system, a six-dimensional pose of the target point in the robot's base coordinate system is formed. The pose of the target point obtained is transformed in the robot's base coordinate system to form the robot's running trajectory and sent to the controller system. The robot is then positioned and moved to the pipe thread measurement position to perform the measurement work.

2. The pipe thread positioning and measurement method based on three-dimensional vision as described in claim 1, characterized in that, The pipe thread measurement system based on 3D vision and robotics includes a robotic measurement system and a 3D vision positioning system. The robot measurement system includes a robot arm, a robot arm controller, and measurement equipment; The three-dimensional vision positioning system includes a three-dimensional vision scanning device and a display interaction system; the three-dimensional vision scanning device is used to acquire the point cloud of the workpiece; the display interaction system is used for image display, image processing and human-computer interaction.

3. The pipe thread positioning and measurement method based on three-dimensional vision as described in claim 1, characterized in that, The hand-eye alignment process is as follows: The calibration plate is fixed in the robot's workspace to ensure that it is within the effective observation range of the 3D vision scanning device; the camera is fixed to the robot's end effector, using an "eye on hand" mounting method. The robot is controlled to drive the end effector to move the camera in different poses, so that the camera can acquire images of the calibration board from multiple perspectives; For each acquisition point, the pose parameters of the calibration plate in the camera coordinate system and the pose parameters of the end effector in the robot base coordinate system are recorded simultaneously for subsequent solving of the hand-eye transformation relationship.

4. The pipe thread positioning and measurement method based on three-dimensional vision as described in claim 1, characterized in that, The preprocessing of the acquired point cloud data includes: Voxel grid downsampling reduces the amount of point cloud data while preserving overall shape features; Statistical filtering is used to remove outliers and noise points, thereby improving point cloud quality; The effective region of the main thread end face is extracted using a Euclidean clustering algorithm.

5. The pipe thread positioning and measurement method based on three-dimensional vision as described in claim 1, characterized in that, Cylindrical model fitting is performed on the preprocessed point cloud, including: The RANSAC algorithm is used to iteratively fit the cylindrical model to obtain preliminary estimates of the cylinder axis, radius, and set of fitted interior points.

6. The pipe thread positioning and measurement method based on three-dimensional vision as described in claim 1, characterized in that, Based on the cylinder fitting results, the near-end face point cloud region is extracted; the extracted end face point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, and a two-dimensional circle fitting is performed to obtain three key geometric parameters: the center coordinates, the normal vector, and the tangent vector, including: Based on the cylinder fitting results, a projection plane perpendicular to the cylinder axis is set. According to the projection distance of the point cloud in the axial direction, a small segment of the point cloud located at the end is selected to obtain a near-end-face annular point cloud. The extracted near-end-face annular point cloud is projected onto a plane perpendicular to the principal axis, and a two-dimensional circle is fitted using the least squares method to obtain three key geometric parameters: the center coordinates, the normal vector, and the tangent vector.

7. A pipe thread positioning and measurement system based on three-dimensional vision, characterized in that: include: The measurement system construction module is configured to: build a pipe thread measurement system based on 3D vision and robotics, and perform hand-eye calibration; The point cloud data acquisition module is configured to: perform multi-angle scanning of the pipe thread end through the constructed pipe thread measurement system to acquire complete three-dimensional point cloud data, wherein the position information of each joint of the robot is recorded in real time during the scanning process; The point cloud data processing module is configured to: preprocess the acquired point cloud data and fit the preprocessed point cloud with a cylindrical model; extract the near-end face point cloud region based on the cylindrical fitting result; project the extracted near-end face point cloud onto a two-dimensional plane perpendicular to the fitting axis and perform two-dimensional circle fitting to obtain three key geometric parameters: the center coordinates, the normal vector, and the tangent vector. The center coordinates of the circle are used as the target point, and its normal vector and tangent vector are determined. The normal vector is defined as the target Z-axis, and the tangent vector is defined as the target X-axis. Together, they form a measurement reference coordinate system. The coordinate transformation module is configured to: transform the reference coordinate system for pipe thread measurement to the robot base coordinate system based on the parameters calibrated by hand and eye, combined with the robot's real-time motion pose and 3D point cloud data, so as to obtain the pose of the target point in the robot base coordinate system; Specifically, the fixed transformation relationship between the camera coordinate system and the robot end effector coordinate system is obtained through the Tsai-Lenz calibration algorithm and denoted as the first homogeneous transformation matrix; when the 3D vision scanning device collects pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded simultaneously, and the second homogeneous transformation matrix is ​​constructed. Combining the first and second homogeneous transformation matrices, the obtained center coordinates, normal vector, and tangent vector are transformed from the camera coordinate system to the robot end effector coordinate system; the Z-axis and X-axis direction vectors are transformed to the robot base coordinate system, and the Y-axis vector is constructed by cross product to form an orthogonal right-handed rotation matrix. The orthogonalization process is then performed using SVD decomposition to obtain the final rotation matrix. The final rotation matrix is ​​decomposed into ZYX Euler angles to extract the rotation angles around the Z, Y, and X axes in the base coordinate system, which are the Euler angles of the target pose. By combining them with the position vector in the base coordinate system, a six-dimensional pose of the target point in the robot's base coordinate system is formed. The path planning module is configured to: transform the pose of the obtained target point in the robot's base coordinate system to form the robot's running trajectory and send it to the controller system; and then, through robot positioning, move to the pipe thread measurement position to perform the measurement work.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the pipe thread positioning and measurement method based on three-dimensional vision as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the pipe thread positioning and measurement method based on three-dimensional vision as described in any one of claims 1-6.

Images