Pipe thread positioning and measuring method and system based on three-dimensional vision

Abstract

The invention provides a pipe thread positioning and measuring method and system based on three-dimensional vision, and belongs to the technical field of industrial automation and three-dimensional vision. According to the method, a measuring system composed of a robot measuring system and a three-dimensional visual locating system is built, and hand-eye calibration is completed; the method comprises the following steps: acquiring a three-dimensional point cloud model of the end part of a pipe thread workpiece, and extracting an end surface normal vector, a tangent vector and a circle center coordinate through preprocessing, cylinder fitting, near-end surface point cloud extraction and end surface circle fitting so as to establish a measurement reference coordinate system; and converting the reference coordinate system to a robot base coordinate system based on a hand-eye calibration result and the tail end pose of the robot, generating a robot moving trajectory, guiding the robot to be positioned to a measurement position, and completing preparation before measurement. According to the invention, automatic high-precision extraction of the pipe thread measurement reference is realized, the problems of non-uniform reference, large error and low efficiency in traditional measurement are solved, the method is suitable for various pipe thread workpieces, and a reliable basis is provided for subsequent high-precision measurement.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial automation and three-dimensional vision, and particularly relates to a pipe thread positioning and measuring method and system based on three-dimensional vision. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Pipe threads are widely used in the connection of pipelines in the fields of petroleum, chemical industry, and mechanical manufacturing, and the machining precision thereof directly affects the sealing performance and connection reliability of the pipelines. In order to ensure the quality of thread connection, it is usually necessary to perform high-precision measurement on the internal thread geometric parameters (such as pitch, thread angle, taper, etc.). However, internal thread measurement often faces two key challenges: firstly, the measurement probe or vision device needs to be accurately aligned with the thread axis; and secondly, the coordinate system of the measurement data must be consistent with the actual geometric position of the thread, otherwise error accumulation will occur, affecting the reliability of the measurement results.

[0004] Therefore, before performing internal thread measurement, it is necessary to accurately obtain the normal vector and center coordinates of the pipe thread end face, and to establish a coordinate system with the thread axis as the reference, which is a prerequisite for realizing high-precision and high-stability internal thread measurement. This reference coordinate is not only used for the attitude adjustment and path planning of the probe, but also serves as the core reference for subsequent measurement data analysis and thread parameter calculation. However, due to installation errors such as eccentricity and inclination of the pipe material during machining clamping or transportation, if thread measurement is directly performed without coordinate calibration, it is easy to cause the measurement reference to be inconsistent, resulting in inaccurate parameter extraction. Especially when using a three-dimensional vision or robot measurement system, the coordinate system of the measurement system often does not coincide with the actual geometry of the workpiece, which further increases the measurement error. SUMMARY

[0005] To overcome the deficiencies of the prior art, the present application provides a pipe thread positioning and measuring method and system based on three-dimensional vision, which extracts the pipe thread end face normal vector, tangent vector, and center coordinates based on three-dimensional point cloud data, combines spatial fitting algorithm and coordinate transformation technology, establishes a measurement reference coordinate system, and provides key technical support for subsequent high-precision automatic measurement of thread parameters, effectively solving the problems of difficult reference acquisition, low registration accuracy, and large measurement error in the existing measurement process.

[0006] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions: The first aspect of the present application provides a pipe thread positioning and measuring method based on three-dimensional vision; A pipe thread positioning and measuring method based on three-dimensional vision, comprising: A pipe thread measurement system based on three-dimensional vision and robot is built, and a hand-eye calibration is performed; A multi-angle scanning is performed on the pipe thread end by the built pipe thread measurement system to obtain complete three-dimensional point cloud data, in which the position information of each joint of the robot is recorded in real time during the scanning process; The obtained point cloud data is preprocessed, and the preprocessed point cloud is fitted with a cylindrical model; according to the fitting result of the cylinder, a near-end surface point cloud region is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, and two-dimensional circle fitting is performed to obtain three key geometric parameters of the center coordinates, normal vector and tangent vector; According to the parameters of the hand-eye calibration, the real-time motion pose of the robot and the three-dimensional point cloud data are combined to convert the pipe thread measurement reference coordinate system to the robot base coordinate system to obtain the pose of the target point in the robot base coordinate system; The obtained pose of the target point in the robot base coordinate system is converted into a robot running track and sent to the controller system, and the robot is positioned and moved to the pipe thread measurement position for measurement work.

[0007] As a further technical solution, the pipe thread measurement system based on three-dimensional vision and robot includes a robot measurement system and a three-dimensional vision positioning system; The robot measurement system includes a robot arm, a robot arm controller and a measurement equipment; The three-dimensional vision positioning system includes a three-dimensional vision scanning device and a display interaction system; the three-dimensional vision scanning device is used to obtain workpiece point cloud; the display interaction system is used for image display, image processing and man-machine interaction.

[0008] As a further technical solution, the process of hand-eye calibration is as follows: The calibration board is fixed in the robot workspace to ensure that it is within the effective observation range of the three-dimensional vision scanning device; the camera is fixed to the end effector of the robot, and the eye-in-hand mounting method is adopted; The robot is controlled to drive the end effector to move the camera in different poses, so that the camera collects images of the calibration board from multiple angles; For each collection point, the pose parameters of the calibration board in the rubber coordinate system and the pose parameters of the end effector in the robot base coordinate system are recorded synchronously, which are used for solving the hand-eye transformation relationship subsequently.

[0009] As a further technical solution, the preprocessed point cloud data includes: The point cloud data is reduced by voxel grid downsampling to retain the overall shape features; Statistical filtering is used to remove outliers and noise points to improve the quality of the point cloud; The effective area of the main thread end face is extracted by using the Euclidean clustering algorithm.

[0010] As a further technical solution, the pre-processed point cloud is fitted with a cylindrical model, including: The RANSAC algorithm is used to iteratively fit the cylindrical model to obtain a preliminary estimated cylindrical axis, radius and fitting inlier set.

[0011] As a further technical solution, according to the cylindrical fitting result, the near-end surface point cloud area is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis to perform two-dimensional circle fitting to obtain three key geometric parameters including the center coordinates, normal vector and tangent vector. On the basis of the cylindrical fitting result, a projection plane perpendicular to the direction of the cylindrical axis is set, and a small section of point cloud located at the end is selected according to the projection distance of the point cloud in the axial direction to obtain the near-end surface annular point cloud; The extracted near-end surface annular point cloud is projected onto a plane perpendicular to the main shaft, and the least square method is used for two-dimensional circle fitting to obtain three key geometric parameters including the center coordinates, normal vector and tangent vector.

[0012] As a further technical solution, according to the parameters of hand-eye calibration, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system in combination with the real-time motion pose of the robot and the three-dimensional point cloud data to obtain the pose of the target point in the robot base coordinate system, including: The fixed transformation relationship between the camera coordinate system and the robot end coordinate system is obtained by the Tsai-Lenz calibration algorithm, which is recorded as the first homogeneous transformation matrix; when the three-dimensional visual scanning device collects the pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded synchronously, and the second homogeneous transformation matrix is constructed; The center coordinates, normal vector and tangent vector obtained are converted from the camera coordinate system to the robot end coordinate system in combination with the first homogeneous transformation matrix and the second homogeneous transformation matrix; the direction vector Z axis and X axis are converted to the robot base coordinate system, the Y axis vector is constructed by cross multiplication, and the SVD decomposition is used for orthogonalization processing to obtain the final rotation matrix; The final rotation matrix is subjected to ZYX Euler angle decomposition to extract the rotation angles of the Z, Y and X axes in the base coordinate system, and the six-dimensional pose of the target point in the robot base coordinate system is formed by combining with the position vector.

[0013] The second aspect of the present application provides a pipe thread positioning and measuring system based on three-dimensional vision.

[0014] A pipe thread positioning and measuring system based on three-dimensional vision, including: The measurement system building module is configured to build a pipe thread measurement system based on three-dimensional vision and a robot, and to perform hand-eye calibration. The point cloud data acquisition module is configured to perform multi-angle scanning on the pipe thread end through the built pipe thread measurement system to acquire complete three-dimensional point cloud data, wherein the position information of each joint of the robot is recorded in real time during the scanning process. The point cloud data processing module is configured to pre-process the acquired point cloud data, and to perform cylindrical model fitting on the pre-processed point cloud; according to the cylindrical fitting result, a near-end surface point cloud area is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is performed, and three key geometric parameters of the center coordinates, normal vector and tangent vector are obtained. The coordinate conversion module is configured to convert the pipe thread measurement reference coordinate system to the robot base coordinate system according to the parameters of the hand-eye calibration, in combination with the real-time motion pose of the robot and the three-dimensional point cloud data, to obtain the pose of the target point in the robot base coordinate system. The path planning module is configured to convert the obtained pose of the target point in the robot base coordinate system into a robot running track and send it to the controller system, so as to reach the pipe thread measurement position through robot positioning and move to perform measurement work.

[0015] The third aspect of the present application provides a computer readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a pipe thread positioning and measurement method based on three-dimensional vision as described in the first aspect of the present application.

[0016] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements the steps of a pipe thread positioning and measurement method based on three-dimensional vision as described in the first aspect of the present application when executing the program.

[0017] The above one or more technical solutions have the following beneficial effects: (1) The present application combines three-dimensional vision system and industrial robot technology, accurately extracts measurement reference information through processing and circle fitting of pipe thread end surface point cloud, and automatically completes the pose conversion from the camera coordinate system to the robot base coordinate system, so as to obtain the spatial pose of the target without manual intervention, and realize the automation and high efficiency of pipe thread detection task.

[0018] (2) Through the high-precision three-dimensional vision system, the robust end surface circle fitting algorithm and the accurate robot hand-eye calibration method, the present application can accurately obtain the center and normal of the pipe thread end surface, realize the real-time conversion of the measurement reference coordinate system in the robot base coordinate system, and ensure the high precision and high repeatability of the detection positioning, thereby providing a reliable foundation for subsequent automatic processing or positioning operation.

[0019] (3) The application has good versatility and robustness, is suitable for pipe thread workpieces of different forms, sizes and mounting modes, and is especially suitable for flexible detection requirements of small and medium batches and multiple varieties, thereby reducing the dependence of a traditional detection system on workpiece consistency.

[0020] Advantages of the additional aspects of the application will be partially given in the following description, partially will become obvious from the following description, or will be understood by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated herein by reference. The illustrations are shown for the purpose of enabling those skilled in the art to implement the application and are not intended to limit the present application in any way.

[0022] Figure 1 A method flowchart of the first embodiment.

[0023] Figure 2 A system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0024] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs.

[0025] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the application.

[0026] The embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0027] Embodiment one The embodiment discloses a pipe thread positioning and measuring method based on three-dimensional vision; As shown in Figure 1 A pipe thread positioning and measuring method based on three-dimensional vision, comprising: Step S1, a pipe thread measuring system based on three-dimensional vision and robot is built, and hand-eye calibration is performed; Step S2, the pipe thread end is scanned at multiple angles by the built pipe thread measuring system, and complete three-dimensional point cloud data is obtained, wherein the joint position information of the robot is recorded in real time during the scanning process; Step S3, the acquired point cloud data is preprocessed, and the preprocessed point cloud is fitted with a cylindrical model; according to the cylindrical fitting result, a near-end surface point cloud region is extracted; the extracted end surface point cloud is projected to a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is performed, and three key geometric parameters of the center coordinates, normal vector and tangent vector are obtained; Step S4, according to the parameters of the hand-eye calibration, combining the real-time motion pose of the robot with the three-dimensional point cloud data, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system, and the pose of the target point in the robot base coordinate system is obtained; Step S5, the obtained pose of the target point in the robot base coordinate system is converted to form a robot running track and sent to the controller system, and through the robot positioning, the pipe thread measurement position is reached for measurement work.

[0028] Specifically, the above steps further include the following contents: Step S1, a pipe thread measurement system based on three-dimensional vision and robot is built, and hand-eye calibration is performed.

[0029] In step S1, the pipe thread measurement system based on three-dimensional vision and robot built contains two parts of a robot measurement system and a three-dimensional vision positioning system.

[0030] Among them, the robot measurement system includes a robot arm, a robot arm controller and a measurement equipment. In this embodiment, the robot measurement system composition device is not limited to use multiple brands of robot arms and measurement devices with similar principles and functions according to different measurement workpieces.

[0031] The three-dimensional vision positioning system includes two parts of a three-dimensional vision scanning device and a display interaction system. The three-dimensional vision scanning device needs to have the function of acquiring workpiece point cloud, its measurement accuracy is higher than 1mm, the depth map frame rate is greater than 1 frame per second, and a high-precision 3D surface structure light camera can be used, but it is not limited to use multiple brands, multiple structures, three-dimensional vision scanning devices with similar principles and same functions, The display interaction system needs to have the functions of image display, image processing and man-machine interaction. The image display device includes but is not limited to a touch display, a projector and the like, the image processing device includes but is not limited to a computer with image processing function, a tablet computer and the like, and the man-machine interaction device includes but is not limited to a mouse, a touch screen, voice recognition and the like.

[0032] Further, in the process of hand-eye calibration of the measurement system, the combination of the robot arm and the three-dimensional visual scanning device includes that the three-dimensional visual scanning device is installed on the robot arm (eye on hand) and the three-dimensional visual scanning device is installed outside the robot arm (eye outside hand). According to the requirements of pipe thread measurement and high precision, the eye-on-hand mode is selected in the embodiment and the hand-eye calibration is performed. The specific process of hand-eye calibration is as follows: By fixing a calibration board, ensuring that it is visible in the robot workspace, the camera is installed at the end of the robot, i.e. eye on hand.

[0033] The robot is controlled to move in different poses, so that the camera observes the calibration board from multiple angles, a total of 20 point positions are shot. Each time the calibration board pose in the camera coordinate system is recorded , and the pose of the end effector in the robot base coordinate system .

[0034] In step S2, the pipe thread end is scanned by the pipe thread measurement system to obtain complete three-dimensional point cloud data, and the joint position information of the robot is recorded in real time during the scanning process.

[0035] In step S2, the three-dimensional model of the pipe thread workpiece end is obtained by using the three-dimensional visual scanning device, the pipe thread end surface point cloud data is obtained by using the three-dimensional model of the pipe thread workpiece end, and the relationship between the workpiece coordinate system and the robot coordinate system is obtained by recording the six-dimensional pose of the robot end in the robot base coordinate system in real time during the process of obtaining the point cloud data. During the process of obtaining the point cloud data, if the three-dimensional point cloud of the workpiece end cannot be obtained by one scanning, the three-dimensional visual scanning device can be used for multiple scanning, and the complete three-dimensional point cloud model of the workpiece end can be obtained by using point cloud splicing, registration, reverse engineering and other methods.

[0036] In addition, in the process of obtaining the three-dimensional model of the pipe thread workpiece end, if there is a CAD model, it can also be used as a reference template for registration and correction with the actual scanning data; or the current end surface point cloud data can be calculated by using the historical model data and the current pose transformation relationship.

[0037] In step S3, the obtained point cloud data is preprocessed, and the preprocessed point cloud is fitted with a cylindrical model; according to the cylindrical fitting result, the near-end surface point cloud region is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, and two-dimensional circle fitting is performed to obtain three key geometric parameters of the center coordinates, normal vector and tangent vector.

[0038] Step S31, the acquired pipe thread end face point cloud data is preprocessed to improve the subsequent fitting accuracy and robustness. Specifically, first, the voxel grid sampling method is used to downsample the point cloud, the basic principle of which is to divide the point cloud space into regular cubic grids, and to represent each voxel with the geometric center or average point of the points in the voxel, thereby effectively reducing the amount of point cloud data while maintaining the overall geometric shape; then, a statistical filtering method is used to remove noise, which identifies and removes outliers by calculating the average distance of a point and its neighboring points and setting a threshold, in order to eliminate isolated points and random noise; further, a clustering segmentation method based on Euclidean distance is used, which divides the point cloud into several sub-clusters according to the spatial distance relationship between points, so as to accurately extract the main area of the thread end face and remove irrelevant point clouds. After the above multi-level preprocessing, the obtained point cloud not only retains the key geometric features of the thread end face, but also significantly improves the data quality and compactness, laying a reliable foundation for subsequent cylindrical model fitting and principal axis extraction.

[0039] The voxel grid is downsampled to reduce the amount of point cloud data and preserve the overall shape features; statistical filtering is used to remove outliers and noise points to improve the quality of the point cloud; Euclidean clustering is used to extract the main area and retain the main thread end face area while removing scattered small clusters.

[0040] Step S32, cylindrical model fitting is performed on the preprocessed point cloud to preliminarily obtain the principal axis direction and cylindrical parameters of the pipe body. In this embodiment, the random sample consensus (RANSAC) method is used for cylindrical model fitting. The process is as follows: First, a cylindrical model is constructed, with the axis of the cylinder being a straight line:

[0041] wherein, is a point on the axis, is a unit directional vector, is the radius.

[0042] A small number of points are randomly sampled from the point cloud to construct candidate cylindrical axis parameters and radius, and the distance of any other point to the cylindrical axis is calculated:

[0043] If the difference between the distance and the radius is less than a predetermined threshold, it is determined to be an inlier. By repeatedly iterating the sampling, the number of inliers is counted, and the model with the most inliers is selected as the optimal cylinder; the inlier set of the model is used for parameter optimization to obtain the geometric parameters of the axis direction, center position and radius of the cylinder. Through the above method, the spatial position and size information of the cylinder can be obtained robustly in the presence of noise and outliers.

[0044] Step S33, according to the cylindrical fitting result, the pipe thread end surface point cloud close to the camera or the specified measurement direction is extracted as the subsequent circle fitting area.

[0045] Specifically, a projection plane perpendicular to the direction of the cylindrical axis is set; a small section of point cloud located at the end (such as ±5mm range) is filtered according to the projection distance of the point cloud in the axial direction, and then a near-end surface annular point cloud is obtained.

[0046] Step S34, projecting the extracted end surface point cloud to the two-dimensional plane perpendicular to the fitting axis, and performing two-dimensional circle fitting by using the least square method to obtain the center coordinates and radius of the two-dimensional circle in the projection plane; Specifically, after projecting the annular point cloud to the main shaft vertical plane, the least square method is used for circle fitting. The mathematical model is:

[0047] For the projection point set , an error function is established by the least square method:

[0048] The above problem is converted into a matrix form:

[0049] The optimal solution parameters ( ) can be obtained by the following formula:

[0050] And then the center of the circle ( ) and the radius :

[0051]

[0052] On this basis, the normal vector is taken as the normal of the projection plane, that is, corresponding to the direction of the cylindrical main shaft; the tangent vector is obtained by rotating the connecting line between the center and a point on the circumference by ninety degrees. This method can efficiently obtain the three key geometric parameters of the center, the normal vector and the tangent vector.

[0053] Further, a local coordinate system is established in the vertical plane of the cylindrical axis, and the two-dimensional center coordinates are mapped back to the three-dimensional space through the linear combination of the basis vectors of the coordinate system, so as to deduce the center coordinates, the end surface normal vector and the tangent vector in the three-dimensional space as the key geometric parameters of the pipe thread end surface. The end surface normal vector is taken as the normal of the projection plane, that is, corresponding to the direction of the cylindrical main shaft; the tangent vector is obtained by rotating the connecting line between the center and a point on the circumference by ninety degrees.

[0054] Step S4, according to the parameters of hand-eye calibration, combining the real-time motion pose of the robot with the three-dimensional point cloud data, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system, and the pose of the target point in the robot base coordinate system is obtained.

[0055] Step S41, the fixed transformation relationship between the camera coordinate system and the robot end coordinate system is obtained by Tsai-Lenz calibration method, denoted as the first homogeneous transformation matrix .

[0056] When the three-dimensional vision scanning device collects the pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded synchronously, including three-dimensional position coordinates and ZYX Euler angles, and the second homogeneous transformation matrix .

[0057] The core of Tsai-Lenz algorithm is to solve the following equation:

[0058] Wherein is the fixed transformation matrix between the camera coordinate system and the end effector coordinate system. Then, the homogeneous transformation matrix can be split into rotation matrix and translation vector two parts:

[0059] Wherein, represents the rotation relationship of the camera coordinate system to the end effector coordinate system, represents the translation relationship between the two coordinate systems. Substituting it into the above formula, we can get: ,

[0060] Wherein is the rotation matrix part of , which represents the rotation relationship of the camera coordinate system relative to the calibration board, is the corresponding translation vector part. is the rotation matrix of the pose change of the robot base coordinate system to the end effector, is the corresponding translation vector part.

[0061] Step S42, taking the center coordinates as the target point position, and determining its normal vector (defined as target Z axis) and tangent vector (defined as target X axis), which together construct the target measurement reference coordinate system (right-handed system XYZ).

[0062] First, the target point and its normal vector and tangent vector are transformed from the camera coordinate system to the end coordinate system:

[0063] wherein, is the end coordinate system; is the camera coordinate system; and further convert it to the robot base coordinate system under the current transformation matrix of the robot end:

[0064] wherein, is the robot base coordinate system.

[0065] Step S43, convert the direction vector (Z axis, X axis) to the robot base coordinate system under the above transformation mode, construct the Y axis vector through the cross product, and form the orthogonal right-hand system rotation matrix. In order to ensure the orthogonality of the rotation matrix, orthogonalization processing is performed using SVD decomposition, and the final rotation matrix is obtained .

[0066] Specifically, the process of orthogonalization processing using SVD decomposition is as follows: Construct the base vector matrix: ; Perform singular value decomposition on the base vector matrix :

[0067] Construct the rotation matrix:

[0068] If , adjust the sign of to ensure that the final rotation matrix satisfies:

[0069] Through this process, the orthogonality and direction consistency of the rotation matrix can be guaranteed, thereby realizing the accuracy of coordinate transformation.

[0070] Step S44, perform ZYX Euler angle decomposition on the rotation matrix , and extract the rotation angles of the Z, Y, and X axes in the base coordinate system, that is, the Euler angles of the target pose. Combine with the position vector to form the complete six-dimensional pose of the target point in the robot base coordinate system.

[0071] Specifically, the final rotation matrix obtained is defined as:

[0072] wherein,​ The element in the first row and the first column of the rotation matrix. The element in the first row and the second column of the rotation matrix. The element in the second row and the first column of the rotation matrix.

[0073] According to the ZYX Euler angle definition, we have:

[0074] If (i.e., a singular pose occurs), a backup formula is used to handle the singular situation. That is:

[0075] When , the rotation angle around the Z-axis and the rotation angle around the X-axis are calculated according to the following formula:

[0076] If (i.e., a singular pose occurs), according to the relationship of the rotation matrix, one of the angles can be selected as zero (or remain the original value), and then the unique solution is obtained from the other angle.

[0077] If :

[0078] If :

[0079] Through the above method, the uncertainty caused by the singular pose can be eliminated, and the calculation of the rotation angle is stable and continuous.

[0080] Finally, by combining the position vector in the base coordinate system with the ZYX Euler angle , the six-dimensional pose of the target point in the robot base coordinate system can be obtained:

[0081] Step S5, the obtained pose of the target point in the robot base coordinate system is converted to form a robot running trajectory and sent to a controller system, and the robot is positioned and moved to the pipe thread measurement position for measurement work by using the controller system.

[0082] Embodiment Two The embodiment discloses a pipe thread positioning and measuring system based on three-dimensional vision. As shown in Figure 2 , a pipe thread positioning and measuring system based on three-dimensional vision comprises: The measurement system building module is configured to build a pipe thread measurement system based on three-dimensional vision and a robot, and perform hand-eye calibration. The point cloud data acquisition module is configured to perform multi-angle scanning on the pipe thread end through the built pipe thread measurement system to acquire complete three-dimensional point cloud data, wherein the position information of each joint of the robot is recorded in real time during the scanning process. The point cloud data processing module is configured to pre-process the acquired point cloud data, and perform cylindrical model fitting on the pre-processed point cloud; according to the cylindrical fitting result, extract the near-end surface point cloud region; project the extracted end surface point cloud to a two-dimensional plane perpendicular to the fitted axis, perform two-dimensional circle fitting, and obtain three key geometric parameters of the center coordinates, normal vector and tangent vector. The coordinate conversion module is configured to convert the pipe thread measurement reference coordinate system to the robot base coordinate system according to the parameters of the hand-eye calibration, combined with the real-time motion pose of the robot and the three-dimensional point cloud data, to obtain the pose of the target point in the robot base coordinate system. The path planning module is configured to convert the obtained pose of the target point in the robot base coordinate system into a robot running track and send it to the controller system, and through robot positioning, move to the pipe thread measurement position to perform measurement work.

[0083] Embodiment three The purpose of this embodiment is to provide a computer-readable storage medium.

[0084] A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of a pipe thread positioning and measurement method based on three-dimensional vision as described in embodiment 1.

[0085] Embodiment four The purpose of this embodiment is to provide an electronic device.

[0086] An electronic device includes a memory, a processor, and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps of a pipe thread positioning and measurement method based on three-dimensional vision as described in embodiment 1.

[0087] The steps and methods involved in the above embodiments two, three and four correspond to embodiment one, and the specific implementation can be referred to the relevant description part of embodiment one. The term "computer-readable storage medium" should be understood to include a single medium or multiple media of one or more instruction sets; it should also be understood to include any medium that can store, encode or carry instruction sets for execution by a processor and make the processor execute any method in the present application.

[0088] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computer devices, or alternatively, they can be realized by program codes executable by the computer devices, so that they can be stored in the storage devices and executed by the computer devices, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0089] The specific embodiments of the present application described above in conjunction with the accompanying drawings are not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for pipe thread positioning and measurement based on three-dimensional vision, characterized in that, The application relates to a pipe thread measurement system based on three-dimensional vision and a robot. The application comprises the following steps: A pipe thread measurement system based on three-dimensional vision and a robot is built, and hand-eye calibration is carried out; A pipe thread end is scanned at multiple angles through the built pipe thread measurement system to obtain complete three-dimensional point cloud data, and the position information of each joint of the robot is recorded in real time during the scanning process; The obtained point cloud data is preprocessed, and the preprocessed point cloud is subjected to cylinder model fitting; according to the cylinder fitting result, a near-end surface point cloud area is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is carried out, and three key geometric parameters, namely a circle center coordinate, a normal vector and a tangent vector, are obtained; According to the parameters of the hand-eye calibration, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system in combination with the real-time motion posture of the robot and the three-dimensional point cloud data, so that the posture of the target point in the robot base coordinate system is obtained; 2. A method of positioning and measuring a tubular thread based on three-dimensional vision as claimed in claim 1, characterized in that, The posture of the target point in the robot base coordinate system is converted into a robot running track and is sent to a controller system, and the robot is positioned and moved to the pipe thread measurement position to carry out measurement work. The pipe thread measurement system based on three-dimensional vision and a robot comprises a robot measurement system and a three-dimensional vision positioning system; The robot measurement system comprises a robot arm, a robot arm controller and a measurement device; 3. A method of positioning and measuring a pipe thread based on three-dimensional vision as claimed in claim 1, characterized in that, The three-dimensional vision positioning system comprises a three-dimensional vision scanning device and a display interaction system; the three-dimensional vision scanning device is used for acquiring workpiece point cloud; and the display interaction system is used for image display, image processing and man-machine interaction. The process of the hand-eye calibration comprises the following steps: A calibration board is fixed in the robot workspace to ensure that the calibration board is in the effective observation range of the three-dimensional vision scanning device; a camera is fixed to the end effector of the robot in an eye-in-hand mode; The robot is controlled to drive the end effector to move the camera at different postures, so that the camera collects images of the calibration board from multiple angles; 4. A method of positioning and measuring a tubular thread based on three-dimensional vision as claimed in claim 1, characterized in that, For each collected point, the posture parameters of the calibration board in the rubber coordinate system and the posture parameters of the end effector in the robot base coordinate system are recorded synchronously, and are used for solving the hand-eye transformation relationship subsequently. The preprocessed point cloud data is subjected to cylinder model fitting, which comprises the following steps: An RANSAC algorithm is used to iteratively fit a cylinder model, so that an estimated cylinder axis, a radius and a fitting in-point set are obtained. According to the cylinder fitting result, a near-end surface point cloud area is extracted; the extracted end surface point cloud is projected onto a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is carried out, and three key geometric parameters, namely a circle center coordinate, a normal vector and a tangent vector, are obtained.

5. A method of positioning and measuring a tubular thread based on three-dimensional vision as claimed in claim 1, characterized in that, On the basis of the cylinder fitting result, a projection plane perpendicular to the direction of the cylinder axis is set, a small amount of point cloud located at the end is selected according to the projection distance of the point cloud in the axial direction, and a near-end surface ring-shaped point cloud is obtained. ​ 6. A method of positioning and measuring a tubular thread based on three-dimensional vision as claimed in claim 1, characterized in that, ​ ​ The intercepted proximal end surface annular point cloud is projected to a plane perpendicular to the main shaft, and three key geometric parameters of the center coordinates, normal vector and tangent vector are obtained by two-dimensional circle fitting using the least square method.

7. A method of positioning and measuring a tubular thread based on three-dimensional vision as claimed in claim 1, characterized in that, According to the parameters of the hand-eye calibration, the real-time motion pose of the robot and the three-dimensional point cloud data are combined, the pipe thread measurement reference coordinate system is converted to the robot base coordinate system, and the pose of the target point in the robot base coordinate system is obtained; A fixed transformation relationship between the camera coordinate system and the robot end coordinate system is obtained by the Tsai-Lenz calibration algorithm, and is recorded as the first homogeneous transformation matrix; when the three-dimensional visual scanning device collects the pipe thread point cloud data, the six-dimensional pose parameters of the robot end effector in the robot base coordinate system are recorded synchronously, and the second homogeneous transformation matrix is constructed; The first homogeneous transformation matrix and the second homogeneous transformation matrix are combined, the obtained center coordinates, normal vector and tangent vector are converted from the camera coordinate system to the robot end coordinate system; the direction vector Z axis and X axis are converted to the robot base coordinate system, the Y axis vector is constructed by cross multiplication, and the final rotation matrix is obtained by orthogonalization processing using SVD decomposition; The final rotation matrix is decomposed into ZYX Euler angles, the rotation angles around the Z, Y and X axes in the base coordinate system are extracted, and the six-dimensional pose of the target point in the robot base coordinate system is formed by combining with the position vector.

8. A three-dimensional vision-based pipe thread positioning and measurement system, characterized by: It comprises: A measurement system building module configured to build a pipe thread measurement system based on three-dimensional vision and a robot, and to perform hand-eye calibration; A point cloud data acquisition module configured to perform multi-angle scanning on the pipe thread end through the built pipe thread measurement system to acquire complete three-dimensional point cloud data, wherein the joint position information of the robot is recorded in real time during the scanning process; A point cloud data processing module configured to preprocess the acquired point cloud data, and to perform cylindrical model fitting on the preprocessed point cloud; according to the cylindrical fitting result, the proximal end surface point cloud region is extracted; the extracted end surface point cloud is projected to a two-dimensional plane perpendicular to the fitting axis, two-dimensional circle fitting is performed, and three key geometric parameters of the center coordinates, normal vector and tangent vector are obtained; A coordinate conversion module configured to convert the pipe thread measurement reference coordinate system to the robot base coordinate system according to the parameters of the hand-eye calibration, combine the real-time motion pose of the robot with the three-dimensional point cloud data, and obtain the pose of the target point in the robot base coordinate system; A path planning module configured to convert the obtained pose of the target point in the robot base coordinate system into a robot running track and send it to a controller system, and move to the pipe thread measurement position for measurement work through robot positioning.

9. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to realize the steps in the pipe thread positioning and measurement method based on three-dimensional vision according to any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps in the pipe thread positioning and measurement method based on three-dimensional vision according to any one of claims 1-7.

Images