Screw / thread bushing mounting method based on visual guidance

Abstract

The invention discloses a screw / thread bushing mounting method based on visual guidance. A visual guidance sensor and a mounting machine are mounted at the tail end of a robot at the same time; the robot moves to the pose1 to collect the point cloud I and then moves to the pose2 to collect the point cloud II; splicing the point cloud I and the point cloud II; segmenting a thread inner wall point cloud and a plane point cloud from the point cloud; fitting the central axis L1 of the cylinder by using the thread inner wall point cloud; fitting a plane by using the plane point cloud, and obtaining an intersection point coordinate P1 of the central axis L1 and the plane; respectively aligning the cylinder central axis L1 and the intersection point coordinate P1 with the standard central axis L0 and the standard intersection point coordinate P0 to obtain an offset matrix; correcting the mounting pose of the robot based on the offset matrix; according to the method, threaded hole point clouds are obtained through symmetrically-arranged collection poses; thread inner wall point cloud is obtained through point cloud segmentation, then the axial direction and the hole center position of a current workpiece threaded hole are accurately obtained, a robot is guided to smoothly complete automatic assembly, and the assembly precision is improved.

Description

Technical Field

[0001] This invention relates to the field of visual guidance, and more specifically to a visual guidance-based screw / threaded sleeve installation method. Background Technology

[0002] Automatic thread insert installation refers to a process in which a robot end effector, based on a pre-taught thread insert trajectory and installation posture, inserts a thread insert (such as a wire thread insert or a self-tapping thread insert) into a threaded hole to enhance thread strength, repair damaged threads, or improve connection performance.

[0003] The automated screw-planting process refers to the process where a robot uses an end effector to install a screw-tightening gun, based on a pre-taught thread-planting trajectory and installation posture, to install a screw into a threaded hole.

[0004] Due to inherent deviations in the machining of threaded holes on workpieces, the center position and axis of the threaded holes vary across different workpieces. Relying solely on the robot's taught installation posture can easily lead to a mismatch between the screw / threaded sleeve installation posture and the current threaded hole posture, resulting in assembly failure. Therefore, both automated threaded sleeve installation and automated screw installation processes require accurate acquisition of the current threaded hole position and thread axial direction.

[0005] Because threaded holes have small diameters and large depths, the point cloud data inside the thread is easily incomplete. Furthermore, the collected point cloud contains a mixture of point clouds from the inner thread wall and planar point clouds, making segmentation difficult. These factors all contribute to inaccurate acquisition of the threaded hole's position and the thread's axial direction. Existing methods use the normal direction of the thread's upper surface to replace the axial direction of the inner thread wall. However, in actual production, due to instability in preceding processes, the axial direction of the inner thread wall does not completely coincide with the normal direction of the planar direction, and the deviation increases with the depth of the threaded hole. Therefore, this method cannot effectively guide the accurate installation of threaded sleeves / screws. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a vision-guided screw / threaded sleeve installation method. The method utilizes symmetrically set acquisition poses to obtain point clouds of threaded holes, which are then stitched together to obtain a complete point cloud of the threaded holes. The point cloud is then segmented to obtain the point cloud of the inner wall of the thread, thereby accurately obtaining the axial direction and center position of the threaded hole in the current workpiece. This guides the robot to successfully complete automatic assembly, improving assembly accuracy.

[0007] The technical solution is as follows:

[0008] A vision-guided screw / threaded sleeve installation method involves simultaneously installing a vision-guided sensor and an installation machine at the end effector of a robot, with their relative positions fixed; the installation machine is either a screw tightening gun or a threaded sleeve installation machine.

[0009] The installation method includes the following steps:

[0010] 1) The robot moves to the acquisition pose 1 according to the taught motion trajectory, and the visual guidance sensor acquires the point cloud I of the area where the threaded hole is located. Then it moves to the acquisition pose 2, and the visual guidance sensor acquires the point cloud II of the area where the threaded hole is located. The acquisition pose 1 and acquisition pose 2 are approximately symmetrical about the theoretical central axis of the threaded hole.

[0011] Based on the pre-acquired conversion relationship between the acquisition pose 1 and the acquisition pose 2, point cloud I and point cloud II are stitched together to obtain the point cloud to be processed.

[0012] 2) Segment the point cloud of the inner wall of the thread and the point cloud of the plane containing the threaded hole from the point cloud to be processed;

[0013] The cylinder's central axis L1 is obtained by fitting the point cloud of the inner wall of the thread; the intersection point P1 of the cylinder's central axis L1 and the fitted plane is obtained by fitting the plane with the planar point cloud.

[0014] 3) Align the cylinder's central axis L1 and intersection point P1 with the pre-stored standard central axis L0 and standard intersection point P0 respectively, and obtain the rotation and translation relationship required for alignment, denoted as the offset matrix;

[0015] The installation machine corrects the robot's installation pose based on the offset matrix, and then installs the screw / threaded sleeve into the threaded hole based on the corrected installation pose.

[0016] Furthermore, at the pose acquisition points pose1 and pose acquisition points pose2, the lateral distance between the visual guidance sensor and the theoretical center position of the threaded hole is... Longitudinal distance is ;

[0017] Where S is the optimal working distance of the sensor; , , H is the theoretical height of the threaded hole, h is the theoretical height of the smooth hole (located at the top of the threaded hole), D is the theoretical diameter of the smooth hole, and d is the theoretical diameter of the threaded hole.

[0018] Furthermore, the process of obtaining the pre-stored standard centerline L0 and standard intersection point coordinates P0 is as follows:

[0019] In the installation scenario of batch screws / threaded sleeves, the robot is taught the installation pose based on the threaded hole on the first workpiece. During this process, step 1) is executed to obtain the point cloud to be processed, and the point cloud of the inner wall of the thread and the point cloud of the plane where the threaded hole is located are segmented from the point cloud to be processed.

[0020] The cylinder's central axis is obtained by fitting the point cloud of the inner wall of the thread, and is denoted as the standard central axis L0. The intersection point coordinates of the standard central axis L0 and the fitted plane are obtained by fitting the plane with the point cloud of the plane, and are denoted as the standard intersection point coordinates P0.

[0021] Preferably, the method for pre-obtaining the transformation relationship between the acquisition pose 1 and the acquisition pose 2 is as follows:

[0022] S1. Set n non-collinear reflective markers around the threaded hole on the first workpiece, where n ranges from 3 to 10.

[0023] The robot moves to the acquisition pose 1 according to the taught motion trajectory, and the visual guidance sensor acquires the grayscale image A of the reflective marker. Then it moves to the acquisition pose 2 and the visual guidance sensor acquires the grayscale image B of the reflective marker.

[0024] S2. Perform the following processing on each reflective marker point:

[0025] Calculate the two-dimensional coordinates W of the reflective marker points in grayscale images A and B, respectively. A and W B ;

[0026] Obtain the two-dimensional coordinates W A The corresponding 3D coordinates W in point cloud I A ', in three-dimensional coordinates W A Construct a 3D bounding box centered on 'M', and fit the selected local point cloud to plane M. A ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor A 'Construct a spatial straight line, and then connect the spatial straight line with plane M' A The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. A '';

[0027] Obtain the two-dimensional coordinates W B The corresponding 3D coordinates W in point cloud II B ', in three-dimensional coordinates W B Construct a 3D bounding box centered on 'M', and fit the selected local point cloud to plane M. B ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor B 'Construct a spatial straight line, and then connect the spatial straight line with plane M' B The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. B '';

[0028] S3, using n pairs of three-dimensional coordinates W A '', three-dimensional coordinates W BPerform a rigid body transformation to obtain the transformation relationship between the acquired pose 'pose1' and the acquired pose 'pose2'.

[0029] Preferably, step 2) involves segmenting the point cloud of the inner wall of the thread and the point cloud of the plane containing the threaded hole from the point cloud to be processed, as follows:

[0030] ① Obtain the normal vector of each point in the point cloud to be processed;

[0031] ② Project all points in the point cloud to be processed onto a two-dimensional image plane to obtain a projected image. Round the sub-pixel points in the projected image, and then perform the following processing on each pixel P:

[0032] Construct an N×N selection box centered on pixel P. Among the N×N pixels selected, multiply the normal vectors of two pixels that belong to the same row and are respectively in the first and Nth columns. Take the average of the results of each multiplication to obtain the mean value I.

[0033] Next, multiply the normal vectors of two pixels belonging to the same column and respectively to the first and Nth rows, and take the mean of the multiplication results to obtain mean II;

[0034] The smaller of the mean I and the mean II is recorded as the normal rate of change. If the normal rate of change is less than the threshold, then point P is recorded as the foreground point; otherwise, point P is recorded as the background point.

[0035] ③ Store the 3D points corresponding to all foreground points in the point cloud of the inner wall of the thread, and store the 3D points corresponding to all background points in the point cloud of the plane where the threaded hole is located, thus completing the point cloud segmentation.

[0036] Preferably, it also includes step ④, obtaining the bounding box of the point cloud of the inner wall of the thread, setting the voxel side length to be smaller than the point cloud spacing, and voxelizing the bounding box;

[0037] Traverse all points in the point cloud of the inner wall of the thread, assign points to corresponding voxels based on their coordinates and voxel boundaries, and confirm the connectivity of the point cloud based on the connectivity of the voxels; divide the point cloud into multiple connected regions.

[0038] In the point cloud of the inner wall of the thread, connected regions with a number of point clouds less than a preset value are filtered out.

[0039] Preferably, the preset value is 20~50.

[0040] Preferably, step ① obtains the normal vectors of each point in the point cloud to be processed in the following way:

[0041] Iterate through all points in the point cloud to be processed, and perform the following processing on each point Q:

[0042] Find the k points closest to point Q and store them in a neighborhood set; find the centroid of the neighborhood set. );

[0043] Calculate matrix C: ; in,( Let be the three-dimensional coordinates of the i-th point in the neighborhood set;

[0044] Perform eigenvalue decomposition on matrix C to obtain three sets of eigenvalues ​​and eigenvectors. Take the eigenvector corresponding to the smallest eigenvalue, and denote it as vector C. ;

[0045] If F If the value is less than 0, then the vector V is inverted and denoted as the normal vector of point Q; otherwise, the vector V is directly denoted as the normal vector of point Q. Here, F represents the direction vector from point Q to point (0, 0, 0).

[0046] Preferably, k takes a value of 5 to 30.

[0047] Preferably, N takes the values ​​3, 5, or 7.

[0048] This method has the following characteristics:

[0049] 1. The point cloud of the threaded hole is obtained by using the symmetrically set acquisition pose. The point cloud is then stitched together to obtain a complete point cloud of the threaded hole. The point cloud is then segmented to obtain the point cloud of the inner wall of the thread, thereby accurately obtaining the axial direction and center position of the threaded hole of the current workpiece. This guides the robot to complete the automatic assembly smoothly and improves the assembly accuracy.

[0050] By limiting the position of the acquisition pose by the diameter and height of the threaded hole, the sensor can acquire as much point cloud as possible at the acquisition pose1 / acquisition pose2.

[0051] 2. Based on the characteristics of high rate of change of the normal vector of the thread point cloud and low curvature variable of the surrounding area, the point cloud of the inner wall of the thread and the planar point cloud containing the threaded hole are segmented from the point cloud to be processed. This eliminates the need for theoretical values ​​to segment the thread point cloud, is not constrained by workpiece machining deviations, and improves the segmentation accuracy of the thread point cloud. Furthermore, based on the connected components of the point cloud, noisy regions are further filtered out. Moreover, compared to existing methods that divide connected components through point cloud clustering, this method uses voxels for connected component filtering, which is more efficient. For a point cloud of 100,000, the clustering time is only 30ms, and it can be implemented on the CPU, exhibiting high efficiency and convenience. Attached Figure Description

[0052] Figure 1 A schematic diagram showing the setting of three reflective markers around a threaded hole;

[0053] Figure 2 This is a schematic diagram of the collected point cloud (I).

[0054] Figure 3This is a schematic diagram illustrating the relationship between the pose1 visual guidance sensor and the pose of the threaded hole.

[0055] Figure 4 A schematic diagram showing the relationship between the pose2 vision-guided sensor and the pose of the threaded hole;

[0056] Figure 5 The diagram within the selection box illustrates the calculation process for obtaining the mean I.

[0057] Figure 6 The diagram within the selection box illustrates the calculation process for obtaining Mean II. Detailed Implementation

[0058] The technical solution of the present invention will be described in detail below with reference to specific embodiments.

[0059] A vision-guided screw / threaded sleeve installation method involves simultaneously installing a vision-guided sensor and an installation machine at the end effector of a robot, with their relative positions fixed; the installation machine can be a screw tightening gun or a threaded sleeve installation machine.

[0060] The installation method includes the following steps:

[0061] 1) The robot moves to the acquisition pose 'pose1' according to the taught motion trajectory, and the vision-guided sensor acquires the point cloud 'I' of the area where the threaded hole is located (e.g., ...). Figure 2 Then move to the acquisition pose 2, and the visual guide sensor acquires the point cloud II of the area where the threaded hole is located; the acquisition pose 1 and acquisition pose 2 are roughly symmetrical about the theoretical central axis of the threaded hole (the difference between β1 and β2 is less than 10°, β1 is the angle between pose 1, the theoretical center of the threaded hole and the theoretical central axis, and β2 is the angle between pose 2, the theoretical center of the threaded hole and the theoretical central axis).

[0062] Based on the pre-acquired conversion relationship between the acquisition pose 1 and the acquisition pose 2, point cloud I and point cloud II are stitched together to obtain the point cloud to be processed.

[0063] 2) Segment the point cloud of the inner wall of the thread and the point cloud of the plane containing the threaded hole from the point cloud to be processed;

[0064] The cylinder's central axis L1 is obtained by fitting the point cloud of the inner wall of the thread; the intersection point P1 of the cylinder's central axis L1 and the fitted plane is obtained by fitting the plane with the planar point cloud.

[0065] 3) Align the cylinder's central axis L1 and intersection point P1 with the pre-stored standard central axis L0 and standard intersection point P0 respectively, and obtain the rotation and translation relationship required for alignment, denoted as the offset matrix;

[0066] The robot's installation pose is corrected based on the offset matrix, and the installation machine installs the screw / threaded sleeve into the threaded hole based on the corrected installation pose.

[0067] For example, in the installation scenario of batch screws / threaded sleeves, the robot is taught the installation pose based on the threaded hole on the first workpiece; that is, the first workpiece is placed at the workstation, and the robot trajectory is taught so that when the robot is in the installation pose, it can accurately install the screws / threaded sleeves into the threaded hole of the first workpiece.

[0068] During this process, step 1) is performed to obtain the point cloud to be processed, and the point cloud of the inner wall of the thread and the point cloud of the plane where the thread hole is located are segmented from the point cloud to be processed.

[0069] The cylinder's central axis is obtained by fitting the point cloud of the inner wall of the thread, and is denoted as the standard central axis L0. The intersection point coordinates of the standard central axis L0 and the fitted plane are obtained by fitting the plane with the point cloud of the plane, and are denoted as the standard intersection point coordinates P0.

[0070] When other workpieces are placed in batches at the workstation, screws / threaded sleeves are installed on the workpiece using steps 1) to 3).

[0071] As a preferred implementation, based on the first workpiece, the transformation relationship between the acquisition pose 1 and the acquisition pose 2 is obtained in advance, specifically as follows:

[0072] S1. Set n non-collinear reflective markers around the threaded hole on the first workpiece (e.g., ... Figure 1 n takes values ​​from 3 to 10;

[0073] The robot moves to the acquisition pose 1 according to the taught motion trajectory, and the visual guidance sensor acquires the grayscale image A of the reflective marker. Then it moves to the acquisition pose 2 and the visual guidance sensor acquires the grayscale image B of the reflective marker.

[0074] S2. Perform the following processing on each reflective marker point:

[0075] Calculate the two-dimensional coordinates W of the reflective marker points in grayscale images A and B, respectively. A and W B ;

[0076] Obtain the two-dimensional coordinates W A The corresponding 3D coordinates W in point cloud I A ', in three-dimensional coordinates W A Construct a 3D bounding box (cylinder / sphere) centered on a point cloud, and fit the selected local point cloud to plane M. A ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor A 'Construct a spatial straight line, and then connect the spatial straight line with plane M'A The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. A '';

[0077] Obtain the two-dimensional coordinates W B The corresponding 3D coordinates W in point cloud II B ', in three-dimensional coordinates W B Construct a 3D bounding box centered on 'M', and fit the selected local point cloud to plane M. B ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor B 'Construct a spatial straight line, and then connect the spatial straight line with plane M' B The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. B '';

[0078] S3, using n pairs of three-dimensional coordinates W A '', three-dimensional coordinates W B Perform a rigid body transformation to obtain the transformation relationship between the acquired pose 'pose1' and the acquired pose 'pose2'.

[0079] In order to acquire a more comprehensive thread point cloud, this solution imposes the following limitations on the acquisition pose1 / acquisition pose2:

[0080] At pose 1 / pose 2, the lateral distance between the visual guidance sensor and the theoretical center position of the threaded hole is... Longitudinal distance is ;

[0081] Where S is the optimal working distance of the sensor; , , H is the theoretical height of the threaded hole, h is the theoretical height of the smooth hole (located at the top of the threaded hole), D is the theoretical diameter of the smooth hole, and d is the theoretical diameter of the threaded hole.

[0082] like Figure 3 This is a schematic diagram showing the relationship between the pose1 visual guidance sensor and the pose of the threaded hole.

[0083] like Figure 4 This is a schematic diagram showing the relationship between the pose2 vision-guided sensor and the pose of the threaded hole.

[0084] To more accurately segment the point cloud of the inner wall of the thread, in step 2) of this embodiment, based on the characteristics of the high rate of change of the normal vector of the thread point cloud and the low curvature variable of the surrounding area, the point cloud of the inner wall of the thread and the plane point cloud where the threaded hole is located are segmented from the point cloud to be processed, as follows:

[0085] ① Obtain the normal vector of each point in the point cloud to be processed;

[0086] ② Project all points in the point cloud to be processed onto a two-dimensional image plane to obtain a projected image. Round the sub-pixel points in the projected image, and then perform the following processing on each pixel P:

[0087] Construct an N×N selection box centered on pixel P. Among the N×N pixels selected, multiply the normal vectors of two pixels that belong to the same row and are respectively in the first and Nth columns. Take the average of the results of each multiplication to obtain the mean value I. Typically, N takes the value 3, 5, or 7.

[0088] Interpretively, the larger the dot product, the smaller the rate of change between the two normal vectors;

[0089] like Figure 5 When N=5, select a 5×5 pixel area, and multiply the normal vectors of the corresponding color blocks by their respective pixels. Specifically, multiply the normal vectors of the two pixels in the first row and first column, the two pixels in the second row and first column, the two pixels in the second row and fifth column, and so on, until the two pixels in the fifth row and first column, the two pixels in the fifth row and fifth column. This yields five multiplication results. The mean value I is obtained by taking the average of these five multiplications. The formula is as follows:

[0090] mean ,in, , These are the normal vectors of the two pixels in the i-th row, 1st column, and 5th column, respectively. for , The included angle;

[0091] The mean I reflects the rate of change of the normal vector at point P along the X direction;

[0092] Next, multiply the normal vectors of two pixels belonging to the same column and respectively to the first and Nth rows, and take the mean of the results to obtain mean II;

[0093] like Figure 6 The normal vectors of the corresponding color blocks are multiplied by a dot product: the normal vectors of the two pixels in the first column, row 1 and row 5, the normal vectors of the two pixels in the second column, row 1 and row 5, and so on, until the normal vectors of the two pixels in the fifth column, row 1 and row 5 are multiplied. The average of these five multiplications is taken to obtain the mean value II. The calculation formula is as follows:

[0094] mean ,in, , These are the normal vectors of two pixels in the j-th column, row 1 and row 5, respectively. for , The included angle;

[0095] Mean II reflects the rate of change of the normal vector at point P along the Y direction;

[0096] Since the orientation of the thread in the image is uncertain, this method obtains the rate of change of the normal vector along the X and Y directions respectively. It ensures that one of the rates of change reflects the rate of change of the normal vector at point P.

[0097] The smaller of the mean I and the mean II is recorded as the normal rate of change. If the normal rate of change is less than the threshold, then point P is recorded as the foreground point; otherwise, point P is recorded as the background point.

[0098] ③ Store the 3D points corresponding to all foreground points in the point cloud of the inner wall of the thread, and store the 3D points corresponding to all background points in the point cloud of the plane where the threaded hole is located, thus completing the point cloud segmentation.

[0099] For explanation purposes, if there are not all pixels in the N×N neighborhood of point P, then the corresponding point will not undergo a dot product operation on its normal vector. For example, if there are no pixels to the left and above the first pixel, the algorithm assumes that the pixel cannot be multiplied, the normal change rate of the pixel is empty, and it is assumed to be a background point. For the third pixel in the first row, there are two pairs of pixels on the left and right sides. The two dot product results can be obtained, and the average value is recorded as the mean I. The mean II cannot be calculated, so the mean I is directly recorded as the normal change rate.

[0100] To further filter out noise and artifacts in the point cloud, step ④ is also included: obtaining the bounding box of the point cloud of the inner wall of the thread, setting the voxel side length to be smaller than the point cloud spacing, and voxelizing the bounding box (dividing the bounding box into several cubes).

[0101] Traverse all points in the point cloud of the inner wall of the thread, assign points to corresponding voxels based on their coordinates and voxel boundaries, and confirm the connectivity of the point cloud based on the connectivity of the voxels; divide the point cloud into multiple connected regions.

[0102] In the point cloud of the inner wall of the thread, connected regions with a number of points less than a preset value (20~50) are filtered out.

[0103] Compared to existing methods that divide connected components by point cloud clustering and use voxels for connected component filtering, this method is more efficient, requiring only 30ms for clustering of 100,000 point clouds.

[0104] The method for obtaining the normal vectors of each point in the point cloud to be processed in step ① is as follows:

[0105] Iterate through all points in the point cloud to be processed, and perform the following processing on each point Q:

[0106] Find the k nearest points to point Q (k ranges from 5 to 30) and store them in a neighborhood set; find the centroid of the neighborhood set. );

[0107] Calculate matrix C: ; in,( Let be the three-dimensional coordinates of the i-th point in the neighborhood set;

[0108] Perform eigenvalue decomposition on matrix C to obtain three sets of eigenvalues ​​and eigenvectors. Take the eigenvector corresponding to the smallest eigenvalue, and denote it as vector C. ;

[0109] If F If the value is less than 0, then the vector V is inverted and denoted as the normal vector of point Q; otherwise, the vector V is directly denoted as the normal vector of point Q. Here, F represents the direction vector from point Q to point (0, 0, 0).

[0110] This method utilizes symmetrically set acquisition poses to obtain point clouds of threaded holes, and then obtains the point cloud of the inner wall of the thread through point cloud segmentation. This allows for accurate acquisition of the axial direction and center position of the threaded hole in the current workpiece, guiding the robot to complete automatic assembly smoothly and improving assembly accuracy.

[0111] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A vision-guided screw / threaded sleeve installation method, characterized in that, A visual guidance sensor and an installation machine are simultaneously installed at the end of the robot, with their relative positions fixed; the installation machine is a screw tightening gun or a threaded sleeve installation machine. The installation method includes the following steps: 1) The robot moves to the acquisition pose 1 according to the taught motion trajectory, and the visual guidance sensor acquires the point cloud I of the area where the threaded hole is located. Then it moves to the acquisition pose 2, and the visual guidance sensor acquires the point cloud II of the area where the threaded hole is located. The acquisition pose 1 and acquisition pose 2 are approximately symmetrical about the theoretical central axis of the threaded hole. Based on the pre-acquired conversion relationship between the acquisition pose 1 and the acquisition pose 2, point cloud I and point cloud II are stitched together to obtain the point cloud to be processed. 2) Segment the point cloud of the inner wall of the thread and the point cloud of the plane containing the threaded hole from the point cloud to be processed; The cylinder's central axis L1 is obtained by fitting the point cloud of the inner wall of the thread; the intersection point P1 of the cylinder's central axis L1 and the fitted plane is obtained by fitting the plane with the planar point cloud. 3) Align the cylinder's central axis L1 and intersection point P1 with the pre-stored standard central axis L0 and standard intersection point P0 respectively, and obtain the rotation and translation relationship required for alignment, denoted as the offset matrix; The installation machine corrects the robot's installation pose based on the offset matrix, and then installs the screw / threaded sleeve into the threaded hole based on the corrected installation pose.

2. The vision-guided screw / threaded sleeve installation method as described in claim 1, characterized in that: At pose 1 / pose 2, the lateral distance between the visual guidance sensor and the theoretical center position of the threaded hole is... Longitudinal distance is ; Where S is the optimal working distance of the sensor; , , H is the theoretical height of the threaded hole, h is the theoretical height of the smooth hole (located at the top of the threaded hole), D is the theoretical diameter of the smooth hole, and d is the theoretical diameter of the threaded hole.

3. The vision-guided screw / threaded sleeve installation method as described in claim 1, characterized in that: The process of obtaining the pre-stored standard centerline L0 and standard intersection point coordinates P0 is as follows: In the installation scenario of batch screws / threaded sleeves, the robot is taught the installation pose based on the threaded hole on the first workpiece. During this process, step 1) is executed to obtain the point cloud to be processed, and the point cloud of the inner wall of the thread and the point cloud of the plane where the threaded hole is located are segmented from the point cloud to be processed. The cylinder's central axis is obtained by fitting the point cloud of the inner wall of the thread, and is denoted as the standard central axis L0. The intersection point coordinates of the standard central axis L0 and the fitted plane are obtained by fitting the plane with the point cloud of the plane, and are denoted as the standard intersection point coordinates P0.

4. The vision-guided screw / threaded sleeve installation method as described in claim 3, characterized in that: The method for pre-obtaining the transformation relationship between the acquisition pose 1 and the acquisition pose 2 is as follows: S1. Set n non-collinear reflective markers around the threaded hole on the first workpiece, where n ranges from 3 to 10. The robot moves to the acquisition pose 1 according to the taught motion trajectory, and the visual guidance sensor acquires the grayscale image A of the reflective marker. Then it moves to the acquisition pose 2 and the visual guidance sensor acquires the grayscale image B of the reflective marker. S2. Perform the following processing on each reflective marker point: Calculate the two-dimensional coordinates W of the reflective marker points in grayscale images A and B, respectively. A and W B ; Obtain the two-dimensional coordinates W A The corresponding 3D coordinates W in point cloud I A ', in three-dimensional coordinates W A 'Using ' as the center, construct a 3D bounding box, and fit the plane M using the selected local point cloud.' A ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor A 'Construct a spatial straight line, and then connect the spatial straight line with plane M' A The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. A ''; Obtain the two-dimensional coordinates W B The corresponding 3D coordinates W in point cloud II B ', in three-dimensional coordinates W B 'Using ' as the center, construct a 3D bounding box, and fit the plane M using the selected local point cloud.' B ;Utilizing the origin of the camera coordinate system and the three-dimensional coordinate W in the vision-guided sensor B 'Construct a spatial straight line, and then connect the spatial straight line with plane M' B The coordinates of the intersection point are denoted as the three-dimensional coordinates W of the reflective marker point. B ''; S3, using n pairs of three-dimensional coordinates W A '', three-dimensional coordinates W B Perform a rigid body transformation to obtain the transformation relationship between the acquired pose 'pose1' and the acquired pose 'pose2'.

5. The vision-guided screw / threaded sleeve installation method as described in claim 1, characterized in that: Step 2) Segment the point cloud of the inner wall of the thread and the point cloud of the plane containing the threaded hole from the point cloud to be processed, as follows: ① Obtain the normal vector of each point in the point cloud to be processed; ② Project all points in the point cloud to be processed onto a two-dimensional image plane to obtain a projected image. Round the sub-pixel points in the projected image, and then perform the following processing on each pixel P: Construct an N×N selection box centered on pixel P. Among the N×N pixels selected, multiply the normal vectors of two pixels that belong to the same row and are respectively in the first and Nth columns. Take the average of the results of each multiplication to obtain the mean value I. Next, multiply the normal vectors of two pixels belonging to the same column and respectively to the first and Nth rows, and take the mean of the multiplication results to obtain mean II; The smaller of the mean I and the mean II is recorded as the normal rate of change. If the normal rate of change is less than the threshold, then point P is recorded as the foreground point; otherwise, point P is recorded as the background point. ③ Store the 3D points corresponding to all foreground points in the point cloud of the inner wall of the thread, and store the 3D points corresponding to all background points in the point cloud of the plane where the threaded hole is located, thus completing the point cloud segmentation.

6. The vision-guided screw / threaded sleeve installation method as described in claim 1, characterized in that: It also includes step ④, obtaining the bounding box of the point cloud of the inner wall of the thread, setting the voxel side length to be smaller than the point cloud spacing, and voxelizing the bounding box; Traverse all points in the point cloud of the inner wall of the thread, assign points to corresponding voxels based on their coordinates and voxel boundaries, and confirm the connectivity of the point cloud based on the connectivity of the voxels; divide the point cloud into multiple connected regions. In the point cloud of the inner wall of the thread, connected regions with a number of point clouds less than a preset value are filtered out.

7. The vision-guided screw / threaded sleeve installation method as described in claim 6, characterized in that: The default value is 20~50.

8. The vision-guided screw / threaded sleeve installation method as described in claim 5, characterized in that: Step ① involves obtaining the normal vectors of each point in the point cloud to be processed as follows: Iterate through all points in the point cloud to be processed, and perform the following processing on each point Q: Find the k points closest to point Q and store them in a neighborhood set; find the centroid of the neighborhood set. ); Calculate matrix C: ; in,( Let be the three-dimensional coordinates of the i-th point in the neighborhood set; Perform eigenvalue decomposition on matrix C to obtain three sets of eigenvalues ​​and eigenvectors. Take the eigenvector corresponding to the smallest eigenvalue, and denote it as vector C. ; If F If the value is less than 0, then the vector V is inverted and denoted as the normal vector of point Q; otherwise, the vector V is directly denoted as the normal vector of point Q. Here, F represents the direction vector from point Q to point (0, 0, 0).

9. The vision-guided screw / threaded sleeve installation method as described in claim 8, characterized in that: k takes values ​​from 5 to 30.

10. The vision-guided screw / threaded sleeve installation method as described in claim 1, characterized in that: N takes values ​​of 3, 5, and 7.

Images