Machine vision-based methods for pose adjustment and positioning of shaft and hole parts

Abstract

This invention discloses a machine vision-based method for adjusting and positioning the pose of shaft-hole type parts. The method includes using a robotic arm to drive a camera to capture images of the part's end face, employing the RANSAC algorithm to remove error points from the image data, determining the normal vector of the part's end face using elliptical features, and then calculating the pitch and horizontal angles of the part's skew. A two-dimensional gimbal compensates for the deviation angles, thus leveling the part. This invention, through the cooperation of a robotic arm and a two-dimensional gimbal, allows the robotic arm to move the camera to a preset position, the camera to capture images of the part's end face, the RANSAC algorithm to remove error points from the image data, and the determination of the normal vector of the part's end face using elliptical features, thereby calculating the part's skew. This improves the accuracy of axial deviation and center positioning, enabling the measurement and adjustment of axial deviations generated during clamping of shaft-hole type parts before press-fitting, thus improving the accuracy and success rate of part press-fitting.

Description

Technical Field

[0001] This invention relates to a method for adjusting the pose of a part and locating its center, specifically a method for adjusting and locating the pose of shaft and hole parts based on machine vision, belonging to the field of workpiece assembly technology. Background Technology

[0002] With the increasing size of modern aerospace products and the ever-increasing demands for safety, efficiency, and reliability, higher requirements are being placed on the positioning, clamping, and installation technologies in the assembly process of shaft and hole parts. The assembly must be reliable in quality, and achieve higher precision and efficiency.

[0003] In the prior art, 1) a system and method for rapid adjustment of the end effector posture of a shaft hole assembly robot disclosed in publication number CN116852382A, wherein the system includes: an image acquisition device for capturing the original image of the hole-shaped part to be assembled; a robot hand-eye calibration module for obtaining the hand-eye transformation matrix between the coordinate system of the image acquisition device and the coordinate system of the robot end effector; a target recognition module for identifying and locating the contour of the hole to be assembled in the original image; a hole pose calculation module for calculating the pose information of the hole to be assembled in the coordinate system of the image acquisition device based on the contour of the hole to be assembled, and calculating the pose information of the hole to be assembled in the coordinate system of the robot end effector; and a robot control command generation module for generating corresponding robot control commands based on the pose information of the hole to be assembled in the coordinate system of the robot end effector. 1) The robot control commands are used to control the robot to adjust the end-effector posture; 2) A method for locating and positioning shaft-hole parts based on cross laser and machine vision, disclosed in CN106767401A, uses cross laser structured light to obtain four points on a spatial circle at one time; calculates the normal vector of the spatial circle based on the four points; establishes a local coordinate system of the spatial circle based on the normal vector, performs dimensionality reduction on the spatial circle contour in this local coordinate system, and uses a nonlinear recursive algorithm to obtain the center and radius parameters of the spatial circle in the local coordinate system; transforms the normal vector and center position of the spatial circle to the robot coordinate system to obtain the position, posture and radius of the spatial circle in the robot coordinate system. The existing technology also uses machine vision to simulate the human eye for part positioning and posture adjustment, which greatly improves the assembly accuracy and automation level.

[0004] However, for some large-sized parts with complex assembly processes, the requirements for assembly accuracy are higher. Existing technology cannot measure and adjust the axial deviation generated during the clamping of shaft and hole parts before press fitting, which leads to low accuracy and success rate of part press fitting in the actual assembly process. Furthermore, the lack of further optimization of axial deviation and center positioning accuracy will also affect the assembly accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based method for adjusting and positioning the posture of shaft and hole parts in order to solve the technical problems of low assembly accuracy and low degree of automation in existing technologies.

[0006] This invention achieves the above objective through the following technical solution: a method for adjusting and positioning the pose of shaft and hole-type parts based on machine vision, comprising the following steps:

[0007] Step 1: The robotic arm moves the camera to the preset position, starts the camera and takes a picture of the left end face of the part to obtain image data of the left end face of the part;

[0008] Step 2: Preprocess the image data, including filtering, thresholding, and edge detection.

[0009] Step 3: Remove error points from the image data using the RANSAC algorithm, and then use the direct least squares method to complete the ellipse fitting.

[0010] Step 4: Determine the normal vector of the end face of the part through the elliptical feature, and then calculate the pitch angle θ1 and horizontal angle θ2 of the part's skewness;

[0011] Step 5: Send the pitch angle θ1 and horizontal angle θ2 to the 2D gimbal for deviation angle compensation to level the part;

[0012] Step 6: Perform ellipse fitting again to determine if the deviation angle meets the requirements. If it does not meet the requirements, repeat step 5 above.

[0013] Step 7: If the deviation angle meets the requirements, then fit a circle to the end face of the part to obtain the center coordinates of the shaft hole of the part.

[0014] Step 8: The robotic arm moves the camera to the preset position, starts the camera and takes a picture of the right end face of the part to obtain the image data of the right end face of the part. Then, steps 2 to 7 are repeated to obtain the center coordinates of the shaft hole of the part, and then the center coordinates of both ends of the shaft hole of the part are obtained.

[0015] As a further aspect of the present invention: in step one, the camera is always located in the XOZ plane, and the end face of the part is adjusted to be parallel to the camera.

[0016] As a further aspect of the present invention: in step two, the filtering process employs bilateral filtering.

[0017] As a further solution of the present invention: In step two, the global maximum inter-class variance method is used for threshold segmentation. By utilizing the grayscale characteristics of the image, the segmentation threshold k is calculated to maximize the variance between the target feature and the non-feature region. The threshold k is set to divide the image h(x,y) into two parts, where the part greater than or equal to k is recorded as 1, which is the required feature grayscale information.

[0018]

[0019] As a further aspect of the present invention: In step two, edge detection uses the Canny operator. By calculating the image gradient, the amplitude and orientation angle of the image are obtained. Then, the non-maximum suppression method is used to divide the gradient into 0°, 45°, 90°, and 135°. Points with amplitude and orientation angle less than those of adjacent points are identified as non-edge feature points and are removed.

[0020] As a further aspect of the present invention: in step three, the image data is repeatedly extracted and iterated using the RANSAC algorithm, and the data is divided into two types: "interior points" that satisfy the fitted curve and "outer points" that do not satisfy the fitted curve.

[0021] Set the number of iterations z and the threshold δ, randomly select 6 data points from the image data, and use the direct least squares method to fit the ellipse;

[0022] Calculate the distance between the unselected points and the fitted ellipse, and mark those points with a distance less than the threshold δ as interior points, and those with a distance greater than the threshold δ as exterior points.

[0023] By iterating k times, we fit an ellipse to all interior points to obtain the equation of the ellipse.

[0024] As a further aspect of the present invention: in step four, the coordinates of the normal line of the end face of the part in the camera coordinate system are obtained by calculating the characteristic equations of the two ellipses inside and outside the shaft hole.

[0025] The projected ellipses C1 and C2 of a spatial circle must both pass through a pair of complex conjugate points, i.e., the equation is:

[0026]

[0027] There must be two complex conjugate solutions, where X is the coordinate of the point of the projected ellipse. This pair of complex conjugate points forms a straight line L. ∞ The coordinates of the normal to the end face of the part in the camera coordinate system are:

[0028]

[0029] Where: K is the camera intrinsic parameter matrix; based on the calculated end-face normal vector, calculate the angle between the projection of the normal vector onto the XY plane and the YZ plane and the y-axis, i.e., the pitch angle θ1 = tan -1 (n3 / n2), horizontal angle θ2=tan -1 (n1 / b2).

[0030] As a further aspect of the present invention: in step five, the two-dimensional gimbal rotates in the pitch and horizontal directions, and sends the pitch angle θ1 and the horizontal angle θ2 to the two-dimensional gimbal through the control platform for compensation.

[0031] As a further aspect of the present invention: In step six, the deviation standard is set to 0.01°. If both the pitch angle θ1 and the horizontal angle θ2 are less than 0.01°, proceed to the next step; if one of the deviation angles is greater than 0.01°, the gimbal is adjusted again.

[0032] As a further aspect of this invention: In step seven, the Iterative Reweighted Least Squares (IRLS) algorithm is used to fit the circle. The IRLS method introduces a distance weight function w(δ), and the weight can be calculated using the distance that has already been calculated. The weight function adopts the Huber weight function, and its expression is as follows:

[0033]

[0034] The parameter γ is the clipping function, which defines which points are outliers; the sample weights are updated until the iteration number k1 is reached; the equation of the circle is (xa). 2 +(yb) 2 =c 2 Establish the error function E that minimizes

[0035]

[0036] Where A = -2a, B = -2b, C = a 2 +b 2 -c 2 By taking the partial derivatives with respect to A, B, and C respectively, and solving the system of equations, we can find a, b, and c, and thus obtain the coordinates of the center of the circle (a, b, c).

[0037] The beneficial effects of this invention are as follows: by cooperating with a robotic arm and a two-dimensional gimbal, the robotic arm moves the camera to a preset position, the camera is activated to take pictures of the end face of the part, the RANSAC algorithm is used to remove error points from the image data, the normal vector of the end face of the part is determined by the elliptical feature, and then the part skew is calculated, which improves the accuracy of axis deviation and center positioning. It can measure and adjust the axis deviation generated when clamping shaft and hole parts before pressing, which greatly improves the accuracy and success rate of part pressing. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method for adjusting and positioning the posture of shaft-hole type parts according to the present invention;

[0039] Figure 2 This is a schematic diagram of the measurement process for shaft and hole type parts of the present invention.

[0040] In the picture: 1. Base, 2. 2D gimbal, 3. Fixture, 4. Parts, 5. Camera, 6. Robotic arm. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1, as Figures 1 to 2 As shown, a machine vision-based method for adjusting and locating the pose of shaft-hole type parts includes the following steps:

[0043] Step 1: Part 4 is clamped on fixture 3. Fixture 3 is placed on base 1 via two-dimensional gimbal 2. Robotic arm 6 is also connected to base 1. Camera 5 is connected to the movable end of robotic arm 6. Robotic arm 6 drives camera 5 to move to a preset position, starts the camera and takes a picture of the left end face of part to obtain image data of the left end face of part.

[0044] Step 2: Preprocess the image data, including filtering, thresholding, and edge detection.

[0045] Step 3: Remove error points from the image data using the RANSAC algorithm, and then use the direct least squares method to complete the ellipse fitting.

[0046] Step 4: Determine the normal vector of the end face of the part through the elliptical feature, and then calculate the pitch angle θ1 and horizontal angle θ2 of the part's skewness;

[0047] Step 5: Send the pitch angle θ1 and horizontal angle θ2 to the 2D gimbal for deviation angle compensation to level the part;

[0048] Step 6: Perform ellipse fitting again to determine if the deviation angle meets the requirements. If it does not meet the requirements, repeat step 5 above.

[0049] Step 7: If the deviation angle meets the requirements, then fit a circle to the end face of the part to obtain the center coordinates of the shaft hole of the part.

[0050] Step 8: The robotic arm moves the camera to the preset position, starts the camera and takes a picture of the right end face of the part to obtain the image data of the right end face of the part. Then, steps 2 to 7 are repeated to obtain the center coordinates of the shaft hole of the part, and then the center coordinates of both ends of the shaft hole of the part are obtained.

[0051] Example 2: In addition to all the technical features in Example 1, this example also includes: In step 1, the camera is always located in the XOZ plane, and the end face of the part is adjusted to be parallel to the camera.

[0052] In step two, bilateral filtering is used, which can effectively remove Gaussian noise and salt-and-pepper noise, and also smooth the image.

[0053] In step two, the global maximum inter-class variance method is used for threshold segmentation. By utilizing the grayscale characteristics of the image, the segmentation threshold k is calculated to maximize the variance between the target feature and the non-feature region. The threshold k is set to divide the image h(x,y) into two parts, where the part greater than or equal to k is recorded as 1, which is the required feature grayscale information.

[0054]

[0055] In step two, edge detection uses the Canny operator. By calculating the image gradient, the amplitude and orientation angle of the image are obtained. Then, the non-maximum suppression method is used to divide the gradient into 0°, 45°, 90°, and 135°. Points with amplitude and orientation angle smaller than those of the adjacent points are identified as non-edge feature points and are removed.

[0056] Example 3: In addition to all the technical features in Example 1, this example also includes: in step 3, image data is repeatedly extracted and iterated using the RANSAC algorithm, and the data is divided into two types: "interior points" that satisfy the fitting curve and "outer points" that do not satisfy the fitting curve.

[0057] Set the number of iterations z and the threshold δ, randomly select 6 data points from the image data, and use the direct least squares method to fit the ellipse;

[0058] Calculate the distance between the unselected points and the fitted ellipse, and mark those points with a distance less than the threshold δ as interior points, and those with a distance greater than the threshold δ as exterior points.

[0059] By iterating k times, we fit an ellipse to all interior points to obtain the equation of the ellipse.

[0060] In step four, by calculating the characteristic equations of the two ellipses inside and outside the shaft hole, the coordinates of the normal to the end face of the part in the camera coordinate system can be obtained; the projected ellipses C1 and C2 of the spatial circle must both pass through a pair of complex conjugate points, i.e., the equation is:

[0061]

[0062] There must be two complex conjugate solutions, where X is the coordinate of the point of the projected ellipse. This pair of complex conjugate points forms a straight line L. ∞ The coordinates of the normal to the end face of the part in the camera coordinate system are:

[0063]

[0064] Where K is the camera intrinsic parameter matrix; based on the calculated end-face normal vector, the angle between the projection of the normal vector onto the XY plane and the YZ plane and the y-axis is calculated, i.e., the pitch angle θ1 = tan -1 (n3 / n2), horizontal angle θ2=tan -1 (n1 / n2).

[0065] In step five, the 2D gimbal can rotate in both the pitch and horizontal directions, and the pitch angle θ1 and horizontal angle θ2 are sent to the 2D gimbal through the control platform for compensation.

[0066] In step six, the deviation standard is set to 0.01°. If both the pitch angle θ1 and the horizontal angle θ2 are less than 0.01°, proceed to the next step; if any deviation angle is greater than 0.01°, the gimbal is adjusted again.

[0067] In step seven, the Iterative Reweighted Least Squares (IRLS) algorithm is used to fit the circle. The IRLS method introduces a distance weight function w(δ), and the weight can be calculated from the already calculated distance. The weight function adopts the Huber weight function, and its expression is as follows:

[0068]

[0069] The parameter γ is the clipping function, which defines which points are outliers; the sample weights are updated until the iteration number k1 is reached; the equation of the circle is (xa). 2 +(yb) 2 =c 2 Establish the error function E that minimizes

[0070]

[0071] Where A = -2a, B = -2b, C = a 2 +b 2 -c 2 By taking the partial derivatives with respect to A, B, and C respectively, and solving the system of equations, we can find a, b, and c, and thus obtain the coordinates of the center of the circle (a, b, c).

[0072] A robotic arm drives a camera to photograph the end face of the part. The RANSAC algorithm is used to remove error points from the image data. Then, the direct least squares method is used to complete the ellipse fitting. The normal vector of the end face of the part is determined by the ellipse features. Then, the pitch angle and horizontal angle of the part are calculated. The two-dimensional gimbal compensates for the deviation angle to level the part. The end face of the part is fitted with a circle to obtain the coordinates of the center of the shaft hole, so as to center and press the part.

[0073] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0074] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for adjusting and positioning the pose of shaft-hole type parts based on machine vision, characterized in that, The method for adjusting and positioning the posture of shaft-hole type parts includes the following steps: Step 1: The robotic arm moves the camera to the preset position, starts the camera and takes a picture of the left end face of the part to obtain image data of the left end face of the part; Step 2: Preprocess the image data, including filtering, thresholding, and edge detection. Step 3: Remove error points from the image data using the RANSAC algorithm, and then use the direct least squares method to complete the ellipse fitting. Step 4: Determine the normal vector of the part's end face using the elliptical feature, and then calculate the pitch angle of the part's skew. and horizontal angle ; In step four, the coordinates of the normal to the end face of the part in the camera coordinate system are obtained by calculating the characteristic equations of the two ellipses inside and outside the shaft hole. Projected ellipse of a spatial circle , They must all pass through a pair of complex conjugate points, i.e., the equation: ； There must be two complex conjugate solutions, where X is the coordinate of a point on the projected ellipse, and this pair of complex conjugate points forms a straight line. The coordinates of the normal to the end face of the part in the camera coordinate system are: ； in: Let be the camera intrinsic parameter matrix; based on the calculated end-face normal vector, calculate the angle between the projection of the normal vector onto the XY plane and the YZ plane and the y-axis, i.e. Horizontal angle ; Step 5: Adjust the pitch angle and horizontal angle Send the data to the 2D gimbal for deviation angle compensation to level the part. Step 6: Perform ellipse fitting again to determine if the deviation angle meets the requirements. If it does not meet the requirements, repeat step 5 above. Step 7: If the deviation angle meets the requirements, fit a circle to the end face of the part and obtain the center coordinates of the shaft hole of the part. Step 8: The robotic arm moves the camera to the preset position, starts the camera and takes a picture of the right end face of the part to obtain the image data of the right end face of the part. Then, steps 2 to 7 are repeated to obtain the center coordinates of the shaft hole of the part, and then the center coordinates of both ends of the shaft hole of the part are obtained.

2. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step one, the camera is always located in the XOZ plane, and the end face of the part is adjusted to be parallel to the camera.

3. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step two, the filtering process employs bilateral filtering.

4. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step two, the global maximum inter-class variance method is used for threshold segmentation. Utilizing the image's grayscale characteristics, a segmentation threshold k is calculated to maximize the variance between the target feature and non-feature regions. This threshold k divides the image h(x,y) into two parts. ； Where k is greater than or equal to k is denoted as 1, representing the required grayscale feature information.

5. The machine vision-based method for adjusting and positioning the pose of shaft-hole type parts according to claim 1, characterized in that: In step two, edge detection uses the Canny operator. By calculating the image gradient, the amplitude and orientation angle of the image are obtained. Then, the non-maximum suppression method is used to divide the gradient into 0°, 45°, 90°, and 135°. Points with amplitude and orientation angle smaller than those of adjacent points are identified as non-edge feature points and are removed.

6. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step three, the image data is repeatedly extracted and iterated using the RANSAC algorithm, and the data is divided into two types: "interior points" that satisfy the fitted curve and "outer points" that do not satisfy the fitted curve. Set the number of iterations z and the threshold Six data points were randomly selected from the image data, and ellipse fitting was performed using the direct least squares method. Calculate the distance between the unselected points and the fitted ellipse, and select points with distances less than a threshold. Points that are not inside are denoted as points that are not outside; By iterating k times, we fit an ellipse to all interior points to obtain the equation of the ellipse.

7. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step five, the two-dimensional gimbal rotates in both the pitch and horizontal directions to adjust the pitch angle. and horizontal angle The compensation is sent to the 2D PTZ camera via the control platform.

8. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step six, the deviation standard is set to 0.01°. If the pitch angle... and horizontal angle If all deviations are less than 0.01°, proceed to the next step; if any deviation angle is greater than 0.01°, adjust the gimbal again.

9. The method for adjusting and positioning the posture of shaft-hole type parts according to claim 1, characterized in that: In step seven, an iterative reweighted least squares algorithm is used to fit the circle. The iterative reweighted least squares algorithm introduces a distance weight function w(δ), and the weight is calculated using the distances that have already been calculated. The weight function adopts the Huber weight function, and its expression is as follows: ； The parameter γ is the clipping function; the sample weights are updated until the required number of iterations is reached. The equation of a circle is: Establish the error function E that minimizes ； in, Take the partial derivatives with respect to A, B, and C respectively, and solve the system of equations to obtain a, b, and c, thereby obtaining the coordinates of the center of the circle (a, b, c).

Images