A grasping method based on plane vision-guided robotic arm

Abstract

The present invention provides a grasping method based on plane vision to guide a robot arm, including placing a part in the area grasped by the robot arm, then using a camera to capture an image of the area grasped by the robot arm and graying the image; performing data type conversion on the image, then subtracting the pixel values ​​of the image from the background image, taking the absolute value, and saving the subtracted image; performing noise reduction, binarization, and black-and-white inversion processing, performing contour outsourcing to obtain the center point coordinates of the contour relative to the image coordinate system, and the deflection angle of the contour; calculating the coordinates and rotation angle of the part contour in the tool coordinate system and sending them to the robot arm, guiding the robot arm to complete grasping the part. The present invention guides the action of the robot arm by accurately identifying the coordinates and rotation angles of the parts to be grasped, and finally realizes accurate grasping of the parts; the present invention can detect the parts in the grasping area, has low requirements for the placement of the parts, fast recognition speed, and grasping efficiency.

Description

Technical Field

[0001] The invention relates to the technical field of robots, in particular to a grasping method based on a plane vision-guided mechanical arm. Background Art

[0002] At present, the degree of automation of production lines in the manufacturing industry is getting higher and higher. By integrating robotic arms and plane vision guidance systems in automated production lines, visual system-guided grasping of metal parts can be achieved.

[0003] However, existing robot arms have low vision levels and are unable to accurately identify the position and posture of product parts, which results in the parts having to be placed as required before the robot arm can identify and grasp the parts, resulting in low grasping efficiency. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a grasping method for a robot arm guided by planar vision.

[0005] The technical solution of the present invention is: a grasping method based on plane vision-guided robotic arm, comprising the following steps:

[0006] S1), placing the part in the area grasped by the robot, and then using the camera to capture an image X1 of the area grasped by the robot;

[0007] S2), graying the image X1 of the area where the robot grasps the part, to obtain a grayed image X2;

[0008] S3), using the astype() function of OpenCV to convert the data types of the grayscale image X2 and the grayscale background image M1 of the robot grasping area in step S2), then subtract the pixel values ​​of the image X2 from the background image M1, take the absolute value, and save the subtracted image X3;

[0009] The part where the absolute value of the pixel value of image X2 is 0 after subtracting the pixel value of background image M1 is used as the background of the robot grasping area, and the part where the absolute value of the pixel value of image X2 is not 0 after subtracting the pixel value of background image M1 is used as the part;

[0010] S4), using the morphology function morphologyEx() of OpenCV to perform noise reduction processing on the image X3 processed in step S3), and using the threshold() function to perform binarization processing and black-and-white inversion processing, and finally obtain the pixel part image X4 of the part;

[0011] S5), use the findContours() function of OpenCV to perform contour wrapping on the image X4 processed in step S4), and obtain the coordinates (x tag ,y tag ), and the deflection angle θ of the profile r ;

[0012] S6), calculate the coordinates (X4) of the contour of the pixel portion image X4 of the part in the tool coordinate system tag ,Y tag ) and the rotation angle θ tag ;

[0013] S7), the obtained coordinates (X tag ,Y tag ) and the rotation angle θ tag Sent to the robotic arm to guide the robotic arm to complete the grasping of parts.

[0014] Preferably, before step S1), the following is included:

[0015] Before placing the parts, use the camera to capture the image of the area where the robot grasps without the parts, and grayscale the image, and use the grayscale processed image as the background image M1, and then save the grayscale processed image, wherein the grayscale processing is as follows:

[0016] Gray=0.299*R+0.587*G+0.114*B.

[0017] Preferably, before step S1), the following steps are included: a point is randomly determined in the camera shooting area as the origin of the tool coordinate system, and the origin of the tool coordinate system should be selected outside the robot gripping area, and a base coordinate system of the robot arm is determined by a robot arm modeling method, and a coordinate point (X) of the origin of the tool coordinate system in the base coordinate system is read by a teach pendant. tool , Y tool ).

[0018] Preferably, in step S2), the formula for graying the image X1 captured by the camera is:

[0019] Gray=0.299*R+0.587*G+0.114*B.

[0020] Preferably, in step S4), the noise reduction process specifically includes the following steps:

[0021] Use the morphology function morphologyEx() of OpenCV to perform a 5*5 median filter on the image X3 processed in step S3), define a data kernel size, and perform a morphological closing operation on the image after the median filter;

[0022] Finally, the image is thresholded, where pixel values ​​greater than 127 become 255 and those less than 127 become 0, thus generating a picture containing only artifacts.

[0023] Preferably, in step S4), the binarization process refers to: converting pixel values ​​in the image greater than 127 to 255, and converting pixel values ​​less than 127 to 0.

[0024] Preferably, in step S4), the black and white inversion process refers to changing the pixel value of 255 to 0, and the pixel value of 0 to 255; wherein, when the pixel is 255, the image is pure black, and when the pixel is 0, it is pure white.

[0025] Preferably, in step S6), the coordinates (X) of the contour of the pixel portion image X4 of the part in the tool coordinate system are calculated. tag ,Y tag ) and the rotation angle θ tag ; Specifically including the following steps:

[0026] S61), calculate the relative displacement, as follows:

[0027]

[0028] Where, X p , Y p are the relative displacements in the x and y directions respectively, (x0, y0) are the coordinates of the origin of the tool coordinate system in the image coordinate system; θ is the angle between the tool coordinate system and the image coordinate system; x err is the error offset in the x direction, y err is the error displacement in the y direction; A = f / H, where f is the focal length and H is the height of the camera;

[0029] S62), coordinates (X tag ,Y tag ) and the rotation angle θ tag The details are as follows:

[0030]

[0031] Among them, (X tool , Y tool ) is the coordinate point of the origin of the tool coordinate system in the base coordinate system; θ tool is the angle between the tool coordinate system and the base coordinate system.

[0032] The beneficial effects of the present invention are:

[0033] 1. The present invention guides the action of the robot arm by accurately identifying the coordinates and rotation angle of the parts to be grasped, and finally realizes accurate grasping of the parts;

[0034] 2. The present invention can detect parts within the grasping area and has low requirements for the placement of parts, thereby improving grasping efficiency;

[0035] 3. The present invention compares the captured part image with the background image to accurately identify the position and posture of the part, and transmits it to the robot arm for grasping. In addition, the photo taking and recognition speeds are fast, further improving the grasping efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of three coordinates of an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of an image coordinate system in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0039] This embodiment provides a grasping method based on plane vision-guided robotic arm, comprising the following steps:

[0040] S1), placing the part in the area grasped by the robot, and then using the camera to capture an image X1 of the area grasped by the robot;

[0041] S2), graying the image X1 of the area where the robot grasps the part, to obtain a grayed image X2; wherein the formula for graying the image X1 captured by the camera is:

[0042] Gray=0.299*R+0.587*G+0.114*B.

[0043] S3), using the astype() function of OpenCV to convert the data types of the grayscale image X2 and the grayscale background image M1 of the robot grasping area in step S2), so that the data format type of the image X2 and the background image M1 is the same; then subtract the pixel values ​​of the image X2 and the background image M1, take the absolute value, and save the subtracted image X3;

[0044] The part where the absolute value of the pixel value of image X2 is 0 after subtracting the pixel value of background image M1 is used as the background of the robot grasping area, and the part where the absolute value of the pixel value of image X2 is not 0 after subtracting the pixel value of background image M1 is used as the part;

[0045] S4), using the morphology function morphologyEx() of OpenCV to perform noise reduction processing on the image X3 processed in step S3), and using the threshold() function to perform binarization processing and black-and-white inversion processing, and finally obtain the pixel part image X4 of the part;

[0046] In this embodiment, the noise reduction process is as follows:

[0047] Use the morphology function morphologyEx() of OpenCV to perform a 5*5 median filter on the image X3 processed in step S3), define a data kernel size, and perform a morphological closing operation on the image after the median filter;

[0048] Finally, the image is thresholded, where pixel values ​​greater than 127 become 255 and those less than 127 become 0, thus generating a picture containing only artifacts.

[0049] The binarization process refers to: changing the pixel values ​​in the image that are greater than 127 to 255, and changing the pixel values ​​that are less than 127 to 0.

[0050] The black and white inversion process refers to changing a pixel value of 255 to 0 and a pixel value of 0 to 255; when the pixel value is 255, the image is pure black, and when the pixel value is 0, the image is pure white.

[0051] S5), use the findContours() function of OpenCV to perform contour wrapping on the image X4 processed in step S4), and obtain the coordinates (x tag ,y tag ), and the deflection angle θ of the profile r ;like Figure 2 As shown;

[0052] S6), calculate the coordinates (X4) of the contour of the pixel portion image X4 of the part in the tool coordinate system tag ,Y tag ) and the rotation angle θ tag ; Specifically including the following steps:

[0053] S61), calculate the relative displacement, as follows:

[0054]

[0055] Where, Xp , Y p are the relative displacements in the x and y directions respectively, (x0, y0) are the coordinates of the origin of the tool coordinate system in the image coordinate system; θ is the angle between the tool coordinate system and the image coordinate system; x err is the error offset in the x direction, y err is the error displacement in the y direction; A = f / H, where f is the focal length and H is the height of the camera;

[0056] S62), coordinates (X tag ,Y tag ) and the rotation angle θ tag The details are as follows:

[0057]

[0058] Among them, (X tool , Y tool ) is the coordinate point of the origin of the tool coordinate system in the base coordinate system; θ tool is the angle between the tool coordinate system and the base coordinate system;

[0059] S7), the obtained coordinates (X tag ,Y tag ) and the rotation angle θ tag Sent to the robotic arm to guide the robotic arm to complete the grasping of parts.

[0060] As a preferred embodiment of the present invention, before step S1), the following steps are also included:

[0061] Before placing the parts, use the camera to capture the image of the area where the robot grasps without the parts, and grayscale the image, and use the grayscale processed image as the background image M1, and then save the grayscale processed image, wherein the grayscale processing is as follows:

[0062] Gray=0.299*R+0.587*G+0.114*B.

[0063] Preferably, before step S1), the following steps are included: a point is randomly determined in the camera shooting area as the origin of the tool coordinate system, and the origin of the tool coordinate system should be selected outside the robot gripping area, and a base coordinate system of the robot arm is determined by a robot arm modeling method, and a coordinate point (X) of the origin of the tool coordinate system in the base coordinate system is read by a teach pendant. tool , Y tool ).like Figure 1 As shown, in this embodiment, the tool coordinate system, the base coordinate system, and the image coordinate system are as follows: Figure 1 shown.

[0064] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A grasping method based on plane vision-guided robotic arm, characterized in that: The following steps are involved: S1), placing the part in the area grasped by the robot, and then using the camera to capture the image X1 of the area grasped by the robot; S2), graying the image X1 of the area where the robot grasps the part, to obtain a grayed image X2; S3), using the astype() function of OpenCV to convert the data types of the grayscale image X2 and the grayscale background image M1 of the robot grasping area in step S2), then subtract the pixel values ​​of the image X2 from the background image M1, take the absolute value, and save the subtracted image X3; The part whose absolute value after subtracting the pixel value of image X2 from the background image M1 is 0 is used as the background of the robot grasping area, and the part whose absolute value after subtracting the pixel value of image X2 from the background image M1 is not 0 is the part; S4), using the morphology function morphologyEx() of OpenCV to perform denoising on the image X3 processed in step S3), and using the threshold() function to perform binarization and black-and-white inversion processing, and finally obtain the pixel part image X4 of the part; the denoising process specifically uses the morphology function morphologyEx() of OpenCV to perform 5*5 median filtering on the image X3 processed in step S3), and defines a data kernel size, and performs morphological closing operation processing on the image after the median filtering; S5), use the findContours() function of OpenCV to perform contour wrapping on the image X4 processed in step S4), and obtain the coordinates of the center point of the contour relative to the image coordinate system (x tag , y tag ), and the deflection angle θ of the profile r ; S6) Calculate the coordinates (X4) of the pixel portion image X4 contour of the part in the tool coordinate system tag , Y tag ) and the rotation angle θ tag ; Specifically including the following steps: S61) Calculate the relative displacement as follows: ； Where, X p ， Y p are the relative displacements in the x and y directions respectively, (x0, y0) are the coordinates of the origin of the tool coordinate system in the image coordinate system; θ is the angle between the tool coordinate system and the image coordinate system; x err is the error offset in the x direction, y err is the error displacement in the y direction; , where f is the focal length and H is the height of the camera; S62), coordinates (X tag , Y tag ) and the rotation angle θ tag The details are as follows: ； Among them, (X tool , Y tool ) is the coordinate point of the origin of the tool coordinate system in the base coordinate system; θ tool is the angle between the tool coordinate system and the base coordinate system; S7) and the obtained coordinates (X tag , Y tag ) and the rotation angle θ tag Send it to the robot arm to guide it to complete the grasping of parts; Before step S1), the following steps are also included: Before placing the parts, use the camera to capture the image of the area where the robot grasps without the parts, and grayscale the image, and use the grayscale processed image as the background image M1, and then save the grayscale processed image, wherein the grayscale processing is as follows: ； By randomly determining a point in the camera shooting area as the origin of the tool coordinate system, and the origin of the tool coordinate system is selected outside the robot gripping area, and the robot arm is used to build The base coordinate system of the robot arm is determined by the model method, and the coordinate point (X tool , Y tool ).

2. According to claim 1, a grasping method based on plane vision-guided robotic arm is characterized by: In step S2), the formula for graying the image X1 captured by the camera is: 。 3. According to claim 1, a grasping method based on plane vision-guided robotic arm is characterized by: In step S4), the binarization process refers to: changing the pixel values ​​in the image that are greater than 127 to 255, and changing the pixel values ​​that are less than 127 to 0.

4. According to claim 3, a grasping method based on plane vision-guided robotic arm is characterized in that: In step S4), the black-and-white inversion process refers to changing the pixel value of 255 to 0, and the pixel value of 0 to 255; wherein, when the pixel is 255, the image is pure black, and when the pixel is 0, it is pure white.

Images