3D visual positioning method and system for industrial robot

Abstract

A 3D visual positioning method for an industrial robot, the method comprising: analyzing a three-dimensional coordinate system corresponding to a grabbed object, and in view of acquired image information, performing determination on coordinate points acquired on the basis of an image; for an abnormal situation, correcting the coordinate points, and further analyzing and generating grabbing points on the basis of the corrected coordinate points; then, performing determination and analysis on the grabbed object on the basis of the generated grabbing points; generating optimal stress points by taking parameters of the grabbed object itself into consideration; and analyzing the optimal stress points and using the optimal stress points as subsequent target grabbing points, so that errors in the subsequent grabbing process are reduced, thereby improving the precision of 3D visual positioning. Further provided is a 3D visual positioning system for an industrial robot.

Description

Industrial robot 3D vision positioning method and system TECHNICAL FIELD

[0001] The present application relates to the technical field of 3D vision positioning, in particular to an industrial robot 3D vision positioning method and system. BACKGROUND

[0002] Industrial robot 3D vision positioning technology is a technology that uses a 3D vision system to accurately obtain the spatial position and attitude of an object to guide the robot to perform precise operations.

[0003] According to the patent with the application number CN202311633829.0, the patent step one: make a tool coordinate tool near the gripper grabbing position, get the conversion of the camera coordinate system to the tool coordinate tool through hand-eye calibration, and record it as step two: place the object to be grabbed and placed in a fixed position, move the robot to align the camera with the object to take a picture, get the object to the camera coordinates, and at the same time get the current robot tool coordinate tool to the robot base coordinate pose transformation The present application uses 3D vision positioning to realize high-precision guidance of the grabbing and placing process of the robot, which can eliminate part of the system error and improve the positioning and guidance accuracy.

[0004] Some existing methods have hand-eye calibration errors, workpiece grabbing position setting errors, robot tool coordinate errors, and other system errors, which make it difficult to achieve high-precision grabbing and placing below millimeters during the grabbing process, limiting the application scenarios of 3D vision and robots.

[0005] SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides an industrial robot 3D vision positioning method and system, which solves the problems of hand-eye calibration errors, workpiece grabbing position setting errors, robot tool coordinate errors, and other system errors.

[0007] To achieve the above purpose, the present application realizes the following technical scheme: an industrial robot 3D vision positioning method, which specifically includes the following steps:

[0008] Obtain the grabbing object and establish a three-dimensional space coordinate system, and obtain the three-dimensional space coordinate points of the grabbing object;

[0009] Obtain the image information of the grabbing object, and obtain the simulated three-dimensional coordinate points of the grabbing object according to the image information, and judge with the three-dimensional space coordinate points;

[0010] For different situations, the simulated three-dimensional coordinate points are corrected to obtain corrected coordinates, and a correction factor is obtained through comprehensive analysis;

[0011] Then the grasping point of the grasping object is analyzed according to the obtained modified coordinates, and the stress height point of the grasping object is calculated according to the grasping point;

[0012] The preselected stress point is obtained by analyzing the stress height point, and the most suitable stress point is obtained by secondary analysis of the preselected stress point.

[0013] Preferably, the specific way of obtaining the three-dimensional space coordinate points of the grasping object is:

[0014] The object to be grasped is denoted as the grasping object, then a three-dimensional space coordinate system is established according to the plane where the grasping object is located, and the corresponding three-dimensional coordinate points of the grasping object are obtained according to the three-dimensional space coordinate system, denoted as (Xn, Yn, Zn), and n = 1, 2, …, 6, and the length, width and height of the grasping object are obtained according to the three-dimensional space coordinate system, and are represented by three-dimensional space coordinate points, then the three-dimensional space coordinate points are integrated to generate the grasping object basic information.

[0015] Preferably, the specific way of generating simulated three-dimensional coordinate points according to the image information of the grasping object is:

[0016] The image coordinate points of the grasping object in the image information are analyzed, then the obtained image coordinate points are converted into a three-dimensional coordinate system to obtain a simulated three-dimensional coordinate system, and the specific way is to realize the marking by normalizing the image coordinate system, that is, to let the coordinates of the image center be (0, 0), and to use the number of pixels in the horizontal and vertical directions to represent the coordinates.

[0017] Preferably, the specific way of judging and comparing the three-dimensional space coordinate points of the grasping object with the simulated three-dimensional coordinate points is:

[0018] When the simulated three-dimensional coordinate points are the same as the three-dimensional space coordinate points, it means that there is no difference between them, and the obtained simulated three-dimensional coordinate points can be used directly, otherwise when there is a difference between them, the simulated three-dimensional coordinate points need to be modified.

[0019] Preferably, the specific way of modifying the simulated three-dimensional coordinate points is:

[0020] The abnormal points of the simulated three-dimensional coordinate points and the three-dimensional space coordinate points are obtained, then the simulated three-dimensional coordinate points are modified to obtain modified coordinates by taking the three-dimensional space coordinate points as the standard, different poses of the grasping image are obtained by taking different photographing directions, the corresponding simulated three-dimensional coordinate points in different photographing directions are compared and analyzed one by one, and the modification factor is calculated according to the comparison result, the modification factor is a weight factor, which is the difference between the coordinates before and after modification.

[0021] Preferably, the specific way of analyzing the grasping point of the grasping object is:

[0022] The center point of the grasping object is obtained, and the center point is marked as the center point coordinate. Then, the center point is taken as the origin, and a perpendicular line is drawn from the origin to different surfaces of the grasping object, which is marked as L. Meanwhile, the length of the perpendicular line is judged. When the lengths of all the perpendicular lines are the same, the hammer point corresponding to the intersection surface of the perpendicular line is obtained, and the obtained hammer point is marked as the grasping point k.

[0023] Preferably, the specific way of analyzing the grasping point is:

[0024] All grasping points k are obtained, and then any one of the grasping points is taken as an analysis object. The stress value of the analysis object is calculated and marked as G. Further, the average grasping force value corresponding to the grasping point k is calculated Meanwhile, the number of clamps is marked as a;

[0025] Then, the average grasping force value The grasping force value corresponding to a single clamp a is calculated and marked as Meanwhile, a plurality of isosceles right triangles are generated according to the grasping point k, and the generated isosceles right triangles are labeled and marked as i, and i = 1, 2, …, 6. Then, the stress point height of the isosceles right triangle i is calculated. The length of the right angle side and the hypotenuse of the isosceles right triangle are obtained and marked as D1 and D2 respectively. Meanwhile, the obtained parameters are substituted into the formula Wherein, a is the angle value of the isosceles right triangle, and the stress point height Fi of the isosceles right triangle i is calculated according to the formula, wherein H is the height of the grasping object.

[0026] Preferably, the specific way of analyzing the preselected stress point according to the stress height point is:

[0027] The stress point height Fi is obtained in descending order, and the height of the industrial robot clamp is obtained and marked as Y. Then, the stress point i corresponding to the stress point height Fi less than 2Y / 3 is screened, and the screened stress point is marked as the preselected stress point. The preselected stress point is analyzed again.

[0028] Preferably, the specific way of analyzing the preselected stress point again is:

[0029] All preselected stress points are obtained, and the corresponding three-dimensional space coordinate point is obtained according to the preselected stress point and marked as (Xo, Yo, Zo). Then, the obtained three-dimensional space coordinate point is substituted into the formula The distance Po between the stress point and the origin is calculated, and (Xv, Yv, Zv) in the formula represents the origin coordinate. The preselected stress point corresponding to the minimum distance Po value is selected, and the selected preselected stress point is marked as the most suitable stress point. The most suitable stress point information is generated.

[0030] The industrial robot 3D visual positioning system comprises an information acquisition unit, an information adaptive processing unit, a grabbing point analysis unit, a grabbing point selection unit and an information output unit.

[0031] The information acquisition unit is used for acquiring length, width and height information of the grabbing object and transmitting the acquired information to the information adaptive processing unit.

[0032] The information adaptive processing unit is used for analyzing image information of the grabbing object in combination with the acquired information, obtaining modified coordinates of the grabbing object by modifying the image information, and transmitting the modified coordinates to the grabbing point analysis unit.

[0033] The grabbing point analysis unit is used for calculating a grabbing point of the grabbing object in combination with the acquired modified coordinates, obtaining a stress point height value according to the grabbing point, and transmitting the stress point height value to the grabbing point selection unit.

[0034] The grabbing point selection unit is used for calculating and analyzing an optimal grabbing point of the grabbing object in combination with the acquired stress point height value, generating optimal grabbing point information, and transmitting the generated optimal grabbing point information to the information output unit.

[0035] The information output unit is used for displaying the acquired optimal grabbing point information to an operator.

[0036] The industrial robot 3D visual positioning method and system provided by the application have the following beneficial effects compared with the prior art.

[0037] The application analyzes a three-dimensional coordinate system corresponding to the grabbing object, judges the coordinate points acquired according to the image in combination with the acquired image information, modifies the coordinate points in the case of abnormality, analyzes and generates a grabbing point according to the modified coordinate points, judges and analyzes the grabbing object according to the generated grabbing point, generates an optimal stress point according to the parameters of the grabbing object itself, analyzes the optimal stress point and takes it as a subsequent grabbing target point, reduces errors in the subsequent grabbing process, and improves the accuracy of 3D visual positioning. BRIEF DESCRIPTION OF DRAWINGS

[0038] Fig. 1 is a method flowchart of the application;

[0039] Fig. 2 is a system principle diagram of the application;

[0040] Fig. 3 is a schematic diagram of a grabbing object of the application. DETAILED DESCRIPTION

[0041] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0042] Please refer to FIG. 1 and FIG. 3, the present application provides an industrial robot 3D visual positioning method, which specifically comprises the following steps:

[0043] Step one: the position of the object to be grabbed is obtained, and the corresponding coordinate information is calculated according to the position, and the specific calculation of coordinate information is as follows:

[0044] The object to be grabbed is denoted as a grabbing object, then a three-dimensional space coordinate system is established according to the plane where the grabbing object is located, specifically the plane where the grabbing object is located is denoted as the X axis, and the grabbing object is a regular shaped object, and it is assumed that the grabbing object is a cuboid or a cube, and the corresponding three-dimensional coordinate points of the grabbing object are obtained according to the three-dimensional space coordinate system, denoted as (Xn, Yn, Zn), and n = 1, 2, …, 6, and the length, width and height of the grabbing object are obtained according to the three-dimensional space coordinate system, and are represented by three-dimensional space coordinate points, then the three-dimensional space coordinate points are integrated to generate the basic information of the grabbing object;

[0045] Step two: the grabbing object is photographed and the image information of the grabbing object is obtained, and the grabbing object is analyzed according to the image information, and the specific analysis method is as follows:

[0046] The image coordinate points of the grabbing object in the image information are analyzed, and specifically the image coordinate points here are marked by normalizing the image coordinate system, in order to facilitate calculation, the physical coordinate system is usually normalized, that is, the coordinates of the image center are (0, 0), and the number of pixels in the horizontal and vertical directions is used to represent the coordinates, then the obtained image coordinate points are converted into a simulated three-dimensional coordinate system, and further the simulated three-dimensional coordinate points of the grabbing image are obtained according to the simulated three-dimensional coordinate system, and the simulated three-dimensional coordinate points are compared and matched with the three-dimensional space coordinate points to determine whether they are the same;

[0047] When the simulated three-dimensional coordinate points and the three-dimensional space coordinate points are the same, it means that there is no difference between them, and the obtained simulated three-dimensional coordinate points can be used directly, otherwise when there is a difference between them, the simulated three-dimensional coordinate points need to be corrected;

[0048] The abnormal points of the simulated three-dimensional coordinate points and the three-dimensional space coordinate points are obtained, and the abnormal points specifically represent the points with coordinate value differences. Then, the simulated three-dimensional coordinate points are corrected based on the three-dimensional space coordinate points to obtain corrected coordinates. Specifically, the (x, y, z) values of the simulated three-dimensional coordinate points are adjusted to make the simulated three-dimensional coordinate points close to or the same as the (x, y, z) values of the three-dimensional space coordinate points. The adjusted (x, y, z) values of the simulated three-dimensional coordinate points are the corrected coordinates. Different photographing directions are adopted to obtain different poses of the captured images. The simulated three-dimensional coordinate points corresponding to different photographing directions are compared and analyzed one by one, and correction factors are calculated based on the comparison results. The correction factors are weight factors, and specifically the differences between the simulated three-dimensional coordinate points and the corrected coordinates.

[0049] Step three: Then, based on the obtained corrected coordinates, the positioning and analysis of the grasping points of the grasping object are performed based on the corrected coordinates. The specific positioning and analysis method is as follows:

[0050] The center point of the grasping object is obtained, and the corrected coordinates corresponding to the center point are obtained as the center point coordinates. Then, the center point is taken as the origin, and the perpendicular lines L are drawn from the origin to different faces of the grasping object. The lengths of the perpendicular lines are judged. When all the lengths of the perpendicular lines are the same, the hammer points corresponding to the perpendicular lines and the intersection surfaces are obtained, and the obtained hammer points are marked as grasping points k. Specifically, the number of grasping points k obtained here is six, and they correspond to six faces of the grasping object.

[0051] Then, the number of grippers of the industrial robot is obtained, and the height of the stress points corresponding to different faces of the grasping object is calculated based on the number of grippers. The specific calculation method is as follows:

[0052] All grasping points k are obtained, and then any one of the grasping points is taken as an analysis object. The stress value of the analysis object is calculated and marked as G, and G here is the gravity of the grasping object as a whole. The average grasping force value corresponding to the grasping point k is further calculated The number of grippers is marked as a. According to actual analysis, the number of grippers is normally 3 or 5. Therefore, when the analysis object is grasped, the optimal grasping point needs to be selected to ensure the stability of grasping. In this application, the analysis is based on the number of grippers being 3.

[0053] Then, the average grasping force value The grasping force value corresponding to a single gripper a is calculated and marked as Meanwhile, a plurality of sets of isosceles right triangles are generated according to the grabbing point k, and the number of the generated isosceles right triangles is 6 sets, and the generated isosceles right triangles are labeled as i, and i = 1, 2, …, 6, then the height of the force point of the isosceles right triangle i is calculated, and the length of the right angle side and the length of the hypotenuse of the isosceles right triangle are denoted as D1 and D2 respectively, and the numerical value of the right angle side D1 is equal, and the obtained parameters are substituted into the formula and the height Fi of the force point of the isosceles right triangle i is calculated according to the formula, wherein H is the height of the grabbing object;

[0054] The calculated force point height Fi is compared and sorted in descending order.

[0055] Step four: according to the obtained force point height sorting information, the grabbing point is selected, and the specific grabbing point selection method is as follows:

[0056] The force point height Fi sorted in descending order is obtained, and the height of the industrial robot gripper is denoted as Y, then the force point height Fi less than 2Y / 3 is selected, and the selected force point is denoted as the preselected force point, and the preselected force point is analyzed again, and the specific analysis method is:

[0057] All the preselected force points are obtained, and the corresponding three-dimensional space coordinate points (Xo, Yo, Zo) are obtained according to the preselected force points, then the obtained three-dimensional space coordinate points are substituted into the formula The distance Po between the force point and the origin is calculated, and (Xv, Yv, Zv) in the formula represents the origin coordinates, and the preselected force point with the minimum distance Po value is selected, and the selected preselected force point is denoted as the most suitable force point, and the most suitable force point information is generated.

[0058] As an embodiment of the present application, please refer to Fig. 2, which focuses on the industrial robot 3D vision positioning system, which comprises: an information acquisition unit, an information adaptive processing unit, a grabbing point analysis unit, a grabbing point selection unit and an information output unit, and the above-mentioned multiple functional units are unidirectionally electrically connected.

[0059] The information acquisition unit is used for acquiring the length, width and height information of the grabbing object, and transmitting the acquired information to the information adaptive processing unit.

[0060] The information adaptive processing unit is used for analyzing the image information of the grabbing object in combination with the obtained information, and obtaining the modified coordinates of the grabbing object by modifying the image information, and transmitting the modified coordinates to the grabbing point analysis unit.

[0061] The grabbing point analysis unit is configured to calculate the grabbing point of the grabbing object according to the obtained modified coordinates, and calculate the stress point height value according to the grabbing point, and transmit the stress point height value to the grabbing point selection unit.

[0062] The grabbing point selection unit is configured to calculate and analyze the most suitable grabbing point of the grabbing object according to the obtained stress point height value, and generate the most suitable grabbing point information, and transmit the generated most suitable grabbing point information to the information output unit.

[0063] The information output unit is configured to display the obtained most suitable grabbing point information to an operator.

[0064] Some data in the above formula are dimensionless for numerical calculation, and the contents not described in detail in the specification all belong to the prior art known by those skilled in the art.

[0065] The above embodiments are only used to illustrate the technical method of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A method for 3D vision positioning of an industrial robot, c h a r a c t e r i s e d in that, The method comprises the following steps: Obtain the grasping object, establish a three-dimensional space coordinate system, and obtain three-dimensional space coordinate points of the grasping object; Obtain image information of the grasping object, and obtain simulated three-dimensional coordinate points of the grasping object according to the image information, and judge whether there is a difference between the three-dimensional space coordinate points and the simulated three-dimensional coordinate points; For the case that there is a difference, correct the simulated three-dimensional coordinate points to obtain corrected coordinates, and comprehensively analyze to obtain a correction factor; According to the obtained corrected coordinates, analyze the grasping point of the grasping object, and calculate the stress height point of the grasping object according to the grasping point; Analyze the stress height point to obtain a preselected stress point, and perform secondary analysis on the preselected stress point to obtain an optimal stress point.

2. The industrial robot 3D vision positioning method according to claim 1, characterized in that, The specific way of obtaining the three-dimensional space coordinate points of the grasping object is: The object to be grasped is denoted as a grasping object, a three-dimensional space coordinate system is established according to the grasping object, and three-dimensional coordinate points corresponding to the grasping object are obtained according to the three-dimensional space coordinate system and denoted as (Xn, Yn, Zn), and n = 1, 2, …, 6. The length, width and height of the grasping object are obtained according to the three-dimensional space coordinate system, and are represented by three-dimensional space coordinate points. The three-dimensional space coordinate points are integrated to generate basic information of the grasping object.

3. The industrial robot 3D vision positioning method of claim 1, wherein, The specific way of generating simulated three-dimensional coordinate points according to image information of the grasping object is: Image coordinate point analysis is performed on the grasping object in the image information, three-dimensional coordinate system conversion is performed on the obtained image coordinate points to obtain a simulated three-dimensional coordinate system, and the image coordinate system is normalized to realize marking, that is, the coordinates of the image center are (0, 0), and the number of pixels in the horizontal and vertical directions is used to represent the coordinates.

4. The industrial robot 3D vision positioning method according to claim 1, characterized in that, The specific way of comparing the three-dimensional space coordinate points of the grasping object with the simulated three-dimensional coordinate points is: When the simulated three-dimensional coordinate points are the same as the three-dimensional space coordinate points, it indicates that there is no difference between them, and the obtained simulated three-dimensional coordinate points can be directly used. Otherwise, when there is a difference between them, the simulated three-dimensional coordinate points need to be corrected.

5. The industrial robot 3D vision positioning method according to claim 4, characterized in that, The specific way of correcting the simulated three-dimensional coordinate points is: Abnormal points of the simulated three-dimensional coordinate points and the three-dimensional space coordinate points are obtained, the simulated three-dimensional coordinate points are corrected according to the three-dimensional space coordinate points to obtain corrected coordinates, different poses of the grasping image are obtained by taking different photographing directions, the corresponding simulated three-dimensional coordinate points in different photographing directions are compared and analyzed one by one, and a correction factor is calculated according to the comparison result.

6. The industrial robot 3D vision positioning method according to claim 1, characterized in that, The specific way of analyzing the grasping point of the grasping object is: The center point of the grasping object is obtained, and the corrected coordinates corresponding to the center point are denoted as center point coordinates. Then, a perpendicular line is drawn from the center point to different surfaces of the grasping object, and the length of the perpendicular line is judged. When the lengths of all the perpendicular lines are the same, the hammer point corresponding to the perpendicular line and the intersection surface is obtained, and the obtained hammer point is denoted as a grasping point k.

7. The industrial robot 3D vision positioning method according to claim 1, characterized in that, The specific way of analyzing the grasping point is: After all the grabbing points k are obtained, any one of the grabbing points is taken as an analysis object, and the stress value of the analysis object is calculated and recorded as G. Further, the average grabbing force value corresponding to the grabbing point k is calculated Meanwhile, the number of clamping jaws is denoted as a; Next, the average grip value is calculated The calculated grip value corresponding to a single jaw a is denoted as ​ According to the capture point k, a plurality of sets of isosceles right triangles are generated, and the generated isosceles right triangles are labeled as i, and i = 1, 2, …, 6. Then, the height of the force point of the isosceles right triangle i is calculated, and the length of the right angle side and the length of the hypotenuse of the isosceles right triangle are respectively marked as D1 and D2. At the same time, the obtained parameters are substituted into the formula And according to the formula to calculate the height of the force point of the isosceles right triangle i, wherein H is the height of the object to be grabbed.

8. The industrial robot 3D vision positioning method according to claim 1, characterized in that, The specific way of analyzing the preselected force point according to the force height point is: The force point height Fi is obtained in descending order, and the height of the industrial robot gripper is recorded as Y. Then, the force point i corresponding to the force point height Fi less than 2Y / 3 is screened, and the screened force point is recorded as the preselected force point. The preselected force point is analyzed again.

9. The industrial robot 3D vision positioning method according to claim 9, characterized in that, The specific way of analyzing the preselected force point again is: All preselected stress points are acquired, and corresponding three-dimensional space coordinate points are acquired according to the preselected stress points and marked as (Xo, Yo, Zo), and then the acquired three-dimensional space coordinate points are substituted into the formula The distance Po between the force point and the origin is calculated, and the formula is (Xv, Yv, Zv) representing the origin coordinates. The preselected force point corresponding to the minimum distance Po value is selected, and the selected preselected force point is recorded as the optimal force point. The optimal force point information is generated.

10. An industrial robot 3D vision positioning system applying the industrial robot 3D vision positioning method according to any one of claims 1-9, characterized in that, The positioning system comprises an information acquisition unit, an information adaptive processing unit, a grabbing point analysis unit, a grabbing point selection unit and an information output unit. The information acquisition unit is used for acquiring the length, width and height information of the object to be grabbed, and The acquired information is transmitted to the information adaptive processing unit. The information adaptive processing unit is used for analyzing the image information of the object to be grabbed in combination with the acquired information, and obtaining the modified coordinates of the object to be grabbed by modifying the image information. The modified coordinates are transmitted to the grabbing point analysis unit. The grabbing point analysis unit is used for calculating the grabbing point of the object to be grabbed in combination with the acquired modified coordinates, and obtaining the force point height value according to the grabbing point. The force point height value is transmitted to the grabbing point selection unit. The grabbing point selection unit is used for calculating and analyzing the optimal grabbing point of the object to be grabbed in combination with the acquired force point height value, and generating the optimal grabbing point information. The generated optimal grabbing point information is transmitted to the information output unit. The information output unit is used for displaying the acquired optimal grabbing point information to the operator.

Images