Calibration method, calibration device and computer readable storage medium

Abstract

The present application discloses a calibration method, a calibration device and a computer-readable storage medium, the method comprising: placing a calibration plate at a fixed first position; manually teaching a robot to move to N different first observation postures in succession; respectively determining the posture description matrix when the robot is in the i-th first observation posture; when the posture of the camera relative to the terminal flange changes, placing the calibration plate at a fixed second position; obtaining the posture description matrix, the posture description matrix, the posture description matrix, the posture description matrix: controlling the robot to move to N second observation postures in succession; determining the posture description matrix of the calibration plate coordinate system in the camera coordinate system; determining the first calibration posture homogeneous matrix of the camera coordinate system in the flange coordinate system according to all the posture description matrices and all the posture description matrices. The calibration method of the present application can simplify the calibration process.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a calibration method, a calibration device, and a computer-readable storage medium. Background Art

[0002] When installing a 3D camera at the end of the robot to complete the task of visually guiding the robot's movement, it is usually necessary to calibrate the relative pose transformation relationship between the camera coordinate system and the robot's flange coordinate system. This process is called eye-on-hand calibration.

[0003] When performing hand-eye calibration with the eye on the hand, a calibration plate is required. However, since the placement of the calibration plate is random, it is impossible to control the robot to observe the calibration plate in the same posture every time. In addition, the relative relationship between the robot's different observation postures cannot be intuitively described for the operator, and the relative relationship between these observation postures determines the accuracy of the hand-eye calibration algorithm. Therefore, the current process of teaching the calibration position takes a lot of time. Summary of the invention

[0004] The present application provides a calibration method, a calibration device and a computer-readable storage medium, which can simplify the calibration process.

[0005] A first aspect of an embodiment of the present application provides a calibration method, which is applied to a robot having a camera and an end tool mounted on an end flange, and the method includes:

[0006] placing the calibration plate at a fixed first position;

[0007] Manually teaching the robot to move to N different first observation postures in sequence, wherein each time the robot is in the first observation posture, the calibration plate is within the acquisition field of view of the camera, and N is greater than or equal to 3;

[0008] Determine respectively when the robot is in each of the first observation postures, the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system of the camera, wherein when the robot is in the i-th first observation posture, the posture description matrix of the calibration plate coordinate system in the camera coordinate system is recorded as the posture description matrix , i is an integer from 1 to N;

[0009] When the position of the camera relative to the end flange changes, placing the calibration plate at a fixed second position;

[0010] Get the pose description matrix of the calibration plate coordinate system in the base coordinate system ;

[0011] Obtain the pose description matrix of the tool coordinate system of the end tool in the flange coordinate system ;

[0012] Determine the pose description matrix of the camera coordinate system in the tool coordinate system ;

[0013] Determine the pose description matrix according to the following formula The corresponding pose description matrix :

[0014] ;

[0015] Control the robot to move to N second observation postures in sequence, wherein when the robot is in the i-th second observation posture, the posture description matrix of the flange coordinate system in the base coordinate system is the posture description matrix ;

[0016] Respectively obtain the second point cloud data of the calibration plate collected by the camera when the robot is in the i-th second observation posture, and respectively determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th second observation posture according to each second point cloud data ;

[0017] According to all the pose description matrices And all the pose description matrices , determine the first calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0018] A second aspect of an embodiment of the present application provides a calibration device, which includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit, respectively. The memory stores program data. The processor implements the steps in the above method by executing the program data in the memory.

[0019] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the above method.

[0020] The beneficial effect is that in the solution of the present application, when the calibration plate is in a fixed first position, multiple posture description matrices of the calibration plate coordinate system {D} in the camera coordinate system {C} are obtained when the calibration plate is in the camera acquisition field of view. Then, after the camera's posture relative to the end flange changes, the multiple posture description matrices , calculate the pose description matrix of the flange coordinate system {F} in the base coordinate system {B} when the calibration plate is in the fixed second position so that the calibration plate is within the camera acquisition field of view , then according to the pose description matrix , control the robot to move to N second observation poses, and determine the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot is in each second observation pose , and finally, according to all the pose description matrices And all the pose description matrices , we can get the first calibration pose homogeneous matrix of the camera coordinate system {C} in the flange coordinate system {F} Through this calibration method, after the position of the camera relative to the end flange changes, the position of the calibration plate can be arbitrarily placed in the robot's workspace, without having to keep the position of the calibration plate consistent with the position in the previous calibration process, which can simplify the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0022] Figure 1 It is a structural schematic diagram of an implementation method of a calibration system of the present application;

[0023] Figure 2 It is a flow chart of an implementation method of the calibration method of the present application;

[0024] Figure 3 It is a structural schematic diagram of an implementation method of the calibration device of the present application;

[0025] Figure 4 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0028] See also Figure 1 , the mechanisms involved in the calibration process of the present application include a robot 110, a camera 120 mounted on the end flange of the robot, an end tool 130 mounted on the end flange of the robot, and a calibration plate 140 placed in the robot's motion space. The camera 120 is a three-dimensional camera that can collect three-dimensional point cloud data, wherein these three-dimensional point cloud data include but are not limited to XYZ coordinates, RGB colors, and normal vector information. The end tool 130 can be a calibration pin or a welding gun, and the present application does not limit the type of the end tool 130. The calibration plate 140 is a plate with a certain plane shape, such as a rectangular plane, a triangular plane, etc., that is, the calibration plate 140 can be a triangular calibration plate or a rectangular calibration plate, wherein the attached figure illustrates the calibration plate 140 as a triangular calibration plate, but the present application does not limit the shape of the calibration plate 140.

[0029] The base coordinate system of the robot 110 Refers to a fixed coordinate system defined on the base of the robot 110, flange coordinate system Refers to a fixed coordinate system defined on the end flange of the robot 110, the camera coordinate system Refers to a fixed coordinate system defined inside the camera 120 and used to describe the coordinates of the three-dimensional point cloud, the calibration plate coordinate system Refers to a fixed coordinate system defined at the center of the calibration plate 140. Tool coordinate system of the end tool 130 Refers to a fixed coordinate system defined on the end tool 130. The tool center point (TCP) of the end tool 130 is the tool coordinate system. The origin of .

[0030] The calibration pose homogeneous matrix refers to the camera coordinate system Relative to flange coordinate system The purpose of this application is to obtain the camera coordinate system of the camera 120 after the posture of the camera 120 on the end flange changes. Relative to flange coordinate system The pose transformation homogeneous matrix of .

[0031] Simultaneous pose transformation homogeneous matrix Representing coordinate system In the coordinate system The pose transformation in, for example, the matrix Indicates the calibration plate coordinate system In the camera coordinate system The pose description in .

[0032] In the prior art, for the calibration of the robot 110 with its eyes on the hand, it is necessary to manually teach the robot to move to an observation posture. This observation posture needs to satisfy the requirement that the camera 120 can collect the three-dimensional point cloud data of the complete calibration plate 140, that is, the calibration plate 140 must be within the collection field of view of the camera 120. At the same time, according to the three-dimensional point cloud data of the calibration plate 140 collected by the camera 120 at the observation posture, the description of the calibration plate coordinate system {D} in the camera coordinate system {C} can be determined. For each observation posture, the description of the calibration plate coordinate system {D} in the camera coordinate system {C} and the description of the flange coordinate system {F} in the base coordinate system {B} can be obtained.

[0033] The robot is taught to move to 10 to 20 different observation positions, and the description of the calibration plate coordinate system {D} in the camera coordinate system {C} and the description of the flange coordinate system {F} in the base coordinate system {B} corresponding to each observation position are obtained. Finally, the equation of the form MX=XN is solved by the Tasi algorithm, where X is the camera coordinate system of the camera 120 to be obtained. Relative to flange coordinate system The process of obtaining the pose transformation matrix by Tasi algorithm belongs to the prior art and will not be described in detail here.

[0034] The present application proposes a simple automatic eye-on-hand hand-eye calibration method, which can effectively ensure the stability and accuracy of the hand-eye calibration results, improve the efficiency of hand-eye calibration, and thus solve problems in practical applications.

[0035] See also Figure 2 , Figure 2 : is a flow chart of an implementation method of the present application, the method comprising:

[0036] S101: placing the calibration plate at a fixed first position.

[0037] Specifically, firstly, the calibration plate 140 is placed in a fixed position and the position of the calibration plate 140 is kept unchanged.

[0038] S102: The manual teaching robot moves to N different first observation positions one after another.

[0039] Each time when the robot 110 is in the first observation posture, the calibration plate 140 is within the acquisition field of view of the camera 120 , and N is greater than or equal to 3.

[0040] Specifically, the robot 110 is manually taught to move so that the robot 110 moves to N different first observation postures (N≥3) in succession, wherein the requirement for the first observation posture is that when the robot 110 is in the first observation posture, the calibration plate 140 is within the acquisition field of view of the camera 120, so that the camera 120 can acquire complete three-dimensional point cloud data of the calibration plate 140.

[0041] In one implementation, the value range of N is 10-20.

[0042] S103: Determine respectively a posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system of the camera when the robot is in each first observation posture.

[0043] When the robot 110 is in the i-th first observation posture, the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} is recorded as the posture description matrix .

[0044] Specifically, for the sake of convenience, the N first observation positions are numbered starting from 1, and the robot is moved to the i-th first observation position, and the calibration plate coordinate system is In the camera coordinate system The pose description matrix under is recorded as the pose description matrix .

[0045] It can be understood that, since the robot moves to each first observation position, the calibration plate 140 is within the acquisition field of view of the camera 120. Therefore, no matter where the calibration plate 140 is located, as long as the calibration plate coordinate system is guaranteed In the camera coordinate system The description matrix is , the calibration plate 140 can be placed within the acquisition field of view of the camera 120.

[0046] It should be noted that both step S102 and step S103 are performed based on the calibration plate 140 being in the fixed first position.

[0047] S104: When the position of the camera relative to the end flange changes, the calibration plate is placed at a fixed second position.

[0048] Specifically, when the position of the camera 120 relative to the end flange changes, the camera coordinate system needs to be re-determined. Relative to flange coordinate system The pose transformation homogeneous matrix of .

[0049] S105: Obtain the pose description matrix of the calibration plate coordinate system in the base coordinate system .

[0050] Specifically, since the calibration plate 140 is in a fixed second position, before the position of the calibration plate 140 is changed next time, the calibration plate coordinate system In the base coordinate system The pose description matrix is is fixed.

[0051] In one implementation, step S105 specifically includes:

[0052] S1051: Manually teach the robot to move so that the tool center point of the end tool contacts the three calibration points on the calibration plate in sequence, wherein the three calibration points are the three vertices of a right triangle.

[0053] Specifically, combined Figure 1 , teach the robot to move so that the tool center point TCP of the end tool 130 successively clicks on the three calibration points on the calibration plate 140 , as well as , three calibration points , as well as are the three vertices of a right triangle, and the calibration points is the right-angled vertex of the right triangle, that is, the calibration point , calibration point The formed straight line and the calibration point , calibration point The straight lines formed intersect.

[0054] S1052: Obtain the position of the tool center point in the tool coordinate system each time the tool center point contacts the calibration point.

[0055] S1053: Make the tool center point contact the calibration point The position of the tool center point at the bottom is determined as the calibration point At the position in the base coordinate system, the tool center point touches the calibration point The position of the tool center point in the base coordinate system is determined as the calibration point At the position in the base coordinate system, the tool center point touches the calibration point The position of the tool center point in the base coordinate system is determined as the calibration point The position in the base coordinate system.

[0056] Specifically, the tool center point TCP of the end tool 130 clicks the calibration point When the tool center point TCP is recorded in the base coordinate system It can be understood that this position is the calibration point. In the base coordinate system Similarly, the calibration point can be obtained in the same way. , In the base coordinate system The next position.

[0057] S1054: Construct the pose description matrix of the calibration plate coordinate system in the base coordinate system according to the following formula ;

[0058]

[0059] in, In the base coordinate system {B}, the calibration point , calibration point The vector formed between In the base coordinate system {B}, based on the calibration point Position and calibration points in the base coordinate system {B} The vector established at the position in the base coordinate system {B}, In the base coordinate system {B}, the calibration point , calibration point The vector formed between In the base coordinate system {B}, based on the calibration point Position and calibration points in the base coordinate system {B} The vector established at the position in the base coordinate system {B}, is the origin and calibration point of the base coordinate system {B} in the base coordinate system {B} The vector formed between is the origin position and calibration point in the base coordinate system {B} based on the base coordinate system {B} The vector established by the position in the base coordinate system {B}.

[0060] Understandably, For vector The model, For vector The model, For vector With vector The magnitude of the vector resulting from the multiplication.

[0061] S106: Obtain the posture description matrix of the tool coordinate system of the end tool in the flange coordinate system .

[0062] In one embodiment, the four-point method can be used for calibration to obtain the posture description matrix of the tool coordinate system {T} in the flange coordinate system {F}: .

[0063] S107: Determine the pose description matrix of the camera coordinate system in the tool coordinate system .

[0064] In one implementation, step S107 specifically includes:

[0065] S1071: Control the robot to move until the camera is directly above the calibration plate, and obtain the position description matrix of the flange coordinate system in the base coordinate system And the first point cloud data of the calibration plate collected by the camera.

[0066] Specifically, the robot 110 can be moved to the top of the calibration plate 140 by manual teaching, and the state at this time is defined as the initial state, and then the position description matrix of the flange coordinate system {F} in the base coordinate system {B} is recorded. At the same time, the three-dimensional point cloud data of the calibration plate 140 collected by the camera 120 in this state is also obtained, and the three-dimensional point cloud data is defined as the first point cloud data.

[0067] S1072: Determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system based on the first point cloud data .

[0068] Specifically, since the first point cloud data is the point cloud data of the calibration plate 140 collected by the camera 120, the position description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} in the initial state can be determined according to the first point cloud data: .

[0069] In one embodiment, step S1072 includes:

[0070] S10721: Extracting contour lines of the plane point cloud data from the first point cloud data.

[0071] Specifically, the contour line of the plane point cloud data is the outer contour of the calibration plate 140. When the calibration plate 140 is a triangular calibration plate, the extracted contour line is a triangle. When the calibration plate 140 is a quadrilateral calibration plate, the extracted contour line is a quadrilateral.

[0072] S10722: Identify the long right-angle sides and short right-angle sides in the contour line.

[0073] It should be noted that when the contour line extracted in step S10721 is a rectangle, one of the long sides of the rectangle is used as the long right-angled side, and the short side that intersects with the long side to form a right angle is used as the short right-angled side; when the contour line extracted in step S10721 is a square, one of the two sides that constitute one of the right angles is used as the long right-angled side, and the other side is used as the short right-angled side.

[0074] S10723: The calibration plate coordinate system is constructed with the intersection of the long right-angled side and the short right-angled side as the origin of the calibration plate coordinate system, the normal vector of the plane point cloud as the Z axis of the calibration plate coordinate system, and the long right-angled side as the X axis of the calibration plate coordinate system, thereby obtaining the pose description matrix .

[0075] Specifically, after extracting the contour line, identify the long right-angle side and the short right-angle side, then use the intersection of the long right-angle side and the short right-angle side as the origin of the calibration plate coordinate system {D}, use the normal vector of the plane point cloud as the Z axis of the calibration plate coordinate system {D}, and use the long right-angle side as the X axis of the calibration plate coordinate system {D} to construct the calibration plate coordinate system {D}. Since the calibration plate coordinate system {D} is established based on the camera coordinate system {C}, after obtaining the calibration plate coordinate system {D}, combined with the known camera coordinate system {C}, the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} can be obtained. .

[0076] S1073: Determine the pose description matrix of the camera coordinate system in the tool coordinate system according to the following formula :

[0077]

[0078] Specifically, the pose description matrix , pose description matrix , pose description matrix And the pose description matrix Substituting the above formula, we can get the pose description matrix of the camera coordinate system {C} in the tool coordinate system {T}: .

[0079] S108: Determine the posture description matrix according to formula 1 The corresponding pose description matrix :

[0080] Formula 1: ;

[0081] Specifically, after obtaining the pose description matrix Then, the pose description matrix obtained above is , pose description matrix , pose description matrix And the pose description matrix Substitute the above formula to get the pose description matrix .

[0082] The above formula can be used to determine each pose description matrix The corresponding pose description matrix .

[0083] S109: Control the robot to move to N second observation positions in sequence.

[0084] Among them, when the robot 110 is in the i-th second observation posture, the posture description matrix of the flange coordinate system in the base coordinate system is the posture description matrix .

[0085] Specifically, it can be seen from the above analysis that no matter where the calibration plate 140 is located, as long as the calibration plate coordinate system is In the camera coordinate system The description matrix is , the calibration plate 140 can be placed within the acquisition field of view of the camera 120.

[0086] The formula in step S108 can be roughly calculated to obtain the pose description matrix of the flange coordinate system {F} in the base coordinate system {B} when the calibration plate 140 is in the fixed second position and the calibration plate 140 is to be within the acquisition field of view of the camera 120. , so we get the pose description matrix Then, the robot 110 is controlled to move so that when the robot is in the i-th second observation posture, the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} is the posture description matrix , that is, when the robot is in the i-th second observation posture, the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} is the posture description matrix .

[0087] Therefore, the above steps can ensure that each time the robot is in the second observation posture, the camera 120 can collect the complete point cloud data of the calibration plate 140.

[0088] S110: respectively obtain the second point cloud data of the calibration plate collected by the camera when the robot is in the i-th second observation posture, and respectively determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th second observation posture according to each second point cloud data .

[0089] Specifically, according to the second point cloud data, determine the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} The above step S107 determines the position description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} according to the first point cloud data The specific process is the same as that of , and the details can be found in the above-mentioned related content, which will not be repeated here.

[0090] S111: Describe the matrix based on all the poses And all the pose description matrices , determine the first calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0091] After getting all the pose description matrices And all the pose description matrices After that, the Tasi algorithm can be used to solve the equation of the form MX=XN, where X is the homogeneous matrix of the first calibration pose required. Specifically, the process includes:

[0092] Determine the first calibration pose homogeneous matrix according to the following formula :

[0093]

[0094]

[0095]

[0096] Wherein, j and k are both integers from 1 to N.

[0097] Among them, the above is obtained by Tasi algorithm The process belongs to the prior art and will not be described in detail here.

[0098] It can be seen from the above that in the solution of the present application, when the calibration plate 140 is in the fixed first position, multiple posture description matrices of the calibration plate coordinate system {D} in the camera coordinate system {C} are obtained when the calibration plate 140 is in the acquisition field of view of the camera 120. Then, after the posture of the camera 120 relative to the end flange changes, according to the multiple posture description matrices , calculate the pose description matrix of the flange coordinate system {F} in the base coordinate system {B} when the calibration plate 140 is in the fixed second position so that the calibration plate 140 is within the camera 120 acquisition field of view , then according to the pose description matrix , control the robot 110 to move to N second observation postures, and determine the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot is in each second observation posture , and finally, according to all the pose description matrices And all the pose description matrices , we can get the first calibration pose homogeneous matrix of the camera coordinate system {C} in the flange coordinate system {F} By using this method, after the position of the camera 120 relative to the end flange changes, the position of the calibration plate 140 can be arbitrarily placed in the working space of the robot 110, without having to keep the position of the calibration plate 140 consistent with the position in the previous calibration process, which can simplify the calibration process.

[0099] In one implementation, step S103 specifically includes:

[0100] S1031: Use manual calibration to determine the reference calibration pose homogeneous matrix of the camera coordinate system relative to the flange coordinate system .

[0101] Specifically, after the calibration plate 140 is placed at the first position, any method in the prior art may be used to determine the camera coordinate system of the camera 120. Relative to flange coordinate system The reference positioning pose homogeneous matrix of .

[0102] S1032: Obtaining the posture description matrix of the calibration plate coordinate system in the base coordinate system when the calibration plate is in the first position .

[0103] Specifically, since the calibration plate 140 is in a fixed first position, before the position of the calibration plate 140 is changed next time, the calibration plate coordinate system In the base coordinate system The pose description matrix is is fixed, wherein the pose description matrix can be determined by any method in the prior art .

[0104] In one embodiment, the pose description matrix is ​​determined The process is the same as that described above for determining the pose description matrix The process is the same as that of , please refer to the detailed description of the relevant content, which will not be repeated here.

[0105] S1033: Obtain the posture description matrix of the flange coordinate system in the base coordinate system when the robot moves to the i-th first observation posture .

[0106] Specifically, each time the robot moves to a first observation posture, the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} is recorded, and the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} corresponding to the i-th first observation posture is recorded as the posture description matrix .

[0107] S1034: Determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th first observation posture according to the following formula :

[0108] .

[0109] Specifically, the reference pose homogeneous matrix , pose description matrix , pose description matrix Substituting the above formula, we can get the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot is in the i-th first observation pose: .

[0110] In one embodiment, step S1031 determines the reference landmark pose homogeneous matrix The process specifically includes:

[0111] S10311: Obtain the third point cloud data of the calibration plate collected by the camera each time the robot is in the first observation posture.

[0112] S10312: Determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system each time the robot is in the i-th first observation posture according to each third point cloud data. .

[0113] Among them, according to the third point cloud data, determine the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} The specific process is the same as the above step S1072. Determine the pose description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} according to the first point cloud data. The specific process is the same as that of , and the details can be found in the above-mentioned related content, which will not be repeated here.

[0114] S10313: Obtain the posture description matrix of the flange coordinate system in the base coordinate system when the robot moves to the i-th first observation posture .

[0115] S10314: Describe the matrix based on all the poses And all the pose description matrices , determine the reference orientation homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0116] Specifically, after obtaining all the pose description matrices And all the pose description matrices Then, the reference calibration pose homogeneous matrix of the camera coordinate system {C} in the flange coordinate system {F} is obtained by the Tasi algorithm. , where the reference positioning pose homogeneous matrix is ​​obtained by Tasi algorithm The specific process belongs to the prior art and will not be introduced here.

[0117] It should be noted that, in other implementations, the position description matrix of the calibration plate coordinate system {D} obtained in the manual calibration process in the camera coordinate system {C} can also be Directly used as the calibration plate coordinate system In the camera coordinate system The pose description matrix is That is to say, step S103 includes: obtaining the third point cloud data of the calibration plate 140 collected by the camera 120 each time the robot 110 is in the first observation posture, and determining the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} each time the robot 110 is in the i-th first observation posture according to each third point cloud data. , and the pose description matrix As .

[0118] In order to better understand the solution of this application, a specific example is provided below for explanation:

[0119] S1: placing the calibration plate 140 at a fixed first position.

[0120] S2: The manual teaching robot 110 moves to N different first observation postures (at each first observation posture, the calibration plate 140 is within the acquisition field of view of the camera 120), and records the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} at the i-th first observation posture .

[0121] S3: Obtain the third point cloud data of the calibration plate 140 collected by the camera 120 each time the robot 110 is in the first observation posture, and determine the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot 110 is in the i-th first observation posture according to the third point cloud data .

[0122] S4: Describe the matrix based on all the poses And all the pose description matrices , determine the reference calibration pose homogeneous matrix of the camera coordinate system {C} in the flange coordinate system {F} .

[0123] S5: Obtaining the posture description matrix of the calibration plate coordinate system {D} in the base coordinate system {B} when the calibration plate 140 is in the first position The specific process includes: teaching the robot to move so that the tool center point TCP of the end tool 130 successively clicks on the three calibration points on the calibration plate 140 , as well as , thereby obtaining the calibration points , , In the base coordinate system The position under the calibration plate coordinate system {D} is constructed according to the following formula to describe the position and posture of the calibration plate coordinate system {D} under the base coordinate system {B} ;

[0124]

[0125] S6: Determine the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot is in the i-th first observation posture according to the following formula :

[0126] .

[0127] Alternatively, you can directly use the pose description matrix in step S3 As a pose description matrix , without executing steps S4 to S6.

[0128] S7: After the position of the camera 120 relative to the end flange changes, the calibration plate 140 is placed in a fixed second position, and the second position is not necessarily the same as the first position.

[0129] S8: Get the pose description matrix of the calibration plate coordinate system {D} in the base coordinate system {B} .

[0130] S9: Obtaining the posture description matrix of the tool coordinate system {T} of the end tool 130 in the flange coordinate system {F} .

[0131] S10: Control the robot 110 to move until the camera 120 is directly above the calibration plate 140, and obtain the position description matrix of the flange coordinate system {F} in the base coordinate system {B} The first point cloud data of the calibration plate 140 collected by the camera 120 is used to determine the position description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} according to the first point cloud data. .

[0132] S11: Determine the pose description matrix of the camera coordinate system {C} in the tool coordinate system {T} according to the following formula :

[0133] .

[0134] S12: Determine the pose description matrix according to the following formula The corresponding pose description matrix :

[0135] ;

[0136] S13: Control the robot 110 to move to N second observation postures in succession, wherein when the robot 110 is in the i-th second observation posture, the posture description matrix of the flange coordinate system {F} in the base coordinate system {B} is the posture description matrix .

[0137] S14: respectively obtain the second point cloud data of the calibration plate 140 collected by the camera 120 when the robot 110 is in the i-th second observation posture, and respectively determine the posture description matrix of the calibration plate coordinate system {D} in the camera coordinate system {C} when the robot 110 is in the i-th second observation posture according to each second point cloud data .

[0138] S15: Describe the matrix based on all the poses And all the pose description matrices , determine the first calibration pose homogeneous matrix of the camera coordinate system {C} in the flange coordinate system {F} .

[0139] See also Figure 3 , Figure 3 1 is a schematic diagram of the structure of an embodiment of the calibration device of the present application. The calibration device 200 includes a processor 210, a memory 220 and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 implements the steps in any of the above-mentioned implementation methods by executing the program data in the memory 220. The detailed steps can be found in the above-mentioned implementations and will not be repeated here.

[0140] The calibration device 200 may be any device with algorithm processing capability, such as a computer, a mobile phone, a robot control cabinet, etc., and is not limited here.

[0141] See also Figure 4 , Figure 4 The computer-readable storage medium 400 stores a computer program 410, which can be executed by a processor to implement the steps in any of the above methods.

[0142] The computer-readable storage medium 400 may specifically be a device that can store the computer program 410, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the computer program 410. The server may send the stored computer program 410 to other devices for execution, or may also execute the stored computer program 410 by itself.

[0143] The above description is only an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A calibration method, characterized in that: The method is applied to a robot having a camera and an end tool mounted on an end flange, and the method comprises: placing the calibration plate at a fixed first position; Manually teaching the robot to move to N different first observation postures in sequence, wherein each time the robot is in the first observation posture, the calibration plate is within the acquisition field of view of the camera, and N is greater than or equal to 3; Determine respectively when the robot is in each of the first observation postures, the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system of the camera, wherein when the robot is in the i-th first observation posture, the posture description matrix of the calibration plate coordinate system in the camera coordinate system is recorded as the posture description matrix , i is an integer from 1 to N; When the position of the camera relative to the end flange changes, placing the calibration plate at a fixed second position; Get the pose description matrix of the calibration plate coordinate system in the base coordinate system ; Obtain the pose description matrix of the tool coordinate system of the end tool in the flange coordinate system ; Determine the pose description matrix of the camera coordinate system in the tool coordinate system ; Determine the pose description matrix according to the following formula The corresponding pose description matrix : ； Control the robot to move to N second observation postures in sequence, wherein when the robot is in the i-th second observation posture, the posture description matrix of the flange coordinate system in the base coordinate system is the posture description matrix ; Respectively obtain the second point cloud data of the calibration plate collected by the camera when the robot is in the i-th second observation posture, and respectively determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th second observation posture according to each second point cloud data ; According to all the pose description matrices And all the pose description matrices , determine the first calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

2. The method according to claim 1, characterized in that The step of respectively determining the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system of the camera when the robot is in each of the first observation postures comprises: Using manual calibration, determine the reference calibration pose homogeneous matrix of the camera coordinate system relative to the flange coordinate system ; Obtaining a posture description matrix of the calibration plate coordinate system in the base coordinate system when the calibration plate is in the first position ; Obtain the posture description matrix of the flange coordinate system in the base coordinate system when the robot moves to the i-th first observation posture ; Determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th first observation posture according to the following formula: : 。 3. The method according to claim 2, characterized in that The manual calibration method is used to determine the reference calibration pose homogeneous matrix of the camera coordinate system relative to the flange coordinate system. The steps include: respectively acquiring the third point cloud data of the calibration plate collected by the camera each time the robot is in the first observation posture; According to each of the third point cloud data, determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system each time the robot is in the i-th first observation posture ; Obtain the posture description matrix of the flange coordinate system in the base coordinate system when the robot moves to the i-th first observation posture ; According to all the pose description matrices And all the pose description matrices , determine the reference mark pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

4. The method according to claim 1, characterized in that: The step of obtaining the position description matrix of the calibration plate coordinate system in the base coordinate system The steps include: Manually teach the robot to move so that the tool center point of the end tool sequentially contacts the three calibration points on the calibration plate, wherein the three calibration points are the three vertices of a right triangle, and the three calibration points are sequentially defined as calibration points , calibration point And calibration points , the calibration point is the right-angled vertex of the right triangle; Obtaining the position of the tool center point in the base coordinate system each time the tool center point contacts the calibration point; The tool center point touches the calibration point When the position of the tool center point in the base coordinate system is determined as the calibration point At the position in the base coordinate system, the tool center point touches the calibration point When the position of the tool center point in the base coordinate system is determined as the calibration point At the position in the base coordinate system, the tool center point touches the calibration point When the position of the tool center point in the base coordinate system is determined as the calibration point The position in the base coordinate system; The pose description matrix of the calibration plate coordinate system in the base coordinate system is constructed according to the following formula : in, is in the base coordinate system, based on the calibration point The position in the base coordinate system, the calibration point The vector established at the position in the base coordinate system, is in the base coordinate system, based on the calibration point The position in the base coordinate system, the calibration point The vector established at the position in the base coordinate system, is a coordinate system based on the origin of the base coordinate system and the calibration point The vector established at the position in the base coordinate system.

5. The method according to claim 1, characterized in that Determining the pose description matrix of the camera coordinate system in the tool coordinate system The steps include: Control the robot to move until the camera is directly above the calibration plate, and obtain the position description matrix of the flange coordinate system in the base coordinate system and first point cloud data of the calibration plate collected by the camera; Determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the first point cloud data ; Determine the pose description matrix of the camera coordinate system in the tool coordinate system according to the following formula ; 。 6. The method according to claim 5, characterized in that The step of determining a pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the first point cloud data The steps include: Extracting contour lines of plane point cloud data from the first point cloud data; Identify the long right-angle side and the short right-angle side in the contour straight line; The calibration plate coordinate system is constructed by taking the intersection of the long right-angled side and the short right-angled side as the origin of the calibration plate coordinate system, taking the normal vector of the plane point cloud as the Z axis of the calibration plate coordinate system, and taking the long right-angled side as the X axis of the calibration plate coordinate system, thereby obtaining the posture description matrix .

7. The method according to claim 1, characterized in that According to all the posture description matrices And all the pose description matrices , determine the first calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system The steps include: The first calibration pose homogeneous matrix is ​​determined according to the following formula: : Wherein, j and k are both integers from 1 to N.

8. The method according to claim 1, characterized in that The end tool is a calibration ejector pin, and the calibration plate is a triangular calibration plate.

9. A calibration device, characterized in that: The calibration device includes a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor implements the steps in the method as described in any one of claims 1 to 8 by executing the program data in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images