Robot hand-eye calibration method and device based on laser displacement sensor

Abstract

The invention discloses a robot hand-eye calibration method and device based on a laser displacement sensor, and the method comprises the steps: controlling an end effector of a robot to move, enabling a laser beam to be projected into a target region of a calibration plate, collecting a two-dimensional image of a laser point, and obtaining the pixel coordinate of the laser point; acquiring a two-dimensional image of the calibration plate including the target area through a two-dimensional image sensor, and calculating to obtain a pixel coordinate of a central point of the target area; a robot end effector is controlled to move randomly, real-time pixel coordinates of the laser points on the calibration plate and corresponding robot end poses are obtained, and a transformation matrix of the light spot pixel coordinates and the robot end poses is calculated based on the real-time pixel coordinates of the laser points on the calibration plate and the corresponding robot end poses; a robot end executor is controlled to move to enable a laser point to coincide with the center point of the target area, the real-time robot end pose is collected, and displacement data measured by a laser displacement sensor are obtained; and calculating calibration parameters according to the data.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot calibration, and in particular relates to a robot hand-eye calibration method and device based on a laser displacement sensor. Background Art

[0002] Hand-eye calibration is an important technology in the field of robotics. It is used to determine the relative position relationship between the robot and the visual sensor, thereby achieving precise visual-guided operation and precise control of the robot's position and posture. At present, most hand-eye calibration methods in the existing technology rely on offline calibration plates or specific calibration workpieces. These methods require manual operation and the calibration process is complicated and easily affected by environmental factors and operator experience.

[0003] At present, the hand-eye calibration method based on laser displacement sensors mainly relies on manual operation, which is specifically manifested in that the operator needs to observe and compare the alignment of the laser point with the calibration target point through the human eye. This method is highly dependent on the operator's experience and visual judgment, and there is a significant risk of error. On the one hand, it is difficult for the human eye to accurately align under micron-level precision requirements, especially in long-term calibration operations, and is easily affected by fatigue and subjective judgment; on the other hand, due to differences in visual ability and experience level among different operators, the consistency of calibration results is poor, which directly affects the accuracy and stability of the robot's subsequent work. In addition, the manual alignment process is cumbersome and inefficient, and it is difficult to meet the needs of modern industry for efficient automation. These shortcomings limit the promotion and application of calibration technology based on laser displacement sensors in high-precision and high-consistency scenarios.

[0004] At present, there are significant technical bottlenecks in the automatic hand-eye calibration method based on laser displacement sensors. The biggest difficulty lies in how to reduce dependence on manual operation and improve calibration accuracy. On the one hand, there are subjective errors in the alignment of laser points by the human eye, especially in complex environments, where factors such as light reflection and occlusion will further aggravate the generation of errors; on the other hand, the calibration requirements between different robot brands and visual sensors vary significantly, resulting in a lack of universality in existing technologies. For example, some brands of robots need to repeatedly adjust the angle and position of the laser point to complete a specific calibration, while other brands of robots may need to achieve real-time calibration in dynamic motion, which places higher requirements on the adaptability of the calibration algorithm. In addition, in the process of automatic calibration, how to efficiently map the spatial relationship between the laser point and the feature points of the calibration plate to the robot hand-eye system is also a major problem in technical implementation, which is to ensure the accuracy of data processing and avoid complex manual intervention. Summary of the invention

[0005] In order to overcome one or more of the above-mentioned technical defects, the present invention provides a robot hand-eye calibration method and device based on a laser displacement sensor, which combines the pixel coordinates of the laser point and the pixel coordinates of the center point of the target area to calculate the alignment error between the robot's end posture and the laser point, improves the calibration accuracy and consistency, and reduces the impact of human intervention on the calibration.

[0006] In order to solve the above problems, the present invention provides a robot hand-eye calibration method based on a laser displacement sensor, comprising:

[0007] Step 1: Control the robot end effector to move and project the laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point;

[0008] Step 2: using a two-dimensional image sensor to collect a two-dimensional image of the calibration plate including the target area, and calculating the pixel coordinates of the center point of the target area;

[0009] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose, and calculate the transformation matrix between the spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose;

[0010] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end position and posture, and obtain the displacement data measured by the laser displacement sensor;

[0011] Repeat steps 1 to 4 to collect multiple sets of data and calculate the calibration parameters.

[0012] Furthermore, the collecting a two-dimensional image of the laser point by a two-dimensional image sensor and obtaining the pixel coordinates of the laser point includes:

[0013] Collect a two-dimensional image of the laser point in the target area;

[0014] The collected two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0015] Furthermore, the collecting of a two-dimensional image of the calibration plate including the target area by a two-dimensional image sensor and calculating the pixel coordinates of the center point of the target area include:

[0016] Acquire a two-dimensional image of the calibration plate;

[0017] Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration plate;

[0018] The Sobel operator is used to perform edge detection on the calibration plate image after grayscale conversion to obtain a binary image;

[0019] Perform Hough circle detection calculation on the binary image, identify the circular target area in the calibration plate, and obtain the center coordinates of the target area.

[0020] Furthermore, the robot end effector is controlled to move randomly, the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture are obtained, and the transformation matrix of the spot pixel coordinates and the robot end posture is calculated based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture, including:

[0021] When the laser point moves randomly on the calibration plate following the action of the robot end effector, the first pixel coordinate of the laser point on the calibration plate is obtained;

[0022] Controlling the robot end effector to perform a first action, obtaining a second pixel coordinate of the laser point on the calibration plate, calculating a difference between the second pixel coordinate and the first pixel coordinate, and obtaining first data;

[0023] Controlling the robot end effector to perform a second action, obtaining a third pixel coordinate of the laser point on the calibration plate, calculating a difference between the third pixel coordinate and the second pixel coordinate, and obtaining second data;

[0024] The transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot is calculated according to the first data and the second data.

[0025] Furthermore, the calculation of calibration parameters includes:

[0026] The rotation matrix and position vector between the robot end flange coordinate system and the robot base coordinate system are obtained through the robot forward kinematics;

[0027] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system;

[0028] According to the transformation matrix between the robot end flange coordinate system and the robot base base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, the relationship between the laser displacement sensor coordinate system and the robot base base coordinate system is calculated.

[0029] The present invention also provides a robot hand-eye calibration device based on a laser displacement sensor, which is used to implement the above-mentioned robot hand-eye calibration method, comprising a laser displacement sensor, a two-dimensional image sensor, a calibration board and a control module, wherein the laser displacement sensor and the two-dimensional image sensor are both fixed on the robot end effector, the calibration board is arranged within the working field of the two-dimensional image sensor, the laser displacement sensor irradiates a laser beam onto the calibration board to form a laser spot as the robot end effector moves, and the laser spot on the calibration board is collected by the two-dimensional image sensor;

[0030] The control module is used to control the movement of the robot end effector, control the laser displacement sensor to emit a laser beam, receive displacement data measured by the laser displacement sensor and receive a two-dimensional image captured by the two-dimensional image sensor, and calculate calibration parameters based on the received data and image.

[0031] The present invention also provides a robot hand-eye calibration device based on a laser displacement sensor, comprising a memory and a processor, wherein the memory stores a program or instruction, and the processor implements the steps of the above method when executing the program or instruction.

[0032] The present invention also provides a readable storage medium on which a program or an instruction is stored. When the program or the instruction is executed by a processor, the steps of the above method are implemented.

[0033] The present invention also provides a computer program product, which is stored in a storage medium and implements the steps of the above-mentioned calibration method when the computer program product is executed by at least one processor.

[0034] The present invention also provides a robot assembly, comprising:

[0035] robot;

[0036] The robot hand-eye calibration device as described above; and / or

[0037] A readable storage medium as described above; and / or

[0038] A computer program product as described above.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The invention discloses a robot hand-eye calibration method and device based on a laser displacement sensor, step 1, control the robot end effector to move to project a laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point; step 2, collect the two-dimensional image of the calibration plate including the target area through the two-dimensional image sensor, and calculate the pixel coordinates of the center point of the target area; step 3, control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture, and calculate the transformation matrix of the spot pixel coordinates and the robot end posture based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture; step 4, control the robot end effector to move to overlap the laser point with the center point of the target area, collect the real-time robot end posture, and obtain the displacement data measured by the laser displacement sensor; repeat steps 1 to 4, collect multiple groups of data, and calculate the calibration parameters. Combined with the pixel coordinates of the laser point and the pixel coordinates of the center point of the target area, the alignment error of the robot end posture and the laser point is calculated to improve the calibration accuracy and consistency, and reduce the influence of human intervention on the calibration. In addition, multiple sets of laser point coordinates are obtained by controlling the random movement of the robot end effector, and an efficient and stable calibration process is formed through transformation matrix solution and calibration, so that this method can adapt to different robot and sensor configurations and achieve cross-platform compatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings, wherein:

[0042] Figure 1 This is a flow chart of a robot hand-eye calibration method based on a laser displacement sensor according to an embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the structure of a robot hand-eye calibration device based on a laser displacement sensor according to an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of various coordinates during calculation in step S5 of the robot hand-eye calibration method based on a laser displacement sensor according to an embodiment of the present invention;

[0045] Figure 4 It is a schematic structural diagram of another robot hand-eye calibration device based on a laser displacement sensor according to an embodiment of the present invention;

[0046] Description of symbols: 1. Robot; 2. Laser displacement sensor; 3. Distance measuring laser beam; 4. Calibration plate; 5. Two-dimensional image sensor; 210. Memory; 220. Processor. DETAILED DESCRIPTION

[0047] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0048] In the description of the present invention, the term "plurality" refers to two or more than two. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship described in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention. The terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0049] In the description of the present invention, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the present invention, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0050] The embodiment of the present invention discloses a robot hand-eye calibration device based on a laser displacement sensor, such as Figure 1 , including a laser displacement sensor 2, a two-dimensional image sensor 5, a calibration board 4 and a control module. The laser displacement sensor 2 and the two-dimensional image sensor 5 are both fixed on the end effector of the robot 1. The calibration board 4 is set within the working field of the two-dimensional image sensor 5. The laser displacement sensor 2 irradiates the laser beam on the calibration board 4 to form a laser point as the end effector of the robot 1 moves. The laser point on the calibration board 4 is collected by the two-dimensional image sensor 5. The control module is used to control the movement of the end effector of the robot 1, control the laser displacement sensor 2 to emit a laser beam, receive the displacement data measured by the laser displacement sensor 2 and receive the two-dimensional image collected by the two-dimensional image sensor 5, and calculate the calibration parameters according to the received data and image. Specifically, the two-dimensional image sensor 5 is a camera.

[0051] In one embodiment, the present invention discloses a robot hand-eye calibration method based on a laser displacement sensor, which is applied to the above-mentioned robot hand-eye calibration device based on a laser displacement sensor, such as Figure 2 , methods include:

[0052] S1. Control the robot end effector to move and project the laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point.

[0053] In this embodiment, step S1 includes:

[0054] Collect a two-dimensional image of the laser point in the target area.

[0055] The collected two-dimensional image is binarized to obtain the pixel coordinates P1 of the laser point, which are the two-dimensional coordinates of the laser point in the two-dimensional image.

[0056] S2. Capture a two-dimensional image of the calibration plate including the target area through a two-dimensional image sensor, and calculate and obtain the pixel coordinates P2 of the center point of the target area.

[0057] In this embodiment, step S2 includes:

[0058] Acquire a 2D image of the calibration plate.

[0059] Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration plate.

[0060] Specifically, the Gaussian filtering formula is:

[0061]

[0062] Assume the initial two-dimensional image matrix is ​​T 1 , the Gaussian filter matrix is ​​T g , perform Gaussian filtering on the initial two-dimensional image to remove noise interference and obtain the Gaussian filtered image T 2 ;

[0063] T 2 =T g T 1

[0064] The Gaussian filtered image T 2 To perform grayscale conversion:

[0065] Grayscale Where (x, y) is the image T 2 The coordinates of each pixel in .

[0066] The Sobel operator is used to perform edge detection on the calibration plate image after grayscale conversion to obtain a binary image.

[0067] Specifically, the Sobel operator is used to perform edge detection calculations, calculate the edges of all closed figures in the two-dimensional image, and obtain the binary image T of edge detection from the initial two-dimensional image. 4The Sobel operator is used to calculate the grayscale approximation of the image brightness function. Using the Sobel operator at any point in the image, the corresponding grayscale vector or its normal vector is generated.

[0068] Assume that the two convolution factors of Sobel are d x d y ,

[0069]

[0070] The gradient magnitude of the image Image gradient direction

[0071] Calculate the binary image after edge detection: T 4 =S·T 2 .

[0072] By extracting the pixel points with the local maximum gradient, we can get a binary image T with only edges. 4 .

[0073] Perform Hough circle detection calculation on the binary image, identify the circular target area in the calibration plate, and obtain the center coordinates of the target area.

[0074] Specifically, the calculation formula for Hough circle detection is the circle equation in the Cartesian space coordinate system: (xa) 2 +(yb) 2 =r 2

[0075] The equation of a circle in Hough transform is:

[0076] Each group (a, b, r) represents a circle passing through the point (x0, y0). By setting a threshold according to the size of r of the target circle, the parameter of r can be determined. Initialize the circle center space N(a, b), and set all N(a, b) = 0. Traverse all non-zero pixels in the Canny edge binary image, draw lines along the gradient direction (vertical direction of the tangent), and add N(a, b) + = 1 for all points (a, b) in the accumulator through which the line segment passes. When there are multiple line segments passing through a certain pixel point, the accumulator recognizes it as a circle. The (a, b) in the accumulator is the center position P2 of the circular target area in the image.

[0077] S3. Control the robot end effector to move randomly in a small range, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose, and calculate the transformation matrix between the calibration plate and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose.

[0078] In this embodiment, step S3 includes:

[0079] The calibration plate is set parallel to the ground. Let the image-calibration plate coordinate system be {P}, let the robot base base coordinate system be {B}, let the center coordinate point o of the circular target area in the calibration plate be known as (uo, vo), that is, P2(a, b) obtained by the accumulator in step S2; establish the coordinate system transformation matrix method:

[0080] The laser point moves randomly on the calibration plate following the movement of the robot end effector. When the laser point hits the lower left corner of the image, the first pixel coordinate a (ua, va) of the laser point on the calibration plate is obtained.

[0081] Control the robot end effector to move △X along the X-axis direction of {B}, obtain the second pixel coordinate b(ub, vb) of the laser point on the calibration plate, calculate the difference between the second pixel coordinate b(ub, vb) and the first pixel coordinate a(ua, va) under {P}, and obtain the X-axis direction and step size under the image-calibration plate coordinate system {P}:

[0082] α=((ub-ua),(vb-va))

[0083] Control the robot end effector to move △Y along the Y-axis direction under {B}, obtain the third pixel coordinate c(uc, vc) of the laser point on the calibration plate, calculate the difference between the third pixel coordinate c(uc, vc) and the second pixel coordinate b(ub, vb) under {P}, and obtain the Y-axis direction and step length under the image-calibration plate coordinate system {P}:

[0084] β=((uc-ub),(vc-vb))

[0085] Assume that there exists a set of n and m values ​​that can make any laser point Pi(ui,vi) under {P} coincide with point o, then:

[0086]

[0087] Among them, the specific coordinate values ​​of the center point o (uo, vo) and the laser point coordinates (ui, vi) are the known results of image recognition; if the above equation has a solution, then there is only one valid solution n, m. It can be seen that as long as △X and △Y are moved n, m times the step length in the robot base coordinate system {B}, the laser point Pi can be made to coincide with the center point o in the image-calibration plate coordinate system {P}, and the transformation matrix R between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot can be calculated.

[0088] S4, control the movement of the robot end effector to make the laser point P1 coincide with the center point P2 of the target area, collect the real-time robot end position and posture, and obtain the displacement data measured by the laser displacement sensor.

[0089] S5. Repeat S1-S4, collect nine sets of data, and calculate the hand-eye calibration parameters based on the laser point calibration method.

[0090] Specifically, the calculation of hand-eye calibration parameters includes:

[0091] like Figure 3 , let the robot base coordinate system be {B}, the robot end flange coordinate system be {E}, the laser displacement sensor coordinate system be {T}, and the origin O be fixed at a certain point L on the emitted laser line.

[0092] The rotation matrix and position vector between the robot end flange coordinate system {E} and the robot base coordinate system {B} are obtained through the robot forward kinematics.

[0093] Coordinate transformation of {T} relative to {B} for:

[0094]

[0095] in, represents the transformation matrix of {E} relative to {B}, It can be obtained by solving the robot forward kinematics in the classic DH model. middle is the rotation matrix of {T) relative to {E}, P is the position vector of {T} under {E}, By decomposing Three vector components. It can be seen that the position calibration of the laser displacement sensor is to calibrate the rotation matrix and position vector between {T} and {E}.

[0096] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system and the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system {E} and the laser displacement sensor coordinate system {T}.

[0097] Assume that the coordinates of the origin of the laser displacement sensor, i.e., the TCP point, under {E} are (x0, y0, z0), the direction vector of the emitted laser line under {E} is (nx, ny, nz), and the distance between the measured theoretical target point and the origin of the laser displacement sensor is the sensor reading. Then, the coordinates of the measured theoretical target point under {E} are:

[0098] x=x 0 +nL

[0099] Where x = (x, y, z) T , x 0 =(x 0 ,y 0 , z0 ) T and n=(n x , n y , n z ) T .

[0100] Then the coordinate value of the theoretical target point under {B} is: Bi =R i (x 0 +n Li )+P i

[0101] matrix is the transformation matrix of {E} relative to {B}, Obtained by solving the robot's forward kinematics, according to the formula, R i is the rotation matrix of {E} relative to {B}, P i is the position vector of {E} under {B}. From the robot forward kinematics, we know that R i is a 3×3 matrix, P i It is a 3×1 matrix, and the transformation matrix represented by the quaternion method can be read out in the robot teaching pendant.

[0102] Assume that each time we hit the same laser point X Bi remain unchanged, then:

[0103] X Bi =X B(i+1)

[0104] R i (x 0 +nL i )+P i =R i+1 (x 0 +nL i+1 )+P i+1

[0105] If the robot flange posture is kept unchanged, the robot flange is adjusted (such as changing Z first, then XY movement) so that the laser always hits the same point, the above formula can be changed to:

[0106] R 0 nL i +P i =R 0 nL i+1 +P i+1

[0107]

[0108] Among them, R 0 , P i+1 , Pi The quaternion can be directly read by the robot's own teaching pendant, and Li and Li+1 can be directly read from the laser displacement sensor. When the constraint point is a point, it is transformed into a constraint problem in the optimization problem, and the direction vector n can be obtained by solving the linear equation group An B.

[0109] A=[R 0 R 0 …R 0 ] T (3(n-1)×3)

[0110]

[0111] Among them, R 0 , P i It can be directly read through the robot teaching pendant and then applied to quaternion calculation, L i It can be directly read from the laser displacement sensor, and the laser sensor origin position x0 and Z-axis direction under {E} can be obtained by calculation.

[0112] According to the quaternion law, the directions of the X and Y axes can be arbitrarily defined according to the actual situation when the right-hand rule and orthogonality are satisfied. The cross product of the flange Z axis (0,0,1) and the common normal of n is γ (γx,γy,γz) with an angle of θ. According to the definition, the position of the laser displacement sensor can be obtained by rotating {E} around γ by θ, and the laser displacement sensor coordinate system {T} can be expressed as:

[0113]

[0114] According to the rotation matrix and position vector between the robot end flange coordinate system {E} and the robot base base coordinate system {B}, and the rotation matrix and position vector between the robot end flange coordinate system {E} and the laser displacement sensor coordinate system {T}, the relationship between the laser displacement sensor coordinate system {T} and the robot base base coordinate system {B} is calculated.

[0115] The present invention combines the pixel coordinates of the laser point and the pixel coordinates of the center point of the target area to calculate the alignment error between the robot's terminal posture and the laser point, improves calibration accuracy and consistency, and reduces the impact of human intervention on calibration. In addition, multiple sets of laser point coordinates are obtained by controlling the robot's end effector to move randomly in a small range, and an efficient and stable calibration process is formed through transformation matrix solution and calibration, so that the method can adapt to different robot and sensor configurations, achieve cross-platform compatibility, and improve versatility and extensibility. The present invention uses an automated algorithm to achieve precise alignment of the laser point with the characteristic point of the calibration plate (the center point of the circular target area of ​​the calibration plate), without relying on human eye observation and comparison, avoiding human errors, and making the calibration result more accurate.

[0116] In one embodiment, if Figure 4 The present invention also discloses a robot hand-eye calibration device based on a laser displacement sensor, comprising a memory 210 and a processor 220, wherein the memory 210 stores a program or instruction, and the processor 220 implements the following steps when executing the program or instruction:

[0117] Step 1: Control the robot end effector to move and project the laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point;

[0118] Step 2: using a two-dimensional image sensor to collect a two-dimensional image of the calibration plate including the target area, and calculating the pixel coordinates of the center point of the target area;

[0119] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose, and calculate the transformation matrix between the spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose;

[0120] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end position and posture, and obtain the displacement data measured by the laser displacement sensor;

[0121] Repeat steps 1 to 4 to collect multiple sets of data and calculate the calibration parameters.

[0122] When the processor 220 executes the program or instruction, it also implements the following steps:

[0123] Collect a two-dimensional image of the laser point in the target area;

[0124] The collected two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0125] When the processor 220 executes the program or instruction, it also implements the following steps:

[0126] Acquire a two-dimensional image of the calibration plate;

[0127] Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration plate;

[0128] The Sobel operator is used to perform edge detection on the calibration plate image after grayscale conversion to obtain a binary image;

[0129] Perform Hough circle detection calculation on the binary image, identify the circular target area in the calibration plate, and obtain the center coordinates of the target area.

[0130] When the processor 220 executes the program or instruction, it also implements the following steps:

[0131] When the laser point moves randomly on the calibration plate following the action of the robot end effector, the first pixel coordinate of the laser point on the calibration plate is obtained;

[0132] Controlling the robot end effector to perform a first action, obtaining a second pixel coordinate of the laser point on the calibration plate, calculating a difference between the second pixel coordinate and the first pixel coordinate, and obtaining first data;

[0133] Controlling the robot end effector to perform a second action, obtaining a third pixel coordinate of the laser point on the calibration plate, calculating a difference between the third pixel coordinate and the second pixel coordinate, and obtaining second data;

[0134] The transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot is calculated according to the first data and the second data.

[0135] When the processor 220 executes the program or instruction, it also implements the following steps:

[0136] The rotation matrix and position vector between the robot end flange coordinate system and the robot base coordinate system are obtained through the robot forward kinematics;

[0137] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system;

[0138] According to the transformation matrix between the robot end flange coordinate system and the robot base base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, the relationship between the laser displacement sensor coordinate system and the robot base base coordinate system is calculated.

[0139] In one embodiment, the present invention further discloses a readable storage medium having a program or an instruction stored thereon, wherein when the program or the instruction is executed by a processor, the following steps are implemented:

[0140] Step 1: Control the robot end effector to move and project the laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point;

[0141] Step 2: using a two-dimensional image sensor to collect a two-dimensional image of the calibration plate including the target area, and calculating the pixel coordinates of the center point of the target area;

[0142] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose, and calculate the transformation matrix between the spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose;

[0143] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end position and posture, and obtain the displacement data measured by the laser displacement sensor;

[0144] Repeat steps 1 to 4 to collect multiple sets of data and calculate the calibration parameters.

[0145] When the program or instruction is executed by the processor, the following steps are also implemented:

[0146] Collect a two-dimensional image of the laser point in the target area;

[0147] The collected two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0148] When the program or instruction is executed by the processor, the following steps are also implemented:

[0149] Acquire a two-dimensional image of the calibration plate;

[0150] Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration plate;

[0151] The Sobel operator is used to perform edge detection on the calibration plate image after grayscale conversion to obtain a binary image;

[0152] Perform Hough circle detection calculation on the binary image, identify the circular target area in the calibration plate, and obtain the center coordinates of the target area.

[0153] When the program or instruction is executed by the processor, the following steps are also implemented:

[0154] When the laser point moves randomly on the calibration plate following the action of the robot end effector, the first pixel coordinate of the laser point on the calibration plate is obtained;

[0155] Controlling the robot end effector to perform a first action, obtaining a second pixel coordinate of the laser point on the calibration plate, calculating a difference between the second pixel coordinate and the first pixel coordinate, and obtaining first data;

[0156] Controlling the robot end effector to perform a second action, obtaining a third pixel coordinate of the laser point on the calibration plate, calculating a difference between the third pixel coordinate and the second pixel coordinate, and obtaining second data;

[0157] The transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot is calculated according to the first data and the second data.

[0158] When the program or instruction is executed by the processor, the following steps are also implemented:

[0159] The rotation matrix and position vector between the robot end flange coordinate system and the robot base coordinate system are obtained through the robot forward kinematics;

[0160] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system;

[0161] According to the transformation matrix between the robot end flange coordinate system and the robot base base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, the relationship between the laser displacement sensor coordinate system and the robot base base coordinate system is calculated.

[0162] In one of the embodiments, the present invention further discloses a computer program product, which is stored in a storage medium. When the computer program product is executed by at least one processor, it implements the steps of the robot hand-eye calibration method based on laser displacement sensor provided in any of the above embodiments, and therefore also includes all the beneficial effects of the robot hand-eye calibration method based on laser displacement sensor provided in any of the above embodiments. To avoid repetition, they are not repeated here.

[0163] In one embodiment, the present invention discloses a robot component, including a robot, a robot hand-eye calibration device as provided in any of the above embodiments; and / or a readable storage medium as in any of the above embodiments and / or a computer program product as provided in any of the above embodiments, and therefore also includes all the beneficial effects of any of the above embodiments, which will not be described again here to avoid repetition.

[0164] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Therefore, any modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A robot hand-eye calibration method based on laser displacement sensor, characterized in that: include: Step 1: Control the robot end effector to move and project the laser beam into the target area of ​​the calibration plate, collect the two-dimensional image of the laser point through the two-dimensional image sensor, and obtain the pixel coordinates of the laser point; Step 2: using a two-dimensional image sensor to collect a two-dimensional image of the calibration plate including the target area, and calculating the pixel coordinates of the center point of the target area; Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose, and calculate the transformation matrix between the spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end pose; Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end position and posture, and obtain the displacement data measured by the laser displacement sensor; Repeat steps 1 to 4 to collect multiple sets of data and calculate the calibration parameters.

2. The robot hand-eye calibration method according to claim 1, characterized in that: The method of collecting a two-dimensional image of the laser point by using a two-dimensional image sensor and obtaining the pixel coordinates of the laser point includes: Collect a two-dimensional image of the laser point in the target area; The collected two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

3. The robot hand-eye calibration method according to claim 1, characterized in that: The method of collecting a two-dimensional image of the calibration plate including the target area by a two-dimensional image sensor and calculating the pixel coordinates of the center point of the target area includes: Acquire a two-dimensional image of the calibration plate; Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration plate; The Sobel operator is used to perform edge detection on the calibration plate image after grayscale conversion to obtain a binary image; Perform Hough circle detection calculation on the binary image, identify the circular target area in the calibration plate, and obtain the center coordinates of the target area.

4. The robot hand-eye calibration method according to claim 1, characterized in that: The robot end effector is controlled to move randomly, the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture are obtained, and the transformation matrix of the spot pixel coordinates and the robot end posture is calculated based on the real-time pixel coordinates of the laser point on the calibration plate and the corresponding robot end posture, including: When the laser point moves randomly on the calibration plate following the action of the robot end effector, the first pixel coordinate of the laser point on the calibration plate is obtained; Controlling the robot end effector to perform a first action, obtaining a second pixel coordinate of the laser point on the calibration plate, calculating a difference between the second pixel coordinate and the first pixel coordinate, and obtaining first data; Controlling the robot end effector to perform a second action, obtaining a third pixel coordinate of the laser point on the calibration plate, calculating a difference between the third pixel coordinate and the second pixel coordinate, and obtaining second data; The transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot is calculated according to the first data and the second data.

5. The robot hand-eye calibration method according to claim 1, characterized in that: The calculation and calibration parameters include: The rotation matrix and position vector between the robot end flange coordinate system and the robot base coordinate system are obtained through the robot forward kinematics; Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system; According to the transformation matrix between the robot end flange coordinate system and the robot base base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, the relationship between the laser displacement sensor coordinate system and the robot base base coordinate system is calculated.

6. A robot hand-eye calibration device based on a laser displacement sensor, used to implement the robot hand-eye calibration method according to any one of claims 1 to 5, characterized in that: It includes a laser displacement sensor, a two-dimensional image sensor, a calibration board and a control module. The laser displacement sensor and the two-dimensional image sensor are both fixed on the robot end effector. The calibration board is set within the working field of the two-dimensional image sensor. The laser displacement sensor irradiates the laser beam on the calibration board to form a laser point as the robot end effector moves. The laser point on the calibration board is collected by the two-dimensional image sensor. The control module is used to control the movement of the robot end effector, control the laser displacement sensor to emit a laser beam, receive displacement data measured by the laser displacement sensor and receive a two-dimensional image captured by the two-dimensional image sensor, and calculate calibration parameters based on the received data and image.

7. A robot hand-eye calibration device based on a laser displacement sensor, comprising a memory and a processor, wherein the memory stores a program or instruction, characterized in that: When the processor executes the program or instruction, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A readable storage medium having a program or instructions stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product, the computer program product being stored in a storage medium, characterized in that: When the computer program product is executed by at least one processor, the steps of the calibration method according to any one of claims 1 to 5 are implemented.

10. A robot assembly, characterized in that: include: robot; The robot hand-eye calibration device as claimed in claim 6 or 7; and / or The readable storage medium according to claim 8; and / or A computer program product as claimed in claim 9.

Images