A robot hand-eye calibration method and device based on a laser displacement sensor

Abstract

The application discloses a robot hand-eye calibration method and device based on a laser displacement sensor, and the method comprises the following steps: controlling a robot end effector to move and project a laser beam in a target region of a calibration board, collecting a two-dimensional image of a laser point, and obtaining pixel coordinates of the laser point; collecting a two-dimensional image of the calibration board including the target region through a two-dimensional image sensor, and calculating pixel coordinates of a center point of the target region; controlling the robot end effector to move randomly, obtaining real-time pixel coordinates of the laser point on the calibration board and corresponding robot end poses, and calculating a transformation matrix of the laser spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose; controlling the robot end effector to move and make the laser point coincide with the center point of the target region, collecting a real-time robot end pose, and obtaining displacement data measured by the laser displacement sensor; and calculating calibration parameters according to the data.

Description

Technical Field

[0001] This invention belongs to the field of robot calibration technology, specifically relating to a robot hand-eye calibration method and device based on a laser displacement sensor. Background Technology

[0002] Hand-eye calibration is an important technology in the field of robotics, used to determine the relative positional relationship between a robot and vision sensors, thereby enabling precise vision-guided operations and accurate control of the robot's position and posture. Currently, most existing hand-eye calibration methods rely on offline calibration boards or specific calibration workpieces. These methods require manual operation, have complex calibration processes, and are easily affected by environmental factors and the operator's experience.

[0003] Currently, hand-eye calibration methods based on laser displacement sensors primarily rely on manual operation. Specifically, operators must visually observe and compare the alignment of the laser point with the target calibration point. This method is highly dependent on the operator's experience and visual judgment, posing a significant risk of error. Firstly, the human eye struggles to achieve precise alignment at the micrometer level, especially during prolonged calibration operations, as it is susceptible to fatigue and subjective judgment. Secondly, differences in visual ability and experience among operators lead to inconsistent calibration results, directly impacting the accuracy and stability of subsequent robot operations. Furthermore, manual alignment is cumbersome and inefficient, failing to meet the demands of modern industry for high-efficiency automation. These drawbacks limit the widespread adoption and application of laser displacement sensor-based calibration technology in high-precision, high-consistency scenarios.

[0004] Currently, automatic hand-eye calibration methods based on laser displacement sensors face significant technical bottlenecks, the biggest challenge being how to reduce reliance on manual operation and improve calibration accuracy. On one hand, the human eye's alignment of laser points is subject to subjective errors, especially in complex environments where factors such as light reflection and occlusion further exacerbate these errors. On the other hand, the calibration requirements of different robot brands and vision sensors vary significantly, resulting in a lack of universality in existing technologies. For example, some robot brands require repeated adjustments to the angle and position of laser points for specific calibrations, while others may require real-time calibration during dynamic movement, placing higher demands on the adaptability of calibration algorithms. Furthermore, efficiently mapping the spatial relationship between laser points and feature points on the calibration board to the robot's hand-eye system during automatic calibration, while ensuring data processing accuracy and avoiding complex manual intervention, is also a major technical challenge. Summary of the Invention

[0005] To overcome one or more of the above-mentioned technical defects, the present invention provides a robot hand-eye calibration method and device based on a laser displacement sensor. By combining the pixel coordinates of the laser point and the pixel coordinates of the center point of the target area, the alignment error between the robot end posture and the laser point is calculated, thereby improving calibration accuracy and consistency and reducing the impact of human intervention on calibration.

[0006] To address the above problems, this invention provides a robot hand-eye calibration method based on a laser displacement sensor, comprising:

[0007] Step 1: Control the robot's end effector to move and project the laser beam onto the target area of ​​the calibration board. Acquire a two-dimensional image of the laser point using a two-dimensional image sensor and obtain the pixel coordinates of the laser point.

[0008] Step 2: Acquire a two-dimensional image of the calibration plate, including the target area, using a two-dimensional image sensor, and calculate the pixel coordinates of the center point of the target area.

[0009] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, and calculate the transformation matrix between the laser spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose.

[0010] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end pose, and obtain the displacement data measured by the laser displacement sensor;

[0011] Repeat steps one through four to collect multiple sets of data and calculate calibration parameters.

[0012] Furthermore, the step of acquiring a two-dimensional image of the laser point using a two-dimensional image sensor and obtaining the pixel coordinates of the laser point includes:

[0013] Acquire a two-dimensional image of the laser point located in the target area;

[0014] The acquired two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0015] Furthermore, the step of acquiring a two-dimensional image of the calibration plate, including the target area, using a two-dimensional image sensor and calculating the pixel coordinates of the center point of the target area includes:

[0016] Acquire two-dimensional images of the calibration plate;

[0017] Gaussian filtering and grayscale conversion are performed on the two-dimensional image of the calibration board;

[0018] The calibration board image after grayscale conversion is subjected to edge detection using the Sobel operator to obtain a binary image.

[0019] Hough circle detection is performed on the binary image to identify circular target regions in the calibration plate and obtain the center coordinates of the target regions.

[0020] Furthermore, the control robot end effector moves randomly to obtain the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose. Based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, the transformation matrix between the laser spot pixel coordinates and the robot end pose is calculated, including:

[0021] When the laser point moves randomly on the calibration board following the movement of the robot's end effector, the first pixel coordinate of the laser point on the calibration board is obtained;

[0022] The robot's end effector is controlled to perform the first action, obtain the second pixel coordinates of the laser point on the calibration plate, calculate the difference between the second pixel coordinates and the first pixel coordinates, and obtain the first data.

[0023] The robot's end effector is controlled to perform a second action, obtain the third pixel coordinate of the laser point on the calibration plate, calculate the difference between the third pixel coordinate and the second pixel coordinate, and obtain the second data.

[0024] Based on the first and second data, calculate the transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot.

[0025] Furthermore, the calculation of calibration parameters includes:

[0026] The rotation matrix and position vector between the robot's end flange coordinate system and the robot's base coordinate system are obtained through the robot's forward kinematics.

[0027] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system.

[0028] Based on the transformation matrix between the robot end flange coordinate system and the robot base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, calculate the relationship between the laser displacement sensor coordinate system and the robot base coordinate system.

[0029] The present invention also provides a robot hand-eye calibration device based on a laser displacement sensor, for implementing the above-mentioned robot hand-eye calibration method, including a laser displacement sensor, a two-dimensional image sensor, a calibration plate and a control module. The laser displacement sensor and the two-dimensional image sensor are both fixed on the robot end effector. The calibration plate is set within the working field of view of the two-dimensional image sensor. The laser displacement sensor illuminates the calibration plate with a laser beam as the robot end effector moves, forming a laser point. The laser point on the calibration plate is collected by the two-dimensional image sensor.

[0030] The control module is used to control the movement of the robot's end effector, control the laser displacement sensor to emit a laser beam, receive the displacement data measured by the laser displacement sensor and receive the two-dimensional image acquired by the two-dimensional image sensor, and calculate calibration parameters based on the received data and images.

[0031] The present invention also provides a robot hand-eye calibration device based on a laser displacement sensor, including a memory and a processor. The memory stores a program or instructions, and the processor executes the program or instructions to implement the steps of the above-described method.

[0032] The present invention also provides a readable storage medium having a program or instructions stored thereon, which, when executed by a processor, implement the steps of the method described above.

[0033] The present invention also provides a computer program product, which is stored in a storage medium and, when executed by at least one processor, implements the steps of the calibration method described above.

[0034] The present invention also provides a robot component, comprising:

[0035] robot;

[0036] The robot hand-eye calibration device as described above; and / or

[0037] The readable storage medium as described above; and / or

[0038] The computer program product described above.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] This invention discloses a robot hand-eye calibration method and apparatus based on a laser displacement sensor. Step 1: Control the robot's end effector to move and project a laser beam onto a target area of ​​a calibration board. A two-dimensional image of the laser point is acquired using a two-dimensional image sensor, and the pixel coordinates of the laser point are obtained. Step 2: A two-dimensional image of the calibration board, including the target area, is acquired using the two-dimensional image sensor, and the pixel coordinates of the center point of the target area are calculated. Step 3: Control the robot's end effector to move randomly, acquiring the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end-eye pose. Based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end-eye pose, the transformation matrix between the laser spot pixel coordinates and the robot end-eye pose is calculated. Step 4: Control the robot's end effector to move so that the laser point coincides with the center point of the target area. The real-time robot end-eye pose is acquired, and the displacement data measured by the laser displacement sensor is obtained. Steps 1 to 4 are repeated to acquire multiple sets of data, and calibration parameters are calculated. By combining the pixel coordinates of the laser point and the pixel coordinates of the center point of the target area, the alignment error between the robot end-eye pose and the laser point is calculated, improving calibration accuracy and consistency, and reducing the impact of human intervention on calibration. Furthermore, by controlling the random movement of the robot's end effector to obtain multiple sets of laser point coordinates, and by solving and calibrating the transformation matrix, an efficient and stable calibration process is formed, enabling this method to adapt to different robot and sensor configurations and achieve cross-platform compatibility. Attached Figure Description

[0041] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0042] Figure 1 This is a flowchart of the robot hand-eye calibration method based on a laser displacement sensor as described in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the robot hand-eye calibration device based on a laser displacement sensor according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the coordinates calculated in step S5 of the robot hand-eye calibration method based on a laser displacement sensor according to an embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram of another robot hand-eye calibration device based on a laser displacement sensor, as described in an embodiment of the present invention.

[0046] Labeling descriptions: 1. Robot; 2. Laser displacement sensor; 3. Ranging laser beam; 4. Calibration plate; 5. Two-dimensional image sensor; 210. Memory; 220. Processor. Detailed Implementation

[0047] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0048] In the description of this invention, the term "multiple" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. The terms "connection," "installation," "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] In the description of this invention, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this invention, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0050] An embodiment of the present invention discloses a robot hand-eye calibration device based on a laser displacement sensor, such as... Figure 1 The system includes a laser displacement sensor 2, a two-dimensional image sensor 5, a calibration plate 4, and a control module. Both the laser displacement sensor 2 and the two-dimensional image sensor 5 are fixed to the end effector of the robot 1. The calibration plate 4 is positioned within the working field of view of the two-dimensional image sensor 5. As the end effector of the robot 1 moves, the laser displacement sensor 2 projects a laser beam onto the calibration plate 4, forming a laser point. This laser point on the calibration plate 4 is then captured by the two-dimensional image sensor 5. The control module controls the movement of the end effector of the robot 1, controls the laser displacement sensor 2 to emit a laser beam, receives the displacement data measured by the laser displacement sensor 2, and receives the two-dimensional image captured by the two-dimensional image sensor 5. It also calculates calibration parameters based on the received data and image. Specifically, the two-dimensional image sensor 5 is a camera.

[0051] In one embodiment, the present invention discloses a robot hand-eye calibration method based on a laser displacement sensor, applied to the aforementioned robot hand-eye calibration device based on a laser displacement sensor, such as... Figure 2 The methods include:

[0052] S1. Control the robot's end effector to move and project the laser beam onto the target area of ​​the calibration plate. Acquire a two-dimensional image of the laser point through a two-dimensional image sensor and obtain the pixel coordinates of the laser point.

[0053] In this embodiment, step S1 includes:

[0054] Acquire a two-dimensional image of the laser point located in the target area.

[0055] The acquired two-dimensional image is binarized to obtain the pixel coordinates P1 of the laser point. The pixel coordinates P1 are the two-dimensional coordinates of the laser point in the two-dimensional image.

[0056] S2. Acquire a two-dimensional image of the calibration plate, including the target area, using a two-dimensional image sensor, and calculate the pixel coordinates P2 of the center point of the target area.

[0057] In this embodiment, step S2 includes:

[0058] Acquire two-dimensional images of the calibration board.

[0059] Gaussian filtering and grayscale conversion are performed on the two-dimensional image of the calibration board.

[0060] Specifically, the Gaussian filtering formula is:

[0061]

[0062] Let the initial two-dimensional image matrix be T1, and the Gaussian filter matrix be T. g Gaussian filtering is applied to the initial two-dimensional image to remove noise interference, resulting in the Gaussian filtered image T2.

[0063] T2 = T g T1

[0064] The Gaussian-filtered image T2 is then converted to grayscale:

[0065] grayscale image Where (x, y) are the coordinates of each pixel in image T2.

[0066] Edge detection was performed on the calibration board image after grayscale conversion using the Sobel operator to obtain a binary image.

[0067] Specifically, the Sobel operator is used for edge detection calculation, calculating the edges of all closed shapes in the 2D image, resulting in a binary edge detection map T4 from the initial 2D image. The Sobel operator is used to approximate the grayscale value of the image's brightness function. Applying the Sobel operator to any point in the image produces either the corresponding grayscale vector or its normal vector.

[0068] Let Sobel's two convolution factors be d. x dy ,

[0069]

[0070] Gradient magnitude of the image Image gradient direction

[0071] The binary image after edge detection is calculated as: T4 = S·T2.

[0072] By extracting the pixels where the gradient reaches its local maximum, we can obtain a binary image T4 that only contains edges.

[0073] Hough circle detection is performed on the binary image to identify circular target regions in the calibration plate and obtain the center coordinates of the target regions.

[0074] Specifically, the equation of the circle in the Hough circle detection formula in the Cartesian coordinate system is: (xa) 2 +(yb) 2 =r 2

[0075] The equation of a circle in the Hough transform is:

[0076] Each set (a, b, r) represents a circle passing through the point (x0, y0). The parameter of r is determined by setting a threshold based on the size of the target circle's r. The center space N(a, b) is initialized, with all N(a, b) = 0. All non-zero pixels in the Canny edge binary image are traversed, and lines are drawn along the gradient direction (perpendicular to the tangent). For all points (a, b) in the accumulator that the line segments pass through, N(a, b) is incremented by 1. When multiple line segments pass through a pixel, the accumulator identifies it as a circle. (a, b) in the accumulator represents the center position P2 of the circular target region in the image.

[0077] S3. Control the robot end effector to move randomly within a small range, obtain the real-time pixel coordinates of the laser points on the calibration board and the corresponding robot end pose, and calculate the transformation matrix between the calibration board and the robot end pose based on the real-time pixel coordinates of the laser points on the calibration board and the corresponding robot end pose.

[0078] In this embodiment, step S3 includes:

[0079] The calibration board is set parallel to the ground. Let the image-calibration board coordinate system be {P}, and the robot base coordinate system be {B}. Let the center coordinates o of the circular target area on the calibration board be (uo, vo), which is P2(a, b) obtained through the accumulator in step S2. The method for establishing the coordinate system transformation matrix is ​​as follows:

[0080] The laser point moves randomly on the calibration plate following the movement of the robot's end effector. When the laser point hits the lower left corner of the image, the first pixel coordinates a(ua, va) of the laser point on the calibration plate are obtained.

[0081] The robot's end effector is controlled to move △X along the X-axis direction of {B} to obtain the second pixel coordinates b(ub, vb) of the laser point on the calibration plate. The difference between the second pixel coordinates b(ub, vb) and the first pixel coordinates a(ua, va) in {P} is calculated to obtain the X-axis direction and step size in the image-calibration plate coordinate system {P}.

[0082] α=((ub-ua),(vb-va))

[0083] The robot's end effector is controlled to move ΔY along the Y-axis direction under {B} to obtain the third pixel coordinate c(uc, vc) of the laser point on the calibration plate. The difference between the third pixel coordinate c(uc, vc) and the second pixel coordinate b(ub, vb) under {P} is calculated to obtain the Y-axis direction and step size under the image-calibration plate coordinate system {P}.

[0084] β=((uc-ub),(vc-vb))

[0085] Suppose there exists a set of n and m values ​​such that any laser point Pi(ui, vi) under {P} coincides with point o, then:

[0086]

[0087] The specific coordinates of the center point o (uo, vo) and the laser point (ui, vi) are known as the results of image recognition. If the above equation has a solution, there is one and only one valid solution n, m. It can be seen that by moving △X and △Y by n and m times the step size in the robot base coordinate system {B}, the coordinates of the laser point Pi and the center point o in the image-calibration plate coordinate system {P} can be made to coincide, and the transformation matrix R between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot can be calculated.

[0088] S4. Control the robot's end effector to move so that the laser point P1 coincides with the center point P2 of the target area, collect the real-time robot end pose, and obtain the displacement data measured by the laser displacement sensor.

[0089] S5. Repeat S1-S4 to collect nine sets of data and calculate hand-eye calibration parameters based on the laser point calibration method.

[0090] Specifically, the calculation of hand-eye calibration parameters includes:

[0091] like Figure 3Let the base coordinate system of the robot be {B}, the coordinate system of the robot end flange be {E}, and the coordinate system of the laser displacement sensor be {T}. The origin O is fixed at a certain point L on the emitted laser line.

[0092] The rotation matrix and position vector between the robot end flange coordinate system {E} and the robot base base coordinate system {B} are obtained through robot forward kinematics.

[0093] Coordinate transformation of {T} relative to {B} for:

[0094]

[0095] in, This represents the transformation matrix of {E} relative to {B}. It can be obtained by solving the robot's forward kinematics in the classic DH model. middle Let P be the rotation matrix of {T} relative to {E}, and let P be the position vector of {T} under {E}. Obtained through decomposition There are three vector components. Therefore, laser displacement sensor position calibration involves calibrating the rotation matrix and position vector between {T} and {E}.

[0096] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, and the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system. Calculate the rotation matrix and position vector between the robot end flange coordinate system {E} and the laser displacement sensor coordinate system {T}.

[0097] Let the coordinates of the origin of the laser displacement sensor, i.e., the TCP point, under {E} be (x0, y0, z0), and the direction vector of the emitted laser line under {E} be (nx, ny, nz). The distance between the measured theoretical target point and the origin of the laser displacement sensor is the sensor reading. Then, the coordinates of the measured theoretical target point under {E} are:

[0098] x = x0 + nL

[0099] Where x = (x, y, z) T x0 = (x0, y0, z0) T and n = (n x n y n z ) T .

[0100] The coordinates of the theoretical target point measured under {B} are: x Bi =R i (x0+n Li)+P i

[0101] matrix Let be the transformation matrix of {E} relative to {B}. Obtained by solving the robot's forward kinematics, according to the formula, R i Let P be the rotation matrix of {E} relative to {B}. i Let R be the position vector of {E} under {B}. From the robot's forward kinematics, we know that R... i Given a 3×3 matrix, P i Given a 3×1 matrix, the transformation matrix represented by quaternions can be read out in the robot teach pendant.

[0102] Assume that the same laser point X is hit each time. Bi If it remains unchanged, then:

[0103] X Bi =X B(i+1)

[0104] R i (x0+nL i )+P i =R i+1 (x0+nL i+1 )+P i+1

[0105] If the robot flange posture remains unchanged, and the robot flange is adjusted (e.g., first changing the Z direction, then moving the XY direction) so that the laser always hits the same point, the above formula can be transformed into:

[0106] R0nL i +P i =R0nL i+1 +P i+1

[0107]

[0108] Among them, R0, P i+1 P i The values ​​can be obtained directly from the quaternions using the robot's built-in teach pendant. Li and Li+1 can be directly read from the laser displacement sensor. When the constraint point is a single point, the problem transforms into a constraint problem within an optimization problem, and the direction vector n can be obtained by solving the linear equation system An B.

[0109] A = [R0 R0…R0] T (3(n-1)×3)

[0110]

[0111] Where R0, P iQuaternion calculations can be performed directly by reading data from the robot teach pendant. i The position of the laser sensor origin x0 and the Z-axis direction under {E} can be obtained directly from the laser displacement sensor.

[0112] According to the quaternion rule, under the conditions of satisfying the right-hand rule and orthogonality, the directions of the X and Y axes can be arbitrarily defined according to actual conditions. Let the cross product of the common normals of the flange's Z-axis (0,0,1) and n be γ(γx,γy,γz), with an angle of θ. By definition, rotating {E} around γ by θ yields the position of the laser displacement sensor. Therefore, the coordinate system {T} of the laser displacement sensor can be represented as:

[0113]

[0114] Based on the rotation matrix and position vector between the robot end flange coordinate system {E} and the robot base base coordinate system {B}, and the rotation matrix and position vector between the robot end flange coordinate system {E} and the laser displacement sensor coordinate system {T}, calculate the relationship between the laser displacement sensor coordinate system {T} and the robot base base coordinate system {B}.

[0115] This invention combines the pixel coordinates of the laser point and the pixel coordinates of the target area's center point to calculate the alignment error between the robot's end effector posture and the laser point, improving calibration accuracy and consistency while reducing the impact of human intervention. Furthermore, by controlling the robot's end effector to move randomly within a small range to obtain multiple sets of laser point coordinates, and through transformation matrix solving and calibration, an efficient and stable calibration process is formed. This allows the method to adapt to different robot and sensor configurations, achieving cross-platform compatibility and improving versatility and scalability. This invention achieves precise alignment between the laser point and the feature point of the calibration board (the center point of the circular target area on the calibration board) through an automated algorithm, eliminating the need for human observation and comparison, avoiding human error, and resulting in more accurate calibration results.

[0116] In one embodiment, such as Figure 4 The present invention also discloses a robot hand-eye calibration device based on a laser displacement sensor, including a memory 210 and a processor 220. The memory 210 stores a program or instructions, and the processor 220 executes the program or instructions to perform the following steps:

[0117] Step 1: Control the robot's end effector to move and project the laser beam onto the target area of ​​the calibration board. Acquire a two-dimensional image of the laser point using a two-dimensional image sensor and obtain the pixel coordinates of the laser point.

[0118] Step 2: Acquire a two-dimensional image of the calibration plate, including the target area, using a two-dimensional image sensor, and calculate the pixel coordinates of the center point of the target area.

[0119] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, and calculate the transformation matrix between the laser spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose.

[0120] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end pose, and obtain the displacement data measured by the laser displacement sensor;

[0121] Repeat steps one through four to collect multiple sets of data and calculate calibration parameters.

[0122] When the processor 220 executes the program or instructions, it also performs the following steps:

[0123] Acquire a two-dimensional image of the laser point located in the target area;

[0124] The acquired two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0125] When the processor 220 executes the program or instructions, it also performs the following steps:

[0126] Acquire two-dimensional images of the calibration plate;

[0127] Gaussian filtering and grayscale conversion are performed on the two-dimensional image of the calibration board;

[0128] The calibration board image after grayscale conversion is subjected to edge detection using the Sobel operator to obtain a binary image.

[0129] Hough circle detection is performed on the binary image to identify circular target regions in the calibration plate and obtain the center coordinates of the target regions.

[0130] When the processor 220 executes the program or instructions, it also performs the following steps:

[0131] When the laser point moves randomly on the calibration board following the movement of the robot's end effector, the first pixel coordinate of the laser point on the calibration board is obtained;

[0132] The robot's end effector is controlled to perform the first action, obtain the second pixel coordinates of the laser point on the calibration plate, calculate the difference between the second pixel coordinates and the first pixel coordinates, and obtain the first data.

[0133] The robot's end effector is controlled to perform a second action, obtain the third pixel coordinate of the laser point on the calibration plate, calculate the difference between the third pixel coordinate and the second pixel coordinate, and obtain the second data.

[0134] Based on the first and second data, calculate the transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot.

[0135] When the processor 220 executes the program or instructions, it also performs the following steps:

[0136] The rotation matrix and position vector between the robot's end flange coordinate system and the robot's base coordinate system are obtained through the robot's forward kinematics.

[0137] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system.

[0138] Based on the transformation matrix between the robot end flange coordinate system and the robot base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, calculate the relationship between the laser displacement sensor coordinate system and the robot base coordinate system.

[0139] In one embodiment, the present invention also discloses a readable storage medium having a program or instructions stored thereon, which, when executed by a processor, perform the following steps:

[0140] Step 1: Control the robot's end effector to move and project the laser beam onto the target area of ​​the calibration board. Acquire a two-dimensional image of the laser point using a two-dimensional image sensor and obtain the pixel coordinates of the laser point.

[0141] Step 2: Acquire a two-dimensional image of the calibration plate, including the target area, using a two-dimensional image sensor, and calculate the pixel coordinates of the center point of the target area.

[0142] Step 3: Control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, and calculate the transformation matrix between the laser spot pixel coordinates and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose.

[0143] Step 4: Control the robot end effector to move so that the laser point coincides with the center point of the target area, collect the real-time robot end pose, and obtain the displacement data measured by the laser displacement sensor;

[0144] Repeat steps one through four to collect multiple sets of data and calculate calibration parameters.

[0145] When the program or instructions are executed by the processor, the following steps are also performed:

[0146] Acquire a two-dimensional image of the laser point located in the target area;

[0147] The acquired two-dimensional image is binarized to obtain the pixel coordinates of the laser point.

[0148] When the program or instructions are executed by the processor, the following steps are also performed:

[0149] Acquire two-dimensional images of the calibration plate;

[0150] Gaussian filtering and grayscale conversion are performed on the two-dimensional image of the calibration board;

[0151] The calibration board image after grayscale conversion is subjected to edge detection using the Sobel operator to obtain a binary image.

[0152] Hough circle detection is performed on the binary image to identify circular target regions in the calibration plate and obtain the center coordinates of the target regions.

[0153] When the program or instructions are executed by the processor, the following steps are also performed:

[0154] When the laser point moves randomly on the calibration board following the movement of the robot's end effector, the first pixel coordinate of the laser point on the calibration board is obtained;

[0155] The robot's end effector is controlled to perform the first action, obtain the second pixel coordinates of the laser point on the calibration plate, calculate the difference between the second pixel coordinates and the first pixel coordinates, and obtain the first data.

[0156] The robot's end effector is controlled to perform a second action, obtain the third pixel coordinate of the laser point on the calibration plate, calculate the difference between the third pixel coordinate and the second pixel coordinate, and obtain the second data.

[0157] Based on the first and second data, calculate the transformation matrix between the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot.

[0158] When the program or instructions are executed by the processor, the following steps are also performed:

[0159] The rotation matrix and position vector between the robot's end flange coordinate system and the robot's base coordinate system are obtained through the robot's forward kinematics.

[0160] Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system.

[0161] Based on the transformation matrix between the robot end flange coordinate system and the robot base coordinate system, and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, calculate the relationship between the laser displacement sensor coordinate system and the robot base coordinate system.

[0162] In one embodiment, the present invention also discloses a computer program product stored in a storage medium. When the computer program product is executed by at least one processor, it implements the steps of the robot hand-eye calibration method based on a laser displacement sensor provided in any of the above embodiments. Therefore, it also includes all the beneficial effects of the robot hand-eye calibration method based on a laser displacement sensor provided in any of the above embodiments. To avoid repetition, these effects will not be repeated here.

[0163] In one embodiment, the present invention discloses a robot component including a robot, a robot hand-eye calibration device as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments and / or a computer program product as provided in any of the above embodiments, and therefore also includes all the beneficial effects of any of the above embodiments, which will not be repeated here to avoid repetition.

[0164] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Therefore, any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A robot hand-eye calibration method based on a laser displacement sensor, characterized by, The method comprises the following steps: Step 1: control the robot end effector to move to project a laser beam in a target area of a calibration board, acquire a two-dimensional image of the laser point through a two-dimensional image sensor, and obtain the pixel coordinates of the laser point; Step 2: acquire a two-dimensional image of the calibration board including the target area through the two-dimensional image sensor, and calculate the pixel coordinates of the center point of the target area; Step 3: control the robot end effector to move randomly, obtain the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, and calculate the transformation matrix of the pixel coordinates of the light spot and the robot end pose based on the real-time pixel coordinates of the laser point on the calibration board and the corresponding robot end pose, comprising: When the laser point moves randomly on the calibration board following the action of the robot end effector, obtain the first pixel coordinates of the laser point on the calibration board; Control the robot end effector to perform a first action, obtain the second pixel coordinates of the laser point on the calibration board, calculate the difference between the second pixel coordinates and the first pixel coordinates, and obtain first data; Control the robot end effector to perform a second action, obtain the third pixel coordinates of the laser point on the calibration board, calculate the difference between the third pixel coordinates and the second pixel coordinates, and obtain second data; According to the first data and the second data, calculate the transformation matrix of the two-dimensional coordinates of the laser point and the three-dimensional coordinates of the robot; Step 4: control the robot end effector to move to coincide the laser point with the center point of the target area, acquire the real-time robot end pose, and obtain the displacement data measured by the laser displacement sensor; Repeat steps 1 to 4 to acquire multiple sets of data and calculate the calibration parameters, comprising: Obtain the rotation matrix and position vector between the robot end flange coordinate system and the robot base coordinate system through robot forward kinematics; Set the coordinates of the origin of the laser displacement sensor in the robot end flange coordinate system, the direction vector of the laser beam emitted by the laser displacement sensor in the robot end flange coordinate system, and calculate the rotation matrix and position vector between the robot end flange coordinate system and the laser displacement sensor coordinate system; According to the transformation matrix between the robot end flange coordinate system and the robot base coordinate system and the transformation matrix between the robot end flange coordinate system and the laser displacement sensor coordinate system, calculate the relationship between the laser displacement sensor coordinate system and the robot base coordinate system.

2. The robot hand-eye calibration method of claim 1, wherein, The method comprises the following steps: Acquire a two-dimensional image of the laser point in the target area; Perform binaryzation processing on the acquired two-dimensional image to obtain the pixel coordinates of the laser point.

3. The robot hand-eye calibration method of claim 1, wherein, The method comprises the following steps: Acquire a two-dimensional image of the calibration board; Perform Gaussian filtering and grayscale conversion on the two-dimensional image of the calibration board; Perform edge detection on the grayscale-converted calibration board image using a Sobel operator to obtain a binary image; Perform Hough circle detection calculation on the binary image to identify the circular target area in the calibration board and obtain the center coordinates of the target area.

4. A robot hand-eye calibration apparatus based on a laser displacement sensor for implementing the robot hand-eye calibration method according to any one of claims 1 to 3, characterized in that, The robot hand-eye calibration device comprises a laser displacement sensor, a two-dimensional image sensor, a calibration board and a control module, the laser displacement sensor and the two-dimensional image sensor are fixed on the robot end effector, the calibration board is arranged in the working field of the two-dimensional image sensor, the laser displacement sensor irradiates a laser beam on the calibration board to form a laser spot with the movement of the robot end effector, and the laser spot on the calibration board is collected by the two-dimensional image sensor. The control module is used for controlling the movement of the robot end effector, controlling the laser displacement sensor to emit the laser beam, receiving the displacement data measured by the laser displacement sensor and the two-dimensional image collected by the two-dimensional image sensor, and calculating the calibration parameters according to the received data and image. 5.A robot hand-eye calibration device based on a laser displacement sensor, comprising a memory and a processor, the memory storing programs or instructions, characterized in that, The processor executes the program or instruction to realize the steps of the method in any one of claims 1-3.

6. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or instruction is executed by the processor to realize the steps of the method in any one of claims 1-3.

7. A computer program product stored in a storage medium, characterized in that, The computer program product is executed by at least one processor to realize the steps of the calibration method in any one of claims 1-3.

8. A robotic assembly comprising: It comprises: a robot; the robot hand-eye calibration device in claim 4 or 5; and / or the readable storage medium in claim 6; and / or the computer program product in claim 7.

Images