Hand-eye calibration method and device, electronic equipment and storage medium
By automatically calculating the rotation center and position information of the robot arm, the problem of large manual calibration error in the prior art is solved, and the accuracy and accuracy of hand-eye calibration are improved.
Patent Information
- Application Number
- CN202410973367.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
AI Technical Summary
The existing hand-eye calibration technology requires manual control of the robotic arm for calibration, resulting in large calibration errors and low accuracy.
By controlling the robotic arm to move the calibration plate to multiple target calibration positions and rotate it, the corresponding images are collected, the rotation center and position information is automatically calculated, the transformation relationship between the camera coordinate system and the base coordinate system is calculated, and the calibration accuracy is improved.
There is no need to manually determine the rotation center, which improves the accuracy of the rotation center and the accuracy of the target transformation relationship, and improves the calibration accuracy of hand-eye calibration.
Smart Images

Figure CN118906049A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of calibration technology, and in particular to a hand-eye calibration method, device, electronic device and storage medium. Background Art
[0002] At present, in order to obtain the relationship between the robot arm and the camera, just like the relationship between the human hand and the eye, the hand-eye calibration method is needed to determine the transformation relationship between the base coordinate system with the reference point on the robot arm as the origin and the camera coordinate system. However, since the commonly used hand-eye calibration technology often requires manual control of the robot arm for calibration, the calibration error is large and the calibration accuracy is not high. Summary of the invention
[0003] The embodiments of the present application disclose a hand-eye calibration method, device, electronic device and storage medium, which can improve the calibration accuracy of hand-eye calibration.
[0004] The present application embodiment discloses a hand-eye calibration method, the method comprising:
[0005] Controlling the mechanical arm to move the calibration plate to a first target calibration position, and controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions;
[0006] Controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position;
[0007] Determining, according to the first image and the second image, a rotation center of the calibration plate corresponding to the first target calibration position;
[0008] According to the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
[0009] As an optional implementation, the camera module is disposed on the robotic arm; before controlling the robotic arm to move the calibration plate to the first target calibration position and controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position, the method further includes:
[0010] Acquire initial posture data of the robotic arm, and control the robotic arm to adjust its posture according to the initial posture data to determine the shooting posture corresponding to the robotic arm;
[0011] The controlling the mechanical arm to move the calibration plate to a first target calibration position, and controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position, comprises:
[0012] Controlling the mechanical arm to move the calibration plate to the first target calibration position;
[0013] The robotic arm is controlled to return to the shooting posture corresponding to the robotic arm, and the camera module arranged on the robotic arm is controlled to collect images of the calibration plate to obtain a first image corresponding to the calibration plate and the first target calibration position.
[0014] As an optional implementation, controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position, includes:
[0015] Controlling the mechanical arm to rotate the calibration plate;
[0016] The robotic arm is controlled to return to the shooting posture corresponding to the robotic arm, and the camera module arranged on the robotic arm is controlled to collect images of the calibration plate to obtain a second image corresponding to the calibration plate and the first target calibration position.
[0017] As an optional implementation manner, before the controlling mechanical arm to move the calibration plate to the first target calibration position and controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position, the method further includes:
[0018] Controlling the mechanical arm to move the calibration plate to a first initial calibration position, and controlling the camera module to capture a third image of the calibration plate corresponding to the first initial calibration position; the first initial calibration position is any one of a plurality of initial calibration positions;
[0019] Controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a fourth image of the calibration plate corresponding to the first initial calibration position;
[0020] Determining, according to the third image and the fourth image corresponding to the first initial calibration position, a rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system;
[0021] Calculate the initial transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers respectively corresponding to the calibration plate and the multiple initial calibration positions, and the posture information respectively corresponding to the robotic arm and the multiple initial calibration positions;
[0022] According to the initial transformation relationship, multiple target calibration positions and position information of the robotic arm corresponding to the multiple target calibration positions are determined.
[0023] As an optional implementation, before controlling the robotic arm to move the calibration plate to the first initial calibration position and controlling the camera module to capture a first image corresponding to the calibration plate and the first initial calibration position, the method further includes:
[0024] Acquire position calibration information; the position calibration information is used to determine the area range of the calibration area in the base coordinate system;
[0025] According to the calibration position information, a plurality of initial calibration positions are randomly generated, and the posture information corresponding to the robot arm and the plurality of initial calibration positions is determined.
[0026] As an optional implementation manner, the calibration plate includes at least two calibration points; and determining, according to the first image and the second image, a rotation center of the calibration plate corresponding to the first target calibration position includes:
[0027] The rotation center of the calibration plate corresponding to the first target calibration position is calculated according to the first image coordinates respectively corresponding to at least two target calibration points of the calibration plate in the first image and the second image coordinates respectively corresponding to the at least two target calibration points in the second image.
[0028] As an optional implementation, the at least two target calibration points are on the same straight line; the calculating, according to the first image coordinates respectively corresponding to the at least two target calibration points of the calibration plate in the first image, and the second image coordinates respectively corresponding to the at least two target calibration points in the second image, the rotation center corresponding to the calibration plate and the first target calibration position comprises:
[0029] Calculating a first straight line on the first image according to first image coordinates respectively corresponding to at least two target calibration points of the calibration plate in the first image;
[0030] Calculating a second straight line on the second image according to first image coordinates corresponding to at least two target calibration points in the first image;
[0031] An intersection point of the first straight line and the second straight line is calculated, and the intersection point of the first straight line and the second straight line is used as a rotation center corresponding to the first target calibration position.
[0032] The present application embodiment discloses a hand-eye calibration device, the device comprising:
[0033] A first moving module is used to control the mechanical arm to move the calibration plate to a first target calibration position, and control the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions;
[0034] A first rotation module, used to control the mechanical arm to rotate the calibration plate, and then control the camera module to capture a second image corresponding to the calibration plate and the first target calibration position;
[0035] A determination module, configured to determine a rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image;
[0036] A calculation module is used to calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions; the base coordinate system is a coordinate system with a reference point on the robotic arm as the origin.
[0037] An embodiment of the present application discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements any one of the hand-eye calibration methods disclosed in the embodiment of the present application.
[0038] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, any one of the hand-eye calibration methods disclosed in the embodiment of the present application is implemented.
[0039] Compared with the related art, the embodiments of the present application have the following beneficial effects:
[0040] The embodiments of the present application provide a hand-eye calibration method, device, electronic device and storage medium, which controls a robotic arm to move a calibration plate to a first target calibration position, and controls a camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions; the robotic arm is controlled to rotate the calibration plate, and then controls the camera module to capture a second image corresponding to the calibration plate and the first target calibration position; according to the first image and the second image, the rotation center corresponding to the calibration plate and the first target calibration position is determined; according to the rotation centers corresponding to the calibration plate and the plurality of target calibration positions, respectively, and the posture information corresponding to the robotic arm and the plurality of target calibration positions, respectively, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated; the base coordinate system is a coordinate system with a reference point on the robotic arm as the origin. By implementing the embodiments of the present application, the calibration plate is moved to each target calibration position and rotated by controlling the robotic arm. The rotation center corresponding to each target calibration position can be automatically calculated based on the first image and the second image corresponding to each target calibration position. There is no need to manually determine the rotation center corresponding to each target calibration position, which improves the accuracy of the determined rotation center. In addition, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated based on the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions, which further improves the accuracy of the calculated target transformation relationship and can improve the calibration accuracy of hand-eye calibration. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1A This is an application scenario diagram of a hand-eye calibration method disclosed in an embodiment of the present application;
[0043] Figure 1B is an application scenario diagram of a hand-eye calibration method disclosed in an embodiment of the present application;
[0044] Figure 2 It is a flowchart of a hand-eye calibration method disclosed in an embodiment of the present application;
[0045] Figure 3 is a flow chart of another hand-eye calibration method disclosed in an embodiment of the present application;
[0046] Figure 4 is a schematic diagram of determining a rotation center in one embodiment;
[0047] Figure 5 is a flow chart of another hand-eye calibration method disclosed in an embodiment of the present application;
[0048] Figure 6 is a schematic diagram of a calibration personnel operation interface in an embodiment;
[0049] Figure 7 is a schematic diagram of the operation process of a calibration person in an embodiment;
[0050] Figure 8 is a schematic diagram of a hand-eye calibration process in an embodiment;
[0051] Fig. 9 It is a structural schematic diagram of a hand-eye calibration device disclosed in an embodiment of the present application;
[0052] Fig.10 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0054] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.
[0055] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image. Both the first image and the second image are images, but they are not the same image.
[0056] Hand-eye calibration refers to the process of moving or rotating a calibration plate by a robotic arm so that the camera module captures the image of the calibration plate, and calculating the transformation relationship between the camera coordinate system and the base coordinate system based on the image information contained in the captured image. Hand-eye calibration can include eye-on-hand calibration and eye-outside-hand calibration. Eye-on-hand calibration refers to the calibration process in which the camera module is set on the robotic arm to capture the image of the calibration plate, and eye-outside-hand calibration refers to the calibration process in which the camera module is set outside the robotic arm to capture the image of the calibration plate.
[0057] For eye-out-of-hand calibration, in current technology, the hand-eye calibration of the robotic arm can be calibrated by laser probe calibration, twelve-point calibration method, etc. The laser probe calibration technology requires the calibration personnel to install the laser probe on the robotic arm, and then manually select nine calibration points evenly, and align the laser probe with the nine calibration points in turn, record the posture of the robotic arm and the pixel coordinates of the calibration points on the image each time the nine calibration points are aligned, and calculate the transformation relationship. However, this technology requires the use of a laser probe, which increases the calibration cost, and requires manual movement of the robotic arm to align with the nine calibration points. The calibration accuracy is poor, and the automatic calibration process cannot be achieved.
[0058] The twelve-point calibration method is to fix the calibration plate at the end of the robotic arm. The calibration personnel control the robotic arm to move at a fixed angle first. Each time the robotic arm moves, a set of images is taken. A total of nine displacements are performed. After obtaining nine sets of images, the robotic arm is controlled to rotate at a fixed angle. Each time it rotates a certain angle, a set of images is recorded. A total of three rotations are performed. After obtaining three sets of images, the transformation relationship between the base coordinate system and the camera coordinate system is calculated based on the twelve sets of images. However, this technology also requires manual control of the displacement and rotation of the mechanical part, and the calibration plate needs to be manually fixed to the end of the robotic arm. The operation is complicated and there is a large calibration error.
[0059] For eye-on-hand calibration, the nine-point calibration method can be used for calibration. The nine-point calibration method is that the calibration personnel controls the robotic arm to grab the calibration plate and move and place it in nine calibration positions in sequence. After placing the calibration plate each time, the robotic arm is controlled to return to the initial posture to take pictures, and nine images are obtained and the posture data of the robotic arm in the initial posture are recorded. Based on the nine images obtained and the posture data of the robotic arm in the initial posture, the transformation relationship between the base coordinate system and the camera coordinate system is calculated. However, this method cannot accurately determine the rotation center of the calibration plate. The calibration personnel need to control the rotation center of the robotic arm through the human eye to coincide with the rotation center of the calibration plate, and then use the robotic arm to move the calibration plate to collect the image of the calibration plate, which also increases the error in the calibration process and affects the accuracy of hand-eye calibration.
[0060] The embodiments of the present application disclose a hand-eye calibration method, device, electronic device and storage medium, which can improve the calibration accuracy of hand-eye calibration.
[0061] See also Figure 1A and Figure 1B , Figure 1A and Figure 1B : is an application scenario diagram of a hand-eye calibration method disclosed in an embodiment of the present application. The hand-eye calibration method is applicable to electronic devices, which may include but are not limited to mobile phones, tablet computers, wearable devices, laptop computers, PCs (Personal Computers), etc., and are not limited in the embodiments of the present application. The electronic device can be connected to the mechanical arm 101 and the camera module 102 in communication, so that the electronic device can control the mechanical arm 101 and obtain the image captured by the camera module 102.
[0062] Specifically, Figure 1A The application scenario shown is an eye-on-hand calibration scenario, where the camera module 102 is disposed on the robotic arm 101. When the robotic arm 101 moves, the camera module 102 moves with the robotic arm 101. The robotic arm 101 can move in the calibration area 103 and rotate the calibration plate 104, which can include a plurality of calibration points 105. Further, the camera module 102 can be disposed at the end of the robotic arm 101. Figure 1B The application scenario shown is a calibration scenario where the eye is outside the hand, and the camera module 102 is arranged outside the mechanical arm 101. When the mechanical arm 101 moves, the camera module 102 remains stationary. The mechanical arm 101 can move and rotate the calibration plate 104 in the calibration area 103, and the calibration plate 104 may include multiple calibration points 105. Further, the camera module 102 can be set to a position where the calibration plate 104 at any position in the calibration area 103 can be photographed, so that the camera module 102 can photograph the calibration plate 104 in the calibration area 103. Among them, the electronic device can be used in the process of hand-eye calibration, and the calibration area 103 can be used to limit the area range where multiple target calibration positions are located. The target calibration position can be a position in the calibration area for controlling the mechanical arm to move the calibration plate to, or it can be a position in the calibration area for controlling the mechanical arm to rotate the calibration plate. Multiple target calibration positions can be set, for example, nine target calibration positions can be set, but it is not limited to this.
[0063] The electronic device controls the mechanical arm 101 to move the calibration plate 104 to the first target calibration position, controls the camera module 102 to collect the first image of the calibration plate 104 corresponding to the first target calibration position, controls the mechanical arm 101 to rotate the calibration plate 104, and then controls the camera module 102 to collect the second image of the calibration plate 104 corresponding to the first target calibration position, and determines the rotation center of the calibration plate 104 corresponding to the first target calibration position according to the first image and the second image. After determining the rotation centers corresponding to the calibration plate 104 and the multiple target calibration positions, the electronic device can calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate 104 and the multiple target calibration positions, and the posture information corresponding to the mechanical arm 101 and the multiple target calibration positions.
[0064] Figure 2 is a flow chart of a hand-eye calibration method disclosed in an embodiment of the present application. Figure 2 The hand-eye calibration method described is applicable to the above electronic devices. Figure 2 As shown, the hand-eye calibration method may include the following steps:
[0065] Step S202, control the robotic arm to move the calibration plate to a first target calibration position, and control the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of the multiple target calibration positions.
[0066] The robot arm can be used to grab objects during operation, and the robot arm may include but is not limited to a four-axis robot arm, a six-axis robot arm, or a linear robot arm, etc., which is not limited in the embodiments of the present application. Among them, the six-axis robot arm can be a robot arm that can move or rotate in three non-parallel directions; the four-axis can be a robot arm that can move or rotate in two non-parallel directions; the linear robot arm can be a robot arm that can move or rotate in one direction.
[0067] When the robotic arm is controlled by an electronic device, the electronic device can collect images of the robotic arm grabbing objects through a camera module, and control the robotic arm according to the collected images, so that the robotic arm can smoothly grab objects during operation, wherein the image can be collected by the camera module. Optionally, the camera module can be arranged on the robotic arm, specifically, the camera module can be arranged at the end of the robotic arm; the camera module can also be arranged outside the robotic arm, specifically, the camera module can be arranged above the calibration area, so that the camera module can capture objects in the working area. Among them, the calibration area can be the area where the robotic arm needs to grab objects during operation. The camera module may include but is not limited to industrial cameras, three-dimensional optical cameras, and depth cameras, etc., which are not limited in the embodiments of the present application.
[0068] In some embodiments, the electronic device can establish a communication connection with the robotic arm and the camera module. The communication connection established between the electronic device and the robotic arm and the camera module can be a wired communication connection or a wireless communication connection, which is not specifically limited. The electronic device can select multiple target calibration positions according to the user clicking on the position in the calibration area displayed by the electronic device, and generate the posture information corresponding to each target calibration position. The electronic device can also generate multiple initial calibration positions according to the calibration parameters input by the user, and calibrate according to the initial calibration positions to obtain the initial transformation relationship, and determine multiple target calibration positions according to the initial transformation relationship, and determine the posture information corresponding to each target calibration position according to the multiple target calibration positions.
[0069] The electronic device can first control the mechanical arm to clamp the calibration plate through the communication connection established with the mechanical arm, and then control the mechanical arm to move the calibration plate to the first target position according to the posture information corresponding to the first target position, and then place the calibration plate at the first target position, and control the camera module to capture the first image corresponding to the calibration plate and the first target calibration position through the communication connection established between the electronic device and the camera module. Among them, the calibration plate can contain multiple calibration points, and the calibration points can be feature points on the calibration plate, such as the corner points of the calibration plate, the pattern on the calibration plate (such as dots, circles, triangles, positive schemes and other identification patterns, etc.).
[0070] As an implementation mode, the electronic device may send a first control signal to the robotic arm based on the posture information corresponding to the first target position to control the robotic arm to move the calibration plate to the first target calibration position, wherein the control signal includes the posture information corresponding to the robotic arm and the first target calibration position, and the posture information may be used to describe the position and posture of the robotic arm when it moves the calibration plate to the first target calibration position. Further, the posture information may be information such as the position and posture of the end of the robotic arm when it moves the calibration plate to the first target calibration position. After moving the calibration plate to the first target calibration position, the electronic device may send a first acquisition signal to the camera module to control the camera module to acquire a first image corresponding to the calibration plate and the first target calibration position, wherein the first image may include the calibration plate at the first target calibration position. The first acquisition signal may be used to indicate that the camera module needs to take a picture.
[0071] Optionally, the first acquisition signal may also include shooting parameters that the camera module needs to adjust during shooting. The shooting parameters may include lens focal length, pixel size, etc., wherein the lens focal length can be used to indicate the focal length of the lens in the camera module, and the pixel size can be used to indicate the number of photosensitive elements in the camera module that need to be used during shooting. The shooting parameters may be manually input into the electronic device by a calibration personnel, or may be parameters preset in the electronic device. After acquiring the shooting parameters, the electronic device may generate a first acquisition signal containing the shooting parameters based on the acquired shooting parameters when sending the first acquisition signal to the camera module.
[0072] By sending control signals to the robotic arm and acquisition signals to the camera module, the electronic device can reasonably control the robotic arm and the camera module. At the same time, through the posture data contained in the control signal and the shooting parameters contained in the acquisition signal, the accuracy of the electronic device's control over the robotic arm and the camera module can be ensured, thereby providing a technical basis for improving the calibration accuracy of hand-eye calibration.
[0073] Step S204, controlling the robotic arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position.
[0074] In some embodiments, after the camera module collects the first image corresponding to the calibration plate and the first target calibration position, the electronic device can send a second control signal to the mechanical arm to control the mechanical arm to rotate the calibration plate, and the second control signal can include the rotation angle of the mechanical arm, wherein the rotation angle can be any angle within the rotation angle interval, or the maximum angle within the rotation angle interval, and the rotation angle interval can be used to limit the range of the rotation angle. The electronic device can obtain the rotation angle interval manually input by the calibration personnel, and determine the rotation angle of the mechanical arm according to the obtained rotation angle interval, and generate a second control signal including the rotation angle according to the rotation angle, and send the second control signal to the mechanical arm to control the mechanical arm to rotate the calibration plate. After the electronic device controls the mechanical arm to rotate the calibration plate through the second control signal, it sends a second acquisition signal to the camera module to control the camera module to collect the second image corresponding to the calibration plate and the first target calibration position, wherein the second image can include the rotated calibration plate. The second acquisition signal can be an acquisition signal different from the first acquisition signal, or it can be an acquisition signal the same as the first acquisition signal.
[0075] Step S206: determining the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image.
[0076] In some embodiments, the rotation center may be the center around which the robot arm rotates the calibration plate, and the rotation center may also be used to indicate the center around which the end of the robot arm rotates. The rotation center may be indicated by image coordinates in a two-dimensional image coordinate system, which may be determined by an image captured by a camera module; the rotation center may also be indicated by coordinates in a camera coordinate system, which may be obtained by transforming image coordinates determined by an image captured by a camera module, wherein the image coordinate system may be a two-dimensional coordinate system established with the image captured by the camera module as a plane.
[0077] The electronic device may determine the rotation center of the calibration plate corresponding to the first target calibration position according to the image position of the calibration plate in the first image and the image position of the calibration plate in the second image. Further, there are multiple calibration points on the calibration plate, and the electronic device may determine the rotation center of the calibration plate corresponding to the first target calibration position according to the image positions of the multiple calibration points in the calibration plate on the first image and the image positions of the multiple calibration points in the calibration plate on the second image.
[0078] The electronic device may convert the calculated rotation centers corresponding to the various target calibration positions into coordinates in the camera coordinate system for representation, so as to obtain the coordinates corresponding to the rotation centers of the various target calibration positions in the camera coordinate system.
[0079] Step S208, calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and multiple target calibration positions; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
[0080] The target transformation relationship between the camera coordinate system and the base coordinate system refers to the transformation between the coordinates in the camera coordinate system and the coordinates in the base coordinate system. The camera coordinate system refers to the coordinate system with the camera as the origin, and the base coordinate system is the coordinate system with the reference point on the robot arm as the origin. The reference point can be the base where the robot arm contacts the platform where the calibration area is located, or it can be the end of the robot arm.
[0081] In some embodiments, the electronic device can record the posture information of the robotic arm corresponding to the first target calibration position when controlling the robotic arm to move the calibration plate to the first target calibration position, or controlling the robotic arm to rotate the calibration plate located at the first target calibration position, thereby obtaining the posture information corresponding to the robotic arm and multiple target calibration positions respectively.
[0082] In some embodiments, the electronic device may construct a calibration equation corresponding to the first target calibration position according to the rotation center corresponding to the calibration plate and the first target calibration position, and the posture information corresponding to the mechanical arm and the first target calibration position, and the calibration equation may include the transformation parameters between the camera coordinate system and the base coordinate system contained in the target transformation relationship. The electronic device constructs calibration equations corresponding to multiple target calibration positions according to the rotation centers corresponding to the multiple target calibration positions, and the posture information corresponding to the mechanical arm and the multiple target calibration positions. Among them, the rotation center corresponding to each target calibration position can be represented by the coordinates in the image coordinate system. According to the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the mechanical arm and the multiple target calibration positions, the calibration equations corresponding to the multiple target calibration positions are obtained, and the solutions corresponding to each calibration equation are solved. According to the solutions corresponding to each calibration equation, the transformation relationship between the image coordinate system and the base coordinate system can be obtained, and then according to the transformation relationship, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated. The camera coordinate system may be a three-dimensional coordinate system with the camera module as the origin, and the base coordinate system may be a coordinate system with the reference point on the robot as the origin. Specifically, the base coordinate system may be a three-dimensional coordinate system with the base of the robot as the origin, and the base of the robot may be the part of the robot that contacts the platform where the calibration area is located. The target transformation relationship may be used to describe the transformation between the camera coordinate system and the base coordinate system. Specifically, the target transformation relationship may be expressed in the form of a matrix or in the form of a function.
[0083] The electronic device obtains calibration equations corresponding to the multiple target calibration positions based on the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions, and obtains the target transformation relationship between the camera coordinate system and the base coordinate system by solving the calibration equations corresponding to the multiple target calibration positions. The target transformation relationship can be calculated based on multiple sets of data, thereby improving the accuracy of calculating the target transformation relationship, thereby improving the calibration accuracy of hand-eye calibration.
[0084] In other embodiments, the electronic device may perform coordinate conversion on the rotation center represented by the coordinates of the image coordinate system corresponding to each target calibration position, and obtain the rotation center represented by the coordinates of the camera coordinate system corresponding to each target calibration position. According to the rotation center represented by the coordinates of the camera coordinate system corresponding to each target calibration position, and the posture information corresponding to the robot arm and the multiple target calibration positions, a corresponding set of equations is generated, and the transformation relationship between the image coordinate system and the base coordinate system is calculated according to the set of equations.
[0085] In an embodiment of the present application, the electronic device controls the mechanical arm to move the calibration plate to a first target calibration position, controls the camera module to collect a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions, controls the mechanical arm to rotate the calibration plate, and then controls the camera module to collect a second image corresponding to the calibration plate and the first target calibration position, determines the rotation center corresponding to the first target calibration position according to the first image and the second image, calculates the position information between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the plurality of target calibration positions, and the position information corresponding to the mechanical arm and the plurality of target calibration positions. The target transformation relationship can be achieved by controlling the robotic arm to move the calibration plate to each target calibration position and rotate it. The rotation center corresponding to each target calibration position can be automatically calculated based on the first image and the second image corresponding to each target calibration position. There is no need to manually determine the rotation center corresponding to each target calibration position, which improves the accuracy of the determined rotation center. In addition, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated based on the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and multiple target calibration positions, which further improves the accuracy of the calculated target transformation relationship and can improve the calibration accuracy of hand-eye calibration.
[0086] See also Figure 3 , Figure 3 1 is a flow chart of another hand-eye calibration method disclosed in an embodiment of the present application. In one embodiment, the hand-eye calibration method further includes the following steps:
[0087] Step S302, obtaining initial posture data of the robotic arm, and controlling the robotic arm to adjust the posture according to the initial posture data to determine the shooting posture corresponding to the robotic arm.
[0088] In some embodiments, the camera module can be set on the mechanical arm. In order to ensure that the calibration plate on the first image and the second image is located at the same position as much as possible, so as to be able to better calculate the rotation center corresponding to each target calibration position based on the first image and the second image, the camera module should select the same position to collect images of the calibration plate, thereby collecting the first image and the second image corresponding to each target calibration position. Therefore, the electronic device can determine the shooting posture corresponding to the mechanical arm according to the initial posture data, thereby ensuring that during the hand-eye calibration process, the camera module on the mechanical arm is located at the same position to collect images of the calibration plate. The shooting posture corresponding to the mechanical arm can be the posture when the camera module on the mechanical arm shoots the calibration plate, and the initial posture data of the mechanical arm can be used to indicate the posture of the mechanical arm when the camera module shoots the calibration plate. The electronic device can manually input the initial posture data through the calibration personnel, or the calibration personnel can select multiple initial posture data stored in the electronic device, so that the electronic device obtains the initial posture data of the mechanical arm. Before calibration, the calibrator can also adjust the robotic arm to a shooting posture, and enable the electronic device to record the current posture data of the robotic arm, thereby obtaining the initial posture data of the robotic arm.
[0089] After receiving the calibration start signal, the electronic device can control the robot arm to adjust its posture according to the acquired initial posture data to determine the shooting posture corresponding to the robot arm, wherein the calibration start signal can be used to indicate the start of hand-eye calibration of the robot arm. Further, the electronic device can send the initial posture data to the robot arm so that the robot arm adjusts its posture according to the initial posture data after receiving the initial posture data.
[0090] When the camera module is set on the robotic arm, the electronic device determines the corresponding shooting posture of the robotic arm based on the acquired initial posture data of the mechanical part, thereby ensuring that during the hand-eye calibration process, the camera modules on the robotic arm are located at the same position to capture images of the calibration plate, so that the electronic device can better determine the rotation center corresponding to each target calibration position based on the first image and the second image corresponding to each target calibration position captured by the camera module.
[0091] In other embodiments, the camera module may be disposed outside the robotic arm. During the hand-eye calibration process, after the electronic device controls the robotic arm to move or rotate the calibration plate, if the robotic arm is not removed in time, the camera module may be blocked, thereby causing the camera module to be unable to capture images of the calibration plate. In order to ensure that the robotic arm does not block the calibration plate when the camera module captures images of the calibration plate, the electronic device may control the robotic arm to adjust its posture based on the acquired initial posture data to determine the corresponding shooting posture of the robotic arm. The shooting posture corresponding to the robotic arm may be a posture that will not block the calibration plate no matter where the calibration plate is moved to the calibration area, and then control the camera module to shoot the calibration plate.
[0092] When the camera module is set outside the mechanical arm, the electronic device controls the mechanical arm to adjust its posture according to the initial posture data to determine the corresponding shooting posture of the mechanical arm, thereby ensuring that the mechanical arm does not block the calibration plate when the camera module collects images of the mechanical arm, and by improving the rationality of the image of the calibration plate collected by the camera module, the accuracy of the rotation center corresponding to each target calibration position can be determined according to the image collected by the camera module.
[0093] Step S304: Control the robotic arm to move the calibration plate to a first target calibration position.
[0094] In some embodiments, the electronic device may send posture data corresponding to the first target calibration position to the robotic arm, so that the robotic arm grabs the calibration plate and moves to the first target calibration position according to the posture data corresponding to the first target calibration position.
[0095] Step S306, controlling the robotic arm to restore to the shooting posture corresponding to the robotic arm, and controlling the camera module arranged on the robotic arm to capture images of the calibration plate, to obtain a first image corresponding to the calibration plate and the first target calibration position.
[0096] In some embodiments, the electronic device can send initial posture data to the robotic arm so that the robotic arm adjusts according to the initial posture data, so that the robotic arm restores the shooting posture corresponding to the robotic arm. After the electronic device controls the robotic arm to restore the shooting posture corresponding to the robotic arm, if the camera module is set on the robotic arm, the electronic device can control the camera module set on the robotic arm to capture the image of the calibration plate to obtain a first image corresponding to the first target calibration position; if the camera module is set outside the robotic arm, the electronic device can control the camera module set outside the robotic arm to capture the image of the calibration plate to obtain a first image corresponding to the first target calibration position.
[0097] Step S308, controlling the robotic arm to rotate the calibration plate.
[0098] In some embodiments, the electronic device can select any angle within the rotation angle range as the rotation angle corresponding to the first target calibration position, and send the rotation angle corresponding to the first target calibration position to the robotic arm, so that the robotic arm grabs the calibration plate and rotates it according to the rotation angle corresponding to the first target calibration position.
[0099] Step S310, controlling the robotic arm to restore to the shooting posture corresponding to the robotic arm, and controlling the camera module arranged on the robotic arm to capture images of the calibration plate, to obtain a second image corresponding to the calibration plate and the first target calibration position.
[0100] In some embodiments, the electronic device may send initial posture data to the robotic arm, so that the robotic arm is adjusted according to the initial posture data, thereby restoring the robotic arm's corresponding shooting posture.
[0101] As an optional implementation, the calibration plate may include at least two calibration points. The calibration points on the calibration plate may be circular. The calibration points on the image may be represented by the coordinates corresponding to the calibration points in the image coordinate system. Furthermore, the calibration points may be represented by the coordinates of the pixels at the center of the calibration points on the image. For example, assuming that the pixels of calibration point A included in the calibration plate on the first image are located in the C1th row and the C2th column on the first image, then calibration point A may be represented by the coordinates (C1, C2). After the electronic device controls the camera module to capture the first image and the second image corresponding to the calibration plate and each target calibration position,
[0102] The rotation center corresponding to the first target calibration position is calculated according to the first image coordinates corresponding to at least two target calibration points of the calibration plate in the first image, and the second image coordinates corresponding to at least two target calibration points in the second image. The calibration plate may further include a characteristic structure, which may be used to determine the position identifier corresponding to each calibration point on each calibration plate, and the position identifier corresponding to each calibration point on the calibration plate may uniquely identify the corresponding calibration point. Specifically, the calibration plate may be a rectangular plate-shaped object, and the characteristic structure on the calibration plate may be a structure with a special symbol printed on any corner of the rectangle of the calibration plate. According to the order of distance between each calibration point on the calibration plate and the characteristic structure, the position identifier corresponding to each calibration point may be determined. For example, assuming that a column of calibration points closest to the characteristic structure on the calibration plate is determined as the calibration points of the first column, and the columns where the calibration points are located are determined by analogy, and a row of calibration points closest to the characteristic structure on the calibration plate is determined as the calibration points of the first row, and the rows where the calibration points are located are determined by analogy, if calibration point A is located in the first row and the third column, the position identifier of the calibration point A can be [1,3]. The electronic device can determine the calibration points corresponding to the at least two position identifiers in the first image according to the at least two preset position identifiers, use the determined at least two calibration points as the target calibration points of the first image, and determine the calibration points corresponding to the at least two position identifiers in the second image, and determine the first image coordinates corresponding to the calibration points determined in the first image according to the positions of the calibration points determined in the first image in the first image. Similarly, the electronic device can determine the calibration points in the second image corresponding to the at least two position identifiers according to the preset at least two position identifiers, use the determined at least two calibration points as the target calibration points of the second image, and determine the calibration points in the second image corresponding to the at least two position identifiers, and determine the second image coordinates corresponding to the calibration points determined in the second image according to the positions of the calibration points determined in the second image in the second image. The electronic device calculates the rotation center corresponding to the first target calibration position according to the determined first image coordinates and the determined second image coordinates.
[0103] The electronic device can calculate the rotation center corresponding to the first target calibration position based on the first image coordinates corresponding to at least two target calibration points of the calibration plate in the first image and the second image coordinates corresponding to at least two target calibration points in the second image. It can automatically calculate the rotation center corresponding to the target calibration position through the captured image, avoid errors caused by manual determination of the rotation center, and improve the calibration accuracy of hand-eye calibration.
[0104] Further, the electronic device can calculate the first straight line on the first image according to the first image coordinates corresponding to the at least two target calibration points of the calibration plate in the first image; calculate the second straight line on the second image according to the first image coordinates corresponding to the at least two target calibration points in the first image; calculate the intersection of the first straight line and the second straight line, and use the intersection of the first straight line and the second straight line as the rotation center corresponding to the first target calibration position. Wherein, the at least two target calibration points in the first image are on the same straight line, and the at least two target calibration points in the second image are also on the same straight line. Specifically, the electronic device can establish the linear equation of the first straight line with the first image coordinates corresponding to the at least two target calibration points of the calibration plate in the first image, thereby obtaining the first straight line on the first image. Similarly, the electronic device can establish the linear equation of the second straight line with the first image coordinates corresponding to the at least two target calibration points in the first image, thereby obtaining the second straight line on the second image. The electronic device can calculate the intersection of the first straight line and the second straight line by solving the linear equation of the first straight line and the equation of the second straight line. The intersection can be represented by coordinates in the image coordinate system, and the intersection of the first straight line and the second straight line can be used as the rotation center corresponding to the first target calibration position.
[0105] The electronic device can determine the straight line through which at least two calibration points pass through the calibration points on the captured image, and calculate the rotation center corresponding to the first target calibration position based on the intersection of the straight lines in the two images, thereby further improving the calibration accuracy of hand-eye calibration.
[0106] Figure 4 is a schematic diagram of determining the rotation center in an embodiment, such as Figure 4 As shown, the calibration plate 401 can be a calibration plate in the first image, the calibration plate 401 in the first image includes a target calibration point 402 and a target calibration point 403, and the straight line 404 is calculated according to the first image coordinates corresponding to the target calibration point 402 and the target calibration point 403 in the first image.
[0107] The calibration plate 405 may be a calibration plate in the second image, and the calibration plate 405 in the second image includes a target calibration point 406 and a target calibration point 407. The straight line 408 is calculated according to the second image coordinates corresponding to the target calibration point 406 and the target calibration point 407 in the second image. The intersection point 409 is the intersection point of the straight line 404 and the straight line 408, and the intersection point 409 may be used as the rotation center corresponding to the first target calibration position.
[0108] Step S312: determining the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image.
[0109] Step S314, calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and multiple target calibration positions; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
[0110] The description of steps S312 to S314 may refer to the relevant description of steps S206 to S208 in the above embodiment, which will not be repeated here.
[0111] In an embodiment of the present application, the electronic device obtains the initial posture data of the robotic arm, and controls the robotic arm to adjust the posture according to the initial posture data to determine the corresponding shooting posture of the robotic arm, controls the robotic arm to move the calibration plate to the first target calibration position, controls the robotic arm to restore the corresponding shooting posture of the robotic arm, and controls the camera module arranged on the robotic arm to capture the image of the calibration plate to obtain a first image corresponding to the calibration plate and the first target calibration position, controls the robotic arm to rotate the calibration plate, controls the robotic arm to restore the corresponding shooting posture of the robotic arm, and controls the camera module arranged on the robotic arm to capture the image of the calibration plate to obtain a second image corresponding to the calibration plate and the first target calibration position, which enables the camera module to capture the image of the calibration plate more reasonably, thereby improving the calibration accuracy of hand-eye calibration.
[0112] Figure 5 FIG. 1 is a flow chart of another hand-eye calibration method disclosed in an embodiment of the present application. Figure 5 As shown, the method comprises the following steps:
[0113] Step S502, control the robotic arm to move the calibration plate to a first initial calibration position, and control the camera module to capture a third image of the calibration plate corresponding to the first initial calibration position; the first initial calibration position is any one of the multiple initial calibration positions.
[0114] The initial calibration position may be the position to which the calibration plate is reached by the manipulator during the pre-calibration process of hand-eye calibration of the manipulator. In one embodiment, before controlling the manipulator to move the calibration plate to the first initial calibration position, the electronic device may obtain calibration parameters, which may be used to indicate the situation in which the electronic device controls the manipulator to move the calibration plate and the angle at which the manipulator rotates the calibration plate. The calibration parameters may include regional parameters of the calibration area, the calibration area ratio, and the rotation angle interval. Among them, the calibration area ratio may be used to describe the range in the calibration area that can be used to determine the initial calibration position. The electronic device may determine the position calibration information based on the regional parameters of the calibration area and the calibration area ratio, wherein the position calibration information calibration position information is used to determine the interval range of the calibration area in the base coordinate system. For example, assuming that the calibration area ratio is 0.8, and the calibration area is an area with a length of 100 cm and a width of 80 cm, then the calibration area can be used to determine the interval of the initial calibration position is an area with a length of 80 cm and a width of 64 cm.
[0115] The electronic device can obtain position calibration information; based on the position calibration information, randomly generate multiple initial calibration positions, and determine the pose information corresponding to the robot arm and the multiple initial calibration positions. Based on the position calibration information, the electronic device randomly selects multiple positions within the interval between the calibration areas in the base coordinate system determined by the position calibration information as multiple initial calibration positions, and determines the pose information corresponding to the multiple initial calibration positions based on the initial calibration positions. The electronic device can automatically generate multiple initial calibration positions based on the acquired position calibration information without manual determination, thereby improving the automation of hand-eye calibration.
[0116] Figure 6 is a schematic diagram of a user operation process in an embodiment, such as Figure 6 As stated, Figure 6 The display interface 600 is a page for the calibration personnel to configure the parameters needed in the calibration process. The display page 600 may include an image display box 601, a parameter configuration box 602, and a "start calibration" button 603. The image display box 601 is an image captured by the camera module. The calibration personnel can observe the situation of the calibration area in the image display box 601, and can also draw an area in the image display box 601 so that the electronic device can obtain the area parameters of the calibration area. The parameter configuration box 602 is an area for the calibration personnel to input the calibration parameters. The "start calibration" button 603 can be used to calibrate the electronic device through the calibration personnel to start hand-eye calibration.
[0117] Figure 7 is a schematic diagram of the operation process of the calibration personnel in an embodiment, such as Figure 7As shown, the calibrator can adjust the camera module to the top of the calibration plate, set the calibration parameters needed during the calibration process, click the "Start Calibration" button, and wait for the hand-eye calibration to be completed.
[0118] Step S504, controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a fourth image corresponding to the calibration plate and the first initial calibration position.
[0119] In some embodiments, the electronic device sends a rotation signal to the robotic arm to control the robotic arm to rotate the calibration plate, and then sends a capture signal to the camera module to control the camera module to capture a fourth image corresponding to the calibration plate and the first initial calibration position.
[0120] Step S506, determining the rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system according to the third image and the fourth image corresponding to the first initial calibration position.
[0121] In some embodiments, the electronic device calculates the rotation center corresponding to the first initial calibration position according to the third image coordinates corresponding to the at least two target calibration points of the calibration plate in the third image, and the fourth image coordinates corresponding to the at least two target calibration points in the fourth image. Further, the electronic device may calculate the third straight line on the third image according to the third image coordinates corresponding to the at least two target calibration points of the calibration plate in the third image; calculate the fourth straight line on the fourth image according to the fourth image coordinates corresponding to the at least two target calibration points in the fourth image; calculate the intersection of the third straight line and the fourth straight line, and use the intersection of the third straight line and the fourth straight line as the rotation center corresponding to the first initial calibration position.
[0122] Step S508, calculating the initial transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and the multiple initial calibration positions, and the posture information corresponding to the robotic arm and the multiple initial calibration positions.
[0123] In some embodiments, the electronic device can obtain calibration equations corresponding to multiple initial calibration positions based on the rotation centers corresponding to the calibration plate and the multiple initial calibration positions, and the posture information corresponding to the robotic arm and the multiple initial calibration positions, and solve the solutions corresponding to each calibration equation. According to the solutions corresponding to each calibration equation, the initial transformation relationship between the camera coordinate system and the base coordinate system can be obtained.
[0124] Step S510, determining multiple target calibration positions and position information corresponding to the robot arm and the multiple target calibration positions respectively according to the initial transformation relationship.
[0125] In some embodiments, the electronic device determines the coordinates of the target calibration positions in the base coordinate system according to the image coordinates in the image coordinate system corresponding to the multiple target calibration positions, and determines the coordinates of the multiple target calibration positions and the mechanical arm and the multiple target calibration positions in the base coordinate system according to the initial transformation relationship, thereby determining the multiple target calibration positions and the posture information corresponding to the mechanical arm and the multiple target calibration positions. The image coordinates in the image coordinate system corresponding to the multiple target calibration positions can be preset.
[0126] Figure 8 is a schematic diagram of a hand-eye calibration process in an embodiment. Figure 8 As shown, the calibration personnel adjust the camera module to the bottom of the calibration plate, set the calibration parameters, and click the "Start Calibration" button to start the hand-eye calibration of the electronic device. The electronic device controls the robotic arm to move the calibration plate to the initial calibration position, and controls the robotic arm to restore the shooting posture while controlling the camera module to shoot, and then controls the robotic arm to move the calibration plate to the next initial calibration position until the calibration plate passes through all the initial calibration positions, and calculates the initial transformation relationship based on the image of the calibration plate, thereby calculating the nine target calibration positions. The electronic device controls the robotic arm to move the calibration plate to the target calibration position, controls the robotic arm to restore the shooting posture and controls the camera module to shoot, then controls the robotic arm to rotate the calibration plate, controls the robotic arm to restore the shooting posture and controls the camera module to shoot, calculates the rotation center corresponding to the current target calibration position, and then controls the robotic arm to move the calibration plate to the next target calibration position until the calibration plate passes through all target calibration positions, calculates the target transformation relationship according to the rotation centers corresponding to the calibration plate and the nine target calibration positions, and completes the calibration.
[0127] Step S512, control the robotic arm to move the calibration plate to a first target calibration position, and control the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of the multiple target calibration positions.
[0128] Step S514, controlling the robotic arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position.
[0129] Step S516: determining the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image.
[0130] Step S518, calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and multiple target calibration positions; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
[0131] The description of steps S512 to S518 may refer to the relevant description of steps S202 to S208 in the above embodiment, which will not be repeated here.
[0132] In an embodiment of the present application, the electronic device controls the mechanical arm to move the calibration plate to a first initial calibration position, controls the camera module to collect a third image of the calibration plate corresponding to the first initial calibration position; the first initial calibration position is any one of a plurality of initial calibration positions, controls the mechanical arm to rotate the calibration plate, and then controls the camera module to collect a fourth image of the calibration plate corresponding to the first initial calibration position; and according to the third image and the second image corresponding to the fourth initial calibration position, determines the rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system; and determines the rotation center of the calibration plate corresponding to the first initial calibration position according to the third image and the second image corresponding to the fourth initial calibration position; and determines the rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system according to the rotation center of the calibration plate corresponding to the plurality of initial calibration positions. The initial transformation relationship between the camera coordinate system and the base coordinate system is calculated, and multiple target calibration positions and the posture information corresponding to the robotic arm and the multiple target calibration positions are determined according to the initial transformation relationship. The electronic device determines the initial transformation relationship through a pre-calibration process, so that the multiple target calibration positions and the posture information corresponding to the robotic arm and the multiple target calibration positions can be automatically determined. There is no need to manually select the calibration position, and then calibrate according to the multiple target calibration positions, which can improve the calibration accuracy of hand-eye calibration.
[0133] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of a hand-eye calibration device disclosed in the embodiment of the present application. The device can be applied to the above electronic devices without specific limitation. Fig. 9 As shown, the hand-eye calibration device 900 may include: a target movement module 901 , a target rotation module 902 , a target determination module 903 , and a target relationship calculation module 904 .
[0134] The target moving module 901 is used to control the mechanical arm to move the calibration plate to a first target calibration position, and control the camera module to collect a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of the multiple target calibration positions;
[0135] The target rotation module 902 is used to control the mechanical arm to rotate the calibration plate, and then control the camera module to capture a second image corresponding to the calibration plate and the first target calibration position;
[0136] The target determination module 903 is used to determine the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image;
[0137] The target relationship calculation module 904 is used to calculate the target transformation relationship between the camera coordinate system and the base coordinate system based on the rotation centers corresponding to the calibration plate and multiple target calibration positions, and the posture information corresponding to the robotic arm and multiple target calibration positions; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
[0138] In one embodiment, the hand-eye calibration device 900 further includes a posture acquisition module:
[0139] The posture acquisition module is used to obtain the initial posture data of the robotic arm and control the robotic arm to adjust the posture according to the initial posture data to determine the corresponding shooting posture of the robotic arm.
[0140] The target movement module 901 is also used to control the robotic arm to move the calibration plate to the first target calibration position; control the robotic arm to restore to the shooting posture corresponding to the robotic arm, and control the camera module arranged on the robotic arm to capture the image of the calibration plate to obtain a first image corresponding to the calibration plate and the first target calibration position.
[0141] In one embodiment, the target rotation module 902 is also used to control the robotic arm to rotate the calibration plate; control the robotic arm to restore to the shooting posture corresponding to the robotic arm, and control the camera module arranged on the robotic arm to capture the image of the calibration plate to obtain a second image corresponding to the calibration position of the calibration plate and the first target.
[0142] In one embodiment, the hand-eye calibration device 900 further includes an initial movement module, an initial rotation module, an initial determination module, an initial relationship determination module, and a target position determination module:
[0143] An initial movement module, used to control the mechanical arm to move the calibration plate to a first initial calibration position, and control the camera module to collect a third image of the calibration plate corresponding to the first initial calibration position; the first initial calibration position is any one of the multiple initial calibration positions;
[0144] An initial rotation module, used to control the mechanical arm to rotate the calibration plate, and then control the camera module to capture a fourth image corresponding to the calibration plate and the first initial calibration position;
[0145] An initial determination module, used to determine the rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system according to the third image and the fourth image corresponding to the first initial calibration position;
[0146] An initial relationship determination module is used to calculate the initial transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and the multiple initial calibration positions, and the posture information corresponding to the robotic arm and the multiple initial calibration positions;
[0147] The target position determination module is used to determine multiple target calibration positions and the posture information corresponding to the robot arm and the multiple target calibration positions according to the initial transformation relationship.
[0148] In one embodiment, the hand-eye calibration device 900 further includes a parameter acquisition module and an initial position determination module:
[0149] An information acquisition module, used to acquire position calibration information; the position calibration information is used to determine the area range of the calibration area in the base coordinate system;
[0150] The initial position determination module is used to randomly generate multiple initial calibration positions according to the calibration position information, and determine the posture information corresponding to the robot arm and the multiple initial calibration positions.
[0151] In one embodiment, the target determination module 903 further includes:
[0152] The center calculation unit is used to calculate the rotation center corresponding to the calibration plate and the first target calibration position according to the first image coordinates corresponding to at least two target calibration points of the calibration plate in the first image, and the second image coordinates corresponding to at least two target calibration points in the second image.
[0153] In one embodiment, the center calculation unit is also used to calculate a first straight line on the first image according to the first image coordinates corresponding to at least two target calibration points of the calibration plate in the first image; calculate a second straight line on the second image according to the first image coordinates corresponding to at least two target calibration points in the first image; calculate the intersection of the first straight line and the second straight line, and use the intersection of the first straight line and the second straight line as the rotation center corresponding to the first target calibration position.
[0154] See also Fig.10 , Fig.10 Schematic diagram of the structure of an electronic device disclosed in the embodiment of the present application. Fig.10 As shown, the electronic device 1000 may include:
[0155] A memory 1001 storing executable program codes;
[0156] A processor 1002 coupled to the memory 1001;
[0157] The processor 1002 calls the executable program code stored in the memory 1001 to execute any hand-eye calibration method disclosed in the embodiments of the present application.
[0158] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by the processor, the processor implements any one of the hand-eye calibration methods disclosed in the embodiment of the present application.
[0159] The embodiments of the present application disclose a computer program product, including a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.
[0160] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.
[0161] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0162] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0163] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0164] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.
[0165] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0166] The above is a detailed introduction to a hand-eye calibration method, device, electronic device and storage medium disclosed in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.
Claims
1. A hand-eye calibration method, characterized in that: The method comprises: Controlling the mechanical arm to move the calibration plate to a first target calibration position, and controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions; Controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position; Determining, according to the first image and the second image, a rotation center of the calibration plate corresponding to the first target calibration position; According to the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions, the target transformation relationship between the camera coordinate system and the base coordinate system is calculated; the base coordinate system is a coordinate system with the reference point on the robotic arm as the origin.
2. The method according to claim 1, characterized in that: The camera module is disposed on the mechanical arm; before the mechanical arm is controlled to move the calibration plate to the first target calibration position and the camera module is controlled to capture a first image corresponding to the calibration plate and the first target calibration position, the method further includes: Acquire initial posture data of the robotic arm, and control the robotic arm to adjust its posture according to the initial posture data to determine the shooting posture corresponding to the robotic arm; The controlling mechanical arm moves the calibration plate to a first target calibration position, and the controlling the camera module collects a first image corresponding to the calibration plate and the first target calibration position, including: Controlling the mechanical arm to move the calibration plate to the first target calibration position; The robotic arm is controlled to return to the shooting posture corresponding to the robotic arm, and the camera module arranged on the robotic arm is controlled to collect images of the calibration plate to obtain a first image corresponding to the calibration plate and the first target calibration position.
3. The method according to claim 2, characterized in that The controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a second image corresponding to the calibration plate and the first target calibration position, comprises: Controlling the mechanical arm to rotate the calibration plate; The robotic arm is controlled to return to the shooting posture corresponding to the robotic arm, and the camera module arranged on the robotic arm is controlled to collect images of the calibration plate to obtain a second image corresponding to the calibration plate and the first target calibration position.
4. The method according to claim 1, characterized in that Before the controlling mechanical arm moves the calibration plate to the first target calibration position and the controlling the camera module to capture a first image corresponding to the calibration plate and the first target calibration position, the method further includes: Controlling the mechanical arm to move the calibration plate to a first initial calibration position, and controlling the camera module to capture a third image of the calibration plate corresponding to the first initial calibration position; the first initial calibration position is any one of a plurality of initial calibration positions; Controlling the mechanical arm to rotate the calibration plate, and then controlling the camera module to capture a fourth image of the calibration plate corresponding to the first initial calibration position; Determining, according to the third image and the fourth image corresponding to the first initial calibration position, a rotation center of the calibration plate corresponding to the first initial calibration position in the camera coordinate system; Calculate the initial transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers respectively corresponding to the calibration plate and the multiple initial calibration positions, and the posture information respectively corresponding to the robotic arm and the multiple initial calibration positions; According to the initial transformation relationship, multiple target calibration positions and position information of the robotic arm corresponding to the multiple target calibration positions are determined.
5. The method according to claim 4, characterized in that Before controlling the mechanical arm to move the calibration plate to the first initial calibration position and controlling the camera module to capture a first image corresponding to the calibration plate and the first initial calibration position, the method further includes: Acquire position calibration information; the position calibration information is used to determine the area range of the calibration area in the base coordinate system; According to the position calibration information, a plurality of initial calibration positions are randomly generated, and the position and posture information corresponding to the robotic arm and the plurality of initial calibration positions are determined.
6. The method according to claim 1, characterized in that The calibration plate includes at least two calibration points; and determining the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image includes: The rotation center of the calibration plate corresponding to the first target calibration position is calculated according to the first image coordinates respectively corresponding to at least two target calibration points of the calibration plate in the first image and the second image coordinates respectively corresponding to the at least two target calibration points in the second image.
7. The method according to claim 6, characterized in that The at least two target calibration points are on the same straight line; the calculation of the rotation center of the calibration plate corresponding to the first target calibration position according to the first image coordinates respectively corresponding to the at least two target calibration points of the calibration plate in the first image and the second image coordinates respectively corresponding to the at least two target calibration points in the second image includes: Calculating a first straight line on the first image according to first image coordinates respectively corresponding to at least two target calibration points of the calibration plate in the first image; Calculating a second straight line on the second image according to first image coordinates corresponding to at least two target calibration points in the first image; An intersection point of the first straight line and the second straight line is calculated, and the intersection point of the first straight line and the second straight line is used as a rotation center corresponding to the first target calibration position.
8. A hand-eye calibration device, characterized in that: include: A target moving module, used to control the mechanical arm to move the calibration plate to a first target calibration position, and control the camera module to capture a first image corresponding to the calibration plate and the first target calibration position; the first target calibration position is any one of a plurality of target calibration positions; A target rotation module, used to control the mechanical arm to rotate the calibration plate, and then control the camera module to capture a second image corresponding to the calibration plate and the first target calibration position; A target determination module, used to determine the rotation center of the calibration plate corresponding to the first target calibration position according to the first image and the second image; The target relationship calculation module is used to calculate the target transformation relationship between the camera coordinate system and the base coordinate system according to the rotation centers corresponding to the calibration plate and the multiple target calibration positions, and the posture information corresponding to the robotic arm and the multiple target calibration positions; the base coordinate system is a coordinate system with a reference point on the robotic arm as the origin.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Determination method and device of rotation center of rotation platform, server and storage medium
CN108038871A
Hand-eye calibration method of industrial robot
CN110103217A
Indoor visual positioning method, system and equipment based on live-action image and storage medium
CN112967341A
Active stereoscopic vision system calibration method
CN113538598A
High-precision compensation method based on fitting circle center
CN115297711A
