A hand-eye calibration method, device, equipment and computer-readable storage medium

Abstract

The present application provides a hand-eye calibration method, which is applied to a hand-eye calibration system. The hand-eye calibration system includes a pin shaft and an industrial camera with a relatively fixed positional relationship. A hole position calibration plate is installed at the hole position of the pin shaft. The hand-eye calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera and a base camera arranged on the opposite side of the hole position calibration plate. Based on this, the base camera and the base calibration plate are controlled to perform N movements. During the movement process, the industrial camera is controlled to photograph the base calibration plate, and the base camera is controlled to photograph the hole position calibration plate. According to the collected calibration plate pictures, the hand-eye relationship between the pin shaft and the industrial camera is calibrated. The present application is completed through an external auxiliary camera and a calibration plate, without the need for redundant operations such as disassembling the mechanism, and can perform fast and accurate hand-eye calibration without a robotic arm. The present application also provides a hand-eye calibration device, equipment, and computer-readable storage medium.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a hand-eye calibration method, device, equipment, and computer-readable storage medium. Background Art

[0002] The hand-eye calibration method is widely used in the field of machine vision. In robot grasping, the computer can analyze the pose information based on the image information obtained by the vision system from the working environment, and then feedback it to the robot to guide the robot to perform operations such as object grasping and moving. In order to convert the pose information in the vision system into the robot system, hand-eye calibration is an important link, and the calibration accuracy has a great impact on the final positioning accuracy of the robot.

[0003] Hand-eye calibration refers to solving the coordinate transformation relationship between the end coordinate system of an industrial robot and the camera coordinate system, or the coordinate transformation relationship between the base coordinate system of an industrial robot and the camera coordinate system. That is, the hand-eye calibration of the robot ultimately aims to solve a hand-eye homogeneous transformation matrix and apply the estimated homogeneous transformation matrix to the robot system. Currently, hand-eye calibration methods are mainly divided into four categories: AX = XB, AX = YB, calibration based on reprojection error, and AXB = YCZ; according to the order of solving unknown parameters, they can also be divided into the separation method and the overall method.

[0004] According to the different relative positions of the camera and the robot, the hand-eye vision system is divided into the Eye-in-Hand system and the Eye-to-Hand system. Among them, the camera of the Eye-in-Hand system is installed at the end-effector of the robotic arm and moves with the robot during the robot's operation; the camera of the Eye-to-Hand system is installed at a fixed position outside the robot body and does not move with the robot during the robot's operation.

[0005] However, for special usage scenarios without a robotic arm, the existing hand-eye calibration methods cannot be directly used for fast and accurate calibration. Summary of the Invention

[0006] This application provides a hand-eye calibration method, device, equipment, and computer-readable storage medium, which can perform fast and accurate hand-eye calibration in a scenario without a robotic arm.

[0007] Specifically, this application is implemented through the following technical solutions:

[0008] In a first aspect, the present application provides a hand-eye calibration method, which is applied to a hand-eye calibration system; the hand-eye calibration system includes a pin shaft and an industrial camera with a relatively fixed positional relationship, a hole calibration plate is installed at the hole position of the pin shaft, the hand-eye calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera, and a base camera arranged on the opposite side of the hole calibration plate; the method includes:

[0009] Controlling the base camera and the base calibration plate to perform N movements, and during the movement process, controlling the industrial camera to photograph the base calibration plate on the opposite side and controlling the base camera to photograph the hole calibration plate on the opposite side, where N is greater than or equal to 1;

[0010] Calibrating the hand-eye relationship between the pin shaft and the industrial camera based on the captured calibration plate images.

[0011] Optionally, the movement mode of the base camera and the base calibration plate includes a translational and / or rotational movement mode.

[0012] Optionally, the first pose relationship between the base camera and the base calibration plate is a constant pose relationship.

[0013] Optionally, the base camera and the base calibration plate are fixed on a fixed rod; the controlling the base camera and the base calibration plate to perform N movements includes:

[0014] Controlling the base camera and the base calibration plate to perform N movements by controlling the fixed rod to perform N movements.

[0015] Optionally, the calibrating the hand-eye relationship between the pin shaft and the industrial camera based on the captured calibration plate images includes:

[0016] Determining the second pose relationship between the base calibration plate and the industrial camera based on the base calibration plate image captured by the industrial camera;

[0017] Determining the third pose relationship between the hole calibration plate and the base camera based on the hole calibration plate image captured by the base camera;

[0018] Determining the pose relationship between the hole calibration plate and the industrial camera based on the first pose relationship, the second pose relationship, and the third pose relationship to achieve the calibration of the hand-eye relationship between the pin shaft and the industrial camera, where the first pose relationship is the pose relationship between the base camera and the base calibration plate.

[0019] Optionally, determining the second pose relationship between the base calibration plate and the industrial camera includes: solving a second coordinate transformation matrix between the coordinate system of the base calibration plate and the coordinate system of the industrial camera;

[0020] Correspondingly, determining the third pose relationship between the hole position calibration plate and the base camera includes: solving a third coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the base camera;

[0021] Correspondingly, determining the pose relationship between the hole position calibration plate and the industrial camera based on the first pose relationship, the second pose relationship, and the third pose relationship includes: determining a fourth coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the industrial camera based on the first coordinate transformation matrix between the coordinate system of the base camera and the coordinate system of the base calibration plate, and the second coordinate transformation matrix and the third coordinate transformation matrix.

[0022] In a second aspect, the present application provides a hand-eye calibration device, which is applied to a hand-eye calibration system; the hand-eye calibration system includes an insertion pin shaft and an industrial camera with a relatively fixed positional relationship, a hole position calibration plate is installed at the hole position of the insertion pin shaft, and the hand-eye calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera, and a base camera arranged on the opposite side of the hole position calibration plate; the device includes:

[0023] A calibration plate photographing unit, configured to control the base camera and the base calibration plate to perform N motions, and during the motion process, control the industrial camera to photograph the base calibration plate on the opposite side, and control the base camera to photograph the hole position calibration plate on the opposite side, where N is greater than or equal to 1;

[0024] A hand-eye relationship calibration unit, configured to calibrate the hand-eye relationship between the insertion pin shaft and the industrial camera based on the calibration plate pictures collected by photographing.

[0025] Optionally, the motion modes of the base camera and the base calibration plate include translational and / or rotational motion modes.

[0026] Optionally, the first pose relationship between the base camera and the base calibration plate is a constant pose relationship.

[0027] Optionally, the base camera and the base calibration plate are fixed on a fixed rod; when the calibration plate photographing unit controls the base camera and the base calibration plate to perform N motions, it is specifically configured to: control the base camera and the base calibration plate to perform N motions by controlling the fixed rod to perform N motions.

[0028] Optionally, the hand-eye relationship calibration unit includes:

[0029] A second pose relationship determination subunit, configured to determine a second pose relationship between the base calibration board and the industrial camera according to a picture of the base calibration board captured by the industrial camera;

[0030] A third pose relationship determination subunit, configured to determine a third pose relationship between the hole position calibration board and the base camera according to a picture of the hole position calibration board captured by the base camera;

[0031] A hand-eye relationship calibration subunit, configured to determine a pose relationship between the hole position calibration board and the industrial camera according to the first pose relationship, the second pose relationship, and the third pose relationship, so as to calibrate the hand-eye relationship between the insertion pin shaft and the industrial camera, where the first pose relationship is the pose relationship between the base camera and the base calibration board.

[0032] Optionally, the second pose relationship determination subunit is specifically configured to solve a second coordinate transformation matrix between the coordinate system of the base calibration board and the coordinate system of the industrial camera;

[0033] The third pose relationship determination subunit is specifically configured to solve a third coordinate transformation matrix between the coordinate system of the hole position calibration board and the coordinate system of the base camera;

[0034] The hand-eye relationship calibration subunit is specifically configured to determine a fourth coordinate transformation matrix between the coordinate system of the hole position calibration board and the coordinate system of the industrial camera according to the first coordinate transformation matrix between the coordinate system of the base camera and the coordinate system of the base calibration board, and the second coordinate transformation matrix and the third coordinate transformation matrix.

[0035] An electronic device includes: a processor and a memory;

[0036] The memory is used to store a computer program;

[0037] The processor is configured to execute the above-mentioned hand-eye calibration method by calling the computer program.

[0038] A computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the above-mentioned hand-eye calibration method is implemented.

[0039] As can be seen from the technical solutions provided by the present application above, the eye-in-hand calibration system includes a plug pin shaft and an industrial camera with a relatively fixed positional relationship. A hole position calibration plate is installed at the hole position of the plug pin shaft. The eye-in-hand calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera, and a base camera arranged on the opposite side of the hole position calibration plate. Based on this, in order to achieve eye-in-hand calibration, the base camera and the base calibration plate can be controlled to move N times, and during the movement process, the industrial camera is controlled to take pictures of the base calibration plate on the opposite side, and the base camera is controlled to take pictures of the hole position calibration plate on the opposite side. Then, according to the calibration plate pictures collected by the shooting, the eye-in-hand relationship between the plug pin shaft and the industrial camera is calibrated. It can be seen that in the special use scenario of shaft-hole fitting, the present application is completed through an external auxiliary camera and a calibration plate, without the need for redundant operations such as disassembling the mechanism, and can quickly and accurately perform eye-in-hand calibration without a robotic arm or a laser positioning device. Description of the Drawings

[0040] Figure 1 It is a schematic structural diagram of Eye-in-Hand shown in the present application;

[0041] Figure 2 It is a schematic diagram of the eye-in-hand calibration system shown in the present application;

[0042] Figure 3A It is one of the schematic diagrams of the actual working conditions of shaft-hole alignment shown in the present application;

[0043] Figure 3B It is another schematic diagram of the actual working conditions of shaft-hole alignment shown in the present application;

[0044] Figure 4 It is a schematic flowchart of a method for eye-in-hand calibration shown in the present application;

[0045] Figure 5 It is a schematic diagram of the conversion of various coordinate systems of Eye-in-Hand shown in the present application;

[0046] Figure 6 It is a schematic diagram of the composition of a device for eye-in-hand calibration shown in the present application;

[0047] Figure 7 It is a schematic structural diagram of an electronic device shown in the present application. Detailed Description of the Embodiments

[0048] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0050] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0051] It should be noted that in the traditional Eye-in-Hand hand-eye calibration method based on a robotic arm, the pose relationship between the end of the robotic arm and the camera fixed at the end of the robotic arm is calibrated, as Figure 1 shown in the structural schematic diagram of Eye-in-Hand, where C represents the camera coordinate system, E represents the end coordinate system, B represents the robotic arm base coordinate system, and K represents the calibration plate coordinate system.

[0052] However, the hand-eye calibration method provided in the embodiments of this application is for a special usage scenario of shaft-hole fitting. In this scenario, the pose relationships among the pin hole, the pin shaft, and the camera need to be calibrated. Therefore, the hand-eye calibration method of the embodiments of this application is used for a vision-based shaft-hole alignment system and is a hand-eye calibration method of Eye-in-Hand without a robotic arm, which can perform fast and accurate calibration without a robotic arm.

[0053] See Figure 2 the schematic diagram of the hand-eye calibration system shown in Figure 3A one of the schematic diagrams of the actual working conditions of shaft-hole alignment shown in Figure 3B and the second schematic diagram of the actual working conditions of shaft-hole alignment shown in Figure 2 and Figures 3A - 3BThe relative fixed position relationship between the inserted pin shaft 2 and the industrial camera 1 shown is only schematic. That is to say, other relative fixed positions can be adopted between the inserted pin shaft 2 and the industrial camera 1 based on the actual application scenario. For example, different settings in terms of the relative distance and / or relative angle between the two.

[0054] In the embodiment of the present application, the industrial camera 1 and the inserted pin shaft 2 are fixed on the first fixing tooling (such as Figure 3B the fixing tooling A in Figure 3B ), and the pin hole 3 is fixed on the second fixing tooling (such as

[0055] the fixing tooling B in Figure 3B ). The calibration result obtained by using the hand-eye calibration method provided by the embodiment of the present application is used to achieve the accurate docking of the inserted pin shaft 2 and the pin hole 3.

[0056] It can be understood that in such a scenario, because there are only Figure 2 and Figures 3A - 3B the fixing tooling A and the fixing tooling B shown, equipment with known pose relationships such as a robotic arm cannot be used, thus lacking the pose information of the industrial camera 1 relative to the inserted pin shaft 2. For this reason, the embodiment of the present application proposes an Eye-in-Hand hand-eye calibration method that does not rely on a robotic arm. This method divides the solution of the hand-eye relationship into two steps: solving the "calibration plate pose calibration" and the "hand-eye pose relationship calibration".

[0056] Next, the hand-eye calibration method provided by the embodiment of the present application will be specifically introduced in conjunction with Figure 2 and Figures 3A - 3B .

[0057] Refer to Figure 4 , which is a schematic flowchart of a hand-eye calibration method provided by the embodiment of the present application. This method is applied to a hand-eye calibration system (such as Figure 2 the hand-eye calibration system shown in

[0058] ); this hand-eye calibration system may include an inserted pin shaft 2 and an industrial camera 1 with a relatively fixed position relationship. A hole position calibration plate 5 is installed at the hole position of the inserted pin shaft 2. This hand-eye calibration system further includes a base calibration plate 7 arranged on the opposite side of the industrial camera 1, and a base camera 6 arranged on the opposite side of the hole position calibration plate 5.

[0058] Before performing the method steps shown in Figure 4 , three operations need to be carried out (the embodiment of the present application does not limit the operation sequence of these three operations), which are respectively: installing a temporary calibration plate 5 at the hole position of the inserted pin shaft 2, which is herein referred to as the hole position calibration plate; calibrating the internal parameters of the industrial camera 1 and the base camera 6; fixing the position of the base camera 6, which replaces the robotic arm base, and the base calibration plate 7.

[0059] It should be noted that the embodiment of the present application does not limit the position relationship between the industrial camera 1 and the inserted pin shaft 2. It can be Figure 2The positional relationship shown has the industrial camera 1 above and the insertion pin shaft 2 below. Of course, it can also be the positional relationship with the insertion pin shaft 2 above and the industrial camera 1 below, or it can be a left - right positional relationship, etc. However, it is necessary to ensure that a base calibration plate 7 is arranged on the opposite side of the industrial camera 1, and a base camera 6 is arranged on the opposite side of the insertion pin shaft 2.

[0060] Figure 4 The hand - eye calibration method shown includes the following steps:

[0061] S401: Control the base camera 6 and the base calibration plate 7 to perform N movements. During the movement process, control the industrial camera 1 to photograph the base calibration plate 7 on the opposite side, and control the base camera 6 to photograph the hole - position calibration plate 5 on the opposite side, where N is greater than or equal to 1.

[0062] In the embodiment of the present application, a base camera 6 is used to simulate and replace the robotic arm base, and the pose relationship between the base camera 6 and the base calibration plate 7 is defined as the first - pose relationship. The first - pose relationship can be a constant - pose relationship M, that is, the base camera 6 and the base calibration plate 7 maintain this constant - pose relationship M both in the initial position setting and in the subsequent movement process (N times). Since the hand - eye calibration introduced later requires the use of the first - pose relationship, when the first - pose relationship is constantly M, it will be more convenient for calibration calculation; of course, the first - pose relationship can also be a non - constant - pose relationship. For example, during the movement process of the base camera 6 and the base calibration plate 7, N pose changes are performed in a preset manner, and each pose relationship between the base camera 6 and the base calibration plate 7 can be known or can be calculated according to the change rule.

[0063] In the embodiment of the present application, the movement mode of the base camera 6 and the base calibration plate 7 can include translational and / or rotational movement modes. That is to say, when controlling the base camera 6 and the base calibration plate 7 to perform N movements, for each movement, it can only perform translational movement, only perform rotational movement, or perform both translational and rotational movements simultaneously. The purpose is to solve the pose relationship between the hole position of the insertion pin shaft 2 and the industrial camera 1 through several translations and / or rotations. Through simulation experiments and actual experiments, it is proved that this method can replace the traditional hand - eye calibration method that needs to rely on a robotic arm, and the finally solved pose parameters can enable the shaft - hole alignment system to successfully dock between the insertion pin shaft 2 and the insertion pin hole 3 within the error range.

[0064] In the embodiment of the present application, the base camera 6 and the base calibration plate 7 can be fixed on a fixed rod, such as Figure 2 the fixed rod shown. By adjusting the angle of the fixed rod, a certain pose relationship (such as a constant - pose relationship, that is, keeping the poses of the base camera 6 and the base calibration plate 7 unchanged) is established between the base camera 6 and the base calibration plate 7.

[0065] Based on this, in one implementation, the "controlling the base camera 6 and the base calibration board 7 to perform N motions" in S401 may include: controlling the fixed rod to perform N motions to control the base camera 6 and the base calibration board 7 to perform N motions. Wherein, each motion of the fixed rod is a translational and / or rotational motion.

[0066] During the motion, control the industrial camera 1 to take pictures of the base calibration board 7 on the opposite side, and control the base camera 6 to take pictures of the hole position calibration board 5 on the opposite side. In actual operation, the base camera 6 and the industrial camera 1 can be used to simultaneously take N pictures of the calibration boards on the opposite side, each collecting N pictures. Specifically, pictures can be taken after each motion is completed, and the corresponding relationship between the two pictures taken corresponding to each motion can be established.

[0067] S402: Calibrate the hand-eye relationship between the insertion pin shaft 2 and the industrial camera 1 based on the captured calibration board pictures.

[0068] In the embodiment of the present application, since the "hand" in the "eye-in-hand" pose calibration is the insertion pin shaft 2 and the "eye" is the industrial camera 1, therefore, what needs to be calibrated is the hand-eye coordinate relationship between the insertion pin shaft 2 and the industrial camera 1 that is relatively fixed to it. For this purpose, four coordinate systems need to be defined, as Figure 5 shown in the schematic diagram of the conversion of each coordinate system of the eye-in-hand. Among them, the coordinate system of the hole position calibration board 5 is used to simulate and replace the traditional end-effector coordinate system of the robotic arm. In the figure, O end -X end Y end Z end is the coordinate system of the hole position calibration board; the base camera 6 arranged on the opposite side of the hole position calibration board 5 is used to simulate and replace the base of the robotic arm, so the coordinate system of the base camera 6 is O base -X base Y base Z base ; the coordinate system of the industrial camera 1 is O cam -X cam Y cam Z cam ; the coordinate system of the base calibration board 7 arranged on the opposite side of the industrial camera 1 is set as O board -X board Y board Z board . In this way, during the N motions of the base camera 6 and the base calibration board 7, the coordinate relationships among the industrial camera 1, the hole position calibration board 5, the base camera 6, and the base calibration board 7 after each motion can be determined.

[0069] Specifically, in an implementation manner of the embodiment of the present application, for "calibrating the hand-eye relationship between the insertion pin shaft 2 and the industrial camera 1 based on the captured calibration plate image", the following steps S4021 - S4023 may be specifically included:

[0070] S4021: Determine the second pose relationship between the base calibration plate 7 and the industrial camera 1 based on the base calibration plate image captured by the industrial camera 1.

[0071] In this implementation manner, for the convenience of distinction, the pose relationship between the base calibration plate 7 and the industrial camera 1 is defined as the second pose relationship. Since the coordinate system of the base calibration plate 7 is O board -X board Y board Z board , and the coordinate system of the industrial camera 1 is O cam -X cam Y cam Z cam , therefore, the second pose relationship may be the pose relationship of the base calibration plate 7 relative to the industrial camera 1 It should be noted that since the base camera 6 and the base calibration plate 7 have moved N times, therefore, the second pose relationship includes N pose relationships of the base calibration plate 7 relative to the industrial camera 1, that is, N

[0072] S4021 may specifically include: Based on the base calibration plate image captured by the industrial camera 1, solve the second coordinate transformation matrix between the coordinate system of the base calibration plate 7 and the coordinate system of the industrial camera 1. Specifically, for the N captured base calibration plate images, solve the pose relationships of the base calibration plate 7 relative to the industrial camera 1 respectively, and solve N Using these N represent the second coordinate transformation matrix between the coordinate system of the base calibration plate 7 and the coordinate system of the industrial camera 1.

[0073] S4022: Determine the third pose relationship between the hole position calibration plate 5 and the base camera 6 based on the hole position calibration plate image captured by the base camera 6.

[0074] In this implementation manner, for the convenience of distinction, the pose relationship between the hole position calibration plate 5 and the base camera 6 is defined as the third pose relationship. Since the coordinate system of the hole position calibration plate 5 is O end -X end Y end Z end , and the coordinate system of the base camera 6 is O base -X base Y base Z base, therefore, the third pose relationship can be the pose relationship of the hole position calibration plate 5 relative to the base camera 6 It should be noted that since the base camera 6 and the base calibration plate 7 have moved N times, therefore, the third pose relationship includes N pose relationships of the hole position calibration plate 5 relative to the base camera 6, that is, N

[0075] S4022 may specifically include: according to the hole position calibration plate pictures taken by the base camera 6, solving the third coordinate transformation matrix between the coordinate system of the hole position calibration plate 5 and the coordinate system of the base camera 6. Specifically, for the N collected hole position calibration plate pictures, the pose relationships of the hole position calibration plate 5 relative to the base camera 6 are solved respectively, and N Using these N represents the third coordinate transformation matrix between the coordinate system of the hole position calibration plate 5 and the coordinate system of the base camera 6.

[0076] It should be noted that the embodiments of the present application do not limit the execution order of steps S4021 and S40232.

[0077] S4023: According to the first pose relationship, the second pose relationship, and the third pose relationship, determine the pose relationship between the hole position calibration plate 5 and the industrial camera 1, so as to realize the hand-eye relationship calibration between the insertion pin shaft and the industrial camera.

[0078] In this implementation manner, when the second pose relationship is determined through S4021 and the third pose relationship is determined through S4022, and since the first pose relationship is the pose relationship between the base camera 6 and the base calibration plate 7 and the first pose relationship is known, therefore, using these three pose relationships, the pose relationship between the hole position calibration plate 5 and the industrial camera 1 can be determined, and at this time, the hand-eye relationship calibration between the insertion pin shaft 2 and the industrial camera 1 is realized.

[0079] S4023 may specifically include: according to the first coordinate transformation matrix between the coordinate system of the base camera 6 and the coordinate system of the base calibration plate 7, and the second coordinate transformation matrix and the third coordinate transformation matrix, determine the fourth coordinate transformation matrix between the coordinate system of the hole position calibration plate 5 and the coordinate system of the industrial camera 1.

[0080] Specifically, since the first pose relationship (such as the above constant pose M) between the base camera 6 and the base calibration plate 7 is known, therefore, the first coordinate transformation matrix between the coordinate system of the base camera 6 and the coordinate system of the base calibration plate 7 (that is, N ) is also known; the second coordinate transformation matrix between the coordinate system of the base calibration plate 7 and the coordinate system of the industrial camera 1 (that is, N ) and the third coordinate transformation matrix between the coordinate system of the hole position calibration plate 5 and the coordinate system of the base camera 6 (i.e., N ) are obtained by solving the pose relationship between the calibration plate and the camera for the collected calibration plate images through the above steps S4021 and S4022 respectively.

[0081] Among them, in the above first coordinate transformation matrix, second coordinate transformation matrix and third coordinate transformation matrix, the elements at the same matrix position correspond to the same movement among the N movements in S401. It can be understood that when there is a constant pose relationship M between the base camera 6 and the base calibration plate 7, it can be a constant value.

[0082] In the working condition of the shaft-hole fit in the embodiment of the present application, the ultimately required pose relationship is the N pose relationships of the hole position calibration plate 5 relative to the industrial camera 1 (i.e., N ). Using this transformation matrix to represent the coordinate transformation matrix between the coordinate system of the hole position calibration plate 5 and the coordinate system of the industrial camera 1, which is defined here as the fourth coordinate transformation relationship, thus completing the hand-eye calibration.

[0083] Specifically, the coordinate transformation relationship corresponding to each movement in the N movements of S401 (the coordinate transformation relationship between the hole position calibration plate 5 and the industrial camera 1) can be solved by the following formula:

[0084]

[0085] It can be seen that the hand-eye calibration result can be solved through matrix calculation, and the method is simple.

[0086] In the hand-eye calibration method provided in the embodiment of the present application, this method is applied to a hand-eye calibration system; the hand-eye calibration system includes an insertion pin shaft and an industrial camera with a relatively fixed positional relationship. A hole position calibration plate is installed at the hole position of the insertion pin shaft. The hand-eye calibration system also includes a base calibration plate arranged on the opposite side of the industrial camera and a base camera arranged on the opposite side of the hole position calibration plate. Based on this, in order to achieve hand-eye calibration, the base camera and the base calibration plate can be controlled to perform N movements, and during the movement process, the industrial camera is controlled to take pictures of the base calibration plate on the opposite side, and the base camera is controlled to take pictures of the hole position calibration plate on the opposite side. Then, based on the collected calibration plate images, the hand-eye relationship between the insertion pin shaft and the industrial camera is calibrated. It can be seen that in the special use scenario of shaft-hole fit in the embodiment of the present application, it is completed through an external auxiliary camera and a calibration plate, without the need for redundant operations such as disassembling the mechanism, and fast and accurate hand-eye calibration can be carried out without a robotic arm or a laser positioning device.

[0087] See Figure 6, which is a schematic diagram of the composition of an eye-in-hand calibration device shown in this application. This device is applied to an eye-in-hand calibration system. The eye-in-hand calibration system includes a pin shaft and an industrial camera with a relatively fixed positional relationship. A hole position calibration plate is installed at the hole position of the pin shaft. The eye-in-hand calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera and a base camera arranged on the opposite side of the hole position calibration plate. The device method includes:

[0088] A calibration plate photographing unit 610, configured to control the base camera and the base calibration plate to perform N movements, and during the movement process, control the industrial camera to photograph the base calibration plate on the opposite side and control the base camera to photograph the hole position calibration plate on the opposite side, where N is greater than or equal to 1;

[0089] An eye-in-hand relationship calibration unit 620, configured to calibrate the eye-in-hand relationship between the pin shaft and the industrial camera according to the calibration plate pictures collected by photographing.

[0090] In an implementation manner of the embodiment of this application, the movement modes of the base camera and the base calibration plate include translational and / or rotational movement modes.

[0091] In an implementation manner of the embodiment of this application, the first pose relationship between the base camera and the base calibration plate is a constant pose relationship.

[0092] In an implementation manner of the embodiment of this application, the base camera and the base calibration plate are fixed on a fixed rod. When the calibration plate photographing unit 610 controls the base camera and the base calibration plate to perform N movements, it is specifically configured to: control the base camera and the base calibration plate to perform N movements by controlling the fixed rod to perform N movements.

[0093] In an implementation manner of the embodiment of this application, the eye-in-hand relationship calibration unit 620 includes:

[0094] A second pose relationship determination subunit, configured to determine the second pose relationship between the base calibration plate and the industrial camera according to the base calibration plate picture photographed by the industrial camera;

[0095] A third pose relationship determination subunit, configured to determine the third pose relationship between the hole position calibration plate and the base camera according to the hole position calibration plate picture photographed by the base camera;

[0096] The hand-eye relationship calibration sub-unit is used to determine the pose relationship between the hole position calibration plate and the industrial camera according to the first pose relationship, the second pose relationship, and the third pose relationship, so as to realize the hand-eye relationship calibration between the insertion pin shaft and the industrial camera. Among them, the first pose relationship is the pose relationship between the base camera and the base calibration plate.

[0097] In an implementation manner of the embodiment of the present application,

[0098] The second pose relationship determination sub-unit is specifically used to solve the second coordinate transformation matrix between the coordinate system of the base calibration plate and the coordinate system of the industrial camera;

[0099] The third pose relationship determination sub-unit is specifically used to solve the third coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the base camera;

[0100] The hand-eye relationship calibration sub-unit is specifically used to determine the fourth coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the industrial camera according to the first coordinate transformation matrix between the coordinate system of the base camera and the coordinate system of the base calibration plate, and the second coordinate transformation matrix and the third coordinate transformation matrix.

[0101] For the implementation process of the functions and roles of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and will not be elaborated here.

[0102] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0103] The embodiment of the present application also provides an electronic device, and the structural schematic diagram of the electronic device is as Figure 7As shown, the electronic device 7000 includes at least one processor 7001, a memory 7002, and a bus 7003. At least one processor 7001 is electrically connected to the memory 7002. The memory 7002 is configured to store at least one computer-executable instruction, and the processor 7001 is configured to execute the at least one computer-executable instruction, thereby performing the steps of any hand-eye calibration method provided in any embodiment or any alternative embodiment of the present application.

[0104] Further, the processor 7001 can be an FPGA (Field-Programmable Gate Array), or other devices with logical processing capabilities, such as an MCU (Microcontroller Unit) or a CPU (Central Processing Unit).

[0105] Applying the embodiments of the present application, in the special use scenario of shaft-hole fitting, it is completed through an external auxiliary camera and a calibration board, without the need for redundant operations such as disassembling the mechanism, and rapid and accurate hand-eye calibration can be performed without a robotic arm or a laser positioning device.

[0106] The embodiments of the present application also provide another computer-readable storage medium storing a computer program, which is used to implement the steps of any hand-eye calibration method provided in any embodiment or any alternative embodiment of the present application when executed by a processor.

[0107] The computer-readable storage medium provided by the embodiments of the present application includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. That is, the readable storage medium includes any medium that stores or transmits information in a form readable by a device (such as a computer).

[0108] Applying the embodiments of the present application, in the special use scenario of shaft-hole fitting, it is completed through an external auxiliary camera and a calibration board, without the need for redundant operations such as disassembling the mechanism, and rapid and accurate hand-eye calibration can be performed without a robotic arm or a laser positioning device.

[0109] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A hand-eye calibration method, characterized in that, The method is applied to a hand-eye calibration system; the hand-eye calibration system includes a pin shaft and an industrial camera with a relatively fixed positional relationship, a hole position calibration plate is installed at the hole position of the pin shaft, the hand-eye calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera, and a base camera arranged on the opposite side of the hole position calibration plate; the method includes: Controlling the base camera and the base calibration plate to perform N movements, and during the movement process, controlling the industrial camera to capture the base calibration plate on the opposite side and controlling the base camera to capture the hole position calibration plate on the opposite side, where N is greater than or equal to 1; Calibrating the hand-eye relationship between the pin shaft and the industrial camera according to the captured calibration plate pictures; Wherein, the base camera and the base calibration plate are fixed on a fixed rod; the controlling the base camera and the base calibration plate to perform N movements includes: Controlling the base camera and the base calibration plate to perform N movements by controlling the fixed rod to perform N movements.

2. The method according to claim 1, characterized in that, The movement modes of the base camera and the base calibration plate include translational and / or rotational movement modes.

3. The method according to claim 1, characterized in that, The first pose relationship between the base camera and the base calibration plate is a constant pose relationship.

4. The method according to any one of claims 1-3, characterized in that, The calibrating the hand-eye relationship between the pin shaft and the industrial camera according to the captured calibration plate pictures includes: Determining the second pose relationship between the base calibration plate and the industrial camera according to the base calibration plate pictures captured by the industrial camera; Determining the third pose relationship between the hole position calibration plate and the base camera according to the hole position calibration plate pictures captured by the base camera; Determining the pose relationship between the hole position calibration plate and the industrial camera according to the first pose relationship, the second pose relationship, and the third pose relationship, so as to realize the calibration of the hand-eye relationship between the pin shaft and the industrial camera, where the first pose relationship is the pose relationship between the base camera and the base calibration plate.

5. The method according to claim 4, characterized in that, The determining the second pose relationship between the base calibration plate and the industrial camera includes: solving the second coordinate transformation matrix between the coordinate system of the base calibration plate and the coordinate system of the industrial camera; Correspondingly, the determining the third pose relationship between the hole position calibration plate and the base camera includes: solving the third coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the base camera; Correspondingly, the determining the pose relationship between the hole position calibration plate and the industrial camera according to the first pose relationship, the second pose relationship, and the third pose relationship includes: determining the fourth coordinate transformation matrix between the coordinate system of the hole position calibration plate and the coordinate system of the industrial camera according to the first coordinate transformation matrix between the coordinate system of the base camera and the coordinate system of the base calibration plate, and the second coordinate transformation matrix and the third coordinate transformation matrix.

6. A hand-eye calibration device, characterized in that, The device is applied to a hand-eye calibration system; the hand-eye calibration system includes a pin shaft and an industrial camera with a relatively fixed positional relationship, a hole position calibration plate is installed at the hole position of the pin shaft, and the hand-eye calibration system further includes a base calibration plate arranged on the opposite side of the industrial camera, and a base camera arranged on the opposite side of the hole position calibration plate; the device includes: A calibration plate photographing unit, configured to control the base camera and the base calibration plate to perform N movements, and during the movement process, control the industrial camera to photograph the base calibration plate on the opposite side, and control the base camera to photograph the hole position calibration plate on the opposite side, where N is greater than or equal to 1; A hand-eye relationship calibration unit, configured to calibrate the hand-eye relationship between the pin shaft and the industrial camera according to the photographed calibration plate pictures; Wherein, the base camera and the base calibration plate are fixed on a fixed rod; when the calibration plate photographing unit controls the base camera and the base calibration plate to perform N movements, it is specifically configured to: control the base camera and the base calibration plate to perform N movements by controlling the fixed rod to perform N movements.

7. The device according to claim 6, characterized in that, The hand-eye relationship calibration unit includes: A second pose relationship determination subunit, configured to determine the second pose relationship between the base calibration plate and the industrial camera according to the base calibration plate picture photographed by the industrial camera; A third pose relationship determination subunit, configured to determine the third pose relationship between the hole position calibration plate and the base camera according to the hole position calibration plate picture photographed by the base camera; A hand-eye relationship calibration subunit, configured to determine the pose relationship between the hole position calibration plate and the industrial camera according to the first pose relationship, the second pose relationship, and the third pose relationship, so as to realize the calibration of the hand-eye relationship between the pin shaft and the industrial camera, where the first pose relationship is the pose relationship between the base camera and the base calibration plate.

8. An electronic device, characterized in that, Includes: A processor and a memory; The memory is used to store a computer program; The processor is configured to execute the hand-eye calibration method according to any one of claims 1-5 by calling the computer program.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the program is executed by the processor, it realizes the hand-eye calibration method according to any one of claims 1-5.

Images