Binocular vision calibration method, calibration device, robot and storage medium

Abstract

The present invention provides a binocular vision calibration method, device and equipment, which relates to the field of robot control technology and is designed to solve the problem of difficulty in adapting to the situation where the relative position between the camera and the robot changes. The binocular vision calibration method includes: obtaining a first spatial transformation relationship between an end effector and a first target fixed on the end effector, a first position of the tip of a followed second target relative to the coordinate system of the second target, a first position of the first target in the camera coordinate system, and a second position of the second target in the camera coordinate system; obtaining a second spatial transformation relationship between the target on the robot arm and the coordinate system of the end of the robot arm according to the first spatial transformation relationship, the first position and the first position, and the second position. The binocular vision calibration method provided by the present invention can avoid the problem of needing to recalibrate when the relative position between the visual system and the robot changes, thereby improving the operation efficiency of the robot.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a binocular vision calibration method, a control device, a robot and a storage medium. Background Art

[0002] At present, the application forms of hand-eye system mainly include two forms: eye in hand and eye outside hand. Eye in hand means that the camera is fixed at the end of the robot. Eye outside hand means that the camera is placed outside the robot body. When the robot moves, it will not move with the robot, and if installed in the right place, it can obtain a global view. The traditional hand-eye system needs hand-eye calibration for three-dimensional spatial positioning. The calibration of the two forms of hand-eye system generally requires the collection of multiple sets of calibration plate photos and the corresponding robot end effector posture. By constructing an equation in the form of AX=XB, the solved X is the hand-eye matrix. The result is greatly affected by the quality of the collected photos and the accuracy of the absolute posture of the robot end effector. The quality of the photo is often related to many factors such as the camera and the environment, and the absolute spatial accuracy of the robot is affected by many factors such as DH calibration accuracy and installation accuracy, so it often leads to the failure to obtain an effective hand-eye matrix in many scenarios. And after the calibration is completed, once the relative position between the camera and the robot changes, it needs to be recalibrated. Summary of the invention

[0003] The first object of the present invention is to provide a binocular vision calibration method to solve the technical problem that it is difficult to adapt to the situation where the relative position between the camera and the robot changes.

[0004] In order to solve the above technical problems, the present invention provides a binocular vision calibration method, comprising:

[0005] The calibration method of binocular vision includes:

[0006] Acquire a first spatial transformation relationship between an end effector and a first target fixed on the end effector, a first position of a tip of a followed second target relative to a coordinate system of the second target, a first pose of the first target in a camera coordinate system, and a second pose of the second target in the camera coordinate system;

[0007] A second spatial transformation relationship between the target on the robot arm and the coordinate system of the end of the robot arm is obtained according to the first spatial transformation relationship, the first position, the first posture, and the second posture.

[0008] The beneficial effects brought by the binocular vision calibration method of the present invention are:

[0009] By adopting the above technical means, the target is added to the end of the robot to realize the pose conversion between the camera coordinate system and the robot end coordinate system. The homogeneous transformation matrix between the target on the end of the robot's mechanical arm and the robot end coordinate system is obtained by calibration. In this way, the pose of the robot end can be expressed in the visual system coordinate system. Since the target is fixed on the end of the robot, as long as the relative pose between the target on the end of the robot and the robot end remains unchanged, there is no need to recalibrate. This avoids the problem of recalibration when the relative pose between the visual system and the robot changes, and improves the operation efficiency of the robot.

[0010] In a preferred technical solution, obtaining the first spatial transformation relationship includes obtaining the first spatial transformation relationship according to the first posture and a third posture matrix of the end effector in the camera coordinate system.

[0011] In a preferred technical solution, obtaining the third pose matrix includes obtaining the third pose matrix according to the second position of the end effector in the camera coordinate system and the first rotation matrix of the end effector in the camera coordinate system.

[0012] In a preferred technical solution, obtaining the second position includes obtaining the second position according to a third position of the end effector relative to the first target and the first posture.

[0013] In a preferred technical solution, the third position is obtained by different rotation matrices of the first target and different positions of the first target in the camera coordinate system when the end effector maintains a fixed point at the center of the circle and the end effector is in different postures.

[0014] In a preferred technical solution, obtaining the first rotation matrix includes obtaining the first rotation matrix according to different positions of the end effector moving along different axes of the tool coordinate system.

[0015] In a preferred technical solution, obtaining the first position includes obtaining different rotation matrices of the second target and the tip of the second target at different positions in the camera coordinate system when the center of the second target remains a fixed point and the second target is in different postures.

[0016] The second object of the present invention is to provide a binocular vision calibration device to solve the technical problem of being difficult to adapt to the situation where the relative position between the camera and the robot changes.

[0017] The binocular vision calibration device provided by the present invention comprises:

[0018] An acquisition module, configured to acquire a first spatial transformation relationship between an end effector and a first target fixed on the end effector, a first position of a tip of a followed second target relative to a coordinate system of the second target, a first pose of the first target in a camera coordinate system, and a second pose of the second target in the camera coordinate system;

[0019] A calibration module is used to obtain a second spatial transformation relationship between a target on the robot arm and a coordinate system of an end of the robot arm according to the first spatial transformation relationship, the first position, the first posture, and the second posture.

[0020] The third object of the present invention is to provide a robot to solve the technical problem that it is difficult to adapt to the situation where the relative position between the camera and the robot changes.

[0021] The robot provided by the present invention includes a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the above-mentioned calibration method is implemented.

[0022] A fourth object of the present invention is to provide a computer-readable storage medium to solve the technical problem that it is difficult to adapt to the situation where the relative position between the camera and the robot changes.

[0023] A computer-readable storage medium stores a computer program, and when the computer program is read and executed by a processor, the above-mentioned calibration method is implemented.

[0024] The binocular vision calibration device, robot and computer-readable storage medium of the present invention can achieve the same technical effect as the above-mentioned binocular vision calibration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments or the background technology description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 A schematic diagram of the physical device composition of a binocular vision calibration method provided by an embodiment of the present invention;

[0027] Figure 2 A schematic flow chart of a binocular vision calibration method provided by one embodiment of the present invention;

[0028] Figure 3A schematic flow chart of a binocular vision calibration method provided by another embodiment of the present invention;

[0029] Figure 4 It is a structural schematic diagram of a binocular vision calibration device in another embodiment of the present invention.

[0030] Description of reference numerals:

[0031] 101- followed target; 102- followed arm; 103- following target; 104- following arm; 105- binocular camera

[0032] 401 - acquisition module; 402 - calibration module. DETAILED DESCRIPTION

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0034] Figure 1 A schematic diagram of the composition of the physical device for applying the binocular vision calibration method provided in an embodiment of the present invention. The physical device is a robot system with eyes outside the hand. Specifically, the NDI binocular camera 105 as a camera is not linked to the robot arm, and the position where the camera is installed can obtain a more global field of view. The end effector of the robot arm is fixedly installed with a target. In this scene, there are two robot arms, one is a followed arm 102 with a followed target 101 fixedly installed, and the other is a following arm 104 with a following target 103 fixedly installed. The following target 103 is the first target O1 described later, and the followed target 101 is the second target O2 described later.

[0035] For each coordinate system, the coordinate system of the first target fixed at the end of the robot is called the O1 coordinate system, the coordinate system of the second target is called the O2 coordinate system, the coordinate system of the robot end effector is called the TCP coordinate system, the camera coordinate system is called the cam coordinate system, and the tip followed by the second target O2 is called the Tip coordinate system.

[0036] Figure 3 A schematic flow chart of a binocular vision calibration method provided by one embodiment of the present invention; Figure 3 As shown, the binocular vision-based calibration method can be applied to the above-mentioned robot device, and the above-mentioned method includes:

[0037] S210, get the third position Different rotation matrices of the second target O2 when the center of the second target O2 remains a fixed point and the second target O2 is in different postures and the tip of the second target O2 is in different positions in the camera coordinate system is obtained.

[0038] Since the target O1 is fixedly connected to the robot end effector, the homogeneous transformation matrix between the target O1 coordinate system and the robot TCP coordinate system is theoretically an invariant matrix. The first position of the target O1 in the camera coordinate system can be directly obtained through the camera. cam H O1 In this way, the center of the robot TCP end circle can be kept at a fixed point, and then the robot TCP can be transformed into different postures, and NDI can be used to read multiple sets of postures of the target O1 on the robot TCP in the camera coordinate system. O1 cam H i (i is the position data serial number, i = 1, 2, 3, 4...). In this way, the center of the robot TCP end, that is, the center of the robot end effector, is kept in the camera coordinate system, that is, the second position cam TCP Multiple groups were obtained without changing P O1 cam H data. Through two sets of data, we can get the following expression:

[0039]

[0040]

[0041] Subtracting the two equations yields:

[0042]

[0043] At this time, we get the form of A*X=B. Since there are multiple sets of data, we can use the least squares method to solve X. The obtained X is the third position

[0044] S220, obtaining the second position cam TCP P: Get the first position cam H O1 , according to the third position of the end effector relative to the first target O1 And the first position cam H O1 Get second position cam TCP P.

[0045] Among them, the first position cam H O1 is the position and posture of the first target O1 in the camera coordinate system; since the third position has been obtained The first posture cam H O1 It can also be read by the camera, so the second position can be calculated cam TCPP, that is:

[0046]

[0047] S230, obtaining a first rotation matrix cam R TCP :According to the different positions of the end effector moving along different axes of the tool coordinate system, the first rotation matrix is ​​obtained cam R TCP .

[0048] While the camera can read the first target O1, select a space point at position P0 in the camera coordinate system, control the robot TCP to move a distance in the positive direction of the tool coordinate system X, and obtain the position P1 of TCP in the camera coordinate system; control the robot TCP to move a distance in the positive direction of the tool coordinate system Z, and obtain the position P2 of TCP in the camera coordinate system. The above data can be used to define the three direction vectors of the TCP coordinate system XYZ in the camera coordinate system, and the x-axis direction vector of the robot TCP coordinate system is:

[0049] X = P1-P0;

[0050] The Z-axis direction vector is:

[0051] Z = P2-P0;

[0052] Then the y-axis direction vector can be obtained as:

[0053] Y = Z × X;

[0054] In order to ensure that the X, Y, and Z axes are perpendicular to each other, redefine the Z direction vector as:

[0055] Z = X × Y;

[0056] Then the first rotation matrix of the robot TCP in the camera coordinate system is cam R TCP for:

[0057] Cam R TCD =[X′Y′Z′]

[0058] S240, obtain the third pose matrix cam H TCP :According to the second position of the end effector in the camera coordinate system cam TCP P and the first rotation matrix of the end effector in the camera coordinate system cam R TCP Get the third pose matrix cam H TCP :

[0059] From this, the third pose matrix of the robot TCP in the camera coordinate system can be constructed cam H TCP :

[0060]

[0061] S250: Obtaining a first spatial transformation relationship Tcp H O1 :According to the first position cam H O1 And the third pose matrix of the end effector in the camera coordinate system cam H TCP Get the first space transformation relationship Tcp H O1 .

[0062] Move the robot TCP to point P0 again, and use the camera to read the first position of the first target O1 on TCP in the camera coordinate system, which is cam H O1 , and the third pose matrix of TCP in the camera coordinate system is cam H TCP , we can obtain:

[0063] Tcp H O1 = cam H TCP . Inverse* cam H O1

[0064] Since the robot TCP and its follower target are fixedly connected, theoretically Tcp H O1 is a fixed matrix.

[0065] S260, get the first position The different rotation matrices of the second target O2 when the center of the second target O2 is kept as a fixed point and the second target O2 is in different postures and the tip of the second target O2 is in different positions in the camera coordinate system are obtained;

[0066] This step is applied to obtain the second position cam TCP The same method as P. That is, by keeping the end center Tip of the mobile second target O2 at a fixed point, and then changing the posture of the mobile second target O2, using NDI to read multiple sets of postures of the mobile target in different postures, the position coordinates of the end center Tip of the mobile second target O2 in the target 02 coordinate system can be calculated. Since the position of the end center Tip of the mobile second target O2 remains unchanged, the multiple sets of postures obtained are shown in the following formula:

[0067]

[0068]

[0069] Subtracting the two equations yields:

[0070]

[0071] At this time, we get the form of C*Y=D. Since there are multiple sets of data, we can solve Y by the least squares method. The obtained Y is

[0072] S270: Obtaining a second spatial transformation relationship TCP P Tip :Get the second target O2 in the camera coordinate system cam H O2 , according to the first spatial transformation relationship Tcp H O1 , first position and cam H O1 , the second position obtains the second spatial transformation relationship between the target on the manipulator and the coordinate system of the end of the manipulator TCP P Tip .

[0073] The coordinates of the tip of the second target O2 in the robot TCP coordinate system can be obtained by calculation: TCP P Tip :

[0074] TCP P Tip =( cam H O1 * O1 H TCP ).inverse* cam H O2 * O2 P Tip (1)

[0075] This system uses the tip of the moving target as the target point and the actual TCP position as the actual value. The two can be expressed in the camera coordinate system as:

[0076] cam P Tip = cam H O2 * O2 P Tip ; (2)

[0077] cam P TCP = cam H O1* O1 H TCP ; (3)

[0078] The binocular vision calibration method provided in this embodiment realizes the posture conversion between the camera coordinate system and the robot end coordinate system by adding a target to the end of the robot. The homogeneous transformation matrix between the target on the end of the robot's mechanical arm and the robot end coordinate system is obtained by calibration. In this way, the posture of the robot end can be expressed in the visual system coordinate system. Since the target is fixed on the end of the robot, as long as the relative posture between the target on the end of the robot and the end of the robot remains unchanged, there is no need to recalibrate. In this way, the problem of needing to recalibrate when the relative posture between the visual system and the robot changes in the past is avoided, thereby improving the operation efficiency of the robot.

[0079] Figure 3 FIG. 5 is a schematic flow chart of another binocular vision calibration method in one embodiment of the present invention, the method comprising:

[0080] S301, keep the center of the robot TCP end at a fixed point, then change the robot TCP to different postures, and use NDI to read multiple sets of postures of the first target O1 on the robot TCP in the camera coordinate system O1 cam H i , then execute S302.

[0081] S302, obtain multiple groups O1 cam H data, solve the third position by least square method Then execute S303.

[0082] S303, get the first position cam H O1 , then execute S304.

[0083] S304, according to cam H O1 and Calculate the second position cam TCP P, then execute S305.

[0084] S305: Obtain a first rotation matrix according to different positions of the end effector moving along different axes of the tool coordinate system cam R TCP , then execute S306.

[0085] S306, according to cam TCP P and cam R TCP Get the third pose matrixcamH TCP , then execute S307.

[0086] S307, according to cam H O1 and camH TCP Get the first space transformation relationship Tcp H O1 , then execute S308.

[0087] S308, obtaining the first position by different rotation matrices of the second target O2 when the center of the second target O2 is kept as a fixed point and the second target O2 is in different postures and the tip of the second target O2 is in different positions of the camera coordinate system Then execute S309.

[0088] S309: Obtain the second position and posture of the second target O2 in the camera coordinate system cam H O2 , then execute S310.

[0089] S310, according to Tcp H O1 , and cam H O1 , cam H O2 Get the second space transformation relationship TCP P Tip

[0090] Figure 4 In one embodiment of the present invention, a binocular vision calibration device is provided. The binocular vision calibration device comprises:

[0091] An acquisition module is used to acquire a first spatial transformation relationship between the end effector and a first target O1 fixed on the end effector. Tcp H O1 , the first position of the tip of the followed second target O2 relative to the coordinate system of the second target O2 The first target O1 is in the camera coordinate system cam H O1 , and the second position of the second target O2 in the camera coordinate system cam H O2 ;

[0092] A calibration module is used to transform the first space Tcp H O1 , first position And the first position cam H O1 , Second position cam H O2Get the second spatial transformation relationship between the target on the robot arm and the coordinate system of the end of the robot arm TCP P Tip .

[0093] The binocular vision calibration device provided in this embodiment realizes the posture conversion between the camera coordinate system and the robot end coordinate system by adding a target to the end of the robot. The homogeneous transformation matrix between the target on the end of the robot's mechanical arm and the robot end coordinate system is obtained by calibration. In this way, the posture of the robot end can be expressed in the visual system coordinate system. Since the target is fixed on the end of the robot, as long as the relative posture between the target on the end of the robot and the end of the robot remains unchanged, there is no need to recalibrate. In this way, the problem of needing to recalibrate when the relative posture between the visual system and the robot changes in the past is avoided, thereby improving the operation efficiency of the robot.

[0094] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the first space transformation relationship Tcp H O1 , according to the first position cam H O1 And the third pose matrix of the end effector in the camera coordinate system camH TCP .

[0095] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the third posture matrix camH TCP , according to the second position of the end effector in the camera coordinate system cam TCP P and the first rotation matrix of the end effector in the camera coordinate system cam R TCP .

[0096] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the second position cam TCP P, according to the third position of the end effector relative to the first target O1 And the first position cam H O1 .

[0097] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the third position It is obtained by different rotation matrices of the first target O1 and different positions of the first target O1 in the camera coordinate system when the end effector maintains a fixed point at the center of the circle and the end effector is in different postures.

[0098] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the first rotation matrix cam R TCP, the first rotation matrix is ​​obtained according to the different positions of the end effector moving along different axes of the tool coordinate system cam R TCP .

[0099] Optionally, as an embodiment, the acquisition module is specifically used to: obtain the first position Different rotation matrices of the second target O2 when the center of the second target O2 remains a fixed point and the second target O2 is in different postures and the tip of the second target O2 is in different positions in the camera coordinate system is obtained.

[0100] A robot further provided in an embodiment of the present invention includes a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the above-mentioned calibration method is implemented.

[0101] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is read and executed by a processor, the calibration method provided in the above embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0102] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the control device through a computer, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above-mentioned method embodiments, wherein the storage medium may be a memory, a disk, an optical disk, etc.

[0103] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

[0104] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0105] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the control device and air conditioner disclosed in the embodiment, since it corresponds to the calibration method of the air conditioner disclosed in the above embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0106] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A binocular vision calibration method, characterized in that: include: Acquire a first spatial transformation relationship between an end effector and a first target fixed on the end effector, a first position of a tip of a followed second target relative to a coordinate system of the second target, a first pose of the first target in a camera coordinate system, and a second pose of the second target in the camera coordinate system; A second spatial transformation relationship between the second target on the followed arm and the coordinate system of the end of the followed arm is obtained according to the first spatial transformation relationship, the first position, the first posture, and the second posture.

2. The binocular vision calibration method according to claim 1, characterized in that: Obtaining the first spatial transformation relationship includes obtaining the first spatial transformation relationship according to the first posture and a third posture matrix of the end effector in the camera coordinate system.

3. The binocular vision calibration method according to claim 2, characterized in that: Obtaining the third pose matrix includes obtaining the third pose matrix according to the second position of the end effector in the camera coordinate system and the first rotation matrix of the end effector in the camera coordinate system.

4. The binocular vision calibration method according to claim 3, characterized in that: Obtaining the second position includes obtaining the second position according to a third position of the end effector relative to the first target and the first posture.

5. The binocular vision calibration method according to claim 4, characterized in that: The third position is obtained by different rotation matrices of the first target and different positions of the first target in the camera coordinate system when the end effector maintains a fixed point at the center of the circle and the end effector is in different postures.

6. The binocular vision calibration method according to claim 3, characterized in that: Obtaining the first rotation matrix includes obtaining the first rotation matrix according to different positions of the end effector moving along different axes of the tool coordinate system.

7. The binocular vision calibration method according to claim 1, characterized in that: Obtaining the first position includes obtaining different rotation matrices of the second target and the tip of the second target at different positions in the camera coordinate system when the center of the second target remains a fixed point and the second target is in different postures.

8. A binocular vision calibration device, characterized in that: include: An acquisition module, configured to acquire a first spatial transformation relationship between an end effector and a first target fixed on the end effector, a first position of a tip of a followed second target relative to a coordinate system of the second target, a first pose of the first target in a camera coordinate system, and a second pose of the second target in the camera coordinate system; A calibration module is used to obtain a second spatial transformation relationship between the second target on the followed arm and the coordinate system of the end of the followed arm according to the first spatial transformation relationship, the first position, the first posture, and the second posture.

9. A robot, characterized in that: It comprises a computer-readable storage medium storing a computer program and a processor, and when the computer program is read and executed by the processor, the calibration method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is read and executed by a processor, the calibration method according to any one of claims 1 to 7 is implemented.

Images