Robot calibration device
Through the robot's own robot arm and depth camera, the coordinate system transformation relationship between each component is automatically determined, which solves the cumbersome calibration problem of external equipment dependence in the existing technology, and realizes efficient and convenient autonomous calibration of robots.
Patent Information
- Application Number
- CN202322992221.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-06
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing robot calibration methods rely on external devices, and the process is cumbersome and inconvenient, making it difficult to achieve efficient unification of the coordinate system of robot components.
A robot calibration device is designed to use the robot's own robot arm and depth camera to automatically determine the conversion relationship between various components by obtaining the robot arm parameters and image data, including the base and end of the robot arm, the depth camera and the chassis. The coordinate system transformation matrix realizes independent calibration.
The robot calibration can be completed without external equipment, simplifying the calibration process, improving efficiency and convenience, reducing sensor measurement noise and error, and enhancing calibration accuracy.
Smart Images

Figure CN222096128U_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of robotics technology, and in particular, to a robot calibration device. Background Art
[0002] Robots, such as inspection robots, generally consist of an autonomous mobile chassis, a robotic arm, and a visual perception module (such as a camera). Each module (or component) works in its corresponding coordinate system. In order for the modules to work together to complete the specified functions and tasks, unifying (calibrating) the coordinate systems of each module is a necessary prerequisite for the normal operation of the robot system.
[0003] The existing calibration method usually adopts an external calibration method, such as using other equipment to measure the robot and then calibrate the robot. However, this method requires the use of external equipment for processing, and the calibration process is very inconvenient. Utility Model Content
[0004] Each embodiment of the present application provides a robot calibration device, which can calibrate the robot more conveniently.
[0005] The technical solution is as follows:
[0006] In a first aspect, the present application provides a robot calibration device, the device is used to calibrate the robot, the robot includes the following components: a first robot arm base, a first robot arm end, a second robot arm base, the second robot arm end and a depth camera arranged on the first robot arm end, the calibration of the robot includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot, the device includes: a parameter data acquisition module, used to acquire robot arm parameters, so as to determine a first conversion relationship between the first robot arm base and the first robot arm end, a second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters. conversion relationship; a first data acquisition module, used to acquire at least three sets of first calibration data, so as to determine the third conversion relationship between the first robotic arm base and the second robotic arm base according to the first calibration data, wherein the first calibration data include the posture data acquired when the first robotic arm end and the second robotic arm end are in a zero-force state and the ends overlap; a second data acquisition module, used to acquire at least three sets of second calibration data, so as to determine the fourth conversion relationship between the first robotic arm end and the depth camera according to the second calibration data, wherein the second calibration data include the first image data of the second robotic arm end taken by the depth camera at the end of the first robotic arm when the end is stationary.
[0007] In a second aspect, the present application provides a network device, comprising: a memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the method described in the first aspect.
[0008] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.
[0009] The beneficial effects of the technical solution provided by this application are:
[0010] The scheme of the present application can be applied in the scene of calibrating a robot, and the robot includes the following components: a base, a first mechanical arm base, a first mechanical arm end, a second mechanical arm base, a second mechanical arm end, and a depth camera arranged on the first mechanical arm end. The first mechanical arm base and the second mechanical arm base are arranged on a movable base, and the coordinate systems of each component are different. The calibration of the robot by this scheme includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot. This scheme can complete the calibration process of the robot by the robot itself, and can calibrate the mechanical arm base and the mechanical arm end according to the mechanical arm parameters, calibrate the two mechanical arm ends according to the data when the two mechanical arm ends overlap, and calibrate the depth camera according to the depth camera on one mechanical arm end to identify the preset point position of the other mechanical arm end, and calibrate the depth camera, and can also identify the fixed point on the chassis according to the depth camera, and then calibrate the chassis, so as to determine the conversion relationship between any two components among the chassis, the first mechanical arm base, the first mechanical arm end, the second mechanical arm base, the second mechanical arm end, and the depth camera, and complete the calibration of the robot. This solution does not require the participation of external equipment, and the robot itself can be directly used to complete the robot calibration, making the calibration process simpler and more convenient.
[0011] Specifically, this solution can obtain the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end, and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; after the robot arm base and the robot arm end are calibrated, the first robot arm end and the second robot arm end can be controlled in a zero-force state, and the first robot arm end and the second robot arm end can be manually adjusted (or otherwise adjusted) so that the first robot arm end and the second robot arm end coincide with each other, and the position data of the two robots are obtained as the first calibration data. Three groups of the first calibration data can be obtained, and then the third conversion relationship between the first robot arm base and the second robot arm base is analyzed. After calibrating the first robotic arm end and the second robotic arm end, the depth camera at the first robotic arm end can be controlled to remain still, and the first image data of a preset point at the second robotic arm end can be captured as the second calibration data. The second robotic arm end can be moved to multiple positions to obtain at least three sets of calibration data, and then the fourth conversion relationship between the first robotic arm end and the depth camera is determined to complete the calibration of the first robotic arm base, the first robotic arm end, the second robotic arm base, the second robotic arm end and the depth camera. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.
[0013] Figure 1 is a schematic structural diagram of a robot according to an embodiment of the present application;
[0014] Figure 2 is a schematic diagram of the steps of a robot calibration method according to an embodiment of the present application;
[0015] Figure 3 It is a flowchart of a singular value decomposition algorithm of an embodiment of the present application;
[0016] Figure 4 is a flowchart of a robot calibration method according to an embodiment of the present application;
[0017] Figure 5 is a schematic structural diagram of a robot calibration device according to an embodiment of the present application;
[0018] Figure 6 It is a structural block diagram of a network device according to an embodiment of the present application;
[0019] Figure 7 This is a structural block diagram of a user equipment according to an embodiment of the present application; DETAILED DESCRIPTION
[0020] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0021] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms, and "multiple" refers to two or more, and other quantifiers are similar. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The term "and / or" used here describes the association relationship of associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0022] The solution of the present application can be applied in the scenario of calibrating a robot. The calibration process of the robot can be completed by the robot itself. The robot includes: a chassis, a first robotic arm (or robotic arm 1) base, a first robotic arm end, a second robotic arm (or robotic arm 2) base, a second robotic arm end, and a depth camera arranged on the first robotic arm end; the calibration of the robot includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot. For the robot, the coordinate system relationship between each component (or module) is as follows: Figure 1 As shown in the figure, the calibration of the inspection robot is to determine the coordinate system O of the center of the mobile chassis. w -X w Y w Z w 、Coordinate system O of robot arm 1 base ab1 -X ab1 Y ab1 Z ab1 、Coordinate system O of robot arm 2 base ab2 -X ab2 Y ab2 Z ab2 、Coordinate system O of robot arm 1 end ae1 -X ae1 Y ae1 Z ae1、Coordinate system O of robot arm 2 end ae2 -X ae2 Y ae2 Z ae2 and the camera coordinate system O c -X c Y c Z c The transformation relationship between the six coordinate systems is to solve the coordinate rotation matrix R and translation matrix T between each coordinate system.
[0023] This solution can calibrate the robot base and the end of the robot according to the robot parameters, calibrate the two ends of the robot according to the data when the two ends of the robot overlap, and calibrate the depth camera according to the depth camera on one end of the robot to identify the preset points of the other end of the robot. It can also identify the fixed points on the chassis according to the depth camera, and then calibrate the chassis, so as to determine the conversion relationship between any two components among the chassis, the first robot base, the first robot end, the second robot base, the second robot end, and the depth camera, and complete the calibration of the robot. This solution does not require the participation of external equipment, and can directly use the robot itself to complete the calibration of the robot, and the calibration process is simpler and more convenient.
[0024] Specifically, this solution can obtain the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end, and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; after the robot arm base and the robot arm end are calibrated, the first robot arm end and the second robot arm end can be controlled in a zero-force state, and the first robot arm end and the second robot arm end can be manually adjusted so that the first robot arm end and the second robot arm end coincide with each other, and the posture data is obtained as the first calibration data. Three groups of the first calibration data can be obtained, and then the third conversion relationship between the first robot arm base and the second robot arm base is analyzed. After calibrating the first and second mechanical arm ends, the depth camera at the first mechanical arm end can be controlled to remain stationary, and the first image data of the preset point at the second mechanical arm end can be captured as the second calibration data. The second mechanical arm end can be moved to multiple positions to obtain at least three sets of calibration data, and then the fourth conversion relationship between the first mechanical arm end and the depth camera is determined to complete the calibration of the first mechanical arm base, the first mechanical arm end, the second mechanical arm base, the second mechanical arm end and the depth camera. Afterwards, multiple fixed points on the chassis can be identified based on the depth camera, and then the chassis can be calibrated.
[0025] Specifically, in an optional example, the robot of this solution can be an inspection robot, or an intelligent inspection robot. In the dual-arm calibration system, according to the arm parameters (such as DH parameters) of the used arm, the rotation matrix and translation matrix between the base coordinate system and the end coordinate system of the arm can be obtained, and then the rotation matrix of the end coordinate system of the arm can be determined to transform the end coordinate system of the arm to the base coordinate system of the arm. ab1 R ae1 , ab2 R ae2 and translation matrix ab1 T ae1 , ab2 T ae2 ,Right now ab1 R ae1 , ab2 R ae2 , ab1 T ae1 , ab2 T ae2 It is known that we only need to determine the rotation matrix that transforms the base coordinates of robot 1 to the base coordinates of robot 2 ab2 R ab1 and translation matrix ab2 T ab1 , the calibration of the dual robotic arms can be completed. During the calibration process, the two robotic arms are placed in the zero-force teaching state, and the two robotic arms are dragged so that the ends of the two robotic arms are absolutely overlapped. The posture data of the ends of the two robotic arms when they overlap are read separately to obtain a set of corresponding point coordinates X in the base coordinate system of robotic arms 1 and 2. i ab1 , X i ab2 Repeat at least three times to obtain three or more corresponding point coordinates, through the coordinate transformation relationship: X i ab2 = ab2 R ab1 X i ab1 + ab2 T ab1 Construct a system of equations and solve the rotation matrix using the SVD method ab2 R ab1 and translation matrix ab2 T ab1 Among them, X i ab2 represents the coordinate value of the i-th corresponding point in the base coordinate system of robot 2, X i ab1 It represents the coordinate value of the i-th corresponding point in the base coordinate system of robot 1. Due to the error of sensor measurement during the calibration process, there is a certain error in the rotation matrix and translation matrix solved by only three corresponding points. By increasing the number of corresponding points, the estimation error of the coordinate transformation matrix can be reduced.
[0026] In the external parameter calibration system of the depth camera, the dual-arm calibration has been completed, that is, the rotation matrix of the robot end coordinates converted to the robot base coordinates ab1 R ae1 , ab2 R ae2 and translation matrix ab1 T ae1 , ab2 T ae2 It is known that the rotation matrix of the base coordinates of robot arm 1 is converted to the base coordinates of robot arm 2 ab2 R ab1 and translation matrix ab2 T ab1 The external parameter calibration of the depth camera is to determine the coordinate system O of the end of the robot 1 ae1 -X ae1 Y ae1 Z ae1 and the coordinate system O of the depth camera c -X c Y c Z c The transformation matrix ae1 R c , ae1 T c During the calibration process, the robot arm 1 equipped with the depth camera remains stationary, and the robot arm 2 is moved so that its end is within the optimal depth measurement range of the depth camera. The RGB image and depth point cloud image collected by the depth camera are used to identify and collect the center of the cross section of the end of the robot arm relative to the depth camera coordinate system O. c -X c Y c Z c The coordinate value X i c At the same time, the end of the robot 2 is collected in the robot 2 base coordinate system O ab2 -X ab2 Y ab2 Z ab2 The coordinate value X i ab2 , collect the end of robot 1 in the robot 1 base coordinate system O ab1 -X ab1 Y ab1 Z ab1 The coordinate value X i ab1 , determine the transformation matrix from the base coordinate system of robot 1 to the end coordinate system. i ab2 By X i ab1 = ab1 R ab2 Xi ab2 + ab1 T ab2 Project it to the base coordinate system of robot arm 1, and then use X i ae1 = ae1 R ab1 X i ab1 + ae1 T ab1 X i ab1 Project it to the coordinate system of the end of the robot 1. At this point, a set of corresponding point coordinates of the depth camera coordinate system and the coordinate system of the end of the robot 1 is obtained. Repeat the above operation to obtain three or more corresponding point coordinates, and use X i ae1 = ae1 R c X i c+ae1 T c Construct a linear equation system, and then use the SVD method to solve the transformation matrix of the depth camera coordinate system and the end coordinate system of the robot 1. ae1 R c and ae1 T c . Complete the external parameter calibration of the depth camera.
[0027] In the overall calibration system, the external parameters of the dual manipulators and the depth camera have been calibrated. Next, the coordinate transformation matrix of the robot chassis base coordinate system and the coordinate systems of manipulators 1, 2, and the depth camera is determined. Since the coordinate transformation relationship between the depth camera, manipulator 1, and manipulator 2 has been determined, the robot chassis coordinate system only needs to determine the transformation relationship with the coordinate system of one of the three modules. This solution can give priority to determining the depth camera coordinate system O c -X c Y c Z c and the robot mobile chassis base coordinate system O w -X w Y w Z w During the calibration process, first start the mapping function of the robot's mobile chassis to complete the mapping of the robot's current environment. Mark three or more position points near the origin of the robot's mobile chassis mapping, and place the QR code recording the position of the robot's mobile chassis at the corresponding marked point position. After the mapping is completed, according to the established map (or environment map), move the robot to the origin of the map. Move the robot arm 1 with the depth camera so that the QR code is within the optimal depth field of view of the depth camera. Through the RGB image and depth map collected by the depth camera, identify and read the robot's mobile chassis position coordinate X in the QR code.i w and the three-dimensional position coordinates X of the QR code relative to the camera i c At this point, a set of coordinate corresponding points of the mobile chassis base coordinate system and the depth camera coordinate system is obtained. Recognize and read the location information and location coordinates of other QR codes again to obtain three or more corresponding point coordinates. i w = w R c X i c + w T c Construct a linear equation system, and then use the SVD method to solve the transformation matrix of the robot's mobile chassis base coordinate system and the depth camera coordinate system. ae1 R c and ae1 T c .
[0028] In a more specific example, Figure 2 As shown in the figure, for the calibration of the intelligent inspection robot system, the specific technical implementation scheme is divided into three parts, namely, the calibration of the dual robotic arm system, the hand-eye calibration of the robotic arm and the depth camera, and the joint calibration of the robot mobile chassis with the dual robotic arms and the depth camera.
[0029] For the calibration of the dual robotic arms in the inspection robot system, the main problem is to solve the coordinate system O ab2 -X ab2 Y ab2 Z ab2 To coordinate system O ab1 -X ab1 Y ab1 Z ab1 The transformation matrix ab1 R ab2 and ab1 T ab2 The end points of robot arms 1 and 2 are manually aligned by zero-force teaching, and the end coordinates (x a ,y a , z a ), the coordinates of robot arm 1 are stored in matrix A a In the example, the coordinates of robot 2 are stored in matrix B. a After repeated many times, we get A a n×3 and B a n×3 Two matrices, n≥3. The transformation matrix can be solved by the singular value decomposition (SVD) algorithm ab1 Rab2 and ab1 T ab2 , the algorithm flow chart is as follows Figure 3 shown.
[0030] For the calibration of the robot arm 1 and the depth camera, the main problem is to solve the camera coordinate system O c -X c Y c Z c To the end coordinate system O of robot arm 1 ae1 -X ae1 Y ab1 Z ae1 The transformation matrix ae1 R c , ae1 T c The key to solving the coordinate transformation matrix using the PnP method is to determine at least 3 corresponding points in the coordinate system O c -X c Y c Z c and O ae1 -X ae1 Y ab1 Z ae1 This solution takes the coordinate origin of the end of the robot arm 2 without a camera as O c -X c Y c Z c Coordinate system and O ae1 -X ae1 Y ab1 Z ae1 The following will introduce in detail how to obtain the coordinate values of corresponding points in the corresponding coordinate system based on two coordinate systems.
[0031] First, solve the coordinate origin of the end of robot 2 in the coordinate system O of the end of robot 1 ae1 -X ae1 Y ab1 Z ae1 The coordinate value under. Let the homogeneous transformation matrix from the base coordinate system of robot 1 to the end coordinate system of robot 1 be The homogeneous transformation matrix from the base coordinate system of robot 2 to the base coordinate system of robot 1 is The homogeneous transformation matrix from the end coordinate system of robot 2 to the base coordinate system of robot 2 is: The homogeneous transformation matrix from the end coordinate system of robot 2 to the end coordinate system of robot 1 can be obtained: for:
[0032]
[0033] in,
[0034]
[0035]
[0036]
[0037] ab1 R ae1 , ab1 T ae1 Obtained from the DH of robot arm 1. ab1 R ab2 , ab1 T ab2 From the first step of calibration, ab2 R ae2 , ab2 R ae2 Obtained from the DH parameters of robot 2.
[0038] According to the homogeneous transformation matrix have:
[0039]
[0040] in, ae1 T ae2 That is, the coordinate value of the origin of the coordinate system of the end of robot 2 in the coordinate system of the origin of the end of robot 1.
[0041] Secondly, solve the coordinate value of the origin of the coordinates of the end of the robot 2 in the camera coordinate system. The origin of the coordinate system of the end of the robot arm is usually the center of the circular surface of the end. Based on the convolutional neural network, a deep learning network model is constructed to collect pictures of the end of the robot arm under different lighting and different backgrounds. Use a rectangular frame to mark the circle of the end of the robot arm in the picture. In the picture, the circle of the end of the robot arm generally appears as a circle and an ellipse. The center of the marked rectangular frame is the intersection of the center of the circle and the major and minor axes of the ellipse. The collected pictures are trained with the constructed deep learning network model to obtain the weights of the convolutional neural network. Based on the deep learning model and the trained weight parameters, the end of the robot arm is detected online, and the pixel point (u x ,u y ), based on the camera's pinhole imaging model and the camera's internal parameters, we have:
[0042]
[0043]
[0044]
[0045] in, It is the depth value at the pixel point (ux, uy) where the origin of the coordinate system of the end of the robot 2 is located under the i-th observation at different positions, that is, the z coordinate value of the origin of the coordinate system of the end of the robot 2 in the camera coordinate system, which can be directly obtained through the depth camera. and u0, v0, d are the x and y coordinates of the origin of the coordinate system of the end of the robot 2 in the camera coordinate system under the i-th observation at different positions. x d y , f are known camera intrinsic parameters.
[0046] At this point, the coordinates of the specified corresponding points in the camera coordinate system and the end coordinate system of the robot 1 can be determined. Through multiple observations at different positions, the matrix A consisting of the coordinates of the corresponding points in the camera coordinate system is obtained. c n×3 and the matrix B formed by the coordinates of the corresponding points in the robot end coordinate system ae n×3 , the camera coordinate system O can be solved by the SVD algorithm c -X c Y c Z c To the end coordinate system O of robot arm 1 ae1 -X ae1 Y ab1 Z ae1 The transformation matrix ae1 R c , ae1 T c , complete the camera's external parameter calibration.
[0047] Based on the solution ae1 R c , ae1 T c , the calibration error is defined by reprojection. In a new observation, the coordinates of the origin of the end coordinate system of robot 2 in the camera coordinate system and the end coordinate system of robot 1 are and Then we have:
[0048]
[0049] By setting the calibration error threshold err threshold , under certain conditions, automatic online calibration of this part can be achieved.
[0050] Considering that the extrinsic calibration of the depth camera is completed on the basis of the dual robotic arm calibration, the transformation matrix between the dual robotic arms is estimated by linear least squares. In the process of extrinsic calibration of the depth camera, the positioning error of the robotic arm and the error in the estimation of the transformation matrix are introduced, which will increase the error of the extrinsic estimation of the depth camera. Therefore, the transformation matrix is determined by a linear method.ae1 R c , ae1 T c After that, ae1 R c , ae1 T c As the initial value, formula (1.4) is used as the objective function, and the corresponding point coordinates are obtained. The transformation matrix is again optimized nonlinearly through the Gauss-Newton method to reduce the positioning error of the robot arm, the estimation error of the transformation matrix between the two robot arms, and the influence of the camera depth observation noise on the camera extrinsic parameter estimation, thereby improving the accuracy of the camera extrinsic parameter estimation.
[0051] For the calibration of the mobile chassis, robotic arm, and depth camera, it is only necessary to complete the calibration of the mobile chassis and the depth camera to determine the coordinate transformation relationship of the mobile chassis relative to the robotic arms 1 and 2. In this part of the calibration, a QR code label is introduced as the corresponding point between the mobile chassis base coordinate system and the camera coordinate system. The QR code label contains the coordinate value of the center of the QR code label relative to the base coordinate system of the mobile robot chassis. The solution of the coordinates of the center of the QR code relative to the camera coordinate system is the same as the solution of the origin of the coordinate system of the end of the robotic arm 1 relative to the camera coordinate system. First, pictures of the QR code are collected at different angles, different lighting, and different backgrounds, and the QR codes in the pictures are marked. The deep learning model is used for training, and the convolutional neural network weights of the QR code are trained. Then, during the calibration process, the content of the QR code is read and the center of the QR code is detected in real time. The coordinates of the center of the QR code in the camera coordinate system are solved by formula (1.3). Since the relative positions of the bases of robot arms 1 and 2 and the center of the mobile chassis are unchanged, the solved coordinate values are projected into the coordinate system of the base of robot arm 1. The corresponding point coordinates are obtained
[0052]
[0053] After obtaining the coordinates of three or more corresponding points, the transformation matrix of the mobile robot chassis coordinate system and the robot arm 1 base coordinate system can be solved by the SVD algorithm. w R ab1 , w T ab1 .
[0054] Once again ae1 R c , ae1 T c As the initial value, take formula (1.4) as the objective function, use the obtained corresponding point coordinates, and use the Gauss-Newton method to transform the matrix again w R ab1 , w T ab1Nonlinear optimization is performed to reduce the influence of the positioning error of the manipulator, the estimation error of the transformation matrix between the two manipulators, the camera depth observation noise, and the positioning error of the mobile robot on the transformation matrix of the mobile robot chassis coordinate system and the manipulator 1 base coordinate system, so as to improve the accuracy of the camera extrinsic parameter estimation. In this scheme, the transformation matrix is optimized by minimizing the reprojection error. In addition to the Gauss-Newton method, the gradient descent method, the Levenberg-Marquardt method, etc. can also be used.
[0055] Then the homogeneous form of the transformation matrix from the base coordinate system of the robot 1 to the mobile chassis coordinate system is: The homogeneous transformation matrix from the end coordinate system of the robot 1 to the base coordinate system of the mobile chassis can be obtained: for:
[0056]
[0057] The homogeneous transformation matrix from the camera coordinate system to the mobile chassis base coordinate system can be obtained for:
[0058]
[0059] The homogeneous transformation matrix from the end coordinate system of the robot 2 to the coordinate system of the mobile chassis can be obtained: for
[0060]
[0061] In this scheme, in addition to the calibration of the dual-arm system, the transformation matrix estimated by the linear method is used as the initial value, and the optimal transformation matrix under certain measurement noise and calibration error is estimated by minimizing the reprojection error. This scheme does not require a calibration target, and can realize the external parameter calibration of the dual-arm system and the depth camera without the help of an external system and calibration plate. The calibration process is simple and does not require complicated manual operations. The scheme of this scheme can be applied to the calibration of multivariate robot systems, providing a quick and easy calibration scheme for complex and multivariate inspection robots. In addition, this scheme has a small number of observations, multiple calibration systems are combined, and the linear and nonlinear estimation of the rotation matrix improves the efficiency of nonlinear optimization and reduces the influence of sensor measurement noise and process estimation errors on the accuracy of the final transformation matrix estimation.
[0062] Specifically, an embodiment of the present application provides a robot calibration method, the method is used to calibrate a robot, the robot includes the following components: a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera disposed on the first robotic arm end, the calibration of the robot includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot, and the conversion relationship between the coordinate systems corresponding to the components of the robot includes a translation matrix and a rotation matrix. Figure 4 As shown, the method includes:
[0063] Step 402: Acquire robot arm parameters to determine a first conversion relationship between a first robot arm base and a first robot arm end, and a second conversion relationship between a second robot arm base and a second robot arm end according to the robot arm parameters.
[0064] Step 404, obtain at least three sets of first calibration data to determine a third conversion relationship between the first robotic arm base and the second robotic arm base based on the first calibration data, wherein the first calibration data includes posture data obtained when the end of the first robotic arm and the end of the second robotic arm are in a zero-force state and the ends overlap.
[0065] Step 406, obtaining at least three sets of second calibration data to determine a fourth conversion relationship between the first robotic arm end and the depth camera based on the second calibration data, wherein the second calibration data includes first image data of the second robotic arm end taken by the depth camera at the first robotic arm end when the depth camera is stationary.
[0066] The solution of the present application can be applied in the scenario of calibrating a robot. The robot includes the following components: a base, a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera arranged on the first robotic arm end. The first robotic arm base and the second robotic arm base are arranged on a movable base. The coordinate systems of each component are different. The calibration of the robot by this solution includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot.
[0067] This solution can complete the robot calibration process through the robot itself. The robot base and the end of the robot can be calibrated according to the robot parameters. The two ends of the robot can be calibrated according to the data when the two ends of the robot overlap. The depth camera on one end of the robot can recognize the preset points of the other end of the robot to calibrate the depth camera. The fixed points on the chassis can also be recognized by the depth camera, and then the chassis can be calibrated, so as to determine the conversion relationship between any two components among the chassis, the first robot base, the first robot end, the second robot base, the second robot end, and the depth camera, and complete the calibration of the robot. This solution does not require the participation of external equipment, and can directly use the robot itself to complete the calibration of the robot, and the calibration process is simpler and more convenient.
[0068] Specifically, this solution can obtain the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end, and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; after the robot arm base and the robot arm end are calibrated, the first robot arm end and the second robot arm end can be controlled in a zero-force state, and the first robot arm end and the second robot arm end can be manually adjusted so that the first robot arm end and the second robot arm end coincide with each other, and the posture data (parameters of the robot arm base and the robot arm end) are obtained as the first calibration data. Three groups of the first calibration data can be obtained, and then the third conversion relationship between the first robot arm base and the second robot arm base is analyzed. After calibrating the first robotic arm end and the second robotic arm end, the depth camera at the first robotic arm end can be controlled to remain still, and the first image data of a preset point at the second robotic arm end can be captured as the second calibration data. The second robotic arm end can be moved to multiple positions to obtain at least three sets of calibration data, and then the fourth conversion relationship between the first robotic arm end and the depth camera is determined to complete the calibration of the first robotic arm base, the first robotic arm end, the second robotic arm base, the second robotic arm end and the depth camera.
[0069] In addition to calibrating the robot arm base, the end of the robot arm and the depth camera, this solution can also calibrate the movable chassis. The fixed points on the chassis can be identified by the depth camera, so as to calibrate the conversion relationship between the chassis and the depth camera. Specifically, as an optional embodiment, the robot also includes a movable chassis, and the method also includes: starting the mobile mapping function of the robot chassis to establish an environmental map of the current environment from the mapping origin through the depth camera; moving the end of the first robot arm so that the depth camera obtains the second image data of the fixed point near the mapping origin, and the fixed point is set to at least three; according to at least three groups of three-dimensional position coordinates and second image data corresponding to at least three fixed points, determine the fifth conversion relationship between the chassis and the depth camera. A two-dimensional code can be set at the fixed point of the chassis, and the fixed point can be the center of the two-dimensional code. The two-dimensional code can carry the three-dimensional position coordinates of the fixed point relative to the chassis, so as to identify the three-dimensional position coordinates by identifying the two-dimensional code, and determine the predicted position coordinates of the fixed point relative to the depth camera by depth recognition, and then use the calibrated conversion relationship to determine the first conversion relationship. Specifically, as an optional embodiment, the fifth conversion relationship between the chassis and the depth camera is determined based on at least three groups of three-dimensional position coordinates corresponding to at least three fixed points and the second image data, including: the fixed point is provided with a two-dimensional code, and the two-dimensional code carries the three-dimensional position coordinates of the fixed point; by identifying the two-dimensional code, the three-dimensional position coordinates of the fixed point relative to the chassis are obtained; the second image data is identified to determine the predicted position coordinates of the fixed point relative to the depth camera; the fifth conversion relationship between the chassis and the depth camera is determined based on at least three groups of three-dimensional position coordinates and the predicted position coordinates corresponding to at least three fixed points.
[0070] The three-dimensional position coordinates and the predicted position coordinates can be transformed by using the first conversion relationship, the second conversion relationship, the third conversion relationship and the fourth conversion relationship, and then solved to determine the fifth conversion relationship. Specifically, as an optional embodiment, the fifth conversion relationship between the chassis and the depth camera is determined based on at least three groups of three-dimensional position coordinates and predicted position coordinates corresponding to at least three fixed points, including: obtaining the first conversion relationship, the second conversion relationship, the third conversion relationship and the fourth conversion relationship; obtaining at least three groups of three-dimensional position coordinates and predicted position coordinates corresponding to at least three fixed points, and converting and solving them based on the first conversion relationship, the second conversion relationship, the third conversion relationship and the fourth conversion relationship to determine the fifth conversion relationship between the chassis and the depth camera.
[0071] For the calibration of the depth camera, the conversion relationship between the depth camera and the end of the first robotic arm can be determined, and then the conversion relationship between the depth camera and other components can be deduced. Specifically, as an optional embodiment, the acquisition of at least three sets of second calibration data to determine the fourth conversion relationship between the end of the first robotic arm and the depth camera based on the second calibration data includes: acquiring first image data of the end of the second robotic arm taken by the depth camera at the end of the first robotic arm when it is stationary, and acquiring the first conversion relationship, the second conversion relationship and the third conversion relationship to form the second calibration data; based on the first image data in the second calibration data, identifying the preset point on the end of the second robotic arm, and determining the predicted coordinate value of the preset point relative to the depth camera; based on the first conversion relationship, the second conversion relationship and the third conversion relationship in the second calibration data, determining the mapping coordinate value of the preset point relative to the end of the first robotic arm; based on at least three sets of preset coordinate values and the mapping coordinate values, determining the fourth conversion relationship between the end of the first robotic arm and the depth camera.
[0072] For the calibration of two robot arm bases, the two robot arm ends can be made to overlap, so as to take the posture data for analysis and determine the conversion relationship between the robot arm bases. Specifically, as an optional embodiment, the acquisition of at least three sets of first calibration data to determine the third conversion relationship between the first robot arm base and the second robot arm base according to the first calibration data includes: controlling the first robot arm end and the second robot arm end to be in a zero force state, and obtaining the first posture data of the first robot arm end and the second posture data of the second robot arm end when the ends overlap, determining a set of first calibration data to form at least three sets of first calibration data; converting and solving the first posture data and the second posture data in each set of first calibration data according to the first conversion relationship and the second conversion relationship to determine the third conversion relationship. When the two robot arm ends overlap, the positions of the robot arm ends finally determined by the posture data should be corresponding (they are the same only after being mapped by the coordinate transformation matrix). Therefore, the position of the robot arm end can be determined by the first posture data and the second posture data, and then the calibration of the two robot arm ends can be performed.
[0073] Specifically, based on the above embodiments, the embodiments of the present application further provide a robot calibration device, which is used to calibrate a robot, wherein the robot comprises the following components: a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera disposed on the first robotic arm end. The calibration of the robot comprises determining the conversion relationship between the coordinate systems corresponding to the components of the robot, such as Figure 5 As shown, the device comprises:
[0074] The parameter data acquisition module 502 is used to acquire the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters.
[0075] The first data acquisition module 504 is used to acquire at least three sets of first calibration data to determine a third conversion relationship between the first robotic arm base and the second robotic arm base based on the first calibration data, wherein the first calibration data includes posture data acquired when the end of the first robotic arm and the end of the second robotic arm are in a zero-force state and the ends overlap.
[0076] The second data acquisition module 506 is used to acquire at least three sets of second calibration data to determine a fourth conversion relationship between the first robotic arm end and the depth camera based on the second calibration data, wherein the second calibration data includes first image data of the second robotic arm end taken by the depth camera at the first robotic arm end when the depth camera is stationary.
[0077] The implementation of the embodiment of the present application is similar to the implementation of the above-mentioned method embodiment. The specific implementation can refer to the specific implementation of the above-mentioned method embodiment, which will not be repeated here.
[0078] The solution of the present application can be applied in the scenario of calibrating a robot. The robot includes the following components: a base, a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera arranged on the first robotic arm end. The first robotic arm base and the second robotic arm base are arranged on a movable base. The coordinate systems of each component are different. The calibration of the robot by this solution includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot.
[0079] This solution can complete the robot calibration process through the robot itself. The robot base and the end of the robot can be calibrated according to the robot parameters. The two ends of the robot can be calibrated according to the data when the two ends of the robot overlap. The depth camera on one end of the robot can recognize the preset points of the other end of the robot to calibrate the depth camera. The fixed points on the chassis can also be recognized by the depth camera, and then the chassis can be calibrated, so as to determine the conversion relationship between any two components among the chassis, the first robot base, the first robot end, the second robot base, the second robot end, and the depth camera, and complete the calibration of the robot. This solution does not require the participation of external equipment, and can directly use the robot itself to complete the calibration of the robot, and the calibration process is simpler and more convenient.
[0080] Specifically, this solution can obtain the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end, and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; after the robot arm base and the robot arm end are calibrated, the first robot arm end and the second robot arm end can be controlled in a zero-force state, and the first robot arm end and the second robot arm end can be manually adjusted so that the first robot arm end and the second robot arm end coincide with each other, and the posture data is obtained as the first calibration data. Three groups of the first calibration data can be obtained, and then the third conversion relationship between the first robot arm base and the second robot arm base is analyzed. After calibrating the first robotic arm end and the second robotic arm end, the depth camera at the first robotic arm end can be controlled to remain still, and the first image data of a preset point at the second robotic arm end can be captured as the second calibration data. The second robotic arm end can be moved to multiple positions to obtain at least three sets of calibration data, and then the fourth conversion relationship between the first robotic arm end and the depth camera is determined to complete the calibration of the first robotic arm base, the first robotic arm end, the second robotic arm base, the second robotic arm end and the depth camera.
[0081] It should be noted that the division of units and / or modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit and / or module in each embodiment of the present application may be integrated into a processing unit and / or module, or each unit and / or module may exist physically separately, or two or more units and / or modules may be integrated into one unit and / or module. The above-mentioned integrated units and / or modules may be implemented in the form of hardware or in the form of software functional units and / or modules.
[0082] If the integrated units and / or modules are implemented in the form of software functional units and / or modules and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0083] In addition, the data transmission device and the data transmission method provided in the above embodiments are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.
[0084] Figure 6 A structural block diagram of a network device is shown according to an exemplary embodiment.
[0085] like Figure 6 As shown, the network device 1100 includes at least: a processor 1110 , a memory 1120 , and a transceiver 1130 .
[0086] The transceiver 1130 is used to receive and send data under the control of the processor 1110 .
[0087] exist Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1110 and memory represented by memory 1120. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1130 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, which may include a wireless channel, a wired channel, an optical cable, and other transmission media.
[0088] The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 can store data used by the processor 1110 when performing operations.
[0089] Optionally, the processor 1110 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor 1110 may also adopt a multi-core architecture. The processor 1110 and the memory 1120 may also be physically arranged separately.
[0090] The processor 1110 calls the computer program stored in the memory 1120 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.
[0091] Figure 7 A structural block diagram of a user equipment is shown according to an exemplary embodiment.
[0092] like Figure 7 As shown, the user equipment 1300 includes at least: a processor 1310 , a memory 1320 , and a transceiver 1330 .
[0093] The transceiver 1330 is used to receive and send data under the control of the processor 1310 .
[0094] exist Figure 7 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1310 and various circuits of memory represented by memory 1320 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1330 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, including transmission media such as wireless channels, wired channels, and optical cables. For different user devices, the user interface 1340 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.
[0095] The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 can store data used by the processor 1310 when performing operations.
[0096] Optionally, the processor 1310 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor 1310 may also adopt a multi-core architecture. The processor 1310 and the memory 1320 may also be physically arranged separately.
[0097] The processor 1310 calls the computer program stored in the memory 1320 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.
[0098] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.
[0099] In addition, a storage medium is provided in an embodiment of the present application, on which a computer program is stored, and when the computer program is executed by the processor, the data transmission method in the above embodiments is implemented. The storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (such as floppy disk, hard disk, tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state drive (SSD)), etc.
[0100] In an embodiment of the present application, a program product is provided, for example, the program product is an FPGA chip or a DSP chip, and the program product includes executable instructions, and the executable instructions are stored in a storage medium. The processor reads the executable instructions from the storage medium, so that when the executable instructions are executed by the processor, the data transmission method in each of the above embodiments is implemented.
[0101] The solution of the present application can be applied in the scenario of calibrating a robot. The robot includes the following components: a base, a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera arranged on the first robotic arm end. The first robotic arm base and the second robotic arm base are arranged on a movable base. The coordinate systems of each component are different. The calibration of the robot by this solution includes determining the conversion relationship between the coordinate systems corresponding to the components of the robot.
[0102] This solution can complete the robot calibration process through the robot itself. The robot base and the end of the robot can be calibrated according to the robot parameters. The two ends of the robot can be calibrated according to the data when the two ends of the robot overlap. The depth camera on one end of the robot can recognize the preset points of the other end of the robot to calibrate the depth camera. The fixed points on the chassis can also be recognized by the depth camera, and then the chassis can be calibrated, so as to determine the conversion relationship between any two components among the chassis, the first robot base, the first robot end, the second robot base, the second robot end, and the depth camera, and complete the calibration of the robot. This solution does not require the participation of external equipment, and can directly use the robot itself to complete the calibration of the robot, and the calibration process is simpler and more convenient.
[0103] Specifically, this solution can obtain the robot arm parameters to determine the first conversion relationship between the first robot arm base and the first robot arm end, and the second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; after the robot arm base and the robot arm end are calibrated, the first robot arm end and the second robot arm end can be controlled in a zero-force state, and the first robot arm end and the second robot arm end can be manually adjusted so that the first robot arm end and the second robot arm end coincide with each other, and the posture data is obtained as the first calibration data. Three groups of the first calibration data can be obtained, and then the third conversion relationship between the first robot arm base and the second robot arm base is analyzed. After calibrating the first robotic arm end and the second robotic arm end, the depth camera at the first robotic arm end can be controlled to remain still, and the first image data of a preset point at the second robotic arm end can be captured as the second calibration data. The second robotic arm end can be moved to multiple positions to obtain at least three sets of calibration data, and then the fourth conversion relationship between the first robotic arm end and the depth camera is determined to complete the calibration of the first robotic arm base, the first robotic arm end, the second robotic arm base, the second robotic arm end and the depth camera.
[0104] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0105] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0106] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0107] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0108] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0109] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A robot calibration device, It is characterized in that The device is used to calibrate a robot, the robot comprising the following components: a first robotic arm base, a first robotic arm end, a second robotic arm base, a second robotic arm end, and a depth camera arranged on the first robotic arm end. The calibration of the robot comprises determining a conversion relationship between coordinate systems corresponding to the components of the robot. The device comprises: A parameter data acquisition module, used to acquire the parameters of the robot arm, so as to determine a first conversion relationship between the first robot arm base and the first robot arm end, and a second conversion relationship between the second robot arm base and the second robot arm end according to the robot arm parameters; A first data acquisition module is used to acquire at least three sets of first calibration data to determine a third conversion relationship between the first robotic arm base and the second robotic arm base according to the first calibration data, wherein the first calibration data includes posture data acquired when the first robotic arm end and the second robotic arm end are in a zero-force state and the ends overlap; The second data acquisition module is used to acquire at least three sets of second calibration data to determine a fourth conversion relationship between the first robotic arm end and the depth camera based on the second calibration data, wherein the second calibration data includes first image data of the second robotic arm end taken by the depth camera at the end of the first robotic arm when the depth camera is stationary.
Citation Information
Cited By
Robot calibration method and device
CN117506897A
Robot calibration method and apparatus
CN117506897B
