Mechanical arm calibration method and device, mechanical arm and storage medium

Abstract

The invention discloses a mechanical arm calibration method and device, a mechanical arm and a storage medium. The mechanical arm calibration method comprises the steps that a # imgabs0 # X = X # imgabs1 # homogeneous transformation matrix equation of the mechanical arm is established; based on a quaternion multiplication algorithm, the homogeneous transformation matrix equation # imgabs2 # X = X # imgabs3 # is converted into an equation # imgabs4 # = # imgabs5 #; based on a cross multiplication algorithm, a # imgabs6 # = # imgabs7 # equation is converted into a matrix multiplication equation; and based on the property of the quaternion and a singular value decomposition algorithm, determining a solution of a target matrix represented by a quaternion expression so as to determine a conversion relation between the camera coordinate system and the end flange coordinate system. Therefore, the solution of the target matrix is obtained through the properties of the target matrix and the quaternion and the singular value decomposition algorithm, and the accuracy of calibrating the conversion relation between the camera coordinate system and the TCP coordinate system can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of oral medicine, and more specifically, to a robot arm calibration method, device, robot arm and storage medium. Background Art

[0002] Due to the high positioning accuracy and repeatability of the robot's operation, the robot is widely popular in the field of precision medicine. For example, oral implant surgery. The robot arm obtains the position and posture of the target position point through the camera fixed at the end, and determines the position and posture of the end of the robot arm in the actual space through the transformation relationship between the camera coordinate system (CCS) and the tool center point coordinate system (TCP) inherent to the end of the robot arm, thereby guiding the robot arm to move to the target position point according to the position and posture of the end of the robot arm in the actual space. Among them, the transformation relationship between the camera coordinate system and the TCP coordinate system needs to be determined through calibration.

[0003] In the related art, a two-step method is used to solve the transformation relationship between the base coordinate system and the TCP coordinate system, the transformation relationship between the TCP coordinate system and the camera coordinate system, and the transformation relationship between the camera coordinate system and the reference array coordinate system to construct a matrix equation to solve the rotation matrix and the translation vector, and then determine the transformation relationship between the camera coordinate system and the TCP coordinate system based on the rotation matrix and the translation vector.

[0004] However, due to the inaccuracy of numerical calculations or noise in the measurement process, additional errors are introduced, which affects the accuracy of determining the transformation relationship between the camera coordinate system and the TCP coordinate system, which in turn affects the performance of the robot when performing precise surgical tasks, such as performing oral implant surgery. Summary of the invention

[0005] In view of the above problems, the present application proposes a robotic arm calibration method, device, robotic arm and storage medium, which can effectively improve the accuracy of determining the conversion relationship between the camera coordinate system and the TCP coordinate system.

[0006] In a first aspect, an embodiment of the present application provides a method for calibrating a robotic arm, the robotic arm calibration comprising: establishing a X=X Homogeneous transformation matrix equation; where, X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system; based on the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equation; Based on the cross multiplication algorithm, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector; the target matrix is ​​converted into a quaternion expression; based on the properties of the quaternion and the singular value decomposition algorithm, the solution of the quaternion expression is determined to determine the transformation relationship between the camera coordinate system and the end flange coordinate system.

[0007] In a second aspect, an embodiment of the present application further provides a robotic arm calibration device, which includes: an establishment module for establishing the A_iX=XB_iAiX=BiX homogeneous transformation matrix equation of the robotic arm; wherein the A_iX=XB_iAiX=BiX homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, the TCP coordinate system, the camera coordinate system and the reference array coordinate system; a first execution module for converting based on a quaternion multiplication algorithm X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equation; The second execution module is used to convert the = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector; a third execution module is used to convert the target matrix into a quaternion expression; and a fourth execution module is used to determine the solution of the quaternion expression based on the properties of the quaternion and the singular value decomposition algorithm to determine the transformation relationship between the camera coordinate system and the end flange coordinate system.

[0008] In a third aspect, an embodiment of the present application also provides a robotic arm, comprising a processor, a memory, and one or more applications; the one or more applications are stored in the memory and configured to be executed by the processor to implement the above-mentioned robotic arm calibration method.

[0009] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a program code is stored, wherein the above-mentioned robot arm calibration method is executed when the program code is run by a processor.

[0010] The technical solution provided by the present application, the robot arm calibration method comprises: establishing a robot arm X=X Homogeneous transformation matrix equation; where, X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system; based on the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equation; Based on the cross multiplication algorithm, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector; the target matrix is ​​converted into a quaternion expression; based on the properties of the quaternion and the singular value decomposition algorithm, the solution of the quaternion expression is determined to determine the transformation relationship between the camera coordinate system and the end flange coordinate system. X=X The homogeneous transformation matrix equation is converted into a matrix multiplication equation to determine the target matrix containing the scalar converted from the X matrix to the quaternion and the vector converted from the X matrix to the quaternion. Then, through the properties of the quaternion and the singular value decomposition algorithm, the solution of the target matrix expressed as a quaternion expression is obtained to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby eliminating the need to use a two-step method to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby effectively improving the accuracy of the calibration result between the camera coordinate system and the TCP coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] Figure 1 A schematic structural diagram of a robotic arm provided in an embodiment of the present application is shown.

[0013] Figure 2 A schematic flow chart of a robotic arm calibration method provided in an embodiment of the present application is shown.

[0014] Figure 3 A schematic structural diagram of a robotic arm calibration device provided in an embodiment of the present application is shown.

[0015] Figure 4A schematic structural diagram of a robotic arm provided in an embodiment of the present application is shown.

[0016] Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0018] In the following description, reference is made to "some embodiments", which describes a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. In the following description, reference is made to the term "multiple" to mean at least two.

[0019] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0021] Due to the high positioning accuracy and repeatability of the robot, the robot is widely popular in the field of precision medicine. For example, oral implant surgery. The robot arm obtains the position and posture of the target position point through the camera fixed at the end, and determines the position and posture of the end of the robot arm in the actual space through the transformation relationship between the camera coordinate system (CCS) and the tool center point coordinate system (TCP) inherent to the end of the robot arm, thereby guiding the robot arm to move to the target position point according to the position and posture of the end of the robot arm in the actual space.

[0022] Among them, the conversion relationship between the position and posture of the position point in the actual space in the TCP coordinate system intrinsic to the end of the robot arm and the base coordinate system (BCS) intrinsic to the robot arm can be directly obtained from the control system of the robot arm, while the conversion relationship between the camera coordinate system and the TCP coordinate system needs to be determined through calibration.

[0023] In the related art, the camera at the end of the robot arm is rigidly connected to the intrinsic TCP at the end of the robot arm. The conversion relationship between the camera coordinate system at the end of the robot arm and the intrinsic TCP coordinate system at the end of the robot arm is directly measured in the hardware design drawings. The measured value is the design value, and the conversion relationship between the camera coordinate system and the TCP coordinate system is solved through the design value. However, there is an error between the design value and the actual object of the assembly. The error is brought into the hand-eye calibration of the base of the robot arm and the camera. Since the result of the hand-eye calibration itself is used to correct the TCP conversion relationship, the correction is localized. When the posture of the end of the robot arm changes significantly, this correction becomes inapplicable.

[0024] In order to improve the applicability of the method for determining the conversion relationship between the camera coordinate system and the TCP coordinate system, in a related technology, the posture of the end of the manipulator is kept unchanged relative to the intrinsic base coordinate system of the manipulator, and the end of the manipulator is controlled to translate to three positions. Through the different descriptions of the three positions in the camera coordinate system and the base coordinate system of the manipulator, the rotation relationship (rotation amount) between the camera coordinate system and the intrinsic base coordinate system of the manipulator is obtained. Then, the origin of the intrinsic TCP coordinate system of the manipulator is kept stationary, and the end of the manipulator fits the sphere and its center by rotating the reference array, and the position of the center of the sphere is used to calculate the translation amount in the conversion relationship of the TCP coordinate system. The conversion relationship between the camera coordinate system and the TCP coordinate system is then determined based on the rotation amount and the translation amount. However, the end of the manipulator must perform a series of specified movements (translation in a specified direction and rotation around a specified center), which has low flexibility and is greatly restricted by the space of the operating room and the placement of instruments.

[0025] In order to reduce the spatial limitation of the method for determining the transformation relationship between the camera coordinate system and the TCP coordinate system, in a related technology, a matrix equation is constructed by solving the transformation relationship between the base coordinate system and the TCP coordinate system, the transformation relationship between the TCP coordinate system and the camera coordinate system, and the transformation relationship between the camera coordinate system and the reference array coordinate system through a two-step method to solve the rotation matrix and the translation vector, and then determine the transformation relationship between the camera coordinate system and the TCP coordinate system according to the rotation matrix and the translation vector. However, due to the inaccuracy of numerical calculations or noise in the measurement process, additional errors will be introduced, which will affect the accuracy of determining the transformation relationship between the camera coordinate system and the TCP coordinate system, and thus affect the performance of the robot when performing precise surgical tasks. For example, performing oral implant surgery.

[0026] In order to improve the above problems, the present application provides a robot arm calibration method, device, robot arm and storage medium. The robot arm calibration method includes: establishing a robot arm X=X Homogeneous transformation matrix equation; where, X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system; based on the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equation; Based on the cross multiplication algorithm, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector; the target matrix is ​​converted into a quaternion expression; based on the properties of the quaternion and the singular value decomposition algorithm, the solution of the quaternion expression is determined to determine the transformation relationship between the camera coordinate system and the end flange coordinate system.

[0027] Thus, by X=X The homogeneous transformation matrix equation is converted into a matrix multiplication equation to determine the target matrix containing the scalar converted from the X matrix to the quaternion and the vector converted from the X matrix to the quaternion. Then, through the properties of the quaternion and the singular value decomposition algorithm, the solution of the target matrix expressed as a quaternion expression is obtained to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby eliminating the need to use a two-step method to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby effectively improving the accuracy of the calibration result between the camera coordinate system and the TCP coordinate system.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0029] See also Figure 1 , Figure 1 FIG. 1 shows a schematic diagram of the structure of a mechanical arm provided in an embodiment of the present application. Figure 1 As shown, the robot arm 100 includes a base coordinate system F1, a TCP coordinate system F2, a camera coordinate system F3 and a reference array coordinate system F4.

[0030] The base coordinate system F1 is the starting point of the coordinate system of the robot 100, and is used to describe information such as the initial position, movement route, and end position of the robot 100. In the control system of the robot 100, the base coordinate system F1 is usually used to determine the posture and position of the robot 100 to ensure that the robot 100 can move along a predetermined route.

[0031] The TCP coordinate system F2 refers to the center point coordinate system of the end effector (such as a gripper, welding gun, etc.) of the robot arm 100. The TCP coordinate system F2 is the basis for programming and motion control of the robot arm 100, and it defines the position and orientation of the end effector.

[0032] The camera coordinate system F3 refers to the coordinate system used by the positioning camera 101 on the end effector of the robot 100. In the integrated application of the robot 100 and the positioning camera 101, the camera coordinate system F3 is usually associated with the position and posture of the end effector of the robot 100, and is used to describe the specific position and orientation of the positioning camera 101 on the robot 100.

[0033] The reference array coordinate system F4 is based on the four reflective balls that the positioning camera 101 can identify within its field of view. The four reflective balls are fixed according to different relative position relationships to form the reference array coordinate system F4.

[0034] When the robot arm 100 needs to move to any target posture, the posture information of the target posture in the camera coordinate system F3 is obtained by positioning the camera 101, and then the posture information of the target posture in the TCP coordinate system F2 is determined according to the conversion relationship between the camera coordinate system F3 and the TCP coordinate system F2, and then the posture information of the target posture in the base coordinate system F1 is determined according to the conversion relationship between the TCP coordinate system F2 and the base coordinate system F1, and then the robot arm 100 is controlled to move to the corresponding position of the target posture according to the posture information of the target posture in the base coordinate system F1.

[0035] In other embodiments, when the robot arm 100 needs to move to any target posture, the posture information of the target posture in the camera coordinate system F3 is obtained by positioning the camera 101, and then the posture information of the target posture in the TCP coordinate system F2 is determined based on the conversion relationship between the camera coordinate system F3 and the TCP coordinate system F2, and then the robot arm 100 is controlled to move to the corresponding position of the target posture based on the posture information of the target posture in the TCP coordinate system F2.

[0036] In some embodiments, the robot arm 100 further includes a controller (not shown in the figure). The controller is used to determine the positions corresponding to different postures in different coordinate systems. The controller is also used to plan the movement path of the robot arm 100 and control the robot arm 100 to move according to the planned movement path.

[0037] The conversion of the posture between the base coordinate system F1 and the TCP coordinate system F2 can be directly obtained from the control system of the robot 100 (for example, the posture of the posture in the TCP coordinate system F2 in the base coordinate system F1 can be determined by the controller).

[0038] However, the transformation of the pose between the camera coordinate system F3 and the TCP coordinate system F2 needs to be obtained through calibration, specifically: See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a robot arm calibration method provided in an embodiment of the present application. Figure 2 As shown, the robot arm calibration method may include steps 210 to 250.

[0039] In step 210, the robot arm is established. X=X Homogeneous transformation matrix equation.

[0040] in, X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system.

[0041] Specifically, in some embodiments, the step of establishing the robot arm X=X The homogeneous transformation matrix equation may further include the following steps: (1) Control the robotic arm to move to multiple target positions.

[0042] (2) Obtain the transformation relationship between the base coordinate system and the end flange coordinate system corresponding to multiple target poses, as well as the transformation relationship between the camera coordinate system and the reference array coordinate system.

[0043] (3) According to the transformation relationship between the base coordinate system and the end flange coordinate system corresponding to multiple target poses, and the transformation relationship between the camera coordinate system and the reference array coordinate system, establish X=X Homogeneous transformation matrix equation.

[0044] Exemplarily, the controller controls the robotic arm to move to the first target posture , and record the first target pose Conversion relationship from base coordinate system F1 to TCP coordinate system F2 , and the first target pose Conversion relationship from camera coordinate system F3 to reference array coordinate system F4 .

[0045] The controller then controls the robot to make any movement to the second target position. , and record the first target pose Conversion relationship from base coordinate system F1 to TCP coordinate system F2 , and the second target pose Conversion relationship from camera coordinate system F3 to reference array coordinate system F4 .

[0046] The controller then controls the robot arm to make any movement to the third target position. , and record the third target posture Conversion relationship from base coordinate system F1 to TCP coordinate system F2 , and the third target pose Conversion relationship from camera coordinate system F3 to reference array coordinate system F4 .

[0047] The controller then controls the robot arm to make any movement to the fourth target position. , and record the fourth target posture Conversion relationship from base coordinate system F1 to TCP coordinate system F2 , and the fourth target pose Conversion relationship from camera coordinate system F3 to reference array coordinate system F4 .

[0048] Since the base coordinate system F1 and the reference array coordinate system F4 remain relatively static during the calibration process, the conversion relationship between the base coordinate system F1 and the reference array coordinate system F4 is fixed. Conversion relationship from base coordinate system F1 to reference array coordinate system F4 is a constant, based on which the following first set of equations is established: ; in, It is the transformation relationship from TCP coordinate system F2 to camera coordinate system F3.

[0049] Rearranging the first set of equations, we can obtain the following second set of equations: * = ; * = ; Rearrange the second set of equations and set = ,Will = , = , ,X= .

[0050] in , , , , , , are all known. Therefore, the second set of equations can be simplified to X=X Homogeneous transformation matrix equation to complete X=X Establishment of homogeneous transformation matrix equation.

[0051] To facilitate the subsequent steps X=X The homogeneous transformation matrix equation is transformed accordingly to obtain X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a quaternion vector target matrix. By solving the target matrix, the transformation relationship between the camera coordinate system F3 and the end flange coordinate system F2 is determined. Specifically: In step 220, based on the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations.

[0052] For example, there is a quaternion ,but .in, is a quaternion The scalar part of is a quaternion The vector part of .

[0053] For example, there is a quaternion as well as If the quaternion and Multiplying them, we get the following equation: = ; Based on this, according to the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations to determine the X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector. By solving the target matrix, the transformation relationship between the camera coordinate system F3 and the end flange coordinate system F2 is determined.

[0054] Further, in some embodiments, the step is based on a quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations can also include the following steps: (1) According to the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted into the first matrix multiplication equation, which is expressed as: ; (2) Arrange the first matrix multiplication equation to obtain = Quaternion differential equations, = The quaternion differential equation is expressed as: ; in, for X=X Homogeneous transformation matrix equation Matrix conversion to quaternion scalar; for X=X Homogeneous transformation matrix equation Matrix conversion to quaternion vector; for X=X Homogeneous transformation matrix equation Matrix conversion to quaternion scalar; for X=X Homogeneous transformation matrix equation Matrix conversion to quaternion vector; for X=X The X matrix in the homogeneous transformation matrix equation is converted to a quaternion scalar; for X=X The X matrix in the homogeneous transformation matrix equation is converted to a vector of quaternions.

[0055] Specifically, the following third set of equations can be obtained from the first matrix multiplication equation: ; Rearranging the third set of equations gives the following fourth set of equations: ; Rearranging the fourth set of equations gives = Quaternion differential equation. In the subsequent steps, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a quaternion vector target matrix. By solving the target matrix, the transformation relationship between the camera coordinate system F3 and the end flange coordinate system F2 is determined. Specifically: In step 230, based on the cross multiplication algorithm, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The homogeneous transformation matrix equation is the target matrix for converting the X matrix into a quaternion scalar and the X matrix into a quaternion vector.

[0056] The cross product algorithm calculates a vector that is perpendicular to both vectors. The direction can be determined by the right-hand rule. Therefore, when the order of the two vectors is reversed, the result of the cross product algorithm must also be reversed. Specifically: ; The cross multiplication algorithm can also be converted into a matrix multiplication form, for example: = =

[0057] like Construct an antisymmetric matrix sk(a) ,but sk(a) for: sk(a)=

[0058] Through the cross multiplication algorithm, = The quaternion differential equation is converted into a matrix multiplication equation. Specifically, in some embodiments, the step is based on a cross multiplication algorithm. = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and a target matrix of the X matrix converted into a quaternion vector, and the following steps may also be included: (1) According to the cross multiplication algorithm, = The quaternion differential equation is converted into the first system of equations, which is expressed as: ; (2) Replace the multiplication equation in the first system with Expressed in the form of an antisymmetric matrix, the second system of equations is obtained, which is expressed as: ; (3) Extract the multiplication equation in the second system of equations In , and get the third-party program group, which is expressed as: ; (4) Convert the third equation group into matrix form to obtain the matrix multiplication equation, which is expressed as: ; Substituting the matrix multiplication equation Determine the target matrix.

[0059] That is, according to the cross product algorithm, the fourth set of equations In the equation Convert to , and Convert to sk( ) . Then we get the second set of equations.

[0060] And the second system of equations In the equation Extract to get the third program group Equation.

[0061] At this point, we can get the third group of equations, and the equations in the third group of equations all contain as well as , convert the third equation group into matrix form, thus obtaining the matrix multiplication equation. Among them, the matrix is the target matrix.

[0062] In step 240, the target matrix is ​​converted into a quaternion representation.

[0063] Target Matrix The quaternion expression of is qx. That is =qx. By solving qx, we can get the conversion relationship between the camera coordinate system and the end flange coordinate system. Specifically: In step 250 , based on the properties of quaternions and a singular value decomposition algorithm, a solution of the quaternion expression is determined to determine the transformation relationship between the camera coordinate system and the end flange coordinate system.

[0064] According to the properties of quaternions, the modulus of quaternions is 1. = =1. Based on the singular value decomposition algorithm, the solution of qx can be solved to determine the transformation relationship between the camera coordinate system and the end flange coordinate system. Specifically: In some embodiments, the step of determining a solution to the quaternion expression based on the properties of the quaternion and the singular value decomposition algorithm to determine the transformation relationship between the camera coordinate system and the end flange coordinate system may also include the following steps: (1) Based on the property of quaternion that the modulus of quaternion is 1, the fourth set of equations is established according to the quaternion expression. The fourth set of equations is expressed as: , In which, replace the matrix multiplication equation The matrix is ​​represented as M matrix; (2) Based on the singular value decomposition algorithm, the fourth system of equations can be transformed into the fifth system of equations, which is expressed as: , (3) Based on the singular value decomposition algorithm, the fifth set of equations is organized into the first equation, which is expressed as: || ||=||Σ ||, where = ; (4) Replace ||Σ in the first equation || Expand to get the second equation, which is expressed as ; (5) Based on the singular value decomposition algorithm and The modulus length is 1, so matrix; (6) Based on the singular value decomposition algorithm and Matrix that determines the solution to the quaternion expression.

[0065] The singular value decomposition algorithm is a method to decompose a matrix into the product of three specific matrices. According to the singular value decomposition algorithm, we can get = . Among them, the matrix is the matrix that needs to be decomposed. These are the three matrices decomposed by the singular value decomposition algorithm. as well as is an orthogonal matrix, is a diagonal matrix, a diagonal matrix The numbers on the diagonal are the eigenvalues ​​of the matrix, which show a decreasing pattern. For example, the diagonal matrix The numbers on the diagonal are , , as well as ,but , , as well as The relationship between > > > .

[0066] So Among them, due to The result is a vector, and U does not change the length of the vector (which can be obtained from the properties of the singular value decomposition algorithm), so it can be directly ignored. ,and is a quaternion whose modulus is 1, and The length of the vector is not changed (due to the properties of the singular value decomposition algorithm).

[0067] Assumptions for ,Right now = , that is to say, = = ,and = =1.

[0068] Then, || Can be equal to the following equation: ||

[0069] Further arrangement can get the following equation: ||

[0070] Further arrangement can get the following equation: || + + +

[0071] Through the diagonal array The properties of , , as well as The relationship between > > > , in order to make || The value of is minimized, then , The value of is minimized, and because The modulus length is 1, then we can determine The specific value of .

[0072] = ; From the above description, we can see that = , thus obtaining =

[0073] Since the singular value decomposition algorithm has the following properties = ,Right now =I. Thus, the following equation can be obtained: = , Thus we can get = = = That is to say, = , that is, the transformation relationship between the camera coordinate system and the end flange coordinate system is Then, by converting the solution of the quaternion qx into a 4x4 homogeneous transformation matrix X, the transformation relationship between the camera coordinate system and the end flange coordinate system is determined to complete the calibration of the robotic arm.

[0074] However, during the calibration process of the robot arm, the accuracy of the solution of the quaternion qx may be affected by other factors such as the environment. In order to further improve the accuracy of the calibration of the robot arm, in some embodiments, the robot arm calibration method may further include the following steps: (1) Substitute the solutions of the quaternion expression into the corresponding values ​​of multiple target poses. X=X Homogeneous transformation matrix equation is used to determine the target homogeneous transformation matrix corresponding to multiple target postures.

[0075] (2) The target posture corresponding to the target homogeneous transformation matrix whose modulus is greater than a preset threshold is determined as an abnormal posture.

[0076] (3) Determine the weight value of abnormal posture.

[0077] (4) Based on the weight value, determine the weighted abnormal posture as well as matrix.

[0078] (5) According to the weighted as well as Matrix, Update X=X Homogeneous transformation matrix equation.

[0079] Determine the first target pose , the second target pose , the third target posture And the fourth target posture The corresponding errors are respectively, that is, the solution of the quaternion qx is brought into the first target pose , the second target pose , the third target posture And the fourth target posture The corresponding X=X Homogeneous transformation matrix equation to obtain the first target pose , the second target pose , the third target posture And the fourth target posture The corresponding X=X The modulus of the homogeneous transformation matrix equation determines the first target pose , the second target pose , the third target posture And the fourth target posture The corresponding errors respectively.

[0080] For example, when the robot arm is in the first target position When , the first target pose is obtained by the above calculation method Corresponding , and Substitution X=X In the homogeneous transformation matrix equation, we can obtain The modulus length can then be used to obtain the first target pose The corresponding error.

[0081] Through the above method, the second target posture is obtained , the third target posture And the fourth target posture The corresponding errors respectively.

[0082] When the error corresponding to the target pose is greater than the preset threshold, it means that the target pose is greatly affected by other factors such as the environment, resulting in a low accuracy of the conversion relationship between the camera coordinate system and the terminal flange coordinate system corresponding to the target pose. This requires recalibration of the conversion relationship between the camera coordinate system and the terminal flange coordinate system corresponding to these target poses (abnormal poses) with low accuracy.

[0083] The weight value of the abnormal posture can be determined according to the modulus length of the homogeneous transformation matrix equation corresponding to the abnormal posture. For example, the weight value of the abnormal posture can be determined by the following formula: ; in, is the weight value corresponding to the abnormal posture; is the modulus of the homogeneous transformation matrix equation corresponding to the abnormal posture; is a constant ( is a positive number, To avoid the denominator of the above formula being 0).

[0084] After determining the weight value corresponding to the abnormal posture, determine the weighted as well as Matrix, updated X=X Homogeneous transformation matrix equation, then updated X=X The homogeneous transformation matrix equation redefines the abnormal posture corresponding to , that is, determine the transformation relationship between the camera coordinate system and the end flange coordinate system corresponding to the abnormal posture.

[0085] Determine the weight values ​​corresponding to multiple abnormal postures, and then determine the average value of the weight values ​​corresponding to multiple abnormal postures. According to the average value and the weighted singular value decomposition algorithm, determine the weighted value of the abnormal posture as well as matrix.

[0086] For example, the weight values ​​corresponding to the multiple abnormal postures are , , then the average value of the weight values ​​corresponding to multiple abnormal postures is W=diag( , ), then the weighted = , weighted = . Then according to the weighted And the weighted renew X=X Homogeneous transformation matrix equation.

[0087] See also Figure 3 , Figure 3 The schematic diagram of the structure of a robot arm calibration device provided in an embodiment of the present application is shown. The robot arm calibration device 300 includes: an establishment module 310, a first execution module 320, a second execution module 330, a third execution module 340 and a fourth execution module 350. Specifically: Establishing module 310, used to establish the homogeneous transformation matrix equation A_iX=XB_iAiX=BiX of the robot arm; wherein the homogeneous transformation matrix equation A_iX=XB_iAiX=BiX is the transformation relationship between the base coordinate system, the TCP coordinate system, the camera coordinate system and the reference array coordinate system.

[0088] The first execution module 320 is used to convert the quaternion multiplication algorithm into X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations.

[0089] The second execution module 330 is used to convert = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The homogeneous transformation matrix equation is the target matrix for converting the X matrix into a quaternion scalar and the X matrix into a quaternion vector.

[0090] The third execution module 340 is used to convert the target matrix into a quaternion expression.

[0091] The fourth execution module 350 is used to determine the solution of the quaternion expression based on the properties of the quaternion and the singular value decomposition algorithm, so as to determine the conversion relationship between the camera coordinate system and the end flange coordinate system.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0093] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.

[0094] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0095] See also Figure 4 , Figure 4 A schematic diagram of the structure of a robotic arm provided in an embodiment of the present application is shown. The robotic arm 400 in the present application may include one or more of the following components: a processor 410, a memory 420, and one or more applications, wherein the one or more applications may be stored in the memory 420 and configured to be executed by one or more processors 410, and the one or more programs are configured to execute the anti-light leakage method as described in the aforementioned method embodiment.

[0096] The processor 410 may include one or more processing cores. The processor 410 uses various interfaces and lines to connect various parts of the entire terminal device 400, and executes various functions of the robotic arm 400 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 420, and calling data stored in the memory 420. Optionally, the processor 410 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 410 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 410, but may be implemented separately through a communication chip.

[0097] The memory 420 may include a random access memory (RAM) or a read-only memory (ROM). The memory 420 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the robot arm 400 during use.

[0098] See also Figure 5 , Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application is shown, wherein the computer-readable medium 500 stores program code, and the program code can be called by a processor to execute the cooking control method described in the above method embodiment.

[0099] The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 for executing any method steps in the above method. These program codes can be read from or written to one or more computer program devices. The program code 510 can be compressed, for example, in a suitable form.

[0100] The technical solution provided by the present application, the robot arm calibration method comprises: establishing a robot arm X=X Homogeneous transformation matrix equation; where, X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system; based on the quaternion multiplication algorithm, X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equation; Based on the cross multiplication algorithm, = The quaternion differential equation is converted to a matrix multiplication equation to obtain an equation containing X=X The X matrix in the homogeneous transformation matrix equation is converted into a quaternion scalar and the X matrix is ​​converted into a target matrix of a quaternion vector; the target matrix is ​​converted into a quaternion expression; based on the properties of the quaternion and the singular value decomposition algorithm, the solution of the quaternion expression is determined to determine the transformation relationship between the camera coordinate system and the end flange coordinate system. X=X The homogeneous transformation matrix equation is converted into a matrix multiplication equation to determine the target matrix containing the scalar converted from the X matrix to the quaternion and the vector converted from the X matrix to the quaternion. Then, through the properties of the quaternion and the singular value decomposition algorithm, the solution of the target matrix expressed as a quaternion expression is obtained to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby eliminating the need to use a two-step method to calibrate the conversion relationship between the camera coordinate system and the TCP coordinate system, thereby effectively improving the accuracy of the calibration result between the camera coordinate system and the TCP coordinate system.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A robot arm calibration method, characterized in that: The method comprises: Building the robotic arm X=X Homogeneous transformation matrix equation; wherein, the X=X The homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, TCP coordinate system, camera coordinate system and reference array coordinate system; Based on the quaternion multiplication algorithm, the X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations; Based on the cross multiplication algorithm, the = The quaternion differential equation is converted to a matrix multiplication equation to obtain a matrix multiplication equation containing the X=X The target matrix of the X matrix converted to the quaternion scalar and the X matrix converted to the quaternion vector in the homogeneous transformation matrix equation; Convert the target matrix into a quaternion expression; Based on the properties of quaternions and a singular value decomposition algorithm, a solution to the quaternion expression is determined to determine a conversion relationship between the camera coordinate system and the end flange coordinate system.

2. The robot arm calibration method according to claim 1, characterized in that: The step of establishing the AiX=BiX homogeneous transformation matrix equation of the robotic arm includes: Controlling the robotic arm to move to multiple target positions respectively; Acquire the conversion relationship between the base coordinate system and the end flange coordinate system and the conversion relationship between the camera coordinate system and the reference array coordinate system respectively corresponding to the multiple target postures; According to the conversion relationship between the base coordinate system and the end flange coordinate system and the conversion relationship between the camera coordinate system and the reference array coordinate system corresponding to the multiple target postures, the X=X Homogeneous transformation matrix equation.

3. The robot arm calibration method according to claim 2, characterized in that: The method further comprises: Substitute the solutions of the quaternion expression into the corresponding X=X A homogeneous transformation matrix equation is used to determine the target homogeneous transformation matrices corresponding to the multiple target postures respectively; The target posture corresponding to the modulus of the target homogeneous transformation matrix being greater than a preset threshold is determined as an abnormal posture; Determining a weight value of the abnormal posture; Based on the weight value, determine the weighted value of the abnormal posture as well as matrix; According to the weighted as well as Matrix, update the X=X Homogeneous transformation matrix equation.

4. The robot arm calibration method according to any one of claims 1 to 3, characterized in that: The quaternion multiplication algorithm is based on X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations, including: According to the quaternion multiplication algorithm, the X=X The homogeneous transformation matrix equation is converted into a first matrix multiplication equation, which is expressed as: ； The first matrix multiplication equation is rearranged to obtain the = Quaternion differential equations, = The quaternion differential equation is expressed as: ； in, For the X=X Homogeneous transformation matrix equation Matrix conversion to quaternion scalar; For the X=X Homogeneous transformation matrix equation Matrix conversion to quaternion vector; For the X=X Homogeneous transformation matrix equation Matrix conversion to quaternion scalar; For the X=X Homogeneous transformation matrix equation Matrix conversion to quaternion vector; For the X=X The X matrix in the homogeneous transformation matrix equation is converted to a quaternion scalar; For the X=X The X matrix in the homogeneous transformation matrix equation is converted to a vector of quaternions.

5. The robot arm calibration method according to claim 4, characterized in that: Based on the cross multiplication algorithm, the = The quaternion differential equation is converted to a matrix multiplication equation to obtain a matrix multiplication equation containing the X=X The target matrix of the X matrix converted to the quaternion scalar and the X matrix converted to the quaternion vector in the homogeneous transformation matrix equation includes: According to the cross multiplication algorithm, the = The quaternion differential equation is converted into the first system of equations, which is expressed as: ； Substitute the multiplication equations in the first system of equations into Expressed in the form of an antisymmetric matrix, the second system of equations is obtained, which is expressed as: ； Extract the multiplication equation in the second set of equations In , and obtain a third-party program group, which is expressed as: ； The third equation group is converted into a matrix form to obtain the matrix multiplication equation, which is expressed as: ； Substituting the matrix multiplication equation Determine the target matrix.

6. The robot arm calibration method according to claim 5, characterized in that: The quaternion expression is expressed as =qx.

7. The robot arm calibration method according to claim 6, characterized in that: The method of determining a solution to the quaternion expression based on the properties of the quaternion and the singular value decomposition algorithm comprises: Based on the property of the quaternion that the modulus of the quaternion is 1, a fourth set of equations is established according to the quaternion expression. The fourth set of equations is expressed as follows: ， Wherein, the matrix multiplication equation The matrix is ​​represented as M matrix; Based on the singular value decomposition algorithm, the fourth system of equations can be converted into a fifth system of equations, and the fifth system of equations is expressed as: ， Based on the singular value decomposition algorithm, the fifth set of equations is organized into a first equation, which is expressed as: || ||=||Σ ||, where = ; Replace the ||Σ || Expand to get the second equation, which is expressed as ; Based on the singular value decomposition algorithm and The modulus length is 1, which determines the matrix; Based on the singular value decomposition algorithm and the A matrix that determines the solution to the quaternion expression.

8. A robot arm calibration device, characterized in that: The device comprises: Establishing a module for establishing the A_iX=XB_iAiX=BiX homogeneous transformation matrix equation of the robotic arm; wherein the A_iX=XB_iAiX=BiX homogeneous transformation matrix equation is the transformation relationship between the base coordinate system, the TCP coordinate system, the camera coordinate system and the reference array coordinate system; The first execution module is used to convert the X=X The homogeneous transformation matrix equation is converted to = Quaternion differential equations; The second execution module is used to convert the = The quaternion differential equation is converted to a matrix multiplication equation to obtain a matrix multiplication equation containing the X=X The target matrix of the X matrix converted to the quaternion scalar and the X matrix converted to the quaternion vector in the homogeneous transformation matrix equation; A third execution module, used for converting the target matrix into a quaternion expression; The fourth execution module is used to determine the solution of the quaternion expression based on the properties of the quaternion and the singular value decomposition algorithm, so as to determine the transformation relationship between the camera coordinate system and the end flange coordinate system.

9. A robotic arm, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors as the robot arm calibration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the robot arm calibration method as described in any one of claims 1-7.

Images