Acupuncture robot synchronous calibration method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for synchronous calibration of an acupuncture robot, including: obtaining joint angle data of an acupuncture robot arm, and obtaining first image information of a hand-eye camera and second image information of a global camera; processing joint angle data, first image information and second image information based on a kinematic model and two-dimensional code posture reconstruction technology to obtain multiple groups of measured posture relationships; constructing a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decoupling the calibration objective function based on a closed solution method of Kronecker product, and solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated; constructing an optimization objective function based on the closed solution, iteratively solving the optimization objective function based on an iterative solution of multi-objective synchronous optimization, and obtaining the optimal solution of the quantity to be calibrated. The present invention improves the accuracy of multi-objective coupling calibration of an acupuncture robot and can be applied to the field of robot control technology.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a method, system, device and storage medium for synchronous calibration of an acupuncture robot. Background Art

[0002] Acupuncture robot is one of the new era products that combines traditional Chinese medicine and modern technology. It is the inheritance and development of traditional acupuncture. Acupuncture robot is generally composed of acupuncture robot arm, visual system (including hand-eye camera and global camera) and acupuncture mechanism. Therefore, system calibration is one of the important prerequisites for precise acupuncture control. The calibration of the visual system includes the calibration of the hand-eye camera and the global camera; unlike non-contact medical robots, the acupuncture needles carried by the acupuncture robot need to be inserted into the patient's skin tissue. Therefore, the relative position relationship calibration of the acupuncture mechanism is also necessary. This means that the system calibration of the acupuncture robot is multi-target coupled. For the multi-target coupled calibration of the acupuncture robot, the traditional method can only split the target problem in a step-by-step manner and solve it one by one. It is impossible to achieve the synchronous calibration of multiple sub-problems, resulting in error transmission between multiple sub-problems, affecting the calibration accuracy of the acupuncture robot, thereby affecting the accuracy of the acupuncture robot control. Summary of the invention

[0003] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0004] To this end, an object of an embodiment of the present invention is to provide a synchronous calibration method for an acupuncture robot, which improves the accuracy of multi-target coupled calibration of the acupuncture robot, thereby improving the accuracy of control of the acupuncture robot.

[0005] Another object of an embodiment of the present invention is to provide a synchronous calibration system for an acupuncture robot.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0007] In a first aspect, an embodiment of the present invention provides a method for synchronous calibration of an acupuncture robot, comprising the following steps:

[0008] Acquire the joint angle data of the acupuncture robot arm, and acquire the first image information of the hand-eye camera and the second image information of the global camera;

[0009] Processing the joint angle data, the first image information, and the second image information based on a kinematic model and a two-dimensional code posture reconstruction technology to obtain multiple sets of measured posture relationships;

[0010] Constructing a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decoupling the calibration objective function based on a closed solution method of the Kronecker product, and solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated;

[0011] An optimization objective function is constructed according to the closed solution, and the optimization objective function is iteratively solved based on an iterative solution method of multi-objective synchronous optimization to obtain an optimal solution for the quantity to be calibrated, and then a synchronous calibration result of the acupuncture robot is determined according to the optimal solution;

[0012] Wherein, the acupuncture robot includes the acupuncture robotic arm, the hand-eye camera, the global camera and acupuncture needles.

[0013] Furthermore, in one embodiment of the present invention, the measured posture relationship includes a first relative posture between the end of the robotic arm and the base of the robotic arm, a second relative posture between the QR code calibration group and the hand-eye camera, and a third relative posture between the local QR code and the global camera, the quantity to be calibrated includes a fourth relative posture between the hand-eye camera and the end of the robotic arm, a fifth relative posture between the local QR code and the end of the robotic arm, a sixth relative posture between the global camera and the base of the robotic arm, and a seventh relative posture between the acupuncture needle calibration column and the end of the robotic arm, the local QR code is arranged on the connecting structure between the acupuncture needle and the end of the robotic arm, the QR code calibration group and the acupuncture needle calibration column are arranged on a multi-target calibration plate, and the end of the acupuncture needle is fixed in the groove of the acupuncture needle calibration column.

[0014] Furthermore, in one embodiment of the present invention, the step of processing the joint angle data, the first image information and the second image information based on the kinematic model and the two-dimensional code posture reconstruction technology specifically includes:

[0015] Processing the joint angle data based on a kinematic model to obtain the first relative posture;

[0016] Processing the first image information based on a two-dimensional code posture reconstruction technology to obtain the second relative posture;

[0017] The second image information is processed based on a two-dimensional code posture reconstruction technology to obtain the third relative posture.

[0018] Further, in one embodiment of the present invention, the calibration objective function includes a first objective function, a second objective function and a third objective function, and the first objective function is:

[0019]

[0020] The second objective function is:

[0021]

[0022] The third objective function is:

[0023]

[0024] in, represents the first relative pose of the i-th group of measured pose relationships, represents the first relative pose of the jth set of measured pose relationships, represents the second relative pose of the i-th group of measured pose relationships, represents the third relative posture of the i-th group of measured posture relationships, T Z represents the fourth relative posture, T X represents the fifth relative posture, T W represents the sixth relative posture, t x represents the seventh relative posture, T Y represents the eighth relative pose between the QR code calibration group and the global camera, i∈{1,2,…,n}, j∈{1,2,…,n}, i≠j, and n represents the total number of groups of measured pose relationships.

[0025] Further, in one embodiment of the present invention, the step of solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated specifically includes:

[0026] According to the first relative posture and the third relative posture, the first objective function is optimized and solved based on matrix vectorization and Kronecker product to obtain a fifth rotation matrix and a fifth translation vector of the fifth relative posture, a sixth rotation matrix and a sixth translation vector of the sixth relative posture, and then a fifth closed solution of the fifth relative posture is determined according to the fifth rotation matrix and the fifth translation vector, and a sixth closed solution of the sixth relative posture is determined according to the sixth rotation matrix and the sixth translation vector;

[0027] Substituting the fifth closed solution into the second objective function to obtain a fourth objective function, optimizing and solving the fourth objective function based on matrix vectorization and Kronecker product according to the second relative posture and the third relative posture, obtaining a fourth rotation matrix and a fourth translation vector of the fourth relative posture, an eighth rotation matrix and an eighth translation vector of the eighth relative posture, and then determining a fifth closed solution of the fourth relative posture according to the fourth rotation matrix and the fourth translation vector, and determining a sixth closed solution of the eighth relative posture according to the eighth rotation matrix and the eighth translation vector;

[0028] The third objective function is optimized and solved according to the reference rotation matrix, the reference translation vector and the first relative posture to obtain the seventh translation vector of the seventh relative posture, and then the seventh closed solution of the seventh relative posture is determined according to the seventh translation vector.

[0029] Furthermore, in one embodiment of the present invention, the optimization objective function is:

[0030]

[0031]

[0032]

[0033] Among them, X represents the optimization variable, L i (X) represents the error function of the i-th group of measured pose relationships, ||a||2 represents the 2-norm of a, ||a|| F represents the F-norm of a, λ1, λ2, λ3, λ4 and λ5 all represent preset weight coefficients, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, R X and t X Respectively represent T X The rotation matrix and translation vector, R Y and t Y Respectively represent T Y The rotation matrix and translation vector, R Z and t Z Respectively represent T Z The rotation matrix and translation vector, R W and t W Respectively represent T W The rotation matrix and translation vector, and Represent the reference rotation matrix and reference translation vector respectively.

[0034] Further, in one embodiment of the present invention, the iterative solution based on multi-objective synchronous optimization iteratively solves the optimization objective function to obtain the optimal solution of the quantity to be calibrated, which specifically includes:

[0035] Calculating the error gradient corresponding to each of the measured posture relationships according to the error function, and determining the full gradient of multiple groups of the measured posture relationships according to each of the error gradients;

[0036] Determine an aggregate gradient for each iteration according to the full gradient and the error gradient;

[0037] The optimization objective function is iteratively solved based on the stochastic gradient descent algorithm, and the optimization variable is updated according to the aggregated gradient until a preset number of iterations is reached to obtain an iterative solution of the optimization variable, and then the optimal solution of the quantity to be calibrated is determined according to the iterative solution.

[0038] In a second aspect, an embodiment of the present invention provides an acupuncture robot synchronous calibration system, comprising:

[0039] A calibration data acquisition module is used to obtain the joint angle data of the acupuncture robot arm, and obtain the first image information of the hand-eye camera and the second image information of the global camera;

[0040] A calibration data processing module, used for processing the joint angle data, the first image information and the second image information based on a kinematic model and a two-dimensional code posture reconstruction technology to obtain multiple groups of measured posture relationships;

[0041] A closed solution module is used to construct a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decouple the calibration objective function based on the closed solution method of the Kronecker product, and solve the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated;

[0042] An iterative solution module, used for constructing an optimization objective function according to the closed solution, iteratively solving the optimization objective function based on an iterative solution method of multi-objective synchronous optimization, obtaining an optimal solution for the quantity to be calibrated, and then determining a synchronous calibration result of the acupuncture robot according to the optimal solution;

[0043] Wherein, the acupuncture robot includes the acupuncture robotic arm, the hand-eye camera, the global camera and acupuncture needles.

[0044] In a third aspect, an embodiment of the present invention provides a synchronous calibration device for an acupuncture robot, comprising:

[0045] at least one processor;

[0046] at least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned acupuncture robot synchronous calibration method.

[0048] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned acupuncture robot synchronous calibration method.

[0049] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0050] The embodiment of the present invention obtains joint angle data of an acupuncture robot arm, and obtains first image information of a hand-eye camera and second image information of a global camera, and then processes the joint angle data, the first image information and the second image information based on a kinematic model and a two-dimensional code pose reconstruction technology to obtain multiple groups of measured pose relationships, and then constructs a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decouples the calibration objective function based on a closed solution method of the Kronecker product, and solves the calibration objective function step by step according to the measured pose relationship to obtain a closed solution of the quantity to be calibrated, and then constructs an optimization objective function based on the closed solution, and iteratively solves the optimization objective function based on an iterative solution of multi-objective synchronous optimization to obtain an optimal solution of the quantity to be calibrated, and then determines the synchronous calibration result of the acupuncture robot according to the optimal solution. The embodiment of the present invention constructs a calibration objective function and solves the calibration objective function step by step based on the closed solution method of the Kronecker product, which can make full use of the known measurement posture relationship to obtain a high-precision closed solution within a limited time, constructs an optimization objective function and iteratively solves the optimization objective function based on an iterative solution of multi-objective synchronous optimization, which can avoid the problems of low-order approximation and error propagation, improve the convergence speed and accuracy of the iterative solution, improve the accuracy of the multi-objective coupled calibration of the acupuncture robot, and thus improve the accuracy of the control of the acupuncture robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of the steps of the synchronous calibration method of an acupuncture robot provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of multi-target coupling calibration of an acupuncture robot provided by an embodiment of the present invention;

[0054] Figure 3A structural block diagram of a synchronous calibration system for an acupuncture robot provided in an embodiment of the present invention;

[0055] Figure 4 This is a structural block diagram of the synchronous calibration device for an acupuncture robot provided in an embodiment of the present invention.

[0056] Figure numerals: 1. Acupuncture robotic arm; 2. Hand-eye camera; 3. Local QR code; 4. Acupuncture needle; 5. Global camera; 6. Multi-target calibration plate; 61. QR code calibration group; 62. Acupuncture needle calibration column. DETAILED DESCRIPTION

[0057] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0058] In the description of the present invention, the meaning of "a plurality" is two or more than two. If there is a description of "a first" or "a second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art.

[0059] Reference Figure 1 The embodiment of the present invention provides a synchronous calibration method for an acupuncture robot, which specifically includes the following steps:

[0060] S101, obtaining joint angle data of the acupuncture robot arm, and obtaining first image information of the hand-eye camera and second image information of the global camera;

[0061] S102, processing the joint angle data, the first image information, and the second image information based on a kinematic model and a two-dimensional code posture reconstruction technology to obtain multiple groups of measured posture relationships;

[0062] S103, constructing a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decoupling the calibration objective function based on a closed solution method of the Kronecker product, and solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated;

[0063] S104, constructing an optimization objective function according to the closed solution, iteratively solving the optimization objective function based on the iterative solution of multi-objective synchronous optimization, obtaining the optimal solution of the quantity to be calibrated, and then determining the synchronous calibration result of the acupuncture robot according to the optimal solution;

[0064] Among them, the acupuncture robot includes an acupuncture robotic arm, a hand-eye camera, a global camera and an acupuncture needle.

[0065] like Figure 2 The figure shows a schematic diagram of an application scenario of the synchronous calibration method for an acupuncture robot provided by an embodiment of the present invention. It can be understood that {0} and {e} represent the base coordinate system and the end coordinate system of the acupuncture robot arm, respectively, {hec} represents the hand-eye camera coordinate system, {gc} represents the global camera coordinate system, {lat} represents the local QR code coordinate system, {gat} represents the coordinate system of the QR code calibration group, and {an} represents the end coordinate system of the acupuncture needle. Figure 2 Specific embodiments of the present invention are described in detail.

[0066] Further as an optional implementation, the measured posture relationship includes a first relative posture between the end of the robot arm and the base of the robot arm, a second relative posture between the QR code calibration group 61 and the hand-eye camera 2, and a third relative posture between the local QR code 3 and the global camera 5. The quantity to be calibrated includes a fourth relative posture between the hand-eye camera 2 and the end of the robot arm, a fifth relative posture between the local QR code 3 and the end of the robot arm, a sixth relative posture between the global camera 5 and the base of the robot arm, and a seventh relative posture between the acupuncture needle calibration column 62 and the end of the robot arm. The local QR code 3 is arranged on the connecting structure between the acupuncture needle 4 and the end of the robot arm, the QR code calibration group 61 and the acupuncture needle calibration column 62 are arranged on the multi-target calibration plate 6, and the end of the acupuncture needle 4 is fixed in the groove of the acupuncture needle calibration column 62.

[0067] Specifically, the first relative posture 0 T e , the second relative posture hec T gat , the third relative posture gc T lat 4. Relative Posture e T hec , Fifth relative posture e T lat , Sixth relative posture 0 T gc And the seventh relative posture e t an like Figure 2 shown.

[0068] As an optional implementation, the step of processing the joint angle data, the first image information and the second image information based on the kinematic model and the two-dimensional code posture reconstruction technology specifically includes:

[0069] S1021. Process the joint angle data based on the kinematic model to obtain a first relative posture;

[0070] S1022, processing the first image information based on a two-dimensional code posture reconstruction technology to obtain a second relative posture;

[0071] S1023. Process the second image information based on the two-dimensional code posture reconstruction technology to obtain a third relative posture.

[0072] Specifically, 0 T e , hec T gat , gc T lat To measure the posture relationship, 0 T e Based on the joint angle data, hec T gat Image acquisition based on hand-eye camera, gc T lat Based on the global camera image acquisition, e T hec , e T lat , 0 T gc as well as e t an In addition, the relative positions between the robot base of the acupuncture robot 1, the global camera 5 and the multi-target calibration plate 6 are fixed, and the end of the acupuncture needle 4 is fixed in the groove of the acupuncture needle calibration column 62 in different postures, that is, the eighth relative position between the QR code calibration group 61 and the global camera 5 gc T gat and the ninth relative position between the acupuncture needle calibration column 62 and the robotic arm base 0 t an is an unknown invariant.

[0073] Specifically, according to Figure 2 Multiple kinematic closed-chain relationships can be obtained, namely:

[0074]

[0075] 0 T e e T lat = 0 T gcgc T lat (2)

[0076]

[0077] For the convenience of expression, let: T W = 0 T gc 、T X = e T lat 、T Y = gc T gat 、T Z = e T hec ,t x = e t an 、T A = 0 T e 、T B = gc T lat 、T C = hec T gat In addition, the multi-target calibration of the acupuncture robot requires obtaining n sets of measurement data, that is, and (i=1,2,K,n). Therefore, equations (1)-(3) can be converted to:

[0078]

[0079]

[0080]

[0081] As an optional implementation, the calibration objective function includes a first objective function, a second objective function and a third objective function, the first objective function is the above formula (5), the second objective function is the above formula (4), and the third objective function is the above formula (6), wherein, represents the first relative pose of the i-th group of measured pose relationships, represents the first relative pose of the jth set of measured pose relationships, represents the second relative pose of the i-th group of measured pose relationships, represents the third relative posture of the i-th group of measured posture relationships, T Z represents the fourth relative posture, T X represents the fifth relative posture, T W represents the sixth relative posture, t x represents the seventh relative posture, TY represents the eighth relative pose between the QR code calibration group and the global camera, i∈{1,2,…,n}, j∈{1,2,…,n}, i≠j, and n represents the total number of groups of measured pose relationships.

[0082] Therefore, the multi-objective calibration problem of the acupuncture robot can be expressed as and The coupling of these three sub-problems. In addition, all three sub-problems involve the coupling of rotation matrices and position vectors.

[0083] Homogeneous transformation matrix ref T cur and It can be expressed as:

[0084]

[0085] Therefore, equations (4)-(6) can be decoupled and expressed as:

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] As an optional implementation, the calibration objective function is solved step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated, which specifically includes:

[0092] S1031. According to the first relative posture and the third relative posture, the first objective function is optimized and solved based on matrix vectorization and Kronecker product to obtain a fifth rotation matrix and a fifth translation vector of a fifth relative posture, and a sixth rotation matrix and a sixth translation vector of a sixth relative posture, and then a fifth closed solution of the fifth relative posture is determined according to the fifth rotation matrix and the fifth translation vector, and a sixth closed solution of the sixth relative posture is determined according to the sixth rotation matrix and the sixth translation vector;

[0093] S1032, substituting the fifth closed solution into the second objective function to obtain a fourth objective function, optimizing and solving the fourth objective function based on matrix vectorization and Kronecker product according to the second relative posture and the third relative posture, obtaining a fourth rotation matrix and a fourth translation vector of the fourth relative posture, an eighth rotation matrix and an eighth translation vector of the eighth relative posture, and then determining a fifth closed solution of the fourth relative posture according to the fourth rotation matrix and the fourth translation vector, and determining a sixth closed solution of the eighth relative posture according to the eighth rotation matrix and the eighth translation vector;

[0094] S1033. Optimize and solve the third objective function according to the reference rotation matrix, the reference translation vector and the first relative posture to obtain the seventh translation vector of the seventh relative posture, and then determine the seventh closed solution of the seventh relative posture according to the seventh translation vector.

[0095] Specifically, first, based on matrix vectorization and Kronecker product, we solve The rotation matrix of the problem. Formula (10) can be vectorized as:

[0096]

[0097] Here, vec(·) represents the matrix vectorization operator.

[0098] Formula (13) can be converted into:

[0099]

[0100] Among them, I 3×3 represents the third-order identity matrix, Represents the symbol for the Kronecker product of matrices.

[0101] The matrix form of formula (14) can be expressed as:

[0102]

[0103] For n sets of measurement data, formula (15) can be expanded to:

[0104]

[0105] Since the measured data generally has errors, equation (16) is usually not true. At this time, the variable x needs to be optimized to minimize the 2-norm error of Ax. In addition, since the rotation matrix is ​​an orthogonal matrix and x is composed of two vectorized rotation matrices, the 2-norm of x has an equality constraint. Therefore, a constrained optimization problem about x is established, that is:

[0106]

[0107] Defining variables and Therefore, problem (17) can be transformed into:

[0108]

[0109] Since the matrix is a symmetric matrix, so its Rayleigh quotient can be expressed as:

[0110]

[0111] Assume λ min and q min Respectively The minimum eigenvalue and the corresponding unit eigenvector of . According to the relevant theorem of Rayleigh quotient, The minimum value of λ min , if and only if Therefore, the optimal solution to the optimization problem (18) is

[0112] In particular, similar to the closed-form solution based on quaternions, the closed-form solution based on Kronecker products also has the problem of sign ambiguity, that is, the equation is still valid. Differently, the sign ambiguity of the former is reflected in the conversion of the rotation matrix of the measured data to the quaternion, so the number of ambiguous cases is related to the amount of measured data, showing an exponential growth trend. However, the sign ambiguity of the latter is reflected in the rotation matrix to be calibrated, so there are only two ambiguous cases, which is independent of the amount of measured data. Since the determinant of the rotation matrix is ​​equal to 1, according to the related properties of the determinant of the square matrix, the sign ambiguity problem can be solved by judging the determinant of the matrix: if the determinant is equal to -1, then the closed solutions of the fifth rotation matrix and the sixth rotation matrix are R X and R W ; If the determinant is equal to -1, then the closed solutions of the fifth rotation matrix and the sixth rotation matrix are -R X and -R W .

[0113] Secondly, solve The translation vector of the problem. The matrix form of formula (11) can be expressed as:

[0114]

[0115] For n sets of data, formula (20) can be expanded to:

[0116]

[0117] Similarly, according to formula (21), an unconstrained optimization problem about y is established, that is:

[0118]

[0119] According to the relevant lemma of generalized inverse matrix, the optimal solution of problem (22) is y * =(B T B) -1 B T b, thereby obtaining the fifth translation vector t X and the sixth translation vector t W So far, T can be determined based on the fifth rotation matrix, the fifth translation vector, the sixth rotation matrix and the sixth translation vector W and T X The closed solution of .

[0120] Then, T X Substituting the closed-form solution into equation (4), The problem can be transformed into one with The problem is similar to the two unknown quantity problem, so the above method is used to obtain T Y and T Z The closed form solution of , the embodiment of the present invention will not be described in detail here.

[0121] Finally, solve The translation vector of the problem. Formula (12) can be converted to:

[0122]

[0123] For n sets of data, formula (23) can be expanded to:

[0124]

[0125] in, Represents the rotation matrix and translation vector of the reference.

[0126] Similarly, according to formula (24), we establish a x unconstrained optimization problem and obtain its closed-form solution.

[0127] After obtaining the closed-form solution of the quantity to be calibrated, it is necessary to construct the optimization objective function based on the closed-form solution and perform iterative solution.

[0128] First, define the optimization variable X = (R W ,R X ,R Y ,R Z ,t W ,t X ,t Y ,t Z ,t x). In addition, according to equations (8)-(12), a multi-objective synchronous optimization error function is designed. The error function of the i-th group of measurement data can be expressed as:

[0129]

[0130]

[0131] Among them, λ1,λ2,λ3,λ4,λ5 represent weight coefficients.

[0132] At this time, the variable X needs to be optimized to minimize the error function (25). In addition, the rotation matrix has an orthogonal constraint. Therefore, a constrained optimization problem about X is established, that is:

[0133]

[0134] As an optional implementation, the optimization objective function can be expressed by the above equations (25), (26) and (27), where X represents the optimization variable, L represents i (X) represents the error function of the i-th group of measured pose relationships, ||a||2 represents the 2-norm of a, ||a|| F represents the F-norm of a, λ1, λ2, λ3, λ4 and λ5 all represent preset weight coefficients, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, R X and t X Respectively represent T X The rotation matrix and translation vector, R Y and t Y Respectively represent T Y The rotation matrix and translation vector, R Z and t Z Respectively represent T Z The rotation matrix and translation vector, R W and t W Respectively represent T W The rotation matrix and translation vector, and Represent the reference rotation matrix and reference translation vector respectively.

[0135] Specifically, problem (27) has nonlinear constraints, and it is difficult to find the optimal solution directly. The penalty function method is used to convert the constraints into penalty terms and add them to the original error function. Thus, the error function can be converted to:

[0136]

[0137]

[0138] Among them, σ1, σ2, σ3, σ4 represent penalty factors.

[0139] Formula (28) can be simplified as:

[0140]

[0141] Here, tr(·) represents the trace of the matrix.

[0142] Finally, problem (27) can be transformed into an unconstrained nonlinear optimization problem, that is:

[0143]

[0144] Since the F-norm of the matrix and the 2-norm of the vector are both strictly convex functions, problem (31) is a strictly convex optimization problem. In addition, the error function (30) is differentiable.

[0145] As an optional implementation, the optimization objective function is iteratively solved based on the iterative solution of multi-objective synchronous optimization to obtain the optimal solution of the quantity to be calibrated, which specifically includes:

[0146] S1041, calculating the error gradient corresponding to each measured posture relationship according to the error function, and determining the full gradient of multiple groups of measured posture relationships according to each error gradient;

[0147] S1042, determining the aggregate gradient of each iteration according to the full gradient and the error gradient;

[0148] S1043. Iteratively solve the optimization objective function based on the stochastic gradient descent algorithm, and update the optimization variables according to the aggregated gradient until a preset number of iterations is reached to obtain an iterative solution of the optimization variables, and then determine the optimal solution of the quantity to be calibrated according to the iterative solution.

[0149] Specifically, first, the gradient of the error function (30) is calculated. The gradient of the i-th set of measurement data can be expressed as:

[0150]

[0151] According to the relevant properties of matrix differentiation, the components of the gradient (32) can be expressed as:

[0152]

[0153] Assume t max represents the number of iterations of the multi-target synchronous calibration iterative solution, X t and α t They represent the optimization variable (the initial value of iteration is X0) and the learning rate at time t, where t=1,2,…,t max Therefore, the full gradient of all measured data at time t can be expressed as:

[0154]

[0155] Each iteration will perform m random gradient descents. Assume that the jth (j = 1, 2, ..., m) random gradient descent selects the ith j Group measurement data, represents the optimization variable of the jth stochastic gradient descent. Therefore, at time t, the aggregate gradient of the jth stochastic gradient descent can be expressed as:

[0156]

[0157] Finally, the optimization variable update equation for the jth stochastic gradient descent can be expressed as:

[0158]

[0159] The optimization objective function is iteratively solved based on the stochastic gradient descent algorithm, and the optimization variables are updated according to the aggregated gradient until the preset number of iterations t is reached. max , the iterative solution of the optimization variable can be obtained, and then the optimal solution of the quantity to be calibrated can be determined according to the iterative solution, so that the multi-objective synchronous calibration of the acupuncture robot can be completed.

[0160] The above describes in detail the step flow of the embodiment of the present invention. It can be understood that the embodiment of the present invention constructs a calibration objective function and solves the calibration objective function step by step based on the closed solution method of the Kronecker product, which can make full use of the known measurement posture relationship to obtain a high-precision closed solution within a limited time, construct an optimization objective function and iteratively solve the optimization objective function based on the iterative solution of multi-objective synchronous optimization, which can avoid the problems of low-order approximation and error propagation, improve the convergence speed and accuracy of the iterative solution, improve the accuracy of the multi-objective coupling calibration of the acupuncture robot, and thus improve the accuracy of the acupuncture robot control.

[0161] Reference Figure 3 The embodiment of the present invention provides a synchronous calibration system for an acupuncture robot, comprising:

[0162] A calibration data acquisition module is used to obtain the joint angle data of the acupuncture robot arm, and obtain the first image information of the hand-eye camera and the second image information of the global camera;

[0163] A calibration data processing module is used to process the joint angle data, the first image information and the second image information based on the kinematic model and the QR code posture reconstruction technology to obtain multiple groups of measured posture relationships;

[0164] The closed solution module is used to construct a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decouple the calibration objective function based on the closed solution method of the Kronecker product, and solve the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated;

[0165] An iterative solution module is used to construct an optimization objective function based on a closed solution, iteratively solve the optimization objective function based on an iterative solution method of multi-objective synchronous optimization, obtain the optimal solution of the quantity to be calibrated, and then determine the synchronous calibration result of the acupuncture robot based on the optimal solution;

[0166] Among them, the acupuncture robot includes an acupuncture robotic arm, a hand-eye camera, a global camera and an acupuncture needle.

[0167] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0168] Reference Figure 4 The embodiment of the present invention provides a synchronous calibration device for an acupuncture robot, comprising:

[0169] at least one processor;

[0170] at least one memory for storing at least one program;

[0171] When the at least one program is executed by the at least one processor, the at least one processor implements the acupuncture robot synchronous calibration method.

[0172] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0173] An embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned acupuncture robot synchronous calibration method.

[0174] A computer-readable storage medium according to an embodiment of the present invention can execute a synchronous calibration method for an acupuncture robot provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0175] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0176] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0177] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0178] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0179] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0180] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0181] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0182] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0183] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0184] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A synchronous calibration method for an acupuncture robot, characterized in that: The following steps are involved: Acquire the joint angle data of the acupuncture robot arm, and acquire the first image information of the hand-eye camera and the second image information of the global camera; Processing the joint angle data, the first image information, and the second image information based on a kinematic model and a two-dimensional code posture reconstruction technology to obtain multiple sets of measured posture relationships; Constructing a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decoupling the calibration objective function based on a closed solution method of the Kronecker product, and solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated; An optimization objective function is constructed according to the closed solution, and the optimization objective function is iteratively solved based on an iterative solution method of multi-objective synchronous optimization to obtain an optimal solution for the quantity to be calibrated, and then a synchronous calibration result of the acupuncture robot is determined according to the optimal solution; Wherein, the acupuncture robot includes the acupuncture robotic arm, the hand-eye camera, the global camera and acupuncture needles.

2. The synchronous calibration method of an acupuncture robot according to claim 1, characterized in that: The measured posture relationship includes a first relative posture between the end of the robot arm and the base of the robot arm, a second relative posture between the QR code calibration group and the hand-eye camera, and a third relative posture between the local QR code and the global camera. The quantity to be calibrated includes a fourth relative posture between the hand-eye camera and the end of the robot arm, a fifth relative posture between the local QR code and the end of the robot arm, a sixth relative posture between the global camera and the base of the robot arm, and a seventh relative posture between the acupuncture needle calibration column and the end of the robot arm. The local QR code is arranged on the connecting structure between the acupuncture needle and the end of the robot arm, the QR code calibration group and the acupuncture needle calibration column are arranged on a multi-target calibration plate, and the end of the acupuncture needle is fixed in the groove of the acupuncture needle calibration column.

3. The synchronous calibration method of an acupuncture robot according to claim 2, characterized in that: The step of processing the joint angle data, the first image information and the second image information based on the kinematic model and the two-dimensional code posture reconstruction technology specifically includes: Processing the joint angle data based on a kinematic model to obtain the first relative posture; The first image information is processed based on the two-dimensional code posture reconstruction technology to obtain the second relative posture; the second image information is processed based on the two-dimensional code posture reconstruction technology to obtain the third relative posture.

4. The synchronous calibration method of an acupuncture robot according to claim 2, characterized in that: The calibration objective function includes a first objective function, a second objective function and a third objective function, wherein the first objective function is: The second objective function is: The third objective function is: in, represents the first relative pose of the i-th group of measured pose relationships, represents the first relative pose of the jth set of measured pose relationships, represents the second relative pose of the i-th group of measured pose relationships, represents the third relative posture of the i-th group of measured posture relationships, T Z represents the fourth relative posture, T X represents the fifth relative posture, T W represents the sixth relative posture, t x represents the seventh relative posture, T Y represents the eighth relative pose between the QR code calibration group and the global camera, i∈ {1,2,…,n}, j∈{1,2,…,n}, i≠j, n represents the total number of groups of measured posture relationships.

5. The synchronous calibration method of an acupuncture robot according to claim 4, characterized in that: The step of solving the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated specifically includes: According to the first relative posture and the third relative posture, the first objective function is optimized and solved based on matrix vectorization and Kronecker product to obtain a fifth rotation matrix and a fifth translation vector of the fifth relative posture, a sixth rotation matrix and a sixth translation vector of the sixth relative posture, and then a fifth closed solution of the fifth relative posture is determined according to the fifth rotation matrix and the fifth translation vector, and a sixth closed solution of the sixth relative posture is determined according to the sixth rotation matrix and the sixth translation vector; Substituting the fifth closed solution into the second objective function to obtain a fourth objective function, optimizing and solving the fourth objective function based on matrix vectorization and Kronecker product according to the second relative posture and the third relative posture, obtaining a fourth rotation matrix and a fourth translation vector of the fourth relative posture, an eighth rotation matrix and an eighth translation vector of the eighth relative posture, and then determining a fifth closed solution of the fourth relative posture according to the fourth rotation matrix and the fourth translation vector, and determining a sixth closed solution of the eighth relative posture according to the eighth rotation matrix and the eighth translation vector; The third objective function is optimized and solved according to the reference rotation matrix, the reference translation vector and the first relative posture to obtain the seventh translation vector of the seventh relative posture, and then the seventh closed solution of the seventh relative posture is determined according to the seventh translation vector.

6. The synchronous calibration method of an acupuncture robot according to claim 4, characterized in that: The optimization objective function is: Among them, X represents the optimization variable, L i (X) represents the error function of the i-th group of measured pose relationships, ||a||2 represents the 2-norm of a, ||a|| F represents the F-norm of a, λ1, λ2, λ3, λ4 and λ5 all represent preset weight coefficients, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, and Respectively The rotation matrix and translation vector, R X and t X Respectively represent T X The rotation matrix and translation vector, R Y and t Y Respectively represent T Y The rotation matrix and translation vector, R Z and t Z Respectively represent T Z The rotation matrix and translation vector, R W and t W Respectively represent T W The rotation matrix and translation vector, and Represent the reference rotation matrix and reference translation vector respectively.

7. The synchronous calibration method of an acupuncture robot according to claim 6, characterized in that: The step of iteratively solving the optimization objective function based on the iterative solution of multi-objective synchronous optimization to obtain the optimal solution of the quantity to be calibrated specifically includes: Calculating the error gradient corresponding to each of the measured posture relationships according to the error function, and determining the full gradient of multiple groups of the measured posture relationships according to each of the error gradients; Determine an aggregate gradient for each iteration according to the full gradient and the error gradient; The optimization objective function is iteratively solved based on the stochastic gradient descent algorithm, and the optimization variable is updated according to the aggregated gradient until a preset number of iterations is reached to obtain an iterative solution of the optimization variable, and then the optimal solution of the quantity to be calibrated is determined according to the iterative solution.

8. A synchronous calibration system for an acupuncture robot, characterized in that: include: A calibration data acquisition module is used to obtain the joint angle data of the acupuncture robot arm, and obtain the first image information of the hand-eye camera and the second image information of the global camera; A calibration data processing module, used for processing the joint angle data, the first image information and the second image information based on a kinematic model and a two-dimensional code posture reconstruction technology to obtain multiple groups of measured posture relationships; A closed solution module is used to construct a calibration objective function according to the quantity to be calibrated of the acupuncture robot, decouple the calibration objective function based on the closed solution method of the Kronecker product, and solve the calibration objective function step by step according to the measured posture relationship to obtain a closed solution of the quantity to be calibrated; An iterative solution module, used for constructing an optimization objective function according to the closed solution, iteratively solving the optimization objective function based on an iterative solution method of multi-objective synchronous optimization, obtaining an optimal solution for the quantity to be calibrated, and then determining a synchronous calibration result of the acupuncture robot according to the optimal solution; Wherein, the acupuncture robot includes the acupuncture robotic arm, the hand-eye camera, the global camera and acupuncture needles.

9. A synchronous calibration device for an acupuncture robot, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the acupuncture robot synchronous calibration method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute an acupuncture robot synchronous calibration method as described in any one of claims 1 to 7 when executed by the processor.

Images