Coordinate self-calibration method, system and media of ultrasound-guided dual-arm puncture robot

Abstract

This invention discloses a coordinate self-calibration method, system, and medium for an ultrasound-guided dual-arm puncture robot, belonging to the field of surgical robots. The method includes: S1, acquiring multiple sets of pose information of the robot and corresponding ultrasound images; S2, constructing a pose closed-loop homogeneous matrix transformation model of the robot: AXP = YBQ; where X is the homogeneous transformation matrix from the first robotic arm end-effector coordinate system {E1} to the ultrasound image coordinate system {I1}, Y is the homogeneous transformation matrix from the first robotic arm base coordinate system {O1} to the second robotic arm base coordinate system {O2}, Q is the homogeneous coordinate of the needle tip in the second robotic arm end-effector coordinate system {E2}, and P is the homogeneous coordinate of the surgical puncture needle tip in the ultrasound image corresponding to each set of pose information; S3, decomposing and linearizing the model using a block-based approach to obtain closed-loop solutions of the homogeneous transformation matrices X, Y, and Q, and further optimizing the accuracy through an iterative algorithm to complete the final calibration. This invention can obtain high-precision calibration results with less calibration time.

Description

Technical Field

[0001] This invention belongs to the field of surgical robots, and more specifically, relates to a coordinate self-calibration method, system and medium for an ultrasound-guided dual-arm puncture robot. Background Technology

[0002] With the gradual maturation of surgical robot technology, research on using robots to replace doctors in performing puncture surgery has been widely carried out. Ultrasound-guided robotic puncture surgery can effectively improve the efficiency and accuracy of surgery while reducing labor costs.

[0003] Current methods for calibrating surgical puncture needle tips require additional measuring equipment (such as binocular cameras, depth cameras, etc.) to identify the poses of the two robotic arms, which generates cumulative errors during the calibration process; in addition, introducing additional equipment at the surgical site makes the process more complicated; at the same time, current calibration methods are difficult to perform intraoperative calibration and have limited efficiency. Summary of the Invention

[0004] In view of the shortcomings and improvement needs of the existing technology, the present invention provides a coordinate self-calibration method, system and medium for an ultrasound-guided dual-arm puncture robot. Its purpose is to achieve the calibration of the surgical puncture needle tip without the introduction of additional equipment, and to reduce the calibration complexity and improve the calibration efficiency.

[0005] To achieve the above objectives, according to one aspect of the present invention, a coordinate self-calibration method for an ultrasound-guided dual-arm puncture robot is provided. The robot includes a first robotic arm, a second robotic arm, an ultrasound probe, and a surgical puncture needle. An ultrasound image is generated through the distal end of the ultrasound probe. The method includes:

[0006] S1. Collect multiple sets of pose information and corresponding ultrasound images of the robot; wherein, each set of pose information includes: homogeneous transformation matrix A from the first robot arm base coordinate system {O1} to its end coordinate system {E1}, and homogeneous transformation matrix B from the second robot arm base coordinate system {O2} to its end coordinate system {E2}.

[0007] S2. Construct the pose closed-loop homogeneous matrix transformation model of the robot: AXP = YBQ; where X is the homogeneous transformation matrix from the first robotic arm end-effector coordinate system {E1} to the ultrasound image coordinate system {I1}, Y is the homogeneous transformation matrix from the first robotic arm base coordinate system {O1} to the second robotic arm base coordinate system {O2}, Q is the homogeneous coordinate of the tip of the surgical puncture needle in the second robotic arm end-effector coordinate system {E2}, and P is the homogeneous coordinate of the tip of the surgical puncture needle in the ultrasound image coordinate system {I1} corresponding to each set of pose information.

[0008] S3. Decompose and linearize the model using a block-based approach to obtain closed-form solutions for the homogeneous transformation matrices X, Y, and Q, thus completing the calibration.

[0009] Furthermore, S3 includes:

[0010] S31. The model is decomposed into equations by using a block-based approach, based on rotation matrices and translation vectors.

[0011] S32. Linearize the decomposed equations to obtain the corresponding linear matrix equations;

[0012] S33. Substitute each set of pose information and the corresponding ultrasound image data into the linear matrix equation to obtain a composite linear matrix equation system.

[0013] S34. Solve the system of equations of the composite linear matrix using the least squares method to obtain closed solutions of the homogeneous transformation matrices X, Y and Q.

[0014] Furthermore, in S32, the decomposed equation is linearized by simultaneously performing matrix straightening operations on both sides of the equation and then exchanging parameters through the Kronecker product.

[0015] Furthermore, it also includes:

[0016] Using the closed solution as the initial value, an objective function is constructed. The objective function is to minimize the error between the coordinates of the puncture needle tip in the second robotic arm base coordinate system {O2} and the coordinates of the puncture needle tip in the second robotic arm base coordinate system {O2} calculated by ultrasound image.

[0017] The objective function is solved using an iterative method to obtain the iterative solution corresponding to the closed solution.

[0018] Furthermore, the objective function is:

[0019]

[0020] Where n is the number of sets of acquired pose information and corresponding ultrasound images, F(R) X R Y , t X , t Y , t Q The expression for ) is:

[0021] F(R X R Y , t X , t Y , t Q ) = R Y R B t Q +R Y t B +tY -R A R X t P -R A t X -t A In the formula, R A R B R X R Y These are the rotation matrices of the homogeneous transformation matrices A, B, X, and Y, respectively, and t A , t B , t X , t Y These are the translation vectors of the homogeneous transformation matrices A, B, X, and Y, respectively, and t P and t Q These are the translation vectors of the homogeneous coordinates P and Q, respectively.

[0022] Furthermore, the iterative method includes:

[0023] Within the Lie algebra range, for F(R) X R Y , t X , t Y , t Q Find the differential to obtain the differential equation:

[0024] dF=R A R X [δω X ] ∧ t P +R A δt X -δt Y -R Y [δω Y ] ∧ (R B t Q +t B )-R Y R B δt Q

[0025] Where, ω X and ω Y Representing the rotation matrix R respectively X and R Y The transformation relationship between the rotation vector, the rotation matrix R, and the rotation vector ω is: R = exp([ω]). ∧ );[·] ∧ Indicates the vector antisymmetry operator;

[0026] Design the update rate Δ of the Gauss-Newton method:

[0027]

[0028] in, The expression for G is:

[0029]

[0030] Using the aforementioned update rate, Gaussian-Newton iterations are performed to obtain the iterative solution corresponding to the closed solution.

[0031] Furthermore, S1 includes:

[0032] S11. Within the shared workspace of the two robotic arms, the first and second robotic arms are moved to make the puncture needle visible in the ultrasound image.

[0033] S12. Record the pose information of the two robotic arms at the current moment and the corresponding ultrasound image, wherein the pose information of the two robotic arms at the current moment is represented by the homogeneous transformation matrix A and the homogeneous transformation matrix B, respectively.

[0034] S13. Mark the distance coordinates of the needle tip and the direction vector v of the needle in the ultrasound image acquired at the current moment, wherein the distance coordinates of the needle tip are represented by the homogeneous coordinates P.

[0035] S14. Repeat S11-S13 to obtain multiple sets of pose information and corresponding ultrasound images.

[0036] Furthermore, it also includes: calculating the puncture direction V of the puncture needle, specifically including:

[0037] Based on the transitive relation of the rotation matrix, the following linear equation is constructed:

[0038] (R A R X ) -1 R Y R B V = v

[0039] Substituting the obtained homogeneous transformation matrices X, Y, and Q, along with the corresponding sets of data, into the linear equations yields a system of composite linear equations.

[0040] The puncture direction V of the puncture needle is obtained by solving the composite linear equations using the least squares method.

[0041] According to another aspect of the present invention, a coordinate self-calibration system for an ultrasound-guided dual-arm puncture robot is provided, for performing the coordinate self-calibration method for an ultrasound-guided dual-arm puncture robot as described in any of the first aspects, comprising:

[0042] The data acquisition module is used to acquire multiple sets of pose information of the robot and corresponding ultrasound images; wherein, each set of pose information includes: homogeneous transformation matrix A from the first robot arm base coordinate system {O1} to its end coordinate system {E1}, and homogeneous transformation matrix B from the second robot arm base coordinate system {O2} to its end coordinate system {E2}.

[0043] The model building module is used to construct the pose closed-loop homogeneous matrix transformation model of the robot: AXP = YBQ; where X is the homogeneous transformation matrix from the first robotic arm end-effector coordinate system {E1} to the ultrasonic image coordinate system {I1}, Y is the homogeneous transformation matrix from the first robotic arm base coordinate system {O1} to the second robotic arm base coordinate system {O2}, Q is the homogeneous coordinate of the surgical puncture needle tip in the second robotic arm end-effector coordinate system {E2}, and P is the homogeneous coordinate of the surgical puncture needle tip in the ultrasonic image coordinate system {I1} corresponding to each set of pose information.

[0044] The model solving module is used to decompose and linearize the model in a block-based manner, obtain closed solutions of the homogeneous transformation matrices X, Y and Q, and complete the calibration.

[0045] According to another aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the first aspects.

[0046] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0047] (1) By directly using the ultrasonic probe and surgical puncture needle of the ultrasonic-guided dual-arm puncture robot to collect multiple sets of pose information of the robotic arm and the corresponding ultrasonic image data, based on the collected data, a pose closed-loop homogeneous matrix transformation model of the ultrasonic-guided dual-arm puncture robot is constructed. The corresponding coordinate transformation relationship and the homogeneous coordinates of the tip of the surgical puncture needle are obtained by solving the model. No external observation equipment or additional calibration tools are needed. The high-precision calibration of the system can be completed using only the conventional configuration of the ultrasonic-guided dual-arm puncture robot, which can reduce the calibration complexity and improve the calibration efficiency.

[0048] (2) Further, by simultaneously performing matrix straightening operations on both sides of the decomposed equation and then exchanging parameters through the Kronecker product, the parameters in the decomposed nonlinear equation will be decoupled, thereby linearizing the nonlinear equation and facilitating the subsequent solution of the closed solution.

[0049] (3) Furthermore, the present invention also provides a specific solution method for the pose closed-loop homogeneous matrix transformation model of a robot; at the same time, by using the obtained initial solution as the initial value to construct the objective function and solve the corresponding iterative solution, the noise during data acquisition and the error caused by the measurement error of the device itself can be reduced, thereby further improving the calibration accuracy.

[0050] (4) Furthermore, in the process of iteratively solving the iterative solution using the Gauss-Newton method, the update rate is designed in combination with the Lie algebra method, and the rotation vector is directly used for iterative update. In each iteration, the rotation matrix obtained by the rotation vector can ensure that the rotation matrix is ​​always orthogonal, which avoids the problem of introducing additional errors by orthogonalizing the results in the existing numerical iteration method, and further improves the accuracy of the calibration results.

[0051] Meanwhile, the update rate of this invention is used directly to calculate the iteration direction and step size, eliminating the need to search for the optimal step size through loops, thus improving the iteration convergence speed and iteration efficiency.

[0052] (5) The data acquisition method of the present invention directly uses existing surgical instruments to complete the acquisition of calibration data in a one-time acquisition process. That is, the data acquisition directly uses the ultrasound probe and surgical puncture instruments used in the operation, without changing the configuration of the surgical site for calibration, such as changing the device at the end of the robotic arm. Furthermore, the data acquisition is a simple repetitive action process. The calibration data can be completed by repeating this simple action several times at once. The acquired data is more accurate and the acquisition method is simpler. Attached Figure Description

[0053] Figure 1 This is a flowchart of the coordinate self-calibration method for the ultrasound-guided dual-arm puncture robot of the present invention.

[0054] Figure 2 This is a schematic diagram of the coordinate systems and their transformation relationships for a dual-arm puncture robot. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0056] In this invention, the terms "first," "second," etc., used in the invention and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0057] like Figure 1and Figure 2 As shown, the ultrasound-guided dual-arm puncture robot of the present invention includes: a first robotic arm, a second robotic arm, a medical ultrasound detection instrument (ultrasound probe), and a surgical puncture needle. The ultrasound probe is connected to the end of the first robotic arm, and the surgical puncture needle is connected to the end of the second robotic arm. An ultrasound image is generated through the end of the ultrasound probe. The two-dimensional ultrasound image generated by the end of the ultrasound probe can be regarded as a coordinate plane attached to an image coordinate system {I1}. In this embodiment of the present invention, the two-dimensional ultrasound image is set to coincide with the XY plane. The medical ultrasound detection instrument is an ultrasound instrument capable of acquiring the actual distance coordinate information of the ultrasound image, such as convex array ultrasound, linear ultrasound, transrectal ultrasound, etc.

[0058] The coordinate self-calibration method includes the following steps:

[0059] S1. Acquire multiple sets of pose information of the ultrasound-guided dual-arm puncture robot and the ultrasound image corresponding to each set of pose information; wherein, each set of pose information includes: homogeneous transformation matrix A from the first robotic arm base coordinate system {O1} to the first robotic arm end coordinate system {E1}, and homogeneous transformation matrix B from the second robotic arm base coordinate system {O2} to the second robotic arm end coordinate system {E2}.

[0060] S2. Construct a closed-loop homogeneous matrix transformation model for the pose of an ultrasound-guided dual-arm puncture robot: AXP = YBQ; where X is the homogeneous transformation matrix from the first robotic arm end-effector coordinate system {E1} to the ultrasound image coordinate system {I1}, Y is the homogeneous transformation matrix from the first robotic arm base coordinate system {O1} to the second robotic arm base coordinate system {O2}, Q is the homogeneous coordinate of the surgical puncture needle tip in the second robotic arm end-effector coordinate system {E2}, and P is the homogeneous coordinate of the surgical puncture needle tip in the ultrasound image coordinate system {I1} corresponding to each set of pose information.

[0061] S3. Decompose and linearize the constructed model using a block-based approach to obtain the closed-form solution of the model, that is, the closed-form solution of X, Y and Q.

[0062] Specifically, S1 includes:

[0063] S11. Within the shared workspace of the two robotic arms, the first and second robotic arms are moved so that the puncture needle can be clearly visualized in the ultrasound image.

[0064] S12. Record the pose information of the two robotic arms at the current moment and the corresponding ultrasound image. The pose information of the two robotic arms at the current moment is represented by homogeneous transformation matrix A and homogeneous transformation matrix B, respectively.

[0065] S13. Mark the distance coordinates of the needle tip and the direction vector of the puncture needle in the ultrasound image acquired at the current moment. The distance coordinates of the needle tip are represented by homogeneous coordinates P, and the direction vector of the puncture needle is denoted as vector v. In this embodiment of the invention, the Z-axis components of homogeneous coordinates P and vector v are 0.

[0066] S14. Repeat steps S11-S13 to obtain multiple sets of pose information and the corresponding ultrasound images for each set of pose information.

[0067] Preferably, in S14, at least sixteen sets of pose information and corresponding ultrasound images are collected; wherein, during the acquisition of each set of data, the posture of the robotic arm is not on the same axis.

[0068] Specifically, S3 includes:

[0069] S31. The constructed model is decomposed into rotation matrices and translation vectors using a block-based approach, as shown in the following equation:

[0070] R A R X t P +R A t X +t A =R Y R B t Q +R Y t B +t Y (1)

[0071] Among them, R A R B R X R Y These are the rotation matrices of the homogeneous transformation matrices A, B, X, and Y, respectively, and t A , t B , t X , t Y These are the translation vectors of the homogeneous transformation matrices A, B, X, and Y, respectively, and t P and t Q These are the translation vectors of the homogeneous coordinates P and Q, respectively.

[0072] S32. Based on the properties of the Kronecker product and matrix straightening, formula (1) is linearized. Specifically, since formula (1) is a typical nonlinear equation, each term has a strong coupling relationship, making it difficult to directly obtain a closed solution. To solve this problem, this invention performs matrix straightening operations on both sides of formula (1) simultaneously, and then uses the properties of the Kronecker product to perform parameter exchange, decoupling the parameters and converting it into a linear equation.

[0073] Matrix straightening is performed simultaneously on both sides of formula (1), and parameters are exchanged using the properties of the Kronecker product to decouple the parameters. The decoupled form is shown in formula (2) below:

[0074]

[0075] Where vec is the matrix straightening operator. Let E represent the Kronecker product, where E is the identity matrix.

[0076] Equation (2) can be expressed as a linear equation as follows:

[0077] M ABP m XYQ =t A (3)

[0078] Among them, M ABP It is based on the known quantity R A t A R B t B and t P The coefficient matrix formed, m XYQ It is composed of unknown quantity R X t X R Y t Y and t Q The matrix to be solved. Specifically, M ABP The expression is:

[0079]

[0080] m XYQ The expression is:

[0081]

[0082] S33. For each set of pose information and corresponding ultrasound image data, a linear matrix equation as shown in formula (3) can be established. The matrix equations formed by each set of data can be arranged into a composite linear equation system as shown in formula (4) below:

[0083]

[0084] In this context, the "~" symbol on a parameter represents a combination of parameters corresponding to multiple sets of data.

[0085] S34. Based on the least squares method, and using the Kronecker product and the inverse transformation of matrix straightening, a closed-form solution for X, Y, and Q can be obtained; specifically, the least squares solution of this system of linear equations is:

[0086]

[0087] The least-squares solution of the linear equation system is obtained by using the Kronecker product and the inverse transformation of matrix straightening, thus yielding closed-form solutions for X, Y, and Q.

[0088] Specifically, during the solution process of the model constructed in this invention, due to t P In an embodiment of the invention, t is 0 for one of its components. P The Z-axis component is 0, making R X The column vector corresponding to the Z-axis cannot be solved. In this embodiment of the invention, the orthogonality of the rotation matrix is ​​used, based on R... X The other two column vectors (the column vectors corresponding to the X-axis and Y-axis) are obtained through cross product.

[0089] As a further preferred design of the present invention, in order to reduce noise during multiple data acquisitions and errors caused by the device's own measurement errors, and to further improve the calibration accuracy, the following steps are also included:

[0090] Using the obtained closed solution as the initial value, the objective function is constructed: Based on the obtained multiple sets of data, the objective function is to minimize the error between the coordinates of the puncture needle tip in the second robotic arm base coordinate system {O2} and the coordinates of the puncture needle tip in the second robotic arm base coordinate system {O2} calculated by ultrasound image.

[0091] The objective function is solved using an iterative method to obtain the iterative solution corresponding to the closed solution.

[0092] In this embodiment of the invention, the constructed objective function is:

[0093]

[0094] Where n is the number of sets of acquired pose information and corresponding ultrasound images, F(R) X R Y , t X , t Y , t Q F(R) is the objective function for each set of data. X R Y , t X , t Y , t Q The expression for ) is:

[0095] F(R X R Y , t X , t Y , t Q ) = R Y R B t Q +R Y tB +t Y -R A R X t P -R A t X -t A (7)

[0096] Among them, (R) Y R B t Q +R Y t B +t Y () represents the coordinates of the tip of the puncture needle in the second robotic arm's base coordinate system {O2}; (R A R X t P +R A t X +t A ) represents the coordinates of the tip of the puncture needle in the second robotic arm base coordinate system {O2}, calculated from the ultrasound image.

[0097] Furthermore, the present invention provides an iterative method for solving the objective function, specifically including:

[0098] Within the Lie algebra range, for F(R) X R Y , t X , t Y , t Q Taking the differential, we get:

[0099] dF=R A R X [δω X ] ∧ t P +R A δt X -δt Y -R Y [δω Y ] ∧ (R B t Q +t B )-R Y R B δt Q (8)

[0100] Where, ω X and ω Y Representing the rotation matrix R respectively X and R Y The transformation relationship between the rotation vector, the rotation matrix R, and the rotation vector ω can be expressed using the exponential product formula:

[0101] R = exp([ω] ∧ )

[0102] [·] ∧ Let represent the antisymmetric vector operator, used to transform vectors into antisymmetric matrices, thus converting the cross product of vectors into a matrix operation. According to the anti-cross law of the cross product, formula (8) can be further simplified to:

[0103] dF = -R A R X [t P ] ∧ δω X +R Y [R B t Q +t B ] ∧ δω Y +R A δt X -δt Y -R Y R B δt Q (9)

[0104] Based on formula (9), the update rate (iteration direction and step size) Δ of the Gauss-Newton method is designed as follows:

[0105]

[0106] in, The expression for G is:

[0107]

[0108] Using this update rate, Gauss-Newton iterations are performed to obtain the final iterative solution of the model.

[0109] Specifically, the method of the present invention further includes: step S4, calculating the puncture direction V of the puncture needle based on the calibration results X, Y, Q and multiple sets of collected data A, B, P and v.

[0110] Specifically, in step S4, for each set of collected data, the following linear equation can be obtained based on the constructed model:

[0111] (R A R X ) -1 R Y R B V = v (12)

[0112] The matrix equations formed by each set of data are arranged into a composite linear equation system, and V can be obtained by solving it using the least squares method.

[0113] Specifically, let D = (R A R X ) -1 R Y R B Then the least squares solution to the matrix equation is:

[0114]

[0115] At this point, the solutions for X, Y, Q, and V have been completely obtained, and the calibration is complete.

[0116] The method of this invention directly uses the ultrasonic probe and surgical puncture needle of an ultrasound-guided dual-arm puncture robot to collect multiple sets of pose information of the robotic arm and corresponding ultrasonic image data. Based on the collected data, a pose closed-loop homogeneous matrix transformation model of the ultrasound-guided dual-arm puncture robot is constructed. Solving the model yields the corresponding coordinate transformation relationship and the homogeneous coordinates of the tip of the surgical puncture needle. No additional calibration tools are required. High-precision calibration of the system can be completed using the conventional configuration of the ultrasound-guided dual-arm puncture robot, which can reduce calibration complexity and improve calibration efficiency.

[0117] Meanwhile, the data acquisition method of this invention directly uses existing surgical instruments to complete the acquisition of calibration data in a single acquisition process. That is, the data acquisition directly uses the ultrasound probe and surgical puncture instruments used in surgery, without changing the configuration of the surgical site for calibration, such as changing the devices at the end of the robotic arms. Furthermore, the data acquisition is a simple repetitive action process, that is, continuously changing the position of the two robotic arms so that the puncture needle is clearly displayed in the ultrasound image. Repeating this simple action several times at once can complete the acquisition of calibration data, resulting in more accurate data and a simpler acquisition method.

[0118] Furthermore, in the process of iteratively solving the iterative solution using the Gauss-Newton method, the update rate is designed in conjunction with the Lie algebra method, and the rotation vector is directly used for iterative update. In each iteration, the rotation matrix calculated by the rotation vector can ensure that the rotation matrix is ​​always orthogonal, avoiding the problem of introducing additional errors by orthogonalizing the results in the existing numerical iterative methods, and further improving the accuracy of the calibration results.

[0119] Meanwhile, the update rate of this invention is used directly to calculate the iteration direction and step size, eliminating the need to search for the optimal step size through loops, thus improving the iteration convergence speed and iteration efficiency.

[0120] According to another aspect of the present invention, a coordinate self-calibration system for an ultrasound-guided dual-arm puncture robot is provided. This system is used to perform the corresponding steps in the aforementioned coordinate self-calibration method for an ultrasound-guided dual-arm puncture robot. The system includes:

[0121] The data acquisition module is used to acquire multiple sets of pose information of the robot and corresponding ultrasound images; wherein, each set of pose information includes: homogeneous transformation matrix A from the first robot arm base coordinate system {O1} to its end coordinate system {E1}, and homogeneous transformation matrix B from the second robot arm base coordinate system {O2} to its end coordinate system {E2}.

[0122] The model building module is used to construct the robot's pose closed-loop homogeneous matrix transformation model: AXP = YBQ; where X is the homogeneous transformation matrix from the first robotic arm end-effector coordinate system {E1} to the ultrasonic image coordinate system {I1}, Y is the homogeneous transformation matrix from the first robotic arm base coordinate system {O1} to the second robotic arm base coordinate system {O2}, Q is the homogeneous coordinate of the surgical needle tip in the second robotic arm end-effector coordinate system {E2}, and P is the homogeneous coordinate of the surgical needle tip in the ultrasonic image coordinate system {I1} corresponding to each set of pose information.

[0123] The model solving module is used to decompose and linearize the model in a block-based manner, obtain the closed solutions of the homogeneous transformation matrices X, Y and Q, and complete the calibration.

[0124] According to another aspect of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements each step in the coordinate self-calibration method of the ultrasound-guided dual-arm puncture robot as described above.

[0125] In summary, the method of this invention does not require external observation equipment or additional calibration tools. It only requires a sufficient amount of pose and ultrasound image data of the dual-arm puncture robot to be collected at once. By establishing a pose closed-loop homogeneous matrix transformation model and solving the model linearly, and then optimizing the accuracy with an iterative algorithm, the accurate pose relationship of each coordinate system of the ultrasound-guided dual-arm puncture surgical robot can be obtained. This invention can obtain high-precision calibration results with less calibration time.

[0126] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A coordinate self-calibration method for an ultrasound-guided dual-arm puncture robot, the robot comprising a first robotic arm, a second robotic arm, an ultrasound probe, and a surgical puncture needle, wherein an ultrasound image is generated through the end of the ultrasound probe, characterized in that... The method includes: S1. Acquire multiple sets of pose information and corresponding ultrasound images of the robot; wherein, each set of pose information includes: the first robotic arm base coordinate system. To its end coordinate system homogeneous transformation matrix Second robotic arm base coordinate system To its end coordinate system homogeneous transformation matrix ; S2. Construct the pose closed-loop homogeneous matrix transformation model of the robot: ;in, The coordinate system of the first robotic arm end effector To the ultrasound image coordinate system The homogeneous transformation matrix, The first robotic arm base coordinate system To the second robotic arm base coordinate system The homogeneous transformation matrix, The coordinate system of the second robotic arm end effector Homogeneous coordinates of the tip of the surgical puncture needle; The ultrasound image coordinate system corresponding to each set of pose information Homogeneous coordinates of the tip of the surgical puncture needle; S3. Decompose and linearize the model using a block-based approach to obtain the homogeneous transformation matrix. , and The closed-loop solution is obtained, and the calibration is completed; S3 includes: S31. The model is decomposed into equations by using a block-based approach, based on rotation matrices and translation vectors. S32. Linearize the decomposed equations to obtain the corresponding linear matrix equations; S33. Substitute each set of pose information and the corresponding ultrasound image data into the linear matrix equation to obtain a composite linear matrix equation system. S34. Solve the system of equations of the composite linear matrix using the least squares method to obtain the homogeneous transformation matrix. , and The closed solution; Also includes: Using the closed solution as initial values, an objective function is constructed. The objective function is to minimize the point of the puncture needle tip in the second robotic arm base coordinate system. The coordinates of the points in the second robotic arm base coordinate system are compared with the needle tip point of the puncture needle calculated from the ultrasound image. Error between coordinate points in the data; The objective function is solved using an iterative method to obtain the iterative solution corresponding to the closed solution.

2. The method according to claim 1, characterized in that, In S32, the decomposed equation is linearized by simultaneously performing matrix straightening operations on both sides and then exchanging parameters through the Kronecker product.

3. The method according to claim 1, characterized in that, The objective function is: in, This refers to the acquired pose information and the corresponding number of ultrasound image sets. The expression is: In the formula, , , , These are homogeneous transformation matrices. , , , The rotation matrix, , , , These are homogeneous transformation matrices. , , , The translation vector, and These are homogeneous coordinates. and The translation vector.

4. The method according to claim 3, characterized in that, The iterative method includes: Within the scope of Lie algebras, for Finding the differential, we obtain the differential equation: in, and Representing rotation matrices respectively and rotation vector, rotation matrix With rotation vector The mutual transformation relationship between them is as follows: ; Indicates the vector antisymmetry operator; Design the update rate of the Gauss-Newton method : in, , The expression is: Using the update rate, Gaussian-Newton iterations are performed to obtain the iterative solution corresponding to the closed solution.

5. The method according to claim 1, characterized in that, S1 includes: S11. Within the shared workspace of the two robotic arms, the first and second robotic arms are moved to make the puncture needle visible in the ultrasound image. S12. Record the pose information of the dual robotic arms at the current moment and the corresponding ultrasound image, wherein the pose information of the dual robotic arms at the current moment is respectively represented by the homogeneous transformation matrix. and the homogeneous transformation matrix express; S13. Mark the distance coordinates of the needle tip and the direction vector of the needle in the ultrasound image acquired at the current moment. The distance coordinates of the needle tip are expressed using the homogeneous coordinates. express; S14. Repeat S11-S13 to obtain multiple sets of pose information and corresponding ultrasound images.

6. The method according to claim 5, characterized in that, Also includes: Calculate the puncture direction of the puncture needle Specifically, it includes: Based on the transitive relationship of the rotation matrix, the following linear equation is constructed: The obtained homogeneous transformation matrix , and Substituting the corresponding sets of data into the linear equation yields a composite linear equation system. The puncture direction of the puncture needle is obtained by solving the composite linear equations using the least squares method. .

7. A coordinate self-calibration system for an ultrasound-guided dual-arm puncture robot, characterized in that, A coordinate self-calibration method for performing the ultrasound-guided dual-arm puncture robot as described in any one of claims 1-6, comprising: The data acquisition module is used to acquire multiple sets of pose information and corresponding ultrasound images of the robot; wherein, each set of pose information includes: the first robotic arm base coordinate system. To its end coordinate system homogeneous transformation matrix Second robotic arm base coordinate system To its end coordinate system homogeneous transformation matrix ; The model building module is used to construct the pose closed-loop homogeneous matrix transformation model of the robot. ;in, The coordinate system of the first robotic arm end effector To the ultrasound image coordinate system The homogeneous transformation matrix, The first robotic arm base coordinate system To the second robotic arm base coordinate system The homogeneous transformation matrix, The coordinate system of the second robotic arm end effector Homogeneous coordinates of the tip of the surgical puncture needle; The ultrasound image coordinate system corresponding to each set of pose information Homogeneous coordinates of the tip of the surgical puncture needle; The model solving module is used to decompose and linearize the model in a block-based manner to obtain the homogeneous transformation matrix. , and The closed solution is obtained, and the calibration is completed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images