Dual-arm cooperative ultrasonic puncture robot and calibration method thereof

Abstract

The application discloses a double-arm cooperative ultrasonic puncture robot and a calibration method thereof. The robot comprises two mechanical arms, a puncture device and an ultrasonic scanning device. The calibration device comprises a calibration cone, a calibration dummy and a reference needle tip disc. An ultrasonic probe and the puncture device are respectively installed at the ends of the two mechanical arms with a calibration cone. Firstly, a world coordinate system is established based on the ultrasonic mechanical arm. Then, the two mechanical arms are controlled to make the calibration cones touch each other, so that the calibration between the two mechanical arms is completed. The needle tip on the reference needle tip disc is touched by the calibration cone carried by the ultrasonic mechanical arm, so that the coordinate measurement of the needle tip is completed. The needle tip wrapped by the calibration dummy is scanned by the ultrasonic probe carried by the ultrasonic mechanical arm. According to different needle tip coordinates, the conversion relationship between the ultrasonic image coordinate system and the world coordinate system is obtained through calibration, so that the robot calibration is completed. The calibration method has the characteristics of accuracy, rapidness and standardization, and has great application value in medical ultrasonic puncture robots.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical ultrasonic puncture robot calibration, and particularly relates to a double-arm cooperative ultrasonic puncture robot and a calibration method thereof. BACKGROUND

[0002] Ultrasound scanning diagnosis is a widely used diagnosis method in the medical field. In recent years, with the continuous progress of ultrasonic imaging technology and the rapid development of related medical means, the diagnosis and treatment method of using robots to assist in ultrasonic puncture has achieved wide application prospects. For example, the ultrasonic puncture robot can flexibly and accurately control the needle entry angle, depth and posture of the puncture needle in a narrow operation space according to the ultrasonic image, which can significantly improve the operation precision and reduce the operation trauma. In particular, the double-arm robot system can separately control the ultrasonic probe and the puncture needle tube, providing higher flexibility and assistance for the operation of the doctor. In order to enable the puncture needle to accurately reach the expected position, calibration is the most important step before the work of the medical ultrasonic puncture robot.

[0003] The key of the calibration of the medical ultrasonic puncture robot is the calibration between the double arms and the calibration between the ultrasonic image and the actual absolute coordinate, that is, through the space conversion matrix, any point in the ultrasonic image and any coordinate of the double-arm end operation tool can be mapped to the actual world coordinate system. The existing calibration methods mainly use optical reference frames, magnetic calibration devices or other sensors. These calibration methods can relatively accurately complete the required calibration in the actual calibration device, but they do not solve the calibration problem from the actual device to the ultrasonic image. These devices are mostly not unified in specification, ignore the calibration between the double arms, the calibration process is complicated, and some calibration systems are very inconvenient due to their large size. SUMMARY

[0004] In view of the deficiencies of the prior art, the application provides a double-arm cooperative ultrasonic puncture robot and a calibration method, which can quickly and standardize the calibration of the position of the medical ultrasonic puncture robot.

[0005] To achieve the above object, the technical scheme adopted by the application is as follows:

[0006] A calibration method of a double-arm cooperative ultrasonic puncture robot, the calibration device used in the method comprises two mechanical arms, a reference needle tip disc, a calibration cone, a calibration phantom, a puncture device and an ultrasonic scanning device, the two mechanical arms are an ultrasonic mechanical arm and a puncture mechanical arm, the reference needle tip disc is provided with a plurality of needle tips for calibration, and the method comprises the following steps:

[0007] Step 1, set up the calibration device, install the first calibration cone and the ultrasonic probe of the ultrasonic scanning device together through the flange at the execution end of the ultrasonic mechanical arm, install the second calibration cone and the puncture device together through the flange at the execution end of the puncture mechanical arm, and keep the relative positions between the reference needle tip disc, the base of the ultrasonic mechanical arm and the base of the puncture mechanical arm unchanged during the entire calibration process;

[0008] Step 2, calibration between the two mechanical arms, establish a world coordinate system based on the base of the ultrasonic mechanical arm, and establish a puncture mechanical arm base coordinate system based on the base of the puncture mechanical arm; operate the ultrasonic mechanical arm and the puncture mechanical arm so that the calibration cones at the ends of the two mechanical arms touch the tips, and the first coordinate transformation matrix between the world coordinate system and the puncture mechanical arm base coordinate system is calculated through coordinate transformation;

[0009] Step 3, establish a puncture coordinate system based on the puncture device, and calculate the second coordinate transformation matrix between the puncture coordinate system and the world coordinate system based on the relative position relationship between the puncture device and the puncture mechanical arm;

[0010] Step 4, reference needle tip disc calibration, operate the ultrasonic mechanical arm so that the tip of the first calibration cone at the end of the ultrasonic mechanical arm touches the multiple needle tips on the reference needle tip disc in turn, and the coordinates of the needle tips in the world coordinate system are calculated through coordinate transformation;

[0011] Step 5, wrap the reference needle tip disc or part of the needle tips on the reference needle tip disc with the calibration dummy, scan through the ultrasonic mechanical arm carrying the ultrasonic probe, obtain the positions of the needle tips in the reference needle tip disc in the ultrasonic image, establish an image coordinate system based on the ultrasonic image, calculate the coordinates of the needle tips in the image coordinate system through the known pixel corresponding to the actual distance;

[0012] Step 6, express the transformation matrix from the world coordinate system to the image coordinate system by setting unknowns, multiply the image coordinates of each needle tip in the reference needle tip disc by the transformation matrix, express the coordinates in the world coordinate system in the form of equations, and equalize the coordinates obtained in step 4, multiple needle tips can list multiple equation groups, solve the unknowns in the transformation matrix from the equation groups, and thus obtain the third coordinate transformation matrix from the world coordinate system to the image coordinate system, completing the calibration of the image coordinate system and the puncture coordinate system in the world coordinate system.

[0013] Further, the solving method in step 2 is as follows:

[0014] The transformation matrix from the world coordinate system to the puncture mechanical arm base coordinate system is expressed by setting unknowns as follows:

[0015] Formula (1)

[0016] Wherein Rw represents the first coordinate transformation matrix from the world coordinate system to the puncture robot base coordinate system, Rw represents the coordinates of the tip of the first calibration cone in the world coordinate system, which can be calculated by the characteristics of the ultrasonic robot itself, Rw represents the coordinates of the tip of the second cone in the puncture robot base coordinate system, which can be calculated by the characteristics of the puncture robot itself.

[0017] Change the posture of the ultrasonic robot and the puncture robot, touch the tip at different positions, get multiple sets of formulas based on formula (1), and calculate the first coordinate transformation matrix.

[0018] Further, in step 2, the coordinates of the tip of the first calibration cone in the world coordinate system are calculated according to the following formula:

[0019] Formula (2)

[0020] Rw represents the coordinates of the tip of the first calibration cone in the first calibration cone coordinate system, Rw represents the coordinate transformation matrix between the world coordinate system and the ultrasonic robot end coordinate system, which is calculated by the forward kinematics of the robot and the joint angle; Rw represents the coordinate transformation matrix between the ultrasonic robot end coordinate system and the first cone coordinate system, which is determined by the mounting relationship of the first calibration cone relative to the ultrasonic robot end or calculated by calibration;

[0021] Similarly, the coordinates of the tip of the second calibration cone in the puncture robot base coordinate system are calculated according to the following formula:

[0022] Formula (3)

[0023] Rw represents the coordinates of the tip of the second calibration cone in the puncture robot base coordinate system, Rw represents the coordinates of the tip of the second calibration cone in the second calibration cone coordinate system, Rw represents the coordinate transformation matrix between the puncture robot base coordinate system and the puncture robot end coordinate system, which is calculated by the forward kinematics of the robot and the joint angle; Rw represents the coordinate transformation matrix between the puncture robot end coordinate system and the second cone coordinate system, which is determined by the mounting relationship of the second calibration cone relative to the puncture robot end or calculated by calibration.

[0024] Further, in step 4, the posture of the ultrasonic robot is changed by joint motion, and at least two ultrasonic robot postures are used to touch the same needle tip, and the coordinates of the needle tip are corrected by at least two postures.

[0025] Further, in step 4, since the forward kinematics of the robot arm and the joint angle are known parameters for the robot arm, the needle tip in the world coordinate system is converted into the needle tip in the coordinate system of the end of the ultrasonic robot arm, and the following two methods are used to obtain the coordinates of the needle tip in the coordinate system of the end of the ultrasonic robot arm.

[0026] Firstly, the coordinates of the tip of the first calibration cone in the coordinate system of the end of the ultrasonic robot arm are directly calculated according to the geometric size of the first calibration cone and the mounting relationship between the end of the ultrasonic robot arm, that is, the coordinates of the needle tip in the coordinate system of the end of the ultrasonic robot arm.

[0027] Secondly, a first cone coordinate system is established, unknown parameters are set to express the transformation matrix between the first cone coordinate system and the coordinate system of the end of the ultrasonic robot arm, and a plurality of needle tips on the reference needle tip disc are collided, the relative positions of the plurality of needle tips are known, the unknown parameters are solved by an equation, and the transformation matrix between the first cone coordinate system and the coordinate system of the end of the ultrasonic robot arm is obtained, so that the coordinates of the needle tip in the coordinate system of the end of the robot arm are obtained.

[0028] Further, in step 4, the first cone coordinate system is established based on the coordinate system of the end of the ultrasonic robot arm, that is, at least one coordinate axis of the first cone coordinate system coincides with the coordinate system of the end of the robot arm.

[0029] Further, in step 4, the image coordinate system takes the uppermost vertex of the image symmetry axis as the origin, and a pixel coordinate system of the ultrasonic image is established based on the same position, the coordinates of the needle tip in the pixel coordinate system are determined according to the pixel position of the needle tip in the ultrasonic image, and the coordinates of the needle tip in the image coordinate system are obtained according to the proportional relationship between the pixel coordinate system and the image coordinate system.

[0030] Further, the calibration phantom is made of a material that can simulate human tissues in ultrasonic images.

[0031] Further, in step 4, the reference needle tip disc has at least four needle tips, three of which need to be calibrated in the world coordinate system by collision, and the remaining needle tips are directly wrapped by the calibration phantom, and the coordinates of the needle tips wrapped by the calibration phantom in the world coordinate system are calculated according to the geometric size relationship between the needle tips on the reference needle tip disc.

[0032] The application also provides a double-arm cooperative ultrasonic puncture robot, comprising a double-arm cooperative ultrasonic puncture device and a controller with an algorithm based on the calibration method, wherein the double-arm cooperative ultrasonic puncture device comprises two mechanical arms, a reference needle tip disc, a calibration cone, a calibration phantom, a puncture device and an ultrasonic scanning device, the two mechanical arms are an ultrasonic mechanical arm and a puncture mechanical arm respectively; a first calibration cone and an ultrasonic probe of the ultrasonic scanning device are installed together on an execution end of the ultrasonic mechanical arm through a flange, and a second calibration cone and the puncture device are installed together on an execution end of the puncture mechanical arm through a flange; and the reference needle tip disc is provided with a plurality of needle tips for calibration.

[0033] The application has the following beneficial effects:

[0034] 1. The unknown parameters in the coordinate transformation matrix can be accurately calculated by establishing an accurate coordinate transformation equation through a simple calibration device and a calibration model, so that calibration can be completed.

[0035] 2. The calibration device and the calibration model are simple, the reference needle tip disc and the calibration cone are simple in structure design and convenient to process, so that the machining precision can be made to be the highest, and the calibration error is small enough.

[0036] 3. The number of needle tips on the reference needle tip disc in the calibration model can be designed according to requirements, each needle tip can provide information for calculating the unknown parameters in the coordinate transformation matrix to complete calibration, and a numerical analysis method can be used to a large number of needle tips to improve the calibration precision by using a numerical iteration method.

[0037] 4. The calibration cone and the reference needle tip disc are processed by a metal process, which has low cost.

[0038] 5. The calibration problem between the two arms is solved, and a reliable double-arm calibration method is provided. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a whole schematic view of the double-arm cooperative ultrasonic puncture robot and the calibration device according to the application;

[0040] Figure 2 is a double-arm calibration schematic view of the double-arm cooperative ultrasonic puncture robot according to the application;

[0041] Figure 3 is a needle tip calibration schematic view of the reference needle tip disc of the double-arm cooperative ultrasonic puncture robot according to the application;

[0042] Figure 4 is an ultrasonic image calibration schematic view of the double-arm cooperative ultrasonic puncture robot according to the application;

[0043] Wherein, 1-ultrasound manipulator, 2-puncture manipulator, 3-first calibration cone, 4-second calibration cone, 5-ultrasound probe, 6-puncture device, 7-calibration phantom, 8-reference needle tip disk, 9-ultrasound manipulator flange, 10-puncture manipulator flange, 11-ultrasound image, 12-imaging point, 13-needle tip, prob1-end coordinate system of ultrasound manipulator, prob2-end coordinate system of puncture manipulator, ref1-coordinate system of first calibration cone, ref2-coordinate system of second calibration cone, ult-coordinate system of ultrasound probe, img-image coordinate system, pic-pixel coordinate system, word-world coordinate system, base-base coordinate system of puncture manipulator, tip-coordinate system of puncture device. DETAILED DESCRIPTION

[0044] In order to make the technical problems to be solved by the present application, the technical solutions more clearly, below, combined with the drawings and examples, a medical ultrasound puncture robot position calibration device and calibration method provided by the present application is described in detail, the explanation of the present application is not limited.

[0045] As shown in Figure 1 The present application provides a dual-arm collaborative ultrasound puncture robot, including dual-arm collaborative ultrasound puncture device 6 and controller (host computer or industrial computer) embedded with specific calibration method algorithm, the dual-arm collaborative ultrasound puncture device 6 includes two manipulators, puncture device 6 and ultrasound scanning device, the ultrasound scanning device includes ultrasound probe 5 and ultrasound image instrument, two manipulators are ultrasound manipulator 1 and puncture manipulator 2 respectively.

[0046] The calibration device includes reference needle tip disk 8, two calibration cones and calibration phantom 7, the first calibration cone 3 and the ultrasound probe 5 of the ultrasound scanning device are installed together through the flange at the execution end of the ultrasound manipulator 1, the second calibration cone 4 and the puncture device 6 are installed together through the flange at the execution end of the puncture manipulator 2; A plurality of needle tips 13 for calibration are provided on the reference needle tip disk 8, preferably more than 4 needle tips 13, the relative positional relationship between the needle tips 13 is known, or the relative positional relationship between the needle tips 13 can be calculated by establishing the reference needle tip disk coordinate system; The positional relationship between the reference needle tip disk 8 and the base of the manipulator does not need to be known, but once the calibration starts, the relative positions of the reference needle tip disk 8 and the two manipulators remain fixed during the calibration process; The calibration phantom 7 is used to wrap on the needle tip 13, which facilitates the ultrasound probe 5 to scan and image the needle tip 13, the ultrasound probe 5 cannot directly scan the needle tip 13, one is easy to break or bend the needle tip 13, the other is, the needle tip 13 alone does not have the background environment of ultrasound imaging.

[0047] The calibration cone is detachably installed on the corresponding flange. During the calibration process, the relative positions of the first calibration cone 3 and the ultrasonic probe 5 and the relative positions of the second calibration cone and the puncture device 6 remain unchanged. After the calibration is completed, the calibration cone is removed, and the dual-arm coordinated ultrasonic puncture robot can be put into use.

[0048] It should be noted that the flange is only a connector name. In order to facilitate connection with the execution end of the mechanical arm, one end is provided as a conventional flange face. For the ultrasonic mechanical arm 1, the other end can be provided as a clamping seat or clamping plate for installing the ultrasonic probe 5 or similar structure. For the puncture mechanical arm 2, the other end can be provided as a base or similar structure for installing the puncture device 6. The calibration cone is detachably installed on the flange through a connector such as a bolt or screw. During installation, the tip of the calibration cone faces outward and has a length greater than the ultrasonic probe 5 or the puncture device 6, so that when the calibration cone collides with the needle tip 13, the ultrasonic probe 5 or the puncture device 6 will not cause interference.

[0049] It should be noted that the puncture device 6 of the present application is a functionally independent sub-module, and most conventional puncture devices 6 in the prior art can be used. The form of the puncture device 6 itself has no effect on the technical problems solved by the present application.

[0050] It should be noted that the calibration cone of the present application is not necessarily a conical shape. It only needs to have a conical tip at the front end that can collide with the needle tip 13 on the reference needle tip disc 8. The geometric relationship of the calibration cone itself is known, so the relative position relationship can be calculated by geometry or the coordinates can be calculated by establishing a cone coordinate system.

[0051] It should be noted that the calibration phantom 7 is made of a material that can simulate human tissue in an ultrasonic image, such as silicone material or the like. The purpose of setting the calibration phantom 7 is to facilitate the use of the ultrasonic probe 5 to scan and image the needle tip 13. The phantom material is very widely selected, as long as it does not interfere with the ultrasonic image.

[0052] It should be noted that the first calibration cone 3 and the second calibration cone 4 are both detachably installed (such as fixed by screws). After calibration, the calibration cone is removed, and the dual-arm coordinated ultrasonic puncture robot can work.

[0053] It should be noted that the order of the calibration between the two mechanical arms in step 2 and the calibration of the reference needle tip disc in step 4 can be exchanged.

[0054] As shown in Figures 1 to 4 The present application also provides a calibration method for a dual-arm coordinated ultrasonic puncture robot, comprising the following steps:

[0055] Step 1, build a calibration device, as shown inFigure 1 As shown, the first calibration cone 3 and the ultrasonic probe 5 of the ultrasonic scanning device are installed together through a flange at the execution end of the ultrasonic mechanical arm 1, and the second calibration cone 4 and the puncture device 6 are installed together through a flange at the execution end of the puncture mechanical arm 2. During the entire calibration process, the relative positions among the reference needle tip disc 8, the base of the ultrasonic mechanical arm 1, and the base of the puncture mechanical arm 2 remain unchanged.

[0056] Step 2, calibration between the two mechanical arms, as shown in Figure 2 As shown, the world coordinate system word is established based on the base of the ultrasonic mechanical arm 1, and the puncture mechanical arm base coordinate system base is established based on the base of the puncture mechanical arm 2. The ultrasonic mechanical arm 1 and the puncture mechanical arm 2 are operated so that the calibration cones at the ends of the two mechanical arms touch the tip, and the first coordinate transformation matrix between the world coordinate system word and the puncture mechanical arm base coordinate system base is calculated through coordinate transformation.

[0057] In this embodiment, a specific calculation method is provided as follows:

[0058] Unknowns are set to express the transformation matrix from the world coordinate system word to the puncture mechanical arm base coordinate system base as follows:

[0059] Formula (1)

[0060] wherein represents the first coordinate transformation matrix from the world coordinate system word to the puncture mechanical arm base coordinate system base, represents the coordinates of the tip of the first calibration cone 3 in the world coordinate system word, which can be calculated through the characteristics of the ultrasonic mechanical arm 1 itself, represents the coordinates of the tip of the second cone in the puncture mechanical arm base coordinate system base, which can be calculated through the characteristics of the puncture mechanical arm 2 itself;

[0061] The poses of the ultrasonic mechanical arm 1 and the puncture mechanical arm 2 are changed, and the tip touching is performed at different positions to obtain multiple sets of formulas based on formula (1), and the first coordinate transformation matrix is calculated.

[0062] In the above calculation process, the calculation formula of the coordinates of the tip of the first calibration cone 3 in the world coordinate system word is as follows:

[0063] Formula (2)

[0064] represents the coordinates of the tip of the first calibration cone 3 in the first calibration cone coordinate system ref1, represents the coordinate transformation matrix between the world coordinate system word and the ultrasound robot end coordinate system prob1, which is obtained by forward kinematics of the robot and joint angle calculation; represents the coordinate transformation matrix between the ultrasound robot end coordinate system prob1 and the first cone coordinate system, which is determined by the mounting relationship of the first calibration cone 3 relative to the ultrasound robot 1 end or calculated by calibration;

[0065] Similarly, the coordinate calculation formula of the tip of the second calibration cone 4 in the puncture robot base coordinate system base is as follows:

[0066] Formula (3)

[0067] represents the coordinate of the tip of the second calibration cone 4 in the puncture robot base coordinate system base, represents the coordinate of the tip of the second calibration cone 4 in the second calibration cone coordinate system ref2, represents the coordinate transformation matrix between the puncture robot base coordinate system base and the puncture robot end coordinate system prob2, which is obtained by forward kinematics of the robot and joint angle calculation; represents the coordinate transformation matrix between the puncture robot end coordinate system prob2 and the second cone coordinate system, which is determined by the mounting relationship of the second calibration cone 4 relative to the puncture robot 2 end or calculated by calibration.

[0068] As a preferred embodiment, the first calibration cone 3 adopts a triangular cone. When establishing the first cone coordinate system, one coordinate axis (such as the Z axis) coincides with the coordinate axis (such as the Z axis) of the ultrasound robot end coordinate system prob1, and the X and Y axes are translated. In this way, there is only one unknown parameter between the coordinate transformation matrices. At this time, two needle tips 13 (the relative position relationship between the two needle tips 13 is known) are selected to touch to calculate the coordinate transformation matrix between the ultrasound robot end coordinate system prob1 and the first cone coordinate system.

[0069] In the same way, the second calibration cone 4 is set, and the coordinate transformation matrix between the puncture robot end coordinate system prob2 and the second cone coordinate system can also be obtained.

[0070] Step 3, establish a puncture coordinate system based on the puncture device 6, based on the relative position relationship between the puncture device 6 and the puncture robot 2, and calculate the second coordinate change matrix between the puncture coordinate system and the world coordinate system word; that is, the coordinates of the puncture needle in the world coordinate system word can be obtained, and the formula is as follows:

[0071] Formula (4)

[0072] wherein, represents the coordinates of the tip of the puncture needle 13 in the world coordinate system word, represents the coordinates of the tip of the puncture needle 13 in the puncture device coordinate system tip, represents the coordinate transformation matrix from the world coordinate system word to the puncture robot base coordinate system base, represents the coordinate transformation matrix from the puncture robot base coordinate system base to the puncture robot end coordinate system prob2, which is calculated by forward kinematics of the robot and joint angles, is the coordinate transformation matrix from the puncture robot end coordinate system prob2 to the puncture needle coordinate system on the puncture robot 2, which is calculated according to the installation position relationship of the puncture device 6 or obtained by calibration.

[0073] Step 4, operate the ultrasonic robot 1, as shown in Figure 3 , make the tip of the first calibration cone 3 at the end of the ultrasonic robot 1 touch the plurality of needle tips 13 on the reference needle tip disc 8 in turn, and calculate the coordinates of the needle tip 13 in the world coordinate system word by coordinate transformation;

[0074] Since the forward kinematics of the robot and the joint angles are known parameters for the robot, the coordinates of the needle tip 13 in the world coordinate system word are converted to the coordinates of the needle tip 13 in the ultrasonic robot end coordinate system prob1, and the formula is as follows:

[0075] Formula (5)

[0076] represents the coordinates of the i th needle tip 13 in the world coordinate system word;

[0077] represents the coordinates of the tip of the first calibration cone 3 in the first calibration cone coordinate system ref1, that is, the coordinates of the i th needle tip 13 in the first calibration cone coordinate system ref1 when the tip of the first calibration cone 3 touches the i th needle tip 13;

[0078] represents the coordinate transformation matrix between the world coordinate system word and the ultrasonic robot end coordinate system prob1, which is calculated by forward kinematics of the robot and joint angles;

[0079] represents the coordinate transformation matrix between the ultrasonic robot end coordinate system prob1 and the first cone coordinate system, which is determined by the installation relationship of the first calibration cone 3 relative to the end of the ultrasonic robot 1 or calculated by calibration.

[0080] The coordinate transformation matrix between the ultrasound robot arm end coordinate system prob1 and the first cone coordinate system There are two acquisition methods as follows.

[0081] The first method is to directly calculate the coordinates of the tip of the first calibration cone 3 in the ultrasound robot arm end coordinate system prob1, i.e., the coordinates of the needle tip 13 in the ultrasound robot arm end coordinate system prob1, through the geometric size of the first calibration cone 3 and the installation relationship between the execution end of the ultrasound robot arm 1.

[0082] The second method is to establish a first cone coordinate system, set unknown parameters to express the transformation matrix between the first cone coordinate system and the ultrasound robot arm end coordinate system prob1, collide a plurality of needle tips 13 on the reference needle tip disc 8, the relative position relationship between the plurality of needle tips 13 is known, solve the unknown parameters through the equation, obtain the transformation matrix between the first cone coordinate system and the ultrasound robot arm end coordinate system prob1, and thus obtain the coordinates of the needle tip 13 in the robot arm end coordinate system (as a known common knowledge, the present application will not be repeated).

[0083] As a preferred embodiment, the posture of the ultrasound robot arm 1 can be changed by joint motion, and at least two ultrasound robot arm 1 postures are used to complete the collision of the same needle tip 13, and the coordinates of the needle tip 13 are corrected through at least two postures.

[0084] Step 5, as shown in Figure 4 The reference needle tip disc 8 or part of the needle tips 13 on the reference needle tip disc are wrapped with the calibration dummy 7, the ultrasound robot arm 1 carries the ultrasound probe 5 to scan, the positions of the needle tips 13 in the reference needle tip disc 8 in the ultrasound image 12 are acquired, the image coordinate system imgimg based on the ultrasound image 12 is established, the coordinates of the needle tips 13 in the image coordinate system imgimg are calculated through the known pixel corresponding to the actual distance; the specific method is as follows:

[0085] Specifically, the image coordinate system imgimg takes the uppermost vertex of the image symmetry axis as the origin, the pixel coordinate system picpic of the ultrasound image 12 is established based on the same position, the coordinates of the needle tips 13 in the pixel coordinate system picpic are determined through the pixel positions of the needle tips 13 in the ultrasound image 12, and the coordinates of the needle tips 13 in the image coordinate system imgimg are obtained through the proportional relationship between the pixel coordinate system picpic and the image coordinate system imgimg.

[0086] Step 6, express the transformation matrix from the world coordinate system word to the image coordinate system img by setting unknown numbers, multiply the image coordinates of each needle point 13 in the needle point disc 8 by the transformation matrix, and express the coordinates in the world coordinate system word, which are equal to the coordinates obtained in step 4. Multiple needle points 13 can list multiple sets of equation groups, solve the unknown numbers in the transformation matrix from the equation groups, and thus obtain the third coordinate transformation matrix from the world coordinate system word to the image coordinate system img, complete the calibration of the image coordinate system img and the puncture coordinate system in the world coordinate system word.

[0087] The coordinate transformation formula from the world coordinate system word to the image coordinate system img is as follows:

[0088] Formula (6)

[0089] wherein, represents the coordinates of the i-th needle point 13 in the world coordinate system word; i

[0090] represents the pixel coordinates of the imaging point 12 of the needle point 13 in the pixel coordinate system pic;

[0091] represents the third coordinate transformation matrix between the image coordinate system img and the pixel coordinate system pic;

[0092] is the coordinate transformation matrix between the world coordinate system word and the image coordinate system img, which is obtained by calibration. The specific solving process is as follows:

[0093] The ultrasound probe 5 scans the end as the coordinate origin, and an ultrasound probe coordinate system ult based on the ultrasound probe 5 is established. Therefore, the conversion relationship formula (6) between the world coordinate system word and the pixel coordinate system pic can be rewritten as follows:

[0094] Formula (7)

[0095] wherein, represents the pixel coordinates of the imaging point 12 of the needle point 13 in the image coordinate system img, which can be obtained by reading the pixel or digital pixel;

[0096] represents the coordinate transformation matrix from the world coordinate system word to the ultrasound mechanical arm end coordinate system prob1, which is obtained by solving the forward kinematics and joint angle of the mechanical arm;

[0097] ​represents the coordinate transformation matrix from the ultrasound probe coordinate system ult to the image coordinate system imgimg, which is an unknown matrix;

[0098] represents the coordinate transformation matrix from the image coordinate system img to the pixel coordinate system pic, that is, the proportion relationship between the real object of the needle tip disc 8 and the image pixel, which can be obtained according to the proportion relationship calibration.

[0099] represents the coordinate transformation matrix from the image coordinate system img to the pixel coordinate system pic, that is, the proportion relationship between the real object of the needle tip disc 8 and the image pixel, which can be obtained according to the proportion relationship calibration.

[0100] The above coordinate transformation matrix is unknown only between the ultrasound probe coordinate system ult and the image coordinate system imgimg, so the unknown number is set to express the coordinate transformation matrix, and the equation is solved , so as to calculate the coordinate transformation matrix between the world coordinate system word and the pixel coordinate system pic .

[0101] The above embodiments are only used for illustrating the present application, but not for limiting the present application. Although the present application is described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications or equivalent replacements of the technical solutions of the present application do not deviate from the spirit and scope of the present application, and should be covered in the scope of the claims of the present application.

Claims

1. A calibration method for a dual-arm collaborative ultrasonic puncture robot, the calibration device comprising two robotic arms, a reference needle tip disk, a calibration cone, a calibration phantom, a puncture device, and an ultrasonic scanning device, wherein the two robotic arms are an ultrasonic robotic arm and a puncture robotic arm, and the reference needle tip disk is provided with multiple needle tips for calibration, characterized in that, Includes the following steps: Step 1: Set up the calibration device. Install the first calibration cone and the ultrasonic probe of the ultrasonic scanning device together at the execution end of the ultrasonic robotic arm through the flange. Install the second calibration cone and the puncture device together at the execution end of the puncture robotic arm through the flange. During the entire calibration process, the relative positions of the reference needle tip plate, the base of the ultrasonic robotic arm, and the base of the puncture robotic arm remain unchanged. Step 2: Calibration between the two robotic arms. Establish a world coordinate system based on the base of the ultrasonic robotic arm and a base coordinate system for the puncture robotic arm based on the base of the puncture robotic arm. Operate the ultrasonic robotic arm and the puncture robotic arm so that the calibration cones at the ends of the two robotic arms touch. Solve the first coordinate transformation matrix between the world coordinate system and the base coordinate system of the puncture robotic arm through coordinate transformation. Step 3: Establish a puncture coordinate system based on the puncture device. Based on the relative positional relationship between the puncture device and the puncture robotic arm, calculate the second coordinate transformation matrix between the puncture coordinate system and the world coordinate system. Step 4: Operate the ultrasonic robotic arm so that the tip of the first calibration cone at its end contacts multiple needle tips on the reference needle tip disk in sequence. Calculate the coordinates of the needle tips in the world coordinate system through coordinate transformation, and convert them to the coordinates of the needle tips in the coordinate system of the ultrasonic robotic arm's end effector. The method for obtaining this coordinates is as follows: The first method involves directly calculating the coordinates of the needle tip in the coordinate system of the ultrasonic robotic arm end effector by using the geometric dimensions of the first calibration cone and the installation relationship between the execution end of the ultrasonic robotic arm. The second method involves establishing a first cone coordinate system, setting unknown parameters to express the transformation matrix between the first cone coordinate system and the ultrasonic robotic arm end coordinate system, colliding multiple needle tips on the reference needle tip disk, and knowing the relative positional relationship between the multiple needle tips. By formulating equations to solve for the unknown parameters, the transformation matrix between the first cone coordinate system and the ultrasonic robotic arm end coordinate system is obtained, thereby obtaining the coordinates of the needle tip in the robotic arm end coordinate system. Step 5: Wrap the reference needle tip disk or the upper part of the needle tip of the reference needle tip disk with a calibration phantom, and scan with an ultrasonic probe carried by an ultrasonic robotic arm to obtain the position of the needle tip in the reference needle tip disk in the ultrasonic image. Establish an image coordinate system based on the ultrasonic image, and calculate the coordinates of the needle tip in the image coordinate system by using the known pixel corresponding to the actual distance. Step 6: Express the transformation matrix from the world coordinate system to the image coordinate system by setting unknowns. Multiply the image coordinates of each needle tip in the reference needle tip disk by the transformation matrix to express the coordinates in the world coordinate system. The coordinates are equal to those obtained in Step 4. Multiple needle tips can form multiple sets of equations. Solve the unknowns in the transformation matrix from the sets of equations to obtain the third coordinate transformation matrix from the world coordinate system to the image coordinate system. This completes the calibration of the image coordinate system and the puncture coordinate system in the world coordinate system.

2. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 1, characterized in that: The solution method in step 2 is as follows: The transformation matrix from the world coordinate system to the base coordinate system of the puncture robot arm is expressed by setting unknowns as follows: Formula (1) in This represents the first coordinate transformation matrix from the world coordinate system to the base coordinate system of the puncture robot arm. , representing the coordinates of the tip of the first calibration cone in the world coordinate system, which can be calculated using the characteristics of the ultrasonic robotic arm itself. The coordinates of the tip of the second cone in the base coordinate system of the puncture robot arm are calculated using the characteristics of the puncture robot arm itself. By changing the posture of the ultrasonic robotic arm and the puncture robotic arm and performing tip contact at different positions, multiple sets of formulas based on formula (1) are obtained, and the first coordinate transformation matrix is ​​obtained by formula calculation.

3. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 2, characterized in that: In step 2, the formula for calculating the coordinates of the tip of the first calibration cone in the world coordinate system is as follows: Formula (2) This represents the coordinates of the tip of the first calibration cone in the first calibration cone coordinate system. The coordinate transformation matrix representing the world coordinate system and the end-effector coordinate system of the ultrasonic robotic arm is obtained by solving the forward kinematics and joint angles of the robotic arm. The coordinate transformation matrix between the coordinate system of the end effector of the ultrasonic robotic arm and the coordinate system of the first cone is determined by the installation relationship of the first calibration cone relative to the end effector of the ultrasonic robotic arm or calculated by calibration. Similarly, the formula for calculating the coordinates of the tip of the second calibration cone in the base coordinate system of the puncture robot arm is as follows: Formula (3) This indicates the coordinates of the tip of the second calibration cone in the base coordinate system of the puncture robot arm. This indicates the coordinates of the tip of the second calibration cone in the second calibration cone coordinate system. The coordinate transformation matrix between the base coordinate system and the end effector coordinate system of the puncture robot arm is calculated using the forward kinematics of the robot arm and the joint angles. The coordinate transformation matrix between the coordinate system of the end effector of the puncture robot arm and the coordinate system of the second cone is determined by the installation relationship of the second calibration cone relative to the end effector of the puncture robot arm or calculated by calibration.

4. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 3, characterized in that: In step 4, the posture of the ultrasonic robotic arm is changed by joint movement, and at least two ultrasonic robotic arm postures are used to complete the contact with the same needle tip, and the coordinates of the needle tip are corrected by at least two postures.

5. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 2, characterized in that: In step 4, the first cone coordinate system is established based on the ultrasonic robotic arm end coordinate system, that is, at least one coordinate axis of the first cone coordinate system coincides with the robotic arm end coordinate system.

6. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 4, characterized in that: In step 4, the image coordinate system takes the uppermost vertex of the midline of the image's axis of symmetry as its origin. Based on the same position, a pixel coordinate system for the ultrasound image is established. The coordinates of the needle tip in the pixel coordinate system are determined by the pixel position of the needle tip in the ultrasound image. The coordinates of the needle tip in the image coordinate system are obtained by the proportional relationship between the pixel coordinate system and the image coordinate system.

7. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 4, characterized in that: The calibration phantom is made of a material that can mimic human tissue in ultrasound images.

8. The calibration method for the dual-arm collaborative ultrasonic puncture robot according to claim 5, characterized in that: In step 4, the reference needle tip disk has at least 4 needle tips, of which three needle tips need to be calibrated in the world coordinate system by touching, and the remaining needle tips are directly wrapped by the calibration phantom. The needle tips wrapped by the calibration phantom are calculated in the world coordinate system according to the geometric dimensional relationship between the needle tips on the reference needle tip disk.

9. A dual-arm collaborative ultrasonic puncture robot, characterized in that: The device includes a dual-arm coordinated ultrasonic puncture device and a controller with a built-in algorithm based on the calibration method of any one of claims 1-8. The dual-arm coordinated ultrasonic puncture device includes two robotic arms, a reference needle tip disk, a calibration cone, a calibration phantom, a puncture device, and an ultrasonic scanning device. The two robotic arms are an ultrasonic robotic arm and a puncture robotic arm, respectively. The first calibration cone and the ultrasonic probe of the ultrasonic scanning device are mounted together at the execution end of the ultrasonic robotic arm via a flange, and the second calibration cone and the puncture device are mounted together at the execution end of the puncture robotic arm via a flange. The reference needle tip disk is provided with a plurality of needle tips for calibration.

Images