Robot end execution mechanism and system for puncture intervention and positioning method

Abstract

According to the puncture intervention robot end executing mechanism and system and the positioning method, the end executing mechanism comprises a clamping module and a guide needle module, and the clamping module comprises a carrying piece body, a development board, a connecting piece, a rotary driving mechanism, a clamping device, an ultrasonic probe and an electromagnetic sensor; the needle guide module comprises a guide rail, a sliding block, a needle guide device and displacement driving mechanisms, the coordinate system of the whole system is unified, meanwhile, calibration is conducted through an electromagnetic-visual registration method, operation planning is conducted after preoperative CT scanning is conducted on an affected part of a patient, and after a target pose is obtained through calculation of a constraint positioning algorithm, the target pose is determined to be the target pose. The rotation driving mechanism drives the needle guiding device to rotate, non-coplanar puncture channel path selection is achieved, the displacement driving mechanism drives the needle guiding device to move, the needle entering angle is adjusted, the puncture process and target spot information are monitored in real time based on real-time ultrasonic images, operation errors are reduced, the puncture precision is improved, and the operation safety is improved. And radiation exposure and postoperative complications of a patient are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a robot end effector mechanism, system and positioning method for puncture intervention. Background Art

[0002] Hepatocellular carcinoma (HCC), as a malignant tumor, has become one of the major global health challenges. Traditional ablation interventional surgery usually relies on preoperative and intraoperative X-ray images. Doctors use 18G puncture needles to puncture the tumor and use microwave or radiofrequency ablation needles for treatment. However, there are many limitations in actual surgery, such as the lack of intuitive matching between the image and the actual location of the tumor, the difficulty in ensuring surgical accuracy, and doctors often rely on experience to judge the puncture point. This method is more invasive to patients and the operation is difficult. The doctor's physiological shaking may cause instability, thereby prolonging the operation time, increasing the risk of radiation exposure, and causing serious postoperative complications.

[0003] At present, there are also some surgical robot positioning systems for a wide range of percutaneous puncture surgeries or specializing in percutaneous puncture surgeries for liver tumors, such as patent documents with application numbers CN201811619721.5 (hereinafter referred to as D1), CN201821342041.9 (hereinafter referred to as D2) and CN202410683509.4 (hereinafter referred to as D3). Among them, D1 is a tumor treatment system with a robotic arm puncture. After system calibration, the robotic arm is translated to the needle entry point based on the lesion localization technology of RGB optical images, preoperative CT images, and intraoperative three-dimensional ultrasound images, and the robotic arm is translated to puncture the patient. The system uses a positioning technology based on RGB image processing. However, the RGB image information is limited to a two-dimensional plane perpendicular to the optical axis of the camera, lacking real-time and accurate three-dimensional positioning information, and thus lacking positioning calculation in three-dimensional space. The positioning system can only locate to the target plane but cannot track the insertion depth of the puncture needle, which will lead to the loss of real-time spatial position information of the needle tip and cannot well confirm the target posture of the robotic arm in three-dimensional space. Moreover, its end-effector cannot plan and locate non-coplanar puncture paths in real time during the positioning process, lacking flexibility and adaptability to complex surgeries. For tumor puncture surgeries that require high precision, there are still large errors and lack of safety.

[0004] D2's precise positioning tumor surgical robot system achieves positioning and surgical operations only through a six-degree-of-freedom robotic arm, a tracking marker component and a far-infrared camera. The content disclosed by D2 emphasizes that the installation position of the tracking marker component improves the accuracy of positioning, but does not provide detailed calculation steps and solutions for precise positioning. Its solution only uses an optical locator for tracking and positioning, does not introduce three-dimensional CT images and real-time ultrasound images, lacks anatomical information of the lesion, cannot monitor the puncture process and target information in real time to adapt to complex surgical situations, and cannot guarantee the accuracy of the surgery and the safety of the patient;

[0005] D3's liver puncture surgical robot achieves positioning and surgical operations only through a six-degree-of-freedom robotic arm, a puncture needle posture adjustment mechanism, an ultrasound imaging system, and an electromagnetic tracking system. However, in the public solution of D3, only the adjustment of coplanar puncture based on ultrasound images during surgery is emphasized, and the preoperative surgical planning and automatic positioning methods are not given in detail. The system relies on manual operation and has a limited degree of automation. Its end actuator can only be adjusted within the ultrasound imaging plane, lacking flexibility and adaptability. It fails to show the precise registration relationship between the system and medical images, and lacks three-dimensional lesion anatomical information, making it impossible to guarantee the accuracy of the surgery and protect the safety of the patient. Summary of the invention

[0006] In view of this, the present invention proposes a puncture intervention robot end effector mechanism, system and positioning method to solve the above problems.

[0007] The technical solution of the present invention is achieved in this way:

[0008] A robot end effector for puncture intervention, comprising a clamping module and a needle guide module, the clamping module comprising a carrier body, a development board, a connector, a rotation drive mechanism, a clamp, an ultrasonic probe and an electromagnetic sensor, the development board and the rotation drive mechanism are arranged in the carrier body, the connector is rotatably connected to the carrier body, the rotation drive mechanism drives the connector to rotate, the clamp is arranged on one side of the carrier body, the electromagnetic sensor is arranged on the outer wall of the clamp, and the ultrasonic probe is arranged on the end of the clamp; the needle guide module comprises a guide rail, a slider, a needle guide and a displacement drive mechanism, one end of the guide rail is connected to the outer wall of the connector, the slider is slidably arranged on the guide rail, the needle guide is connected to the slider, and the displacement drive mechanism is arranged on the guide rail for driving the slider to move along the guide rail; the development board is data-connected to the rotation drive mechanism and the displacement drive mechanism respectively.

[0009] Furthermore, the mounting component body includes a first integrated mounting component and a second integrated mounting component, the connecting component is located between the first integrated mounting component and the second integrated mounting component, the development board and the rotation drive mechanism are arranged in the first integrated mounting component, and the clamp is connected to the second integrated mounting component.

[0010] Furthermore, the rotation drive mechanism includes a Mouhe mounting column, a bearing, a first gear, a first servo motor and a second gear. The Mouhe mounting column is arranged on the first integrated mounting component, the bearing sleeve is arranged on the outside of the Mouhe mounting column and passes through the connecting component, the first gear sleeve is arranged on the outside of the bearing, the first servo motor is arranged in the first integrated mounting component, and its output shaft is connected to the second gear, the second gear is meshed with the first gear, and the development board is data connected to the first servo motor.

[0011] Furthermore, the displacement drive mechanism includes a rack, a mounting plate, a second servo motor and a third gear. The rack is arranged at the bottom of the guide rail, one end of which is connected to the outer wall of the connecting piece, one side of the mounting plate is connected to the slider, the guide rail and the rack are located inside the mounting plate, the second servo motor is arranged on the mounting plate, and its output shaft is connected to the third gear, the third gear is meshed with the rack, and the development board is data connected to the second servo motor.

[0012] A system including the robot end effector for puncture intervention, further including a robotic arm moving platform, a serial robotic arm, an operating table, an electromagnetic positioning system, an RGB-D camera and an ultrasonic imaging system, wherein one end of the serial robotic arm is arranged on the robotic arm moving platform, and the other end is connected to the end effector, the ultrasonic imaging system is arranged on the robotic arm moving platform, and the end effector, the electromagnetic positioning system and the RGB-D camera are located above the operating table.

[0013] A positioning method based on a system including the puncture intervention robot end effector, comprising the following steps:

[0014] Step S1, performing a preoperative CT scan on the affected part of the patient;

[0015] Step S2, constructing a calibration closed loop according to the coordinate system of the electromagnetic positioning system, the robot arm moving platform and the end actuator;

[0016] Step S3, registering the RGB-D camera into the calibration closed loop based on the electromagnetic-vision registration method;

[0017] Step S4, performing surgical planning through a graphical user interface and registering the preoperative CT image with the real-time data of the intraoperative RGB-D camera;

[0018] Step S5, using a constrained positioning algorithm to calculate the target pose;

[0019] Step S6: drive the needle guide to move to accurately align the needle insertion path.

[0020] Furthermore, the specific steps of step S2 are:

[0021] Step S21, controlling the serial robot arm to perform multiple groups of translational movements along the robot arm moving platform coordinate system and the end effector coordinate system;

[0022] Step S22, expressing the coordinate system of the manipulator mobile platform in the coordinate system of the electromagnetic positioning system through closed-loop calibration, and updating the coordinate system of the end actuator to the coordinate system of the built-in electromagnetic sensor;

[0023] Step S23: define the needle guide coordinate system through the auxiliary positioning needle equipped by the electromagnetic positioning system, further update the end actuator coordinate system to the needle guide coordinate system, and construct a calibration closed loop.

[0024] Furthermore, the specific steps of step S3 are:

[0025] Step S31: Perform internal calibration on the RGB-D camera to obtain the focal length f x 、f y And the main point (c x 、c y ), use an RGB-D camera to shoot the chessboard calibration plate, and use the chessboard corner point detection algorithm to obtain the two-dimensional pixel coordinates (x, y) of a corner point;

[0026] Step S32: obtain the depth information d of the corresponding corner point position on the depth image by scale reduction, and obtain the three-dimensional point cloud coordinates of the corner point by the following formula:

[0027] Z=d

[0028]

[0029] The 3D coordinate information of the 3D corner point relative to the RGB-D camera is p i (X, Y, Z), by repeating the above steps, we can obtain multiple sets of 3D data of corner points in the RGB-D camera coordinate system and form a point cloud set C 1＝ {p1,p2,p3…p n};

[0030] Step S33: Use the electromagnetic positioning system to assist the positioning needle to manually collect the three-dimensional data q of a corner point relative to the electromagnetic positioning system coordinate system. i (X, Y, Z), which is a point cloud dataset C of multiple groups of corner points expressed in the coordinate system of the electromagnetic positioning system 2＝ {q1,q2,q3…q n}, each corresponding point in the two point cloud datasets satisfies the following transformation relationship:

[0031] MT R ·p i =q i

[0032] MT R is the homogeneous transformation matrix from the RGB-D camera coordinate system to the electromagnetic positioning system coordinate system;

[0033] Step S34: Use the Levenberg-Marquart algorithm to minimize the equation f1=||MT R ·p i -q i ||Estimated MT R , registering the RGB-D camera into the calibration closed loop.

[0034] Furthermore, the specific steps of step S5 are:

[0035] Step S51: convert the needle insertion path planned by the preoperative CT scan into the coordinate system of the robot arm moving platform. The needle diameter is expressed as the needle insertion point in the coordinate system of the robot arm moving platform. Point to the puncture target The vector whose unit vector is The unit vector of the x-axis of the electromagnetic positioning system in the coordinate system of the manipulator moving platform is: The unit vector from the puncture target point l2 to the origin of the electromagnetic positioning system is

[0036] Step S52: When the auxiliary positioning needle is used to define the needle guide, the needle guide is shallowly inserted and the first set of point data is collected. Then, the auxiliary positioning needle is deeply inserted along the needle guide, and its y-axis is ensured to face the electromagnetic positioning system, and the second set of point data is collected. The vector of the needle guide is obtained through the two sets of points. The coordinate system of the positioning needle at the time of deep insertion is used as the coordinate system of the needle guide, and the unit vectors along the x, y, and z axes of the needle guide are obtained. Calculate the direction cosine parameters (Cosα, Cosβ, Cosγ) between the needle guide vector and the x, y, z axes of the needle guide coordinate system;

[0037] Step S53: y-axis unit vector in target posture The expression in the robot moving platform coordinate system is solved by the following formula:

[0038]

[0039] Find the only Then, it is expressed by the following equation

[0040]

[0041] From the above, we can solve the two corresponding groups and They are and The following constraints determine a unique set of and

[0042]

[0043] This calculates the only Confirm the target posture;

[0044] Step S54: Due to the target location A certain distance from the needle insertion point is required, so l3 is defined by the following formula:

[0045]

[0046] Where d is the distance between l3 and l1, so the target pose defined in the manipulator mobile station coordinate system is defined by the following homogeneous transformation matrix:

[0047]

[0048] The rotation matrix in the homogeneous transformation is converted into Euler angle representation (Φ, θ, Ψ), and the target pose is

[0049] Furthermore, the method further comprises the following steps:

[0050] Step S7: Perform real-time ultrasound scanning, and after fine-tuning the needle guide, clamp the ablation needle on the needle guide holder, and perform needle insertion under the guidance of real-time ultrasound images.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention provides a robot end effector for puncture intervention. The rotary drive mechanism can drive the needle guide to rotate along the rotation axis of the clamp to achieve non-coplanar puncture channel path selection. At the same time, the displacement drive mechanism can drive the slider and the needle guide to slide along the guide rail to adjust the needle insertion angle. The puncture process and target point information are monitored in real time based on real-time ultrasound images during surgery to reduce surgical errors, improve puncture accuracy, improve surgical safety, and reduce radiation exposure and postoperative complications of patients.

[0053] The present invention also provides a system and positioning method including the above-mentioned robot end effector for puncture intervention. After the patient lies on the operating table, the electromagnetic positioning system can obtain the position information of the electromagnetic sensor in real time, collect spatial position data for a specific algorithm, and the RGB-D camera is used to register the patient's real-time spatial information. The ultrasonic imaging system can provide the patient's real-time anatomical structure to guide the serial robot arm to automatically fine-tune the puncture posture to improve the puncture accuracy. Before the surgical puncture, the various coordinate systems defined before or during the operation are expressed in the coordinate system of the robot arm moving platform through the electromagnetic-visual registration method, and the preoperative C The planning information in T enables the robot to automatically perform precise alignment according to the planning information through the "spatial constraint positioning algorithm", and then can realize non-coplanar puncture channel path selection based on the end effector, and monitor the puncture process and target target information in real time based on the real-time ultrasound image during the operation, so as to reduce surgical errors, improve puncture accuracy, improve surgical safety, reduce patients' radiation exposure and postoperative complications. At the same time, the doctor can observe the puncture situation in real time through the ultrasound image, fine-tune the puncture needle direction, and minimize the risk until the needle guide is released after reaching the puncture target, completing the robot-assisted positioning percutaneous puncture ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 It is a structural schematic diagram of a puncture intervention robot end effector of the present invention;

[0056] Figure 2 It is an exploded view of the parts of the end effector of a puncture intervention robot of the present invention;

[0057] Figure 3 It is a structural schematic diagram of a main body of a carrying part of a robot end effector for puncture intervention according to the present invention;

[0058] Figure 4 A schematic diagram of a system including a puncture intervention robot end effector of the present invention;

[0059] Figure 5 A schematic diagram of the relationship between various coordinate systems of a system including a puncture intervention robot end effector of the present invention;

[0060] Figure 6It is an overall flow chart of a positioning method of a system including a robot end effector for puncture intervention of the present invention;

[0061] Figure 7 A flowchart of a system calibration method for positioning a system including a robot end effector for puncture intervention according to the present invention;

[0062] Figure 8 A pseudo code calculation flow chart of a constraint positioning algorithm of a positioning method of a system including a robot end effector for puncture intervention according to the present invention;

[0063] In the figure, 1. Robotic arm moving platform; 2. tandem robot arm; 3. end actuator; 300. Mouhe mounting column; 301. Development board; 302. Clamp; 303. Electromagnetic sensor; 304. Ultrasonic probe; 305. First integrated mounting part; 306. First gear; 307. Bearing; 308. Connector; 309. Second integrated mounting part; 310. First servo motor; 311. Second gear; 312. Second servo motor; 313. Third gear; 314. Mounting plate; 315. Rack; 316. Slider; 317. Needle guide; 318. Guide rail; 4. Operating table; 5. Electromagnetic positioning system; 6. RGB-D camera; 7. Ultrasonic imaging system. DETAILED DESCRIPTION

[0064] In order to better understand the technical content of the present invention, a specific embodiment is provided below, and the present invention is further described in conjunction with the accompanying drawings.

[0065] See also Figures 1 to 3 The present invention provides a puncture intervention robot end effector, including a clamping module and a guide needle module, the clamping module includes a carrier body, a development board 301, a connecting piece 308, a rotation drive mechanism, a clamp 302, an ultrasonic probe 304 and an electromagnetic sensor 303, the development board 301 and the rotation drive mechanism are arranged in the carrier body, the connecting piece 308 is rotatably connected to the carrier body, the rotation drive mechanism drives the connecting piece 308 to rotate, the clamp 302 is arranged on one side of the carrier body, and the electromagnetic sensor 303 is arranged The ultrasonic probe 304 is arranged on the outer wall of the clamp 302, and the guide needle module includes a guide rail 318, a slider 316, a needle guide 317 and a displacement driving mechanism. One end of the guide rail 318 is connected to the outer wall of the connecting piece 308, the slider 316 is slidably arranged on the guide rail 318, the needle guide 317 is connected to the slider 316, and the displacement driving mechanism is arranged on the guide rail 318 for driving the slider 316 to move along the guide rail 318; the development board 301 is respectively connected to the data of the rotation driving mechanism and the displacement driving mechanism.

[0066] The present invention provides a puncture intervention robot end effector, which includes two parts, namely, a clamping module based on an ultrasound probe 304, and a guide needle module for installing an ablation needle, wherein the carrier body is used to install a development board 301 and a rotation drive mechanism, and a connecting member 308 is provided on the carrier body, and the rotation drive mechanism can drive the connecting member 308 to rotate, and the outer wall of the connecting member 308 is connected to the guide needle module, so that the guide needle module can rotate along the rotation axis ( Figure 1 The ultrasonic probe 304 is rotated along the dotted line in the figure to realize the selection of non-coplanar puncture channel paths, and a clamp 302 is provided at one end of the carrier body for clamping the ultrasonic probe 304, and the electromagnetic sensor 303 is installed on the outer wall of the clamp 302. The ultrasonic probe 304 can perform real-time ultrasonic scanning on the patient to provide the anatomical structure of the liver tumor, and the electromagnetic sensor 303 can provide the necessary posture information of the end actuator 3.

[0067] On one side of the connector 308 is a needle guide module, a needle guide 317 is connected to a slider 316, the slider 316 is located on a guide rail 318 and can slide on the guide rail 318, and the displacement drive mechanism can drive the slider 316 to slide along the guide rail 318, thereby adjusting the angle of the needle insertion, and real-time monitoring of the puncture process and target target information based on real-time ultrasound images during the operation, so as to reduce surgical errors, improve puncture accuracy, improve surgical safety, and reduce patients' radiation exposure and postoperative complications.

[0068] Furthermore, the mounting component body includes a first integrated mounting component 305 and a second integrated mounting component 309 , the connecting component 308 is located between the first integrated mounting component 305 and the second integrated mounting component 309 , the development board 301 and the rotation drive mechanism are arranged in the first integrated mounting component 305 , and the clamp 302 is connected to the second integrated mounting component 309 .

[0069] The first integrated mounting component 305 and the second integrated mounting component 309 are a complete structure, wherein the first integrated mounting component 305 is located above the second integrated mounting component 309, and can be used to place the development board 301 and install the rotating drive mechanism, and the connecting component 308 is located between the first integrated mounting component 305 and the second integrated mounting component 309, and can be rotated under the drive of the rotating drive mechanism to adjust the position of the entire guide needle module.

[0070] Furthermore, the rotation drive mechanism includes a Mouhe mounting column 300, a bearing 307, a first gear 306, a first servo motor 310 and a second gear 311. The Mouhe mounting column 300 is arranged on the bottom surface of the first integrated mounting component 305. The bearing 307 is sleeved on the outside of the Mouhe mounting column 300 and passes through the connecting component 308. The first gear 306 is sleeved on the outside of the bearing 307. The first servo motor 310 is arranged in the first integrated mounting component 305, and its output shaft passes through the bottom surface of the first integrated mounting component 305 and is connected to the second gear 311. The second gear 311 is meshed with the first gear 306. The development board 301 is data-connected to the first servo motor 310.

[0071] The Mouhe mounting column 300 is a cylinder larger than 35 mm extending downward concentrically with the first integrated mounting component 305. It passes through the first gear 306, the bearing 307 and the middle part of the connecting component 308, limits the rotation to the concentric circle, and is limited by the second integrated mounting component 309, wherein the inner ring of the bearing 307 will be connected to the outer wall of the Mouhe mounting column 300, and the outer ring of the bearing 307 passes through the connecting component 308. At the same time, the height of the bearing 307 will be greater than the height of the connecting component 308, and the part located above the connecting component 308 is sleeved with the first gear 306. Driven by the first servo motor 310 in the first integrated mounting component 305, the second gear 311 will rotate, and drive the outer ring of the bearing 307 and the connecting component 308 to rotate through the first gear 306, thereby driving the guide needle module to rotate for fine-tuning.

[0072] Furthermore, the displacement drive mechanism includes a rack 315, a mounting plate 314, a second servo motor 312 and a third gear 313. The rack 315 is arranged at the bottom of the guide rail 318, and one end of the rack 315 is connected to the outer wall of the connecting piece 308. One side of the mounting plate 314 is connected to the slider 316. The guide rail 318 and the rack 315 are located inside the mounting plate 314. The second servo motor 312 is arranged on the mounting plate 314, and its output shaft is connected to the third gear 313. The third gear 313 is meshed with the rack 315, and the development board 301 is data-connected to the second servo motor 312.

[0073] The rack 315 and the guide rail 318 are in an arc structure, and the mounting plate 314 is similar to a U-shaped structure. The rack 315 and the guide rail 318 both pass through the interior of the mounting plate 314. After the slider 316 is slidably set on the guide rail 318, the upper part of the mounting plate 314 is connected to the top side of the slider 316. When the second servo motor 312 is started, it can drive the third gear 313 to rotate, and the third gear 313 is engaged with the rack 315. Since the end of the rack 315 is fixedly connected to the connecting piece 308, the slider 316 can move along the arc guide rail 318 to adjust the needle insertion angle, and the puncture needle can be guided into the patient's body through the needle guide 317.

[0074] When the end actuator 3 reaches the target posture, the ultrasonic probe 304 can capture the image with the best quality. According to the real-time feedback of the ultrasonic image and the control method, the first servo motor 310 drives the second gear 311 to drive the first gear 306 to rotate, so that the connecting part 308 drives the guide needle module to rotate as a whole, and drives the guide needle 317 to rotate along the rotation axis of the supporting body to realize the selection of non-coplanar puncture channel paths. At the same time, the second servo motor 312 can drive the third gear 313 to rotate, and the slider 316 can slide on the arc guide rail 318 through the fixed rack 315, so as to adjust the angle of the needle insertion, and monitor the puncture process and target target information in real time based on the real-time ultrasonic image during the operation, so as to reduce surgical errors, improve puncture accuracy, improve surgical safety, and reduce patients' radiation exposure and postoperative complications.

[0075] See also Figure 4 A system including the robot end effector for puncture intervention is shown, and also includes a robotic arm moving platform 1, a serial robotic arm 2, an operating table 4, an electromagnetic positioning system 5, an RGB-D camera 6 and an ultrasonic imaging system 7. One end of the serial robotic arm 2 is arranged on the robotic arm moving platform 1, and the other end is connected to the end effector 3. The ultrasonic imaging system 7 is arranged on the robotic arm moving platform 1, and the end effector 3, the electromagnetic positioning system 5 and the RGB-D camera 6 are located above the operating table 4.

[0076] When the end effector 3 is used in the treatment process, it can be installed on the serial robot arm 2. The serial robot arm 2 includes a six-degree-of-freedom robot arm, which can drive the end effector 3 to adjust its position so as to approach the patient on the operating table 4. The movement of the serial robot arm 2 can be achieved through the robot arm moving platform 1. The electromagnetic positioning system 5 located on one side of the operating table 4 can obtain the posture information of the electromagnetic sensor 303 on the clamp 302, and collect spatial posture data for a specific algorithm. The RGB-D camera 6 can be used to register the patient's real-time spatial information, and the ultrasonic imaging system 7 can provide the patient's real-time anatomical structure to guide the serial robot arm 2 to automatically fine-tune the puncture posture to improve the puncture accuracy.

[0077] The serial robot arm 2 of the present invention is a type of robot arm combination with a universal structure. The six-degree-of-freedom robot arm is a commercial UR5 collaborative robot arm, and a two-degree-of-freedom end actuator 3 is installed at its end. The flexible movement of the six-degree-of-freedom serial robot arm 2 automatically operates the end actuator 3 to perform ultrasonic scanning and puncture positioning tasks, and the additional two degrees of freedom on the end actuator 3 are used to fine-tune the needle guide 317, thereby reducing puncture errors and improving surgical accuracy.

[0078] See also Figure 5 As shown in the coordinate system diagram, the system of the present invention is designed to have seven main coordinate systems, namely the robot arm moving platform 1 coordinate system {W} (i.e., the world coordinate system), the end actuator 3 coordinate system {E}, the electromagnetic positioning system 5 coordinate system {M}, the RGB-D camera 6 coordinate system {R}, the coordinate system {C}, the intraoperative ultrasound image coordinate system {U}, and the patient coordinate system {P}, wherein the ① ring is defined as the "main ring", and the calibration ring includes the hand-eye calibration and the definition of the needle guide 317 coordinates, and the ② ring is a general term for all the "secondary rings" except the "main ring". Through the calibration of the ultrasound probe 304, the "system calibration method based on the electromagnetic locator" and the preoperative-intraoperative image registration, the ultrasound image {U}, the patient {P} (expressed under the RGB-D camera 6 {R}) and the preoperative CT image {C} can be included in the "main ring" to achieve closed-loop calibration, and the coordinate system is placed under the world coordinate system {W}.

[0079] See also Figure 6-8 A positioning method based on a system including the puncture intervention robot end effector 3 is shown, comprising the following steps:

[0080] Step S1, performing a preoperative CT scan on the affected part of the patient;

[0081] Step S2: In order to obtain the conversion relationship between the coordinate system {M} of the electromagnetic positioning system 5 and the coordinate system {W} of the manipulator mobile platform 1, a calibration closed loop is constructed according to the coordinate systems of the electromagnetic positioning system 5, the manipulator mobile platform 1 and the end actuator 3. The specific steps are as follows:

[0082] Step S21, controlling the serial robot arm 2 to perform multiple groups of translational movements along the coordinate system {W} of the robot arm moving platform 1 and the coordinate system of the end effector 3;

[0083] Step S22, expressing the coordinate system {W} of the robot arm mobile platform 1 in the coordinate system {M} of the electromagnetic positioning system 5 through closed-loop calibration, and updating the coordinate system of the end actuator 3 from the original end coordinate system {E1} of the serial robot arm 2 to the coordinate system {E2} of the built-in electromagnetic sensor 303;

[0084] Step S23, since the coordinate system of the needle guide 317 can be defined by the auxiliary positioning needle equipped with the electromagnetic positioning system 5, the coordinate system of the end actuator 3 can be further updated to the needle guide 317 coordinate system {E3}, and finally the {E3} coordinate system is defined as the coordinate system {E} of the end actuator 3. Therefore, the coordinate system {M} of the electromagnetic positioning system 5 and the coordinate system {E} of the end actuator 3 can be unified and expressed under the coordinate system {W} of the robot arm moving platform 1, and a closed-loop conversion relationship based on the electromagnetic positioning system 5 and the serial robot arm 2 is established.

[0085] Step S3: In order to register the RGB-D camera 6 into the calibration closed loop, the present invention proposes an electromagnetic-vision registration method, and the specific steps are as follows:

[0086] Step S31: After fixing the RGB-D camera 6 and the electromagnetic positioning system 5, the focal length f is obtained by performing internal calibration on the RGB-D camera 6. x 、f y And the main point (c x 、c y ), use the RGB-D camera 6 to shoot the 6*10 chessboard calibration plate from the front, and use the chessboard corner point detection algorithm to obtain the two-dimensional pixel coordinates (x, y) of a corner point;

[0087] Step S32: obtain the depth information d of the corresponding corner point position on the depth image by scale reduction, and obtain the three-dimensional point cloud coordinates of the corner point by the following formula:

[0088] Z=d

[0089]

[0090] The three-dimensional coordinate information of the three-dimensional corner point relative to the RGB-D camera 6 is p i (X, Y, Z), by repeating the above steps, multiple sets of corner point 3D data in the RGB-D camera 6 coordinate system are obtained and formed into a point cloud set C 1＝ {p1,p2,p3…p n};

[0091] Step S33: Use the electromagnetic positioning system 5 to assist the positioning needle to manually collect the three-dimensional data q of a corner point relative to the coordinate system of the electromagnetic positioning system 5. i (X, Y, Z), which constitutes the point cloud data set C of multiple groups of corner points expressed in the 5-coordinate system of the electromagnetic positioning system 2＝ {q1,q2,q3…q n}, each corresponding point in the two point cloud datasets satisfies the following transformation relationship:

[0092] MT R ·p i =q i

[0093] MT R is the homogeneous transformation matrix from the coordinate system of the RGB-D camera 6 to the coordinate system of the electromagnetic positioning system 5;

[0094] Step S34: Use the Levenberg-Marquart algorithm to minimize the equation f1=||MT R ·p i -q i ||Estimated MT R , registering the RGB-D camera 6 into the calibration closed loop. This method is different from the traditional hand-eye calibration method. It is equivalent to using a lightweight auxiliary positioning pin to replace the heavy robotic arm, making the camera registration process more convenient and efficient.

[0095] Step S4, planning ablation needle intervention information based on preoperative CT images through a graphical user interface, and then registering the preoperative CT images with the real-time data of the intraoperative RGB-D camera 6;

[0096] Step S5: In order to calculate the target posture that aligns the needle guide 317 with the preoperative planned needle insertion path, the present invention proposes a constrained positioning algorithm based on the electromagnetic positioning system 5 to calculate the target posture. The specific steps are:

[0097] Step S51: After the patient is registered during the operation, the needle insertion path planned by the preoperative CT is converted to the coordinate system {W} of the robot arm moving platform 1. The needle diameter is expressed as the needle insertion point in the coordinate system {W} of the robot arm moving platform 1. Point to the puncture target The vector whose unit vector is The unit vector of the 5x axis of the electromagnetic positioning system in the coordinate system {W} of the manipulator mobile platform 1 is The unit vector from the puncture target point l2 to the origin of the electromagnetic positioning system 5 is

[0098] Step S52: When the auxiliary positioning needle is used to define the needle guide 317, the needle guide 317 is shallowly inserted and the first set of point data is collected. Then, the auxiliary positioning needle is deeply inserted along the needle guide 317, and its y-axis is ensured to face the electromagnetic positioning system 5, and the second set of point data is collected. The vector of the needle guide 317 is obtained through the two sets of points. The coordinate system of the positioning needle at the time of deep insertion is used as the coordinate system of the needle guide 317, and the unit vectors along the x, y, and z axes of the needle guide 317 are obtained. Calculate the direction cosine parameters (Cosα, Cosβ, Cosγ) between the needle guide 317 vector and the x, y, z axes of the needle guide 317 coordinate system;

[0099] Step S53: During the puncture process, the electromagnetic positioning system 5 needs to constantly track the electromagnetic sensor 303 on the end actuator 3. When the electromagnetic sensor 303 faces the electromagnetic positioning system 5 and is closest to it, the error of its posture is the lowest. Therefore, the y-axis unit vector in the target posture is The expression in the coordinate system of the robot moving platform 1 is solved by the following formula:

[0100]

[0101] Find the only Then, it is expressed by the following equation

[0102]

[0103] From the above, we can solve the two corresponding groups and They are and The following constraints determine a unique set of and

[0104]

[0105] This calculates the only Confirm the target posture;

[0106] Step S54: Due to the target location A certain distance from the needle insertion point is required, so l3 is defined by the following formula:

[0107]

[0108] Where d is the distance between l3 and l1, so the target pose defined in the coordinate system of the manipulator mobile station 1 is defined by the following homogeneous transformation matrix:

[0109]

[0110] The rotation matrix in the homogeneous transformation is converted into Euler angle representation (Φ, θ, Ψ), and the target pose is

[0111] Based on the above, the coordinate transformation relationship between the world coordinate system {W}, the electromagnetic positioning system 5 coordinate system {M}, the end effector 3 coordinate system {E}, the intraoperative ultrasound image {U}, the preoperative CT image {C}, the RGB-D camera 6 coordinate system {R}, and the patient coordinate system {P} can be efficiently and accurately calculated by methods such as "electromagnetic-visual registration". By expressing each coordinate system defined before or during surgery in the world coordinate system {W}, the planning information in the preoperative CT can be obtained based on this, so that the robot can automatically perform precise alignment according to the planning information through the "spatial constraint positioning algorithm", realize the non-coplanar puncture channel path selection based on the end effector 3, and monitor the puncture process and target target information in real time based on the intraoperative real-time ultrasound image, so as to reduce surgical errors, improve puncture accuracy, improve surgical safety, and reduce patients' radiation exposure and postoperative complications.

[0112] Step S6: drive the needle guide 317 to move to accurately align the needle insertion path.

[0113] Step S7, perform real-time ultrasound scanning, and after fine-tuning the needle guide 317, clamp the ablation needle on the needle holder of the needle guide 317, and perform needle insertion under the guidance of real-time ultrasound images.

[0114] After precise alignment through the constraint positioning algorithm, the movement path of the serial robot arm 2 and the needle guide 317 can be planned to accurately align the needle diameter. At the same time, the doctor can observe the puncture situation in real time through the ultrasound image and fine-tune the puncture needle direction to minimize the risk. The needle guide 317 is released after reaching the puncture target, completing the robot-assisted positioning percutaneous puncture ablation surgery.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A robot end effector for puncture intervention, characterized in that: It includes a clamping module and a needle guide module, the clamping module includes a carrier body, a development board, a connector, a rotation drive mechanism, a clamp, an ultrasonic probe and an electromagnetic sensor, the development board and the rotation drive mechanism are arranged in the carrier body, the connector is rotatably connected to the carrier body, the rotation drive mechanism drives the connector to rotate, the clamp is arranged on one side of the carrier body, the electromagnetic sensor is arranged on the outer wall of the clamp, and the ultrasonic probe is arranged on the end of the clamp; the needle guide module includes a guide rail, a slider, a needle guide and a displacement drive mechanism, one end of the guide rail is connected to the outer wall of the connector, the slider is slidably arranged on the guide rail, the needle guide is connected to the slider, and the displacement drive mechanism is arranged on the guide rail for driving the slider to move along the guide rail; the development board is data-connected with the rotation drive mechanism and the displacement drive mechanism respectively.

2. The robot end effector for puncture intervention according to claim 1, characterized in that: The mounting component body includes a first integrated mounting component and a second integrated mounting component, the connecting component is located between the first integrated mounting component and the second integrated mounting component, the development board and the rotation drive mechanism are arranged in the first integrated mounting component, and the clamp is connected to the second integrated mounting component.

3. The robot end effector for puncture intervention according to claim 2, characterized in that: The rotary drive mechanism includes a Mouhe mounting column, a bearing, a first gear, a first servo motor and a second gear. The Mouhe mounting column is arranged on the first integrated mounting component. The bearing sleeve is arranged on the outside of the Mouhe mounting column and passes through the connecting component. The first gear sleeve is arranged on the outside of the bearing. The first servo motor is arranged in the first integrated mounting component. Its output shaft is connected to the second gear, and the second gear is meshed with the first gear. The development board is data connected to the first servo motor.

4. The robot end effector for puncture intervention according to claim 1, characterized in that: The displacement drive mechanism includes a rack, a mounting plate, a second servo motor and a third gear. The rack is arranged at the bottom of the guide rail, one end of which is connected to the outer wall of the connecting piece, one side of the mounting plate is connected to the slider, the guide rail and the rack are located inside the mounting plate, the second servo motor is arranged on the mounting plate, the output shaft of which is connected to the third gear, the third gear is meshed with the rack, and the development board is data connected to the second servo motor.

5. A system comprising the robot end effector for puncture intervention according to any one of claims 1 to 4, characterized in that: It also includes a robotic arm moving platform, a tandem robotic arm, an operating table, an electromagnetic positioning system, an RGB-D camera and an ultrasonic imaging system. One end of the tandem robotic arm is arranged on the robotic arm moving platform, and the other end is connected to the end actuator. The ultrasonic imaging system is arranged on the robotic arm moving platform, and the end actuator, the electromagnetic positioning system and the RGB-D camera are located above the operating table.

6. A positioning method based on the system of claim 5, characterized in that: The following steps are involved: Step S1, performing a preoperative CT scan on the affected part of the patient; Step S2, constructing a calibration closed loop according to the coordinate system of the electromagnetic positioning system, the robot arm moving platform and the end actuator; Step S3, registering the RGB-D camera into the calibration closed loop based on the electromagnetic-vision registration method; Step S4, performing surgical planning through a graphical user interface and registering the preoperative CT image with the real-time data of the intraoperative RGB-D camera; Step S5, using a constrained positioning algorithm to calculate the target pose; Step S6: drive the needle guide to move to accurately align the needle insertion path.

7. The positioning method according to claim 6, characterized in that: The specific steps of step S2 are: Step S21, controlling the serial robot arm to perform multiple groups of translational movements along the robot arm moving platform coordinate system and the end effector coordinate system; Step S22, expressing the coordinate system of the manipulator mobile platform in the coordinate system of the electromagnetic positioning system through closed-loop calibration, and updating the coordinate system of the end actuator to the coordinate system of the built-in electromagnetic sensor; Step S23: define the needle guide coordinate system through the auxiliary positioning needle equipped by the electromagnetic positioning system, further update the end actuator coordinate system to the needle guide coordinate system, and construct a calibration closed loop.

8. The positioning method according to claim 6, characterized in that: The specific steps of step S3 are: Step S31: Perform internal calibration on the RGB-D camera to obtain the focal length f x 、f y And the main point (c x 、c y ), use an RGB-D camera to shoot the chessboard calibration plate, and use the chessboard corner point detection algorithm to obtain the two-dimensional pixel coordinates (x, y) of a corner point; Step S32: obtain the depth information d of the corresponding corner point position on the depth image by scale reduction, and obtain the three-dimensional point cloud coordinates of the corner point by the following formula: Z=d The 3D coordinate information of the 3D corner point relative to the RGB-D camera is p i (X, Y, Z), by repeating the above steps, we can obtain multiple sets of 3D data of corner points in the RGB-D camera coordinate system and form a point cloud set C 1＝ {p1,p2,p3…p n }; Step S33: Use the electromagnetic positioning system to assist the positioning needle to manually collect the three-dimensional data q of a corner point relative to the electromagnetic positioning system coordinate system. i (X, Y, Z), which is a point cloud dataset C of multiple groups of corner points expressed in the coordinate system of the electromagnetic positioning system 2＝ {q1,q2,q3…q n }, each corresponding point in the two point cloud datasets satisfies the following transformation relationship: MT R ·p i =q i MT R is the homogeneous transformation matrix from the RGB-D camera coordinate system to the electromagnetic positioning system coordinate system; Step S34: Use the Levenberg-Marquart algorithm to minimize the equation f1=||MT R ·p i -q i ||Estimated MT R , registering the RGB-D camera into the calibration loop.

9. The positioning method according to claim 6, characterized in that: The specific steps of step S5 are: Step S51: convert the needle insertion path planned by the preoperative CT scan into the coordinate system of the robot arm moving platform. The needle diameter is expressed as the needle insertion point in the coordinate system of the robot arm moving platform. Point to the puncture target The vector whose unit vector is The unit vector of the x-axis of the electromagnetic positioning system in the coordinate system of the manipulator moving platform is: The unit vector from the puncture target point l2 to the origin of the electromagnetic positioning system is Step S52: When the auxiliary positioning needle is used to define the needle guide, the needle guide is shallowly inserted and the first set of point data is collected. Then, the auxiliary positioning needle is deeply inserted along the needle guide, and its y-axis is ensured to face the electromagnetic positioning system, and the second set of point data is collected. The vector of the needle guide is obtained through the two sets of points. The coordinate system of the positioning needle at the time of deep insertion is used as the coordinate system of the needle guide, and the unit vectors along the x, y, and z axes of the needle guide are obtained. Calculate the direction cosine parameters (Cosα, Cosβ, Cosγ) between the needle guide vector and the x, y, z axes of the needle guide coordinate system; Step S53: y-axis unit vector in target posture The expression in the robot moving platform coordinate system is solved by the following formula: Find the only Then, it is expressed by the following equation From the above, we can solve the two corresponding groups and They are and The following constraints determine a unique set of and This calculates the only Confirm the target posture; Step S54: Due to the target location A certain distance from the needle insertion point is required, so l3 is defined by the following formula: Where d is the distance between l3 and l1, so the target pose defined in the manipulator mobile station coordinate system is defined by the following homogeneous transformation matrix: The rotation matrix in the homogeneous transformation is converted into Euler angle representation (Φ, θ, Ψ), and the target pose is 10. The positioning method according to claim 6, characterized in that: The following steps are also included: Step S7: Perform real-time ultrasound scanning, and after fine-tuning the needle guide, clamp the ablation needle on the needle guide holder, and perform needle insertion under the guidance of real-time ultrasound images.

Images