A system of a robotic end effector comprising a puncture intervention

Abstract

The system of the present application comprises a robot end effector of puncture intervention, the end effector comprises a clamping module and a guide needle module, the clamping module comprises a mounting body, a development board, a connecting piece, a rotary drive mechanism, a clamp, an ultrasonic probe and an electromagnetic sensor, the guide needle module comprises a guide rail, a sliding block and a guide needle device displacement drive mechanism, the coordinate system of the whole system is unified, electromagnetic-visual registration method is used for calibration, and the patient's affected part is scanned by CT before operation, then operation planning is carried out, after the target pose is calculated by constraint positioning algorithm, the rotary drive mechanism drives the guide needle device to rotate, the non-coplanar puncture channel path selection is realized, the displacement drive mechanism drives the guide needle device to move, the needle entry angle is adjusted, the puncture process and target point information are monitored in real time based on real-time ultrasonic image, the operation error is reduced, the puncture precision is improved, the operation safety is improved, the radiation exposure of the patient is reduced and the postoperative complications are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, in particular to a system of a robot end effector containing puncture intervention. BACKGROUND

[0002] Hepatocellular carcinoma (HCC) as a malignant tumor has become one of the important health challenges in the world, and the traditional ablation intervention surgery usually relies on preoperative and intraoperative X-ray images. Doctors use 18G puncture needles to puncture the tumor and use microwave or radiofrequency ablation needles for treatment. However, there are many limitations in actual surgery, such as the matching between the image and the actual position of the tumor is not intuitive enough, the surgical precision is difficult to guarantee, and the doctor often relies on experience to determine the puncture point. This method has greater invasiveness to the patient and high difficulty of surgery. The physiological tremor of the doctor may lead to instability, thereby prolonging the operation time, increasing the risk of radiation exposure, and causing serious postoperative complications.

[0003] Currently, there are also some surgical robot positioning systems for extensive percutaneous puncture surgery or specialized in percutaneous puncture surgery of liver tumors, such as patent documents with application numbers CN201811619721.5 (hereinafter referred to as D1), CN201821342041.9 (hereinafter referred to as D2) and CN202410683509.4 (hereinafter referred to as D3). The tumor treatment system of the mechanical arm of D1 uses lesion positioning technology based on RGB optical images, preoperative CT images and intraoperative three-dimensional ultrasound images to translate the mechanical arm to the needle insertion point. The mechanical arm translates to puncture the patient. This system uses positioning technology based on RGB image processing. However, RGB image information is limited to a two-dimensional plane perpendicular to the camera optical axis, lacking real-time and accurate three-dimensional positioning information, and further lacking positioning calculation in three-dimensional space. The positioning system can only be positioned to the target point plane and cannot track the puncture needle insertion depth, which will lead to the loss of real-time spatial position information of the needle tip and the inability to well confirm the target pose of the mechanical hand in three-dimensional space. Moreover, the end effector thereof cannot real-time plan and position the non-coplanar puncture path during positioning, lacks flexibility and adaptability to complex surgery, still has a large error for tumor puncture surgery with high precision requirements, and lacks safety.

[0004] The precise positioning tumor operation robot system of D2 is only realized by a six-degree-of-freedom mechanical arm, a tracking marker assembly and a far infrared camera to realize positioning and operation, the installation position of the tracking marker assembly is emphasized in the disclosed content of D2 to improve the positioning accuracy, but the calculation steps and schemes of precise positioning are not given in detail, the scheme only uses an optical positioner to track and position, does not introduce three-dimensional CT images and real-time ultrasound images, the anatomical information of the lesion is missing, the puncture process and target information cannot be monitored in real time to adapt to complex operation conditions, and the precision of the operation and the safety of the patient cannot be guaranteed;

[0005] The liver puncture operation robot of D3 is only realized by a six-degree-of-freedom mechanical arm, a puncture needle posture adjusting mechanism, an ultrasonic imaging system and an electromagnetic tracking system to realize positioning and operation, however, in the scheme disclosed by D3, only the adjustment of in-plane puncture based on an intraoperative ultrasound image is emphasized, and the preoperative operation planning and automatic positioning method are not given in detail, the system relies on manual operation, the degree of automation is limited, and the end effector mechanism can only be adjusted in the ultrasonic imaging plane, lacks flexibility and strain capacity, cannot show the accurate registration relationship between the system and medical images, and lacks three-dimensional anatomical information of the lesion, so that the precision of the operation and the safety of the patient cannot be guaranteed. SUMMARY

[0006] Therefore, the present application provides a system comprising a robot end effector for puncture intervention to solve the above problems.

[0007] The technical scheme of the present application is as follows:

[0008] The system comprising a robot end effector for puncture intervention comprises a clamping module and a guide needle module, the clamping module comprises a carrier body, a development board, a connecting piece, a rotary drive mechanism, a clamp, an ultrasonic probe and an electromagnetic sensor, the development board and the rotary drive mechanism are arranged in the carrier body, the connecting piece is rotatably connected with the carrier body, the rotary drive mechanism drives the rotation of the connecting piece, the clamp is arranged on one side of the carrier body, the electromagnetic sensor is arranged on the outer wall of the clamp, and the ultrasonic probe is arranged at the end of the clamp; the guide needle module comprises a guide rail, a sliding block, a guide needle device and a displacement drive mechanism, one end of the guide rail is connected with the outer wall of the connecting piece, the sliding block is slidably arranged on the guide rail, the guide needle device is connected with the sliding block, and the displacement drive mechanism is arranged on the guide rail and used to drive the sliding block to move along the guide rail; and the development board is data-connected with the rotary drive mechanism and the displacement drive mechanism respectively.

[0009] Further, the carrying member body comprises a first comprehensive carrying member and a second comprehensive carrying member, the connecting member is located between the first comprehensive carrying member and the second comprehensive carrying member, the development board and the rotary driving mechanism are arranged in the first comprehensive carrying member, and the gripper is connected with the second comprehensive carrying member.

[0010] Further, the rotary driving mechanism comprises a combined mounting column, a bearing, a first gear, a first servo motor and a second gear, the combined mounting column is arranged on the first comprehensive carrying member, the bearing is sleeved outside the combined mounting column and penetrates through the connecting member, the first gear is sleeved outside the bearing, the first servo motor is arranged in the first comprehensive carrying member, an output shaft of the first servo motor is connected with the second gear, the second gear is engaged with the first gear, and the development board is in data connection with the first servo motor.

[0011] Further, the displacement driving mechanism comprises a rack, a carrying plate, a second servo motor and a third gear, the rack is arranged at the bottom of the guide rail and connected with the outer wall of the connecting member at one end, one side of the carrying plate is connected with the sliding block, the guide rail and the rack are located inside the carrying plate, the second servo motor is arranged on the carrying plate and an output shaft of the second servo motor is connected with the third gear, the third gear is engaged with the rack, and the development board is in data connection with the second servo motor.

[0012] Further, the system further comprises a mechanical arm moving table, a serial mechanical arm, an operating table, an electromagnetic positioning system, an RGB-D camera and an ultrasonic imaging system, one end of the serial mechanical arm is arranged on the mechanical arm moving table, the other end is connected with an end effector, the ultrasonic imaging system is arranged on the mechanical arm moving table, and the end effector, the electromagnetic positioning system and the RGB-D camera are located above the operating table.

[0013] Further, the positioning method of the system comprises the following steps:

[0014] Step S1, performing preoperative CT scanning on the patient's affected part;

[0015] Step S2, constructing a calibration closed loop according to the coordinate systems in which the electromagnetic positioning system, the mechanical arm moving table and the end effector are located;

[0016] Step S3, registering the RGB-D camera into the calibration closed loop based on an electromagnetic-visual registration method;

[0017] Step S4, performing surgery planning through a graphical user interface and registering the preoperative CT image with the real-time data of the intraoperative RGB-D camera;

[0018] Step S5, calculating a target pose by using a constraint positioning algorithm;

[0019] Step S6, driving the guide needle device to move to accurately align the needle insertion path.

[0020] Further, the specific steps of the step S2 are:

[0021] Step S21, control the serial mechanical arm to move along the mechanical arm moving platform coordinate system and the end effector coordinate system for multiple sets of translation movement;

[0022] Step S22, express the mechanical arm moving platform coordinate system under the electromagnetic positioning system coordinate system through closed-loop calibration, and update the end effector coordinate system to the built-in electromagnetic sensor coordinate system;

[0023] Step S23, define the guide needle coordinate system through the auxiliary positioning needle equipped by the electromagnetic positioning system, further update the end effector coordinate system to the guide needle coordinate system, and construct the calibration closed loop.

[0024] Further, the specific steps of the step S3 are:

[0025] Step S31, after the intrinsic calibration of the RGB-D camera, the focal length f x , f y , and the principal point (c x , c y ) are obtained, the checkerboard calibration board is photographed by using the RGB-D camera, and the two-dimensional pixel coordinates (x, y) of an angle point are obtained by using the checkerboard grid angle point detection algorithm;

[0026] Step S32, the depth information d of the corresponding angle point position on the depth image is obtained through scale reduction, and the three-dimensional point cloud coordinates of the angle point are obtained through the following formula:

[0027] Z=d

[0028]

[0029]

[0030] The three-dimensional coordinate information of the three-dimensional angle point relative to the RGB-D camera is p i (X, Y, Z), a plurality of sets of three-dimensional data of the angle point under the RGB-D camera coordinate system are obtained by repeating the above steps, and a point cloud data set C1={p1, p2, p3…p n} is formed;

[0031] Step S33, the three-dimensional data q i (X, Y, Z) of a certain angle point relative to the electromagnetic positioning system coordinate system are manually collected by using the auxiliary positioning needle of the electromagnetic positioning system, and a plurality of sets of point cloud data sets C2={q1, q2, q3…q n} expressed under the electromagnetic positioning system coordinate system are formed, and each corresponding point in the two point cloud data sets satisfies the following conversion relationship:

[0032] MT R ·p i =q i

[0033] wherein MT R is a homogeneous transformation matrix from the RGB-D camera coordinate system to the electromagnetic positioning system coordinate system;

[0034] Step S34, using Levenberg-Marquart algorithm to minimize equation f1 = ||MT R ·p i -q i ||estimates MT R , and registers the RGB-D camera to the calibration closed loop.

[0035] Further, the specific steps of step S5 are:

[0036] Step S51, converting the needle insertion path planned by preoperative CT into the coordinate system of the mechanical arm moving table, the needle insertion path being expressed as a vector from needle insertion point l1(x l1 ,y l1 ,z l1 ) to puncture target point l2(x l2 ,y l2 ,z l2 ) in the coordinate system of the mechanical arm moving table, and the unit vector being The unit vector of the electromagnetic positioning system x-axis in the coordinate system of the mechanical arm moving table is The unit vector from the puncture target point l2 to the origin of the electromagnetic positioning system is

[0037] Step S52, when using the auxiliary positioning needle to define the needle guide, shallowly inserting the needle guide and collecting a first group of point data, then deeply inserting the auxiliary positioning needle along the needle guide, and ensuring that the y-axis of the needle guide faces the electromagnetic positioning system, and collecting a second group of point data, obtaining the vector of the needle guide from the two groups of points Taking the coordinate system of the positioning needle at the moment of deep insertion as the coordinate system of the needle guide, and obtaining the unit vectors along the x, y, and z axes of the needle guide Calculating the directional cosine parameters (Cosα, Cosβ, Cosγ) between the needle guide vector and the x, y, and z axes of the needle guide coordinate system;

[0038] Step S53, the y-axis unit vector in the target pose The expression in the coordinate system of the mechanical arm moving table is solved by the following formula:

[0039]

[0040] Solving the unique After, the following equation is expressed :

[0041]

[0042] Two groups of corresponding And Respectively And A unique set of And :

[0043]

[0044] Thus, the unique Confirm the target posture;

[0045] Step S54, since the target position Need to have a certain needle guide distance with the needle entry point, so l3 is defined by the following formula:

[0046]

[0047] Where d is the distance between l3 and l1, so the target pose WT defined in the robot arm moving platform coordinate system target Defined by the following homogeneous transformation matrix:

[0048]

[0049] Convert the rotation matrix in the homogeneous transformation to Euler angle representation (Φ,θ,Ψ), and the target pose is

[0050] Further, the positioning method of the system further comprises the following steps:

[0051] Step S7, after real-time ultrasound scanning and fine adjustment of the needle guide, the ablation needle is clamped on the needle rack of the needle guide, and real-time ultrasound image guided needle insertion is performed.

[0052] Compared with the prior art, the beneficial effects of the present application are:

[0053] The present application provides a kind of system of robot end effector comprising puncture intervention, rotating drive mechanism can be set to drive needle guide to rotate along the rotating shaft of holder, realize the selection of non-coplanar puncture channel path, displacement drive mechanism can drive slider and needle guide to slide along guide rail, adjust the angle of needle insertion, based on real-time ultrasound image in surgery to real-time monitoring puncture process and target point information, to reduce surgical error, improve puncture precision, improve surgical safety, reduce patient's radiation exposure and postoperative complications;

[0054] The application also provides a system and positioning method of the robot end effector for the puncture intervention, after the patient lies on the operating table, the electromagnetic positioning system can obtain the pose information of the electromagnetic sensor in real time, the spatial pose data is collected for a specific algorithm, the RGB-D camera is used for registering the real-time spatial information of the patient, the ultrasonic imaging system can provide the real-time anatomical structure of the patient, guiding the automatic fine adjustment of the puncture pose of the serial mechanical arm to improve the puncture precision, and before the surgical puncture, the electromagnetic-visual registration method is used to unify the various coordinate systems defined before or during the operation to the expression under the mechanical arm moving table coordinate system, so that the planning information in the preoperative CT can be obtained based on this, so that the robot can automatically perform accurate alignment through the "spatial constraint positioning algorithm" according to the planning information, and then the non-coplanar puncture channel path selection can be realized based on the end effector, the real-time monitoring of the puncture process and the target point information is realized based on the real-time ultrasonic image, so as to reduce the surgical error, improve the puncture precision, improve the surgical safety, reduce the radiation exposure of the patient and the postoperative complications, at the same time, the doctor can observe the puncture situation in real time through the ultrasonic image, fine-tune the puncture needle, reduce the risk to the minimum, and release the needle guide until the puncture target point is reached, and the robot-assisted positioning percutaneous puncture ablation surgery is completed. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only preferred embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0056] Figure 1 It is a structural schematic diagram of a puncture intervention robot end effector of the application.

[0057] Figure 2 It is an exploded view of a puncture intervention robot end effector of the application.

[0058] Figure 3 It is a structural schematic diagram of a mounting part body of a puncture intervention robot end effector of the application.

[0059] Figure 4 It is a schematic diagram of a system comprising a puncture intervention robot end effector of the application.

[0060] Figure 5 It is a schematic diagram of the relationship between the coordinate systems of a system comprising a puncture intervention robot end effector of the application.

[0061] Figure 6A general flowchart of a positioning method of a system of a robot end effector comprising a puncture intervention;

[0062] Figure 7 A flowchart of system calibration of a positioning method of a system of a robot end effector comprising a puncture intervention;

[0063] In the figure, 1, mechanical arm moving table; 2, serial mechanical arm; 3, end effector; 300, mechanical installation column; 301, development board; 302, gripper; 303, electromagnetic sensor; 304, ultrasonic probe; 305, first comprehensive carrier; 306, first gear; 307, bearing, 308, connecting piece; 309, second comprehensive carrier; 310, first servo motor; 311, second gear; 312, second servo motor; 313, third gear; 314, carrier plate; 315, rack; 316, sliding block; 317, needle guide; 318, guide rail; 4, operating table; 5, electromagnetic positioning system; 6, RGB-D camera; 7, ultrasonic imaging system. DETAILED DESCRIPTION

[0064] In order to better understand the technical content of the present application, a specific embodiment is provided below, and the present application is further described in conjunction with the accompanying drawings.

[0065] Reference Figures 1 to 3 The present application provides a kind of puncture intervention robot end effector, including clamping module and needle guide module, the clamping module includes carrier main body, development board 301, connecting piece 308, rotary drive mechanism, gripper 302, ultrasonic probe 304 and electromagnetic sensor 303, the development board 301 and rotary drive mechanism are arranged in carrier main body, the connecting piece 308 is rotatably connected with carrier main body, the rotary drive mechanism drives the rotation of connecting piece 308, the gripper 302 is arranged in one side of carrier main body, the electromagnetic sensor 303 is arranged on the outer wall of gripper 302, the ultrasonic probe 304 is arranged in the end of gripper 302;The needle guide module includes guide rail 318, sliding block 316, needle guide 317 and displacement drive mechanism, one end of guide rail 318 is connected with the outer wall of connecting piece 308, the sliding block 316 is slidably arranged on guide rail 318, the needle guide 317 is connected with sliding block 316, the displacement drive mechanism is arranged on guide rail 318, for driving sliding block 316 moves along guide rail 318;The development board 301 is respectively connected with rotary drive mechanism and displacement drive mechanism data.

[0066] This invention discloses a robotic end effector for puncture intervention, comprising two main parts: a clamping module based on an ultrasound probe 304, and a guide needle module for mounting an ablation needle. A mounting body is used to mount a development board 301 and a rotary drive mechanism. A connector 308 is provided on the mounting body, and the rotary drive mechanism can drive the connector 308 to rotate. The outer wall of the connector 308 is connected to the guide needle module, thereby allowing the guide needle module to rotate along the rotation axis of the mounting body. Figure 1 The dotted line in the image is rotated to select a non-coplanar puncture channel path. A clamp 302 is set at one end of the main body of the mounting component to clamp the ultrasound probe 304. An electromagnetic sensor 303 is installed on the outer wall of the clamp 302. The ultrasound probe 304 can perform real-time ultrasound scanning on the patient to provide the anatomical structure of the liver tumor, while the electromagnetic sensor 303 can provide the necessary positional information of the end effector 3.

[0067] On one side of the connector 308 is a guide needle module. The guide needle 317 is connected to the slider 316. The slider 316 is located on the guide rail 318 and can slide on the guide rail 318. The displacement drive mechanism can drive the slider 316 to slide along the guide rail 318, thereby adjusting the angle of needle insertion. Based on the intraoperative real-time ultrasound image, the puncture process and target point information are monitored in real time to reduce surgical errors, improve puncture accuracy, improve surgical safety, and reduce patient radiation exposure and postoperative complications.

[0068] Furthermore, the main body of the mounting component includes a first integrated mounting component 305 and a second integrated mounting component 309, the connecting component 308 is located between the first integrated mounting component 305 and the second integrated mounting component 309, the development board 301 and the rotary drive mechanism are disposed in the first integrated mounting component 305, and the clamp 302 is connected to the second integrated mounting component 309.

[0069] The first integrated mounting component 305 and the second integrated mounting component 309 form a complete structure. The first integrated mounting component 305 is located above the second integrated mounting component 309 and can be used to place the development board 301 and install the rotary drive mechanism. The connector 308 is located between the first integrated mounting component 305 and the second integrated mounting component 309 and can rotate under the drive of the rotary drive mechanism to adjust the position of the entire guide pin module.

[0070] Further, the rotating driving mechanism comprises a combined mounting column 300, a bearing 307, a first gear 306, a first servo motor 310 and a second gear 311. The combined mounting column 300 is arranged on the bottom surface of the first combined mounting part 305. The bearing 307 is sleeved outside the combined mounting column 300 and is arranged through the connecting part 308. The first gear 306 is sleeved outside the bearing 307. The first servo motor 310 is arranged in the first combined mounting part 305, and the output shaft thereof is connected with the second gear 311 through the bottom surface of the first combined mounting part 305. The second gear 311 is engaged with the first gear 306. The development board 301 is connected with the first servo motor 310 in data.

[0071] The combined mounting column 300 is a column with a diameter greater than 35 mm, which extends downward along the concentric circle of the first combined mounting part 305. The rotating is limited on the concentric circle through the first gear 306, the bearing 307 and the middle part of the connecting part 308. The inner ring of the bearing 307 is connected with the outer wall of the combined mounting column 300, and the outer ring of the bearing 307 is arranged through the connecting part 308. The height of the bearing 307 is greater than the height of the connecting part 308, and the part of the bearing 307 above the connecting part 308 is sleeved with the first gear 306. The second gear 311 is rotated under the driving of the first servo motor 310 in the first combined mounting part 305, and the bearing 307 and the connecting part 308 are driven to rotate through the first gear 306, so as to drive the rotating of the guide pin module to fine tune.

[0072] Further, the displacement driving mechanism comprises a rack 315, a mounting plate 314, a second servo motor 312 and a third gear 313. The rack 315 is arranged at the bottom of the guide rail 318 and is connected with the outer wall of the connecting part 308 at one end. The mounting plate 314 is connected with the sliding block 316 at one side. The guide rail 318 and the rack 315 are arranged inside the mounting plate 314. The second servo motor 312 is arranged on the mounting plate 314, and the output shaft thereof is connected with the third gear 313. The third gear 313 is engaged with the rack 315. The development board 301 is connected with the second servo motor 312 in data.

[0073] The rack 315 and the guide rail 318 are in an arc structure, the mounting plate 314 is similar to a U-shaped structure, the rack 315 and the guide rail 318 pass through the inside of the mounting plate 314, after the sliding block 316 is slidingly arranged on the guide rail 318, the upper portion of the mounting plate 314 is connected with the top side of the sliding block 316, when the second servo motor 312 is started, it can drive the third gear 313 to rotate, the third gear 313 is engaged with the rack 315, because the end portion of the rack 315 is fixedly connected with the connecting piece 308, so that the sliding block 316 can move along the arc guide rail 318, the needle entry angle is adjusted, and the needle guide 317 can guide the puncture needle to penetrate into the patient's body.

[0074] When the end effector 3 reaches the target pose, the ultrasonic probe 304 can capture the optimal quality image, according to the real-time feedback ultrasonic image and the control method, after the first servo motor 310 drives the second gear 311 to drive the first gear 306 to rotate, the connecting piece 308 drives the needle guide module to rotate as a whole, drives the needle guide 317 to rotate along the rotating shaft of the mounting body, realizes the selection of the non-coplanar puncture channel path, and meanwhile the second servo motor 312 can drive the third gear 313 to rotate, and through the fixedly arranged rack 315, the sliding block 316 is slidingly arranged on the arc guide rail 318, so that the needle entry angle is adjusted, the puncture process and the target point information are monitored in real time based on the real-time ultrasonic image during the operation, so as to reduce the operation error, improve the puncture precision, improve the operation safety, reduce the radiation exposure of the patient and the postoperative complications.

[0075] Referring to Figure 4 The system shown in the figure further comprises a mechanical arm moving table 1, a serial mechanical arm 2, an operating table 4, an electromagnetic positioning system 5, an RGB-D camera 6 and an ultrasonic imaging system 7, one end of the serial mechanical arm 2 is arranged on the mechanical arm moving table 1, the other end is connected with the end effector 3, the ultrasonic imaging system 7 is arranged on the mechanical arm moving table 1, and the end effector 3, the electromagnetic positioning system 5 and the RGB-D camera 6 are located above the operating table 4.

[0076] When the end effector 3 is applied in the treatment process, it can be installed on the serial mechanical arm 2, the serial mechanical arm 2 comprises a six-degree-of-freedom mechanical arm, can drive the end effector 3 to adjust the position, so as to approach the patient on the operating table 4, the movement of the serial mechanical arm 2 can be realized through the mechanical arm moving table 1, the electromagnetic positioning system 5 located on one side of the operating table 4 can obtain the pose information of the electromagnetic sensor 303 on the gripper 302, and the spatial pose data is collected for a specific algorithm, and the RGB-D camera 6 can be used for registering the real-time spatial information of the patient, and the ultrasonic imaging system 7 can provide the real-time anatomical structure of the patient, so as to guide the serial mechanical arm 2 to automatically fine-tune the puncture pose instrument and improve the puncture precision.

[0077] The series mechanical arm 2 of the present application is a kind of general-purpose mechanical arm combination, six degrees of freedom mechanical arm is the UR5 collaborative mechanical arm of commercial, its end installs two degrees of freedom end effector 3, through the flexible movement of six degrees of freedom series mechanical arm 2, automatic operation end effector 3 is carried out ultrasound scanning and puncture positioning task, and the additional two degrees of freedom on end effector 3 is used for fine adjustment needle guide 317, reduces puncture error, improves surgical precision.

[0078] Referring to Figure 5 The system of the present application designs seven main coordinate systems, which are mechanical arm moving platform 1 coordinate system {W} (i.e. world coordinate system), end effector 3 coordinate system {E}, electromagnetic positioning system 5 coordinate system {M}, RGB-D camera 6 coordinate system {R}, coordinate system {C}, intraoperative ultrasound image coordinate system {U} and patient coordinate system {P}, wherein, ① ring is "main ring", the calibration ring contains the definition of hand-eye calibration and needle guide 317 coordinates, ② ring is the general term of "secondary ring" except "main ring", through ultrasound probe 304 calibration, "electromagnetic positioner-based system calibration method" and preoperative-intraoperative image registration, ultrasound image {U}, patient {P} (expressed under RGB-D camera 6 {R}) and preoperative CT image {C} can be included in "main ring", realize closed-loop calibration, and the coordinate system is unified in world coordinate system {W}.

[0079] Referring to Figures 6-7 A positioning method of a system based on the robot end effector 3 containing the puncture intervention, comprising the following steps:

[0080] Step S1, preoperative CT scanning is carried out on the patient's affected part;

[0081] Step S2, in order to obtain the conversion relationship between electromagnetic positioning system 5 coordinate system {M} and mechanical arm moving platform 1 coordinate system {W}, according to the coordinate system of electromagnetic positioning system 5, mechanical arm moving platform 1 and end effector 3, a calibration closed loop is constructed, and the specific steps are as follows:

[0082] Step S21, control series mechanical arm 2 to move along mechanical arm moving platform 1 coordinate system {W} and end effector 3 coordinate system in multiple groups;

[0083] Step S22, through closed-loop calibration, express mechanical arm moving platform 1 coordinate system {W} in electromagnetic positioning system 5 coordinate system {M}, and update end effector 3 coordinate system from original end coordinate system {E1} of series mechanical arm 2 to built-in electromagnetic sensor 303 coordinate system {E2};

[0084] Step S23, since the coordinate system of the needle guide 317 can be defined by the auxiliary positioning needle equipped by the electromagnetic positioning system 5, the coordinate system of the end effector 3 can be further updated to the coordinate system {E3} of the needle guide 317, and finally the coordinate system {E3} is defined as the coordinate system {E} of the end effector 3, so that the coordinate system {M} of the electromagnetic positioning system 5 and the coordinate system {E} of the end effector 3 can be unified to be expressed under the coordinate system {W} of the mechanical arm moving table 1, and a closed-loop conversion relationship based on the electromagnetic positioning system 5 and the serial mechanical arm 2 is established.

[0085] Step S3, in order to register the RGB-D camera 6 into the calibration closed loop, the electromagnetic-vision registration method is proposed, and the specific steps are as follows:

[0086] Step S31, after fixing the RGB-D camera 6 and the electromagnetic positioning system 5, the intrinsic calibration of the RGB-D camera 6 is performed to obtain the focal length f x , f y and the principal point (c x , c y ), the front of the RGB-D camera 6 is used to shoot a 6*10 chessboard calibration board, and a certain corner point two-dimensional pixel coordinate (x, y) is obtained by using a chessboard corner point detection algorithm;

[0087] Step S32, the depth information d of the corresponding corner point position on the depth image is obtained by scale reduction, and the three-dimensional point cloud coordinates of the corner point are obtained by the following formula:

[0088] Z=d

[0089]

[0090] The three-dimensional coordinate information of the three-dimensional corner point relative to the RGB-D camera 6 is p i (X, Y, Z), a plurality of groups of three-dimensional data of the corner point under the coordinate system of the RGB-D camera 6 are obtained by repeating the above steps, and a point cloud data set C1={p1, p2, p3…p n} is formed;

[0091] Step S33, the three-dimensional data q i (X, Y, Z) of a certain corner point relative to the coordinate system of the electromagnetic positioning system 5 are manually collected by using the auxiliary positioning needle of the electromagnetic positioning system 5, and a plurality of groups of point cloud data sets C2={q1, q2, q3…q n} expressed under the coordinate system of the electromagnetic positioning system 5 are formed, and each corresponding point in the two point cloud data sets satisfies the following conversion relationship:

[0092] MT R ·p i =q i

[0093] wherein MT R is the homogeneous transformation matrix from the RGB-D camera 6 coordinate system to the electromagnetic positioning system 5 coordinate system;

[0094] Step S34, using the Levenberg-Marquart algorithm to minimize the equation f1 = ||MT R ·p i -q i ||estimates MT R , the registration of the RGB-D camera 6 into the calibration loop is different from the traditional hand-eye calibration method, which is formally equivalent to replacing the heavy mechanical arm with a light auxiliary positioning needle, making the camera registration process more convenient and efficient.

[0095] Step S4, planning the ablation needle intervention information on the basis of the preoperative CT through the graphical user interface, and then registering the preoperative CT image with the real-time data of the RGB-D camera 6 in the operation;

[0096] Step S5, in order to calculate the target pose of the needle guide 317 aligning with the preoperative planned needle insertion path, the present application proposes a constraint positioning algorithm based on the electromagnetic positioning system 5 to calculate the target pose, and the specific steps are as follows:

[0097] Step S51, after the intraoperative patient registration, the preoperative CT planned needle insertion path is converted to be expressed in the mechanical arm moving table 1 coordinate system {W}, and the needle insertion path is expressed as the vector pointing to the puncture target point l2 in the mechanical arm moving table 1 coordinate system {W}, and the unit vector is The unit vector of the electromagnetic positioning system 5 x-axis in the mechanical arm moving table 1 coordinate system {W} is The unit vector from the puncture target point l2 to the origin of the electromagnetic positioning system 5 is

[0098] Step S52, when the needle guide 317 is defined using the auxiliary positioning needle, the needle guide 317 is shallowly inserted, and the first group of point data is collected, then the auxiliary positioning needle is deeply inserted along the needle guide 317, and it is ensured that the needle guide y-axis faces the electromagnetic positioning system 5, and the second group of point data is collected, and the vector of the needle guide 317 is obtained from the two groups of points The coordinate system of the positioning needle at the deep insertion moment is taken as the coordinate system of the needle guide 317, and the unit vectors along the x, y, z axes of the needle guide 317 are obtained in combination with The directional cosine parameters (Cosα, Cosβ, Cosγ) between the needle guide 317 vector and the x, y, z axes of the needle guide 317 coordinate system are calculated;

[0099] ​​Step S53, during the puncture process, the electromagnetic positioning system 5 needs to track the electromagnetic sensor 303 on the end effector 3 at all times, when the electromagnetic sensor 303 faces the electromagnetic positioning system 5 and is closest to it, the error of the pose acquired is the lowest, so the y-axis unit vector in the target pose The expression in the mechanical arm moving platform 1 coordinate system is solved by the following formula:

[0100]

[0101] Solve the unique After that, the following equation is used to express :

[0102]

[0103] From the above, two sets of corresponding and are solved respectively and A unique set of and is determined by the following constraint condition:

[0104]

[0105] From which the unique is calculated Confirm the target attitude;

[0106] Step S54, since the target position A certain needle guide distance needs to exist between the needle entry point, so l3 is defined by the following formula:

[0107]

[0108] Where d is the distance between l3 and l1, so the target pose WT defined in the mechanical arm moving platform 1 coordinate system target is defined by the following homogeneous transformation matrix:

[0109]

[0110] Convert the rotation matrix in this homogeneous transformation to Euler angle representation (Φ, θ, Ψ), and the target pose is

[0111] Based on the above, the coordinate transformation relationships between the world coordinate system {W}, the electromagnetic positioning system coordinate system {M}, the end effector coordinate system {E}, the intraoperative ultrasound image {U}, the preoperative CT image {C}, the RGB-D camera coordinate system {R}, and the patient coordinate system {P} can be efficiently and accurately calculated using methods such as "electromagnetic-visual registration". By expressing each coordinate system defined preoperatively or intraoperatively under the world coordinate system {W}, the planning information from the preoperative CT scan can be obtained. This allows the robot to automatically and accurately align itself using a "spatial constraint positioning algorithm" based on the planning information. The end effector 3 enables the selection of non-coplanar puncture channel paths, and real-time monitoring of the puncture process and target information is achieved using intraoperative ultrasound images. This reduces surgical errors, improves puncture accuracy, enhances surgical safety, and reduces patient radiation exposure and postoperative complications.

[0112] The input to the constraint positioning algorithm is the needle entry point expressed in the coordinate system {W} of the robotic arm's moving stage 1. Puncture target And the unit vector of the 5x axis of the electromagnetic potential system in the coordinate system {W} of the robotic arm moving stage 1. The unit vector from the puncture target l2 to the origin of the electromagnetic positioning system 5 is The guide needle 317 direction cosine parameters (Cosα, Cosβ, Cosγ) are output as the target position pose in the coordinate system {W} of the robotic arm moving stage 1. The specific process is as follows:

[0113] Define the needle insertion vector and target position. Target location Simultaneously define the homogeneous transformation matrix.

[0114] Solve the y-axis component in the target pose

[0115] Solve the x-axis component in the target pose Two solutions

[0116] Based on the right-handed system, using express

[0117] like hour, Assigned to WT target The xyz components in;

[0118] otherwise, Assigned to WT target The xyz components in;

[0119] WT targetConvert to Euler angle expression (Φ, θ, Ψ), output target pose is

[0120] Step S6, drive the needle guide 317 to move for precise alignment of the needle insertion path.

[0121] Step S7, after real-time ultrasound scanning and fine adjustment of the needle guide 317, the ablation needle is clamped on the needle rack of the needle guide 317, and real-time ultrasound image guided needle insertion is performed.

[0122] After precise alignment by the constraint positioning algorithm, the motion path of the serial manipulator 2 and the needle guide 317 can be planned, the needle diameter is precisely aligned, the doctor can observe the puncture situation in real time through the ultrasound image, fine adjustment of the puncture needle is performed, the risk is minimized, and after the target point is reached, the needle guide 317 is released, and the robot assisted positioning percutaneous ablation surgery is completed.

[0123] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A system comprising a robotic end effector for a puncture intervention, characterized in that, The end effector comprises a clamping module and a needle guide module, the clamping module comprises a carrier body, a development board, a connecting piece, a rotary drive mechanism, a clamp, an ultrasonic probe and an electromagnetic sensor, the development board and the rotary drive mechanism are arranged in the carrier body, the connecting piece is rotationally connected with the carrier body, the rotary drive mechanism drives the connecting piece to rotate, the clamp is arranged on one side of the carrier body, the electromagnetic sensor is arranged on the outer wall of the clamp, and the ultrasonic probe is arranged at the end of the clamp; the needle guide module comprises a guide rail, a sliding block, a needle guide and a displacement drive mechanism, one end of the guide rail is connected with the outer wall of the connecting piece, the sliding block is slidingly arranged on the guide rail, the needle guide is connected with the sliding block, and the displacement drive mechanism is arranged on the guide rail and used to drive the sliding block to move along the guide rail; the development board is connected with the rotary drive mechanism and the displacement drive mechanism in data. The carrier body comprises a first comprehensive carrier and a second comprehensive carrier, the connecting piece is located between the first comprehensive carrier and the second comprehensive carrier, the development board and the rotary drive mechanism are arranged in the first comprehensive carrier, and the clamp is connected with the second comprehensive carrier. The rotary drive mechanism comprises a combined mounting column, a bearing, a first gear, a first servo motor and a second gear, the combined mounting column is arranged on the first comprehensive carrier, the bearing is sleeved outside the combined mounting column and penetrates through the connecting piece, the first gear is sleeved outside the bearing, the first servo motor is arranged in the first comprehensive carrier, an output shaft of the first servo motor is connected with the second gear, the second gear is engaged with the first gear, and the development board is connected with the first servo motor in data. Further comprising a mechanical arm moving table, a serial mechanical arm, an operating table, an electromagnetic positioning system, an RGB-D camera and an ultrasonic imaging system, one end of the serial mechanical arm is arranged on the mechanical arm moving table, the other end is connected with the end effector, the ultrasonic imaging system is arranged on the mechanical arm moving table, and the end effector, the electromagnetic positioning system and the RGB-D camera are located above the operating table. The positioning method of the system comprises the following steps: Step S1, performing CT scanning on the patient's affected part before operation; Step S2, constructing a calibration closed loop according to the coordinate systems in which the electromagnetic positioning system, the mechanical arm moving table and the end effector are located; Step S3, registering the RGB-D camera into the calibration closed loop based on an electromagnetic-visual registration method; Step S4, performing operation planning through a graphical user interface, and registering the preoperative CT image with the real-time data of the RGB-D camera during operation; Step S5, calculating a target pose by using a constraint positioning algorithm; Step S6, driving the needle guide to move to accurately align the needle insertion path.

2. A system of a robotic end effector incorporating a puncture intervention according to claim 1, wherein, The displacement driving mechanism comprises a rack, a mounting plate, a second servo motor and a third gear, the rack is arranged at the bottom of the guide rail, one end of the rack is connected with the outer wall of the connecting piece, one side of the mounting plate is connected with the sliding block, the guide rail and the rack are located inside the mounting plate, the second servo motor is arranged on the mounting plate, the output shaft of the second servo motor is connected with the third gear, the third gear is engaged with the rack, and the development board is in data connection with the second servo motor.

3. A system including a robotic end effector for puncture intervention according to claim 1, wherein, The specific steps of the step S2 are: Step S21, controlling the serial mechanical arm to move along the mechanical arm moving platform coordinate system and the end effector coordinate system in multiple groups of translation movements; Step S22, expressing the mechanical arm moving platform coordinate system under the electromagnetic positioning system coordinate system through closed-loop calibration, and updating the end effector coordinate system to the built-in electromagnetic sensor coordinate system; Step S23, defining the guide needle device coordinate system through the auxiliary positioning needle equipped by the electromagnetic positioning system, further updating the end effector coordinate system to the guide needle device coordinate system, and constructing a calibration closed loop.

4. A system including a robotic end effector for puncture intervention according to claim 1, wherein, The specific steps of the step S3 are: Step S31, after the intrinsic calibration of the RGB-D camera, the focal length is obtained 、 , and the principal point ( 、 ), a chessboard calibration board is photographed using the RGB-D camera, and the two-dimensional pixel coordinates of an angle point ( 、 ) are obtained using a chessboard corner detection algorithm; Step S32, obtain the depth information of the corresponding corner point position on the depth image through scale reduction The three-dimensional point cloud coordinates of the corner point are obtained through the following formula: The three-dimensional coordinate information of the three-dimensional corner point relative to the RGB-D camera is A plurality of sets of three-dimensional data of the corner point in the RGB-D camera coordinate system are obtained by repeating the above steps, and a point cloud data set is formed ; Step S33, manually collecting three-dimensional data of a certain corner point relative to the electromagnetic positioning system coordinate system using electromagnetic positioning system assisted positioning needle , a point cloud data set composed of multiple groups of corner points expressed in the electromagnetic positioning system coordinate system , each corresponding point in the two point cloud data sets satisfies the following conversion relationship: wherein is a homogeneous transformation matrix from the RGB-D camera coordinate system to the electromagnetic positioning system coordinate system; Step S34, minimize the equation using Levenberg-Marquart algorithm || ||estimate register the RGB-D camera into the calibration loop.

5. A system including a robotic end effector for puncture intervention according to claim 1, wherein, The specific steps of the step S5 are: Step S51, convert the preoperative CT planned needle entry path into the mechanical arm moving table coordinate system, the needle entry path is expressed as a needle entry point in the mechanical arm moving table coordinate system The vector pointing to the puncture target point , the unit vector of which is , the electromagnetic positioning system The unit vector of the axis in the mechanical arm moving table coordinate system is , the puncture target point The unit vector pointing to the origin of the electromagnetic positioning system is ; Step S52, when the guide needle is defined using the auxiliary positioning needle, the guide needle is inserted shallowly, and a first set of point data is collected, then the auxiliary positioning needle is inserted deeply along the guide needle, and the guide needle is ensured to be in the coordinate system of the auxiliary positioning needle The axis faces the electromagnetic positioning system, and a second set of point data is collected, and the vector of the guide needle is obtained via the two sets of points The coordinate system of the positioning needle at the deep insertion moment is taken as the coordinate system of the guide needle, and the unit vector along the guide needle axis is obtained The direction cosine parameters between the guide needle vector and the guide needle coordinate system axis are calculated ; Step S53, the target pose in the Axis unit vector The expression in the mechanical arm moving table coordinate system is solved by the following formula: solved uniquely After that, the following equation is expressed : From the above, two sets of corresponding With , respectively and , determined by the following constraints on a unique set and : From this, the unique , the target pose is confirmed; Step S54, since the target position There is a certain needle guide distance from the entry point, so Defined by the following formula: wherein is with the distance between the robot base frame and the robot joint frame, thus the target pose defined in the robot base frame is defined by the following homogeneous transformation matrix: Converting the rotation matrix in the homogeneous transformation to Euler angle representation , the target pose is .

6. A system including a robotic end effector for puncture intervention according to claim 1, wherein, The positioning method of the system further comprises the following steps: Step S7, after real-time ultrasonic scanning and fine adjustment of the guide needle device, clamping the ablation needle on the needle rack of the guide needle device, and carrying out real-time ultrasonic image guided needle insertion.

Images