An end-effector guide for a puncture robot and its interaction method

Abstract

This invention discloses an end-effector guidance device and its interaction method for a puncture robot. The end-effector guidance device includes: a display screen, a robotic arm docking seat, a connecting arm, an optical positioning workpiece, an ultrasonic probe, and a controller. The controller is electrically connected to the display screen, the electric gripper, the guide gripper, the ultrasonic probe, and the optical positioning workpiece. The ultrasonic probe integrated at the end of the robotic arm monitors the situation within the puncture site in real time. The positioning workpiece is used to locate the spatial position of the puncture needle. The display screen then shows the real-time situation within the puncture site, allowing the surgeon to fine-tune the end-effector position in a timely manner, reducing the impact of error factors and making the surgery faster, more precise, and safer. This invention can be widely applied in the field of robotics.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to an end-effector guidance device for a puncture robot and its interaction method. Background Technology

[0002] In numerous cases of puncture and interventional surgical robot design, robotic arms are widely used as the guiding actuators for puncture procedures. Utilizing robotic arms to deliver and position puncture instruments typically involves attaching the puncture device to the end of the robotic arm. Based on the pre-planned puncture path, the device is delivered to the patient's surface under the guidance of a navigation device, thus providing guidance for the puncture procedure. However, many factors can lead to errors in positioning, navigation, and puncture execution during this process, such as individual patient differences, tissue deformation due to respiration, deformation of flexible tissues, and minute patient movements. Most of these factors cannot be completely eliminated using this navigation and positioning method. Because of the low level of physician involvement during the procedure, the impact of these errors cannot be eliminated, resulting in slow, inaccurate, and unsafe surgeries. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide an end-effector guide device and its interaction method for a puncture robot, which provides fine-tuning of the puncture end position and real-time display of ultrasound effects, enabling doctors to deeply participate in the on-site operation, reducing the impact of error factors, and making the surgery faster, more accurate, and safer.

[0004] In a first aspect, embodiments of the present invention provide an end effector for a puncture robot, comprising a display screen, a robotic arm docking seat, an electric gripper, a guide gripper, a connecting arm, an optical positioning workpiece, an ultrasonic probe, and a controller, wherein the controller is electrically connected to the display screen, the electric gripper, the guide gripper, the ultrasonic probe, and the optical positioning workpiece, respectively.

[0005] The ultrasound probe is used to acquire real-time ultrasound images of the puncture needle inserted into the body and input the ultrasound images into the display screen. The ultrasound probe is located at the first end of the connecting arm.

[0006] The display screen is used to display the ultrasound images and the operating status of the puncture robot, as well as to perform end-effector pose fine-tuning control on the end-effector guide device.

[0007] The second end of the connecting arm is connected to the docking seat of the robotic arm, and the display screen is disposed on the connecting arm. The second end is the opposite end of the first end.

[0008] The electric gripper is used to receive drive information input from the display screen, and to clamp or release the puncture needle according to the drive information. The electric gripper is disposed on the connecting arm.

[0009] The guide gripper is disposed on the connecting arm. The guide gripper is used to control the direction of the puncture needle clamped by the electric gripper. The guide gripper and the ultrasound probe are located on the same guide axis.

[0010] The optical positioning workpiece is disposed on the connecting arm and is used to locate the position and orientation information of the puncture needle. The position and orientation information is used to determine the spatial orientation of the puncture needle in the ultrasound image.

[0011] Optionally, the display screen includes a touch screen, which includes a control interface and a display interface, wherein the control interface and the display interface are on the same interface.

[0012] Optionally, the guide gripper includes a left guide gripper and a right guide gripper, the left guide gripper being used to control the electric gripper to clamp the puncture needle to the right, and the right guide gripper being used to control the electric gripper to clamp the puncture needle to the left.

[0013] Optionally, the optical positioning workpiece includes a passive marker and a support, wherein the passive marker is disposed on the support.

[0014] Secondly, embodiments of the present invention provide an interaction method for a puncture robot, the method being applied to the aforementioned end-effector guidance device, comprising:

[0015] The position information of the puncture needle is obtained by positioning the workpiece using the optical positioning method.

[0016] The ultrasound image information of the ultrasound probe is obtained through the ultrasound probe;

[0017] The ultrasound image information and the pose information are fused to obtain an ultrasound image with the puncture needle, and the ultrasound image with the puncture needle is sent to the display screen for display.

[0018] Acquire first control information, output second control information based on the first control information, and control the movement of the puncture needle based on the second control information.

[0019] Optionally, it also includes a heartbeat sensor connected to the controller, which also has one or more functions such as heartbeat acquisition, abnormal warning, and status synchronization.

[0020] Optionally, the controller, including the robotic arm control processor, controls the movement of the puncture needle according to the second control information, specifically including:

[0021] The robotic arm control processor receives the second control information and sends the second control information to the robotic arm so that the robotic arm controls the electric gripper and guide gripper according to the second control information. The robotic arm is connected to the end-effector.

[0022] Optionally, the controller further includes a message storage and forwarding server and a visual touch interaction processor, and after sending the second control information to the robotic arm, the method further includes:

[0023] The robotic arm control processor receives feedback information sent by the robotic arm;

[0024] The robotic arm control processor sends the feedback information to the message storage and forwarding server;

[0025] The message storage and forwarding server stores the feedback information and forwards it to the display screen through the visual touch interaction processor.

[0026] Optionally, the method further includes the visual touch interaction processor and the robotic arm control processor establishing a topological relationship through a handshake, and the visual touch interaction processor and the robotic arm control processor dissolving the topological relationship by waving.

[0027] Thirdly, embodiments of the present invention provide a robot, characterized in that it includes a robotic arm, the end of which is connected to the aforementioned end effector, and the aforementioned method is applied to the robot.

[0028] The implementation of the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention monitor the situation inside the puncture and intervention in real time by using an ultrasonic probe integrated at the end of the robotic arm, and display the situation inside the puncture and intervention in real time on a display screen. Doctors can make fine adjustments to the position of the puncture end in a timely manner through the display screen, reducing the impact of error factors, making the surgery faster, more accurate and safer. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the end effector of a puncture robot provided in an embodiment of the present invention;

[0030] Figure 2 This is an interface diagram of the display screen provided in an embodiment of the present invention;

[0031] Figure 3 This is a structural block diagram of an interaction method for a puncture robot provided in an embodiment of the present invention;

[0032] Figure 4 This is a flowchart of an interaction method for a puncture robot provided in an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of an information processing method provided in an embodiment of the present invention;

[0034] Figure reference numerals: 1. Ultrasonic probe; 2. Left guide gripper; 3. Passive marker; 4. Connecting arm; 5. Robotic arm docking seat; 6. Display screen; 7. Electric gripper; 8. Right guide gripper; 9. Guide axis; 10. Connecting arm; 11. Rocker-type operation mode switch; 12. Drag-type operation mode switch; 13. Rocker-type forward and backward control button along the puncture path; 14. Puncture depth display; 15. Target lesion; 16. Puncture path direction; 17. Ultrasonic image; 18. Rocker-type spatial posture control button; 19. Rocker-type self-rotation button around the puncture path; 20. Drag-type spatial position control switch button; 21. Drag-type spatial posture control switch button; 22. Reset button; 23. Electric gripper opening and closing button; 24. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0036] like Figure 1 As shown, this embodiment of the invention provides an end effector for a puncture robot, including a display screen, a robotic arm docking seat, an electric gripper, a guide gripper, a connecting arm, an optical positioning workpiece, an ultrasonic probe, and a controller. The controller is electrically connected to the display screen, the electric gripper, the guide gripper, the ultrasonic probe, and the optical positioning workpiece, respectively.

[0037] The ultrasound probe is used to acquire real-time ultrasound images of the puncture needle inserted into the body and input the ultrasound images into the display screen. The ultrasound probe is set at the first end of the connecting arm.

[0038] The display screen is used to show ultrasound images and the operating status of the puncture robot, as well as to perform fine-tuning control of the end-effector's pose.

[0039] The second end of the connecting arm is connected to the docking seat of the robotic arm, and the display screen is set on the connecting arm. The second end is the opposite end of the first end.

[0040] The electric gripper is used to receive drive information input from the display screen and clamp or release the puncture needle according to the drive information. The electric gripper is set on the connecting arm.

[0041] The guide gripper is mounted on the connecting arm. The guide gripper is used to control the direction of the puncture needle clamped by the electric gripper. The guide gripper and the ultrasonic probe are located on the same guide axis.

[0042] An optical positioning workpiece is mounted on the connecting arm to locate the position and orientation information of the puncture needle. This position and orientation information is used to determine the spatial orientation of the puncture needle in the ultrasound image.

[0043] Optionally, the guide gripper includes a left guide gripper and a right guide gripper, the left guide gripper being used to control the electric gripper to clamp the puncture needle to the right, and the right guide gripper being used to control the electric gripper to clamp the puncture needle to the left.

[0044] Specifically, the electric gripper 7 holds the puncture needle and receives drive information input from the display screen 6. The puncture needle is clamped or released according to the drive information. The guide gripper receives control information input from the display screen 6 and guides and positions the puncture needle through the left guide gripper 2 and the right guide gripper 8. Then, the ultrasound probe 1 acquires real-time ultrasound images of the puncture needle inserted into the body and inputs the ultrasound images into the display screen 6. The doctor performs real-time fine-tuning control of the end-point guide device based on the ultrasound images, thereby adjusting the puncture path of the puncture needle.

[0045] Reference Figure 2 Optionally, the display screen 6 includes a touch screen, which includes a control interface and a display interface, and the control interface and the display interface are on the same interface.

[0046] Specifically, the control interface includes control buttons, which are used to control one or more of the following: puncture path, puncture depth, spatial position, spatial orientation, opening and closing of the motorized gripper, spatial displacement, and repositioning. The display interface includes ultrasound images, which show the location of the target lesion and the direction of the puncture path.

[0047] In one specific embodiment, the robotic arm end effector integrates an ultrasound probe for monitoring the intraoperative condition during puncture; the puncture instrument guide device uses an electric gripper, which can be adapted to different models of gripper ends depending on the different interventional consumables; the robotic arm end effector integrates a real-time ultrasound image display screen and a touch-enabled virtual joystick control screen.

[0048] Specifically, the touch interface offers a rich set of control interaction methods, allowing users to control the puncture intervention through the control buttons on the display screen. These include controls for the puncture instruments: a joystick-style spatial posture control button 19 (end-drag position control), a drag-and-drop spatial posture control switch button 22 (end-drag position control), a reset button 23 (returning to the last planned position), and an electric gripper opening / closing button 24 (controlling the opening and closing of the electric gripper). For displaying information during the puncture procedure, there are: puncture depth display 15 (current puncture depth), target lesion 16 (target lesion location in ultrasound imaging), puncture path direction 17 (preview of the puncture path in ultrasound imaging), and ultrasound imaging 18 (real-time display and local magnification of ultrasound images are available). Based on these control interaction methods, physicians can comprehensively grasp the actual puncture situation, improving surgical accuracy and safety.

[0049] Optionally, the optically positioned workpiece includes a passive marker and a support, with the passive marker disposed on the support.

[0050] Reference Figure 1 Specifically, passive marker 3 is set on the support 4 to locate the spatial pose of the distal puncture needle in the ultrasound image. This involves transforming the ultrasound image plane to the physical space coordinate system established by the optical positioning workpiece. The surgical site to be punctured and the puncture insertion point are then determined in this physical space coordinate system. The plane containing the central axis of the puncture needle is also transformed to the physical space coordinate system, providing real-time feedback on the position coordinates of the needle tip. The spatial pose is then controlled via the display screen to adjust the puncture needle, ensuring accurate puncture of the surgical site.

[0051] Implementing the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention provide an end effector for a puncture robot, comprising: a display screen for displaying ultrasound images and the operating status of the puncture robot, and for fine-tuning the end effector pose of the end effector through the display screen; a robotic arm docking seat, the first end of which is detachably connected to the robotic arm of the puncture robot; a connecting arm, the second end of which is connected to the second end of the robotic arm docking seat, the display screen being disposed on the connecting arm, the second end being the opposite end of the first end; and an electric gripper for receiving... The system receives driving information input via the display screen and clamps or releases the puncture needle according to this information. The electric gripper is located on the upper surface of the connecting arm. A guide gripper, comprising a left guide gripper and a right guide gripper, is also located on the upper surface of the connecting arm, at the first end of the electric gripper. This guide gripper guides and positions the puncture needle. An ultrasound probe, located at the first end of the connecting arm, acquires real-time ultrasound images of the puncture needle within the inserted body and inputs these images into the display screen. The ultrasound probe integrated into the end of the robotic arm monitors the puncture site in real time and displays the results on the screen. This allows the surgeon to fine-tune the puncture needle's position via the display screen, reducing the impact of errors and making the procedure faster, more precise, and safer.

[0052] like Figure 3 and Figure 4 As shown, this embodiment of the invention also provides an interaction method for a puncture robot, the method being applied to the aforementioned end-effector guidance device, comprising:

[0053] S100: Obtain the position information of the puncture needle by optically positioning the workpiece.

[0054] S200. Acquire ultrasound image information from the ultrasound probe using the ultrasound probe.

[0055] S300: The ultrasound image information and pose information are fused to obtain an ultrasound image with the puncture needle, and the ultrasound image with the puncture needle is sent to the display screen for display.

[0056] S400: Obtain first control information, output second control information according to the first control information, and control the movement of the puncture needle according to the second control information.

[0057] Optionally, the controller includes a robotic arm control processor that controls the movement of the puncture needle according to the second control information, specifically including:

[0058] S410, the controller includes a robotic arm control processor, which controls the movement of the puncture needle according to the second control information, specifically including:

[0059] The robotic arm control processor receives the second control information and sends the second control information to the robotic arm so that the robotic arm controls the electric gripper and guide gripper according to the second control information. The robotic arm is connected to the end-effector.

[0060] S420: The robotic arm controls the electric gripper and guide gripper according to the second control information.

[0061] Optionally, the controller further includes a message storage and forwarding server and a visual touch interaction processor. After sending the second control information to the robotic arm, the method further includes:

[0062] S430, the robotic arm control processor receives feedback information sent by the robotic arm;

[0063] The S440 robotic arm control processor sends feedback information to the message storage and forwarding server.

[0064] The S450 message storage and forwarding server stores feedback information and forwards it to the display screen via a visual touch interaction processor.

[0065] Specifically, the display screen outputs control signals to the visual touch interaction processor on the processor side through the user's control output. The visual touch interaction processor acquires the ultrasonic image information of the ultrasonic probe and inputs the ultrasonic image information into the display screen. The optically positioned workpiece inputs the end-effector pose information into the message storage and forwarding server on the processor side for storage. The message storage and forwarding server stores the ultrasonic image input by the visual touch interaction processor and forwards the robotic arm control information input by the visual touch interaction processor to the robotic arm control processor. The robotic arm control processor controls the electric gripper through the robotic arm and sends the robotic arm feedback information input by the robotic arm to the message storage and forwarding server for storage and forwarding to the display screen.

[0066] Optionally, it also includes a heartbeat sensor connected to the controller, which also has one or more functions such as heartbeat acquisition, abnormal warning, and status synchronization.

[0067] In one specific embodiment, a message storage and forwarding server records intraoperative control and feedback information in real time. This server, acting as a distributed centralized node in the entire interaction mechanism, plays a role in data distribution, ensuring interaction stability, and guaranteeing the order of information storage. In the end-effector control module, ultrasound image display and physician touchscreen control are both on a single touchscreen. Therefore, this information is integrated into the visual touch interaction processor, serving as a distributed node in the entire interaction mechanism. The electric gripper is controlled via the open I / O interface of the robotic arm. Thus, the electric gripper control and robotic arm control are integrated into the robotic arm control processor, serving as another distributed node in the entire interaction mechanism. The optical positioning workpiece positions the puncture needle at the end of the robotic arm, obtaining end-effector pose information, which is then sent to the message storage and forwarding server for storage and forwarding. The message storage and forwarding server retrieves and stores the ultrasound images input by the visual touch interaction processor and forwards the robotic arm control information input by the visual touch interaction processor to the robotic arm control processor. The robotic arm control processor controls the electric gripper through the robotic arm and sends the robotic arm feedback information input by the robotic arm to the message storage and forwarding server for storage and forwarding, ultimately outputting it to the display screen for display.

[0068] like Figure 5 As shown, in a specific embodiment, this invention also provides an information processing method applied to the above-described interaction method, comprising: module A saving generated first information to a storage server, and the storage server returning a storage ID of the first information to module A. Module A encrypting the first information and forwarding it through a forwarding server. Module B receiving the first information, decrypting and using the first information to generate second information. The storage server receiving the second information and updating the first information to indicate that it has been used.

[0069] Optionally, the method further includes the visual touch interaction processor and the robotic arm control processor establishing a topological relationship through a handshake, and the visual touch interaction processor and the robotic arm control processor dissolving the topological relationship by waving.

[0070] In one specific embodiment, the storage server uses MongoDB, and the forwarding server uses RabbitMQ. The control module is designed based on a distributed architecture, thus decoupling it from the surgical planning and surgical robot main control software. Based on this architecture, the module can define clear external interaction interfaces, reducing the difficulty and cost of developing and maintaining the entire surgical robot. Regarding the interaction mechanism, firstly, a distributed data flow mechanism is defined, such as... Figure 4As shown, data interaction between nodes is broken down into two steps: storage and distribution. MongoDB is used as the storage medium, and RabbitMQ is used as the message distribution medium. Message data is encrypted and decrypted to ensure communication data security. Secondly, centralized service software such as heartbeat service, anomaly warning service, and state synchronization service need to be deployed on the storage and distribution servers to ensure stable and reliable node interaction. Thirdly, establishing and dissolving topology relationships between nodes requires a handshake and teardown process to verify the identity and state of nodes, ensuring secure interaction.

[0071] The operating principle of this invention is as follows: During the surgery, the physician will be positioned facing the touchscreen. To minimize the physician's need to consider spatial relationships when manipulating the puncture tip, control commands are generated based on the coordinate system of the robotic arm docking seat (flange) for joystick-based spatial posture control and displacement. This allows the physician to control the swing and displacement of the puncture tip without having to visualize the relationship between the robotic arm base coordinate system and the intended direction, achieving a "what you see is what you get" posture control effect. Joystick-based control of forward and backward puncture movement is based on the current puncture guide path vector relative to the operating table, moving it downwards or upwards. To make the joystick control method more comprehensive and allow the robot to adjust the position of the ultrasound probe on the patient's surface, control buttons for rotation along the puncture guide vector are designed, with clockwise and counterclockwise rotation buttons, allowing adjustment of the ultrasound probe position without changing the puncture path. The drag-and-drop control method requires a drag force sensor designed outside the module, and control commands are also generated based on the coordinate system of the robotic arm docking seat (flange).

[0072] Implementing the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention provide an interaction method for a puncture robot, the method being applied to the aforementioned end-effector, comprising: obtaining the pose information of the puncture needle through optical positioning workpiece positioning; obtaining ultrasound image information of the ultrasound probe through the ultrasound probe; fusing the ultrasound image information and the pose information to obtain an ultrasound image with the puncture needle, and sending the ultrasound image with the puncture needle to a display screen for display; obtaining first control information, outputting second control information according to the first control information, and controlling the movement of the puncture needle according to the second control information. By using a distributed architecture formed by sending information to a message storage and forwarding server for storage and forwarding, the development and maintenance of the surgical robot is greatly reduced in difficulty and development cost, while also enabling better and more complete digital recording of the surgical process.

[0073] This invention provides a robot, including a robotic arm, with the aforementioned end effector connected to the end of the robotic arm, and the aforementioned interaction method applied to the robot.

[0074] It is evident that the contents of the above-described device and interaction method embodiments are applicable to this robot embodiment. The specific functions implemented by this robot embodiment are the same as those of the above-described device and interaction method embodiments, and the beneficial effects achieved are also the same as those achieved by the above-described device and interaction method embodiments.

[0075] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. An end effector guide for a puncture robot, characterized in that, Includes display screen, robotic arm docking station, electric gripper, The device includes a guide gripper, a connecting arm, an optical positioning workpiece, an ultrasonic probe, and a controller, wherein the controller is electrically connected to the display screen, the electric gripper, the guide gripper, the ultrasonic probe, and the optical positioning workpiece, respectively. The ultrasound probe is used to acquire real-time ultrasound images of the puncture needle inserted into the body and input the ultrasound images into the display screen. The ultrasound probe is located at the first end of the connecting arm. The display screen is used to display the ultrasound images and the operating status of the puncture robot, as well as to perform end-effector pose fine-tuning control on the end-effector guide device. The second end of the connecting arm is connected to the docking seat of the robotic arm, and the display screen is disposed on the connecting arm. The second end is the opposite end of the first end. The electric gripper is used to receive drive information input from the display screen, and to clamp or release the puncture needle according to the drive information. The electric gripper is disposed on the connecting arm. The guide gripper is disposed on the connecting arm. The guide gripper is used to control the direction of the puncture needle clamped by the electric gripper. The guide gripper and the ultrasound probe are located on the same guide axis. The optical positioning workpiece is disposed on the connecting arm and is used to locate the position and orientation information of the puncture needle. The position and orientation information is used to determine the spatial orientation of the puncture needle in the ultrasound image.

2. The end-guide device according to claim 1, characterized in that, The display screen includes a touch screen, which includes a control interface and a display interface, and the control interface and the display interface are on the same interface.

3. The end-guide device according to claim 1, characterized in that, The guide gripper includes a left guide gripper and a right guide gripper, which are used to guide and position the puncture needle.

4. The end-guide device according to claim 1, characterized in that, The optical positioning workpiece includes passive markers and a support, with the passive markers disposed on the support.

5. A robot, characterized in that, The invention includes a robotic arm, the end of which is connected to an end-effector as described in any one of claims 1-4.

Images