A remote surgical robot for retinal vascular puncture

Abstract

This application discloses a remote surgical robot for retinal vascular puncture, including a master control device, a 5G communication module, and an execution robot device. The execution robot device is remotely controlled by the master hand. The execution robot device includes two mechanisms: a SCARA mechanism and an RCM mechanism. The SCARA mechanism is used to locate the puncture position of the injection needle. The RCM mechanism has three motors that control the tumbling, pitching, and rotation of the injection needle surface. Tumbling and pitching determine the puncture angle of the injection needle, while rotation adjusts the needle tip orientation along the longitudinal axis of the injection needle. The master control device can independently control either the SCARA mechanism or the RCM mechanism via a foot pedal. A puncture motor is installed on the end effector to control the puncture, and a force feedback structure is installed on the injection needle. The master control device senses changes in force during puncture to determine whether the puncture was successful. This setup enables rapid and effective remote control of the puncture position, angle, orientation, and force feedback.

Description

Technical Field

[0001] This invention belongs to the field of ophthalmic surgical robot technology, and particularly relates to the field of robots for fundus vascular puncture, specifically a remote surgical robot for fundus vascular puncture. Background Technology

[0002] Occlusion of blood vessels in the retina is a serious ophthalmic emergency. Currently, the common clinical approach is to inject thrombolytic drugs into the affected blood vessels for targeted treatment and patency. This procedure is a microsurgical operation requiring the surgeon to perform precise manipulations under a microscope using ophthalmic instruments. However, due to the small size of the eyeball and the delicate and fragile structure of its tissues, the injection of medication for retinal vascular occlusion demands exceptional hand-eye coordination and a keen sense of precision to avoid surgical failure.

[0003] The most scarce element in this type of surgery is the availability of highly skilled doctors and the precise technique for puncturing blood vessels during drug injection. This is because the diameter of blood vessels in the retina is approximately 80 μm, while the average tremor in the hands of a typical ophthalmologist reaches 156 μm. Therefore, if the doctor's skill level is not high, the risk of puncture failure is extremely high.

[0004] Therefore, given the scarcity of highly skilled doctors, if patients feel that the location of the doctor would greatly increase their medical costs, and that traveling to a place other than their place of residence would also affect their recovery, then the scarcity of highly skilled doctors is not an option to practice in the patient's location due to the scarcity of medical resources.

[0005] To address the above issues, there is a need for a device that allows patients and doctors to inject drugs to block retinal blood vessels without leaving their usual work or living locations. Summary of the Invention

[0006] To address the aforementioned issues, this application discloses a remote surgical robot for retinal vascular puncture. The robot is remotely controlled via 5G communication technology to perform the puncture procedure. High-speed data transmission ensures delay-free, high-quality transmission of surgical images and immediate response to control commands. Furthermore, to facilitate operation by the surgeon, the robotic arm is divided into two mechanisms: a SCARA mechanism and an RCM mechanism. The SCARA mechanism, controlled by four servo motors, includes a vertical movement mechanism and a horizontal movement structure. The first end of the horizontal movement structure is combined with the vertical movement structure, and the second end connects to the RCM mechanism. The SCARA mechanism drives the RCM mechanism, which houses the injection needle, to change its three-dimensional position, thus locating the puncture point. Once the needle tip position is located, its position remains unchanged until the puncture action occurs. The RCM mechanism has three motors that control the needle's roll, pitch, and rotation. Roll and pitch determine the puncture angle, and a rotating structure at the RCM's execution end, allowing rotation along the needle's longitudinal axis, determines the needle tip orientation. The dual-mechanism setup effectively locates the puncture position and angle, stopping movement upon successful detection. This movement is controlled by the master control module. Once the puncture position and angle are found, a puncture motor on the end effector moves the injection needle along the puncture direction. A force feedback structure on the injection needle allows the master control device to sense changes in force during puncture, indicating successful puncture. Furthermore, to enable step-by-step control of the SCARA and RCM mechanisms, a foot pedal is incorporated into the master control device. This alternating mechanism controls the master's switching between SCARA and RCM mechanisms, maximizing ease and speed of position finding. The choice between SCARA and RCM mechanisms also ensures maximum ease of operation for the master. 5G technology transmits the movement paths of the SCARA and RCM mechanisms and the puncture end effector in real time, ensuring hand-eye coordination for the operator.

[0007] The specific technical solution is as follows:

[0008] This application also discloses a remote surgical robot for retinal vascular puncture, which includes a master control device, a 5G communication module, and an execution robot device.

[0009] The main hand control device includes a main hand execution structure, a foot pedal switching structure, and a video display structure;

[0010] The 5G communication module includes a dedicated cloud network and a data transmission terminal;

[0011] The robotic device includes a puncture actuation structure, a motion control module, and a video acquisition module;

[0012] The puncture execution structure is used to set the injection needle and perform the puncture action.

[0013] The motion control module comprises two mechanisms: the SCARA mechanism and the RCM mechanism. The SCARA mechanism uses four servo motors for control and includes a Z-axis movement mechanism and a horizontal movement structure. The first end of the horizontal movement structure is combined with the vertical movement structure, and the second end of the horizontal movement structure is connected to the RCM mechanism. The SCARA mechanism drives the three-dimensional position change of the RCM mechanism, which contains the injection needle, to determine the puncture position of the injection needle. Once the injection needle position is determined, the position of the injection needle tip will not change until the puncture action occurs. The RCM mechanism has three motors: a roll motor, a pitch motor, and a motor for rotating the injection needle. The roll motor and pitch motor determine the puncture angle of the injection needle. The orientation of the injection needle tip is determined by a rotating structure at the execution end of the RCM that allows rotation along the longitudinal axis of the injection needle. The injection needle is positioned at the RCM point.

[0014] The video acquisition module consists of a video acquisition card mounted on an ophthalmic microscope. It acquires video signals in real time during the operation, processes them, and then sends them to the cloud via the control module.

[0015] The video acquisition module collects and sends back video stream information of the injection needle position and angle in real time. Based on the sent video stream information, the master hand execution structure remotely controls the SCARA mechanism to determine the puncture position; controls the RCM mechanism to determine the puncture angle; and controls the puncture execution structure to perform the puncture action. The master hand switches between the SCARA mechanism and the RCM mechanism through a foot pedal switching structure.

[0016] Furthermore, the SCARA mechanism includes a horizontal movement structure and a vertical movement structure. The horizontal movement structure includes three servo motors: an A motor rotating along the A-axis, a B motor rotating along the B-axis, and a C motor rotating along the C-axis. A motor and B motors are connected by a first connecting plate, and B motors and C motors are connected by a second connecting plate. The vertical movement structure includes a vertical base with a lifting track, a lifting structure connected to the C motor that moves along the lifting track, and a D motor rotating along the D-axis that drives the lifting structure to move along the lifting track. The A motor is connected to the RCM mechanism mounting plate. This configuration ensures horizontal movement using the three servo motors and the connecting plates, and vertical movement using the vertical movement structure. When the main hand operates the structure, only simple motion transmission is needed to adjust the horizontal and vertical positions.

[0017] Furthermore, the RCM mechanism includes a tumbling motor rotating along the E-axis mounted on the equipment plate, a parallelogram structure, a parallelogram structure rotating with the tumbling motor, a pitch guide rail, a moving block extending from the parallelogram structure and moving within the pitch guide rail, the moving block being hinged to the parallelogram structure, a lead screw mounted within the pitch guide rail, the moving block mounted on the lead screw, a pulley one mounted on the side of the lead screw near the tumbling motor, and a pitch motor mounted below the rail on the side near the tumbling motor, with the F-axis as the rotation axis, a pulley two extending from the pitch motor, and a belt connecting pulley one and pulley two; rotation of the pitch motor in different directions drives rotation of the lead screw in different directions, and different directions... The lead screw rotates, causing the moving block to move in different directions, thus forming a parallelogram structure with different pitch angles. A rotating structure is set at the end of the parallelogram structure, and a motor for rotating the injection needle is set inside the rotating structure. The axis of the motor for rotating the injection needle is the G-axis. A puncture execution structure is set inside the rotating structure, and the needle tip of the puncture execution structure is set at the RCM point. The E-axis and G-axis pass through the RCM point, and the E-axis is parallel to the pitch guide rail. Through this setting, the pitch motor enables the parallelogram structure to pitch around the RCM point. The rotation of the E-axis and G-axis ensures that the rotation does not change the position of the RCM point, thereby achieving accurate adjustment of the puncture angle after the puncture position is found.

[0018] Furthermore, a puncture motor is installed inside the puncture execution structure. The rotation drives the injection needle to perform the puncture action. The main hand execution structure controls the puncture path and sends the puncture path to the puncture execution structure through the 5G network, controlling the puncture motor to rotate to execute the puncture path.

[0019] Furthermore, one of the biggest challenges in injecting drugs into retinal blood vessels is that even with accurate positioning and angle, it's not guaranteed that the puncture will penetrate the blood vessel. This requires the operator to perceive the difference between penetration and non-penetration. Therefore, the operator needs to be able to receive the force during the puncture. Specifically, a micro-force measurement module is installed, consisting of a Bragg grating on the outside of the injection needle. This module measures the force on the needle in real time based on the deformation of the grating caused by the force during the puncture. The measured force is processed by a fiber optic demodulator and sent to the cloud, then transmitted to the operator. The operator senses the force during the puncture to determine if the needle has penetrated the blood vessel. In practical use, the needle deforms during injection, causing the optical fiber to deform as well. This changes the period of the grating inside the fiber, leading to a change in the output wavelength. The wavelength shift inside the fiber is calculated based on the output wavelength, thus determining the needle's deformation and the magnitude of the force. The direction of the force is obtained by fusing data from the three optical fibers, achieving precise force feedback.

[0020] Furthermore, the injection needle is connected to the syringe on the microfluidic pump via a soft tube, allowing for micro-injection. Once the injection needle punctures and enters the blood vessel, as long as the patient's position remains unchanged, the stopped motor and other mechanisms ensure the precise and stable position of the injection needle. Therefore, manual injection is no longer necessary. By controlling the speed of the microfluidic pump, the safe injection of thrombolytic drugs and other related medications into the retinal blood vessels can be guaranteed. This pathway has been established, and the type of medication can be changed at the injection pump end, saving the difficulty of re-locating blood vessels and enabling highly precise injection of multiple medications.

[0021] Furthermore, the lifting structure of the vertical moving structure of the SCARA mechanism and the microfluidic pump are both fixedly mounted on the base, and the base is then fixed to a high platform that meets the height requirements. This arrangement can effectively fix the structure to the bedside. Placing the SCARA mechanism and the microfluidic pump on the same base facilitates the integration and operation of the structure, and there is no need to install excessively long flexible tubes, as long as the follow-up requirements are met.

[0022] Technical effect

[0023] By utilizing a dedicated 5G network, real-time transmission of video and operator signals is effectively guaranteed, significantly reducing latency. The motion control module is divided into two mechanisms: the SCARA mechanism and the RCM mechanism. The operator can independently control these two mechanisms via a foot-operated switching mechanism, greatly reducing the difficulty of finding the puncture site. Finding the puncture site through the SCARA mechanism and the puncture angle through the RCM mechanism allows for more efficient and accurate location of the puncture position and angle. This is primarily because the signals received when the operator controls the SCARA and RCM mechanisms are simple linear signals, greatly reducing signal transmission latency and simplifying signal conversion between the two ends, further ensuring real-time signal transmission. Furthermore, the use of both SCARA and RCM mechanisms improves the accuracy of the movement distance, ensuring the effectiveness of retinal vascular puncture. During operation, to better achieve the desired needle insertion into the surgical site, the operator can repeatedly switch between the SCARA and RCM mechanisms via the foot-operated switching mechanism, finding the optimal position and angle through multiple attempts.

[0024] The rotating structure contains a puncture execution structure, in which the needle tip of the puncture execution structure is set at the RCM point; the E-axis and G-axis pass through the RCM point, and the E-axis is parallel to the pitch guide rail to ensure that the position of the needle tip will not change regardless of whether it is a pitch operation, a flip operation or a rotation operation.

[0025] The RCM mechanism, equipped with three motors—a tumbling motor, a pitching motor, and a needle rotation motor—enables the needle to tumble, pitch, and rotate. This addresses the shortcomings of needle angle control. In particular, the needle rotation motor controls the needle's rotation, allowing for the adjustment of the needle tip's orientation to achieve an optimal puncture direction and ensuring a high success rate for retinal vascular puncture. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process structure of the robot of the present invention;

[0027] Figure 2 This is a schematic diagram of the overall structure of the motion control module of the present invention;

[0028] Figure 3 This is a schematic diagram of the SCARA mechanism structure of the present invention;

[0029] Figure 4 This is a schematic diagram of the vertical seat portion of the SCARA mechanism structure of the present invention;

[0030] Figure 5 This is a schematic diagram of the front side view of the RCM mechanism structure of the present invention;

[0031] Figure 6 This is a schematic diagram of the upper side view of the RCM mechanism structure of the present invention;

[0032] Figure 7 This is a schematic diagram of the puncture execution structure of the present invention, which includes a motor for rotating the injection needle.

[0033] Figure 8 This is a schematic diagram of the puncture execution structure of the present invention;

[0034] Figure 9 This is a schematic diagram of the main hand execution structure of the present invention;

[0035] Figure 10 This is a schematic diagram of the operational process of the robot device of the present invention;

[0036] Figure 11 This is a flowchart illustrating the signal transmission and corresponding time between the main control device and the execution robot in this invention.

[0037] Explanation of main figure symbols

[0038] 1. Main hand control device; 11. Main hand execution structure; 12. Foot pedal switching structure; 13. Video display structure;

[0039] 2. 5G communication module; 21. Cloud-based dedicated network; 22. Data transmission terminal; 221. CPE device;

[0040] 3. Execution Robot Device; 31. Puncture Execution Structure; 311. Puncture Motor; 312. Fiber Bragg Grating; 313. Grating Demodulator; 314. Injection Needle; 315. Microfluidic Pump; 316. Flexible Tube; 32. Motion Control Module; 321. SCARA Mechanism; 3211. Motor A; 3212. Motor B; 3213. Motor C; 3214. Motor D; 3215. Connecting Plate 1; 3216. Connecting Plate 2; 3217. Vertical Seat; 3218. Lifting... 3219. Lifting structure; 322. RCM mechanism; 3221. Tilting motor; 3222. Pitch motor; 32221. Belt pulley one; 32222. Belt pulley two; 32223. Annular belt; 3223. Motor for rotating injection needle; 3224. Setting plate; 32251. Parallelogram structure; 32252. Moving block; 32253. Pitch guide rail; 32254. Moving block; 32255. Lead screw; 33. Video acquisition module;

[0041] 4. Motor control structure. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] In this document, “illustrative” means “serving as an example, illustration or description”, and any illustration or implementation described herein as “illustrative” should not be construed as a more preferred or advantageous technical solution.

[0044] To keep the drawings concise, only the parts relevant to this application are shown schematically in each drawing, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, in some drawings, only one of the components with the same structure or function is shown schematically, or only one of them is labeled.

[0045] In this document, unless otherwise expressly specified and limited, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance; unless otherwise specified or explained, the term "multiple" refers to two or more; the terms "connected," "fixed," etc., should be interpreted broadly. For example, "connected" can be a fixed connection, a detachable connection, an integral connection, or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0046] refer to Figure 1 A remote surgical robot for retinal vascular puncture includes a master control device 1, a 5G communication module, and an execution robot device 3.

[0047] The master control device 1 includes a master execution structure 11, a foot pedal switching structure 12, and a video display structure 13.

[0048] The 5G communication module includes a dedicated cloud network 21 and a data transmission terminal 22. The dedicated cloud network 21 uses tunneling protocol technology to achieve information encryption, access control, and network security protection between the master control device 1 and the execution robot device 3. The data transmission terminal 22 consists of two 5G CPE devices 221, located at the master control device 1 and the execution robot device 3 respectively. These devices can send control, feedback, and video stream information from both ends to the dedicated cloud network 21, and can also read data sent by the other end from the dedicated cloud network 21 and transmit it to the other end. Dual-end communication via the 5G CPE devices 221 utilizes the high bandwidth and low latency of 5G to achieve remote surgical operations. The 5G CPE devices 221 can achieve a maximum transmission rate of 1167Mbps and have four built-in signal amplifiers, greatly enhancing wireless coverage and signal strength while ensuring secure and stable network connections. Dual-end communication via two 5G CPE devices 221 allows for the priority selection of the TCP transmission protocol and transmission method to transmit the surgical images from the robot end to the master control end in high definition and in real time.

[0049] The robotic device 3 includes a puncture execution structure 31, a motion control module 32, and a video acquisition module 33;

[0050] refer to Figure 7 and Figure 8The puncture execution structure 31 is used to set the injection needle 314 and perform the puncture action. A puncture motor 311 is installed inside the puncture execution structure 31, which rotates to drive the injection needle 314 to perform the puncture action. The master execution structure 11 controls the puncture path and sends the puncture path to the puncture execution structure 31 in real time via a 5G network, controlling the rotation of the puncture motor 311 to execute the puncture path. More specifically, the puncture path signal is converted into the rotation speed and number of revolutions of the puncture motor 311, thereby enabling the master to control the puncture path. A Bragg grating is installed on the injection needle 314 for real-time force measurement. Three fiber optic gratings 312 are closely attached to the upper part of the injection needle 314 tip at 120° intervals. The fiber optic gratings 312 contain phase gratings, which form a narrow-band transmission filter inside the fiber core. When a beam of light passes through the grating, wavelengths that meet certain conditions will be reflected, while the remaining wavelengths will continue to be transmitted through the grating. During injection, the injection needle 314 deforms, causing the optical fiber to deform as well. This leads to a change in the grating period inside the fiber, resulting in a change in the output wavelength. The wavelength shift inside the fiber is calculated based on the output wavelength, thus determining the needle's deformation and the magnitude of the force. The direction of the force is obtained through data fusion from the three optical fibers, achieving precise force feedback. The micro-force measurement module includes this force detection part and an instrument that receives the optical fiber output signal and calculates the force. A high-precision, high-resolution grating demodulator 313 processes the Bragg grating measurement data. The grating demodulator 313 is used for demodulating signals from various types of grating sensors and acquiring sensor data. It has a selectable scanning frequency between 1 and 1000 Hz, ensuring the accuracy of the micro-force data. The received real-time force is transmitted back to the main actuator 11 via a 5G network. This force feedback allows the operator to understand the force situation during the puncture process, ensuring a successful puncture.

[0051] refer to Figure 8 The injection needle 314 is connected to the syringe on the microfluidic pump 315 via a flexible tube 316. Micro-injection is performed via the microfluidic pump 315. Once the injection needle 314 punctures and enters the blood vessel, as long as the patient's position remains unchanged, the stopped motor and other mechanisms ensure the precise and stable position of the injection needle 314. Therefore, manual injection is no longer necessary. By controlling the speed of the microfluidic pump 315, the safe injection of thrombolytic drugs and other related medications into the retinal vessels can be guaranteed. This pathway is established; simply changing the type of drug at the injection pump end eliminates the difficulty of re-locating blood vessels and allows for highly precise injection of multiple drugs. The microfluidic pump 315 can achieve a minimum injection volume of 1 μL and an injection flow rate from 0.05 nL / min to 520 μL / min; this flow control meets the micro-injection requirements of the retina.

[0052] refer to Figure 2-7The motion control module 32 includes two mechanisms: a SCARA mechanism 321 and an RCM mechanism 322. The SCARA mechanism 321 is controlled by four servo motors and includes a Z-axis movement mechanism and a horizontal movement structure. The first end of the horizontal movement structure is combined with the vertical movement structure, and the second end of the horizontal movement rod structure is connected to the RCM mechanism 322. The SCARA mechanism 321 drives the RCM mechanism 322, which is equipped with an injection needle 314, to change its three-dimensional position, thereby determining the puncture position of the injection needle 314. After the position of the injection needle 314 is determined, the position of the tip of the injection needle 314 will not change until the puncture action occurs. The RCM mechanism 322 has three motors: a tumbling motor, a pitching motor 3222, and a needle rotation motor 3223. The tumbling motor and the pitching motor 3222 determine the puncture angle of the needle 314. The needle tip orientation of the needle 314 is determined by setting a rotating structure at the execution end of the RCM to achieve rotation along the longitudinal axis of the needle 314. The needle 314 is set at the RCM point position.

[0053] refer to Figure 3 and Figure 4 The SCARA mechanism 321 includes a horizontal movement structure and a vertical movement structure. The horizontal movement structure includes three servo motors: an A motor 3211 rotating along the A-axis, a B motor 3212 rotating along the B-axis, and a C motor 3213 rotating along the C-axis. The A motor 3211 and the B motor 3212 are connected by a first connecting plate 3215, and the B motor 3212 and the C motor 3213 are connected by a second connecting plate 3216. The vertical movement structure includes a vertical seat 3217 with a lifting rail 3219, a lifting structure 3218 connected to the C motor 3213, and the lifting structure 3218 moving along the lifting rail 3219. A D motor 3214 rotating along the D-axis is also provided, which drives the lifting structure 3218 to move along the lifting rail 3219. The A motor 3211 is connected to the RCM mechanism 322 mounting plate 3224. This setup ensures horizontal movement using three servo motors and their connecting plates, and vertical movement using a vertical movement structure. When the main hand operates structure 11, only simple motion transmission is needed to adjust the horizontal and vertical positions. The four AD motors 3214, through internal encoders and feedback devices, allow for precise control of their position, speed, and acceleration. A braking device is also included to provide additional safety protection.

[0054] refer to Figures 5-7The RCM mechanism 322 includes a tumbling motor rotating along the E-axis mounted on a device plate, a parallelogram structure 32251, a parallelogram structure rotating with the tumbling motor, a pitch guide rail 32253, a moving block 32254 32252 extending from the parallelogram structure 32251 and moving within the pitch guide rail 32253, the moving block 32254 32252 being hinged to the parallelogram structure 32251, and the pitch guide rail... A lead screw 32255 is installed inside 32253, and a moving block 32254 32252 is installed on the lead screw 32255. A pulley 32221 is installed on the side of the lead screw 32255 near the tumbling motor, and a pitch motor 3222 is installed below the track on the side near the tumbling motor. The rotating shaft is the F-axis. A pulley 32222 extends from the pitch motor 3222, and a ring belt 32223 connects the pulley 32221 and the pulley 32222. The rotation of motor 3222 in different directions drives the lead screw 32255 to rotate in different directions. The rotation of the lead screw 32255 in different directions drives the moving blocks 32254 and 32252 to move in different directions, thereby forming a parallelogram structure 32251 with different pitch angles. A rotating structure is fixedly installed at the end of the parallelogram structure 32251. A motor 3223 for rotating the injection needle is installed inside the rotating structure. The axis of the motor 3223 for rotating the injection needle is the G-axis. A puncture execution structure 31 is installed inside the rotating structure. The needle tip of the puncture execution structure 31 is set at the RCM point. The E-axis and G-axis pass through the RCM point, and the E-axis is parallel to the pitch guide rail 32253. Through this arrangement, the pitch motor 3222 enables the parallelogram structure 32251 to pitch around the RCM point. The rotation of the E-axis and G-axis ensures that the rotation does not change the position of the RCM point, thereby achieving accurate adjustment of the puncture angle after finding the puncture position. The RCM mechanism 322 not only allows for adjustment of the pitch angle but also the tilt angle. Most importantly, the needle rotation motor 3223 enables adjustment of the needle tip orientation of the injection needle 314, ensuring optimal puncture angle and needle tip orientation for successful insertion. Each motor's adjustment is controlled by its speed and direction, making signal transmission simple and convenient. The tilt motor is a servo motor with a maximum speed of 6000 rpm, achieving high-precision positioning with high efficiency and low noise. The pitch motor 3222 and the needle rotation motor 3223 are brushed motors, featuring a lightweight and compact design while maintaining high power, an efficiency approaching 90%, low energy consumption, and quiet operation.

[0055] A motor control structure 4 is set up to connect to the motor control signal. The motor control structure 4 controls the rotation of each axis motor according to the motor signal, thereby controlling the changes in each position and angle.

[0056] refer to Figure 10At any given time, the master actuator 11 can only control one of the RCM and SCARA mechanisms, and the switching is performed by the foot pedal switching structure 12: pressing the left button controls the SCARA mechanism 321; pressing the right button controls the RCM mechanism. The motors in both the RCM and SCARA mechanisms 321 are connected to an 8-axis motor control structure 4. The control data on the master actuator 11 is input into a control card, which then controls the corresponding motor to move according to the input data. When the operator presses the button on the main hand actuator 11, the main hand actuator 11 begins recording motion data. While the operator is holding down the button, the main hand actuator 11 moves, recording the hand displacement, movement speed, angle, and angular velocity in real time. This data enters the main hand actuator 11 control unit, undergoes simple processing, and is then sent to the execution robot device 3 via UDP protocol. The execution robot device 3 control unit further processes this data: first, it extracts the displacement, speed, and angle data collected by the main hand actuator 11; after filtering and debouncing, it converts the data commands into the robot's joint angles and end effector pose; finally, it uses inverse kinematics algorithms to calculate the motor control signals, such as the rotation direction, rotation speed, and rotation distance of each axis motor in the RCM or SCARA mechanism 321. These motor control signals are then transmitted to the motor control structure 4, which controls the movement of each axis motor based on these signals. When the operator releases the button on the main hand actuator 11, no information will be transmitted to the motor regardless of movement, and the motor will remain fixed at the last position reached. The operator's hand displacement and speed are converted into motor rotation information, thus enabling the main hand actuator 11 to control the 8-axis motion mechanism. During actual surgery, the operator first presses the left button on the foot pedal and uses the main hand actuator 11 to control the SCARA mechanism 321. At this time, no motor in the RCM mechanism 322 will move. The operator controls the injection needle 314 to move above the injection target. Once the injection needle 314 has reached the target position, the operator can press the right button on the foot pedal and use the main hand actuator 11 to control the RCM mechanism 322. At this time, none of the motors on the SCARA mechanism 321 will move, and the position of the needle will not change regardless of the movement of the RCM mechanism 322. At this point, the operator can control the RCM mechanism 322 to change the direction and angle of needle insertion. Once all positions, directions, and angles are determined, the operator locks each axis motor of the RCM mechanism 322 through software, leaving only the motor controlling the movement of the injection needle 314, and controls the motor to move the needle up and down to complete the injection task.Of course, the specific adjustment process also needs to consider the need for the injection needle 314 to enter the surgical position. This requires repeated switching of the foot pedal switching structure 12 and multiple adjustments to the control of the RCM or SCARA mechanism 321 to ensure that the angle and position are appropriate.

[0057] The video acquisition module 33 is a USB video acquisition module. A video acquisition card is mounted on the ophthalmic microscope, which acquires the video stream information obtained by the ophthalmic microscope in real time during surgery. After processing, the data is sent to the cloud by the control module. Data transmission via USB improves data transmission efficiency and further avoids latency issues. The camera of the video acquisition module 33 has a maximum resolution of 2160*3840, built-in dual microphones, and connects to the video acquisition card via USB, enabling high-definition image acquisition and high-speed transmission.

[0058] The video acquisition module 33 collects and transmits back in real-time video stream information regarding the position and angle of the injection needle 314. Based on this video stream information, the master hand execution structure 11 remotely controls the SCARA mechanism 321 to determine the puncture position; controls the RCM mechanism 322 to determine the puncture angle; and controls the puncture execution structure 31 to perform the puncture action. The foot pedal switching structure 12 allows the master hand to switch between control of the SCARA mechanism 321 and the RCM mechanism 322. Real-time video stream transmission ensures hand-eye coordination.

[0059] The lifting structure 3218 of the vertical moving structure of the SCARA mechanism 321 and the microfluidic pump 315 are both fixedly mounted on the base, and the base is then fixed to a platform that meets the height requirements (the platform is not shown). This arrangement can effectively fix the structure to the bedside. Placing the SCARA mechanism 321 and the microfluidic pump 315 on the same base facilitates the integration and operation of the structure, and there is no need to set an excessively long flexible tube 316, as long as the follow-up requirements are met.

[0060] refer to Figure 9The haptic device of the master hand actuator 11 uses a high-precision pen-shaped end effector, which provides high-precision position capture and maintains accurate gravity compensation in translation and positioning space. The haptic device renders high contact force at a rate of 4kHz and combines passive and actuating parts to improve haptic transparency. The master hand actuator 11 can optionally use an Omega.6 haptic feedback device. When the foot pedal switching structure 12 switches to control the SCARA mechanism 321, it controls the pen-shaped end effector to perform forward, backward, left, right, and up-down movements. However, because the range of motion of the retinal blood vessels is small, the complete stroke in a single direction only reaches a set distance for the SCARA mechanism 321. The position is continuously adjusted by repeated single-direction movements. For example, when the pen-shaped end effector performs a complete forward-backward movement, the SCARA mechanism 321 moves forward by 1 mm; when it moves backward-forward, the SCARA mechanism 321 moves backward by 1 mm. The movement distance of the main hand actuator 11 in one direction is positively correlated with the movement distance of the SCARA mechanism 321. Through repeated micro-adjustments, the SCARA mechanism 321 moves with the injection needle 314 to the appropriate position.

[0061] To ensure the accuracy of transmitted data, multiple methods are used for data processing. After obtaining the operator's action data, the main hand actuator 11 processes the action data: first, it extracts the action information, and then places it into the corresponding position in a pre-defined communication protocol. This protocol contains the sequence number of the transmitted data and an error checksum for a data frame. The main hand actuator 11 sends the frame data containing position information, sequence number, and error checksum to the execution robot device 3. If various situations occur during transmission that cause errors in the data frame or transmission failure, the execution robot device 3 will detect a mismatch in the sequence number or an error checksum. After receiving the data, the control structure of the execution robot device 3 first checks whether the sequence number and error checksum correspond. If they do not correspond, it means the data frame is incorrect. The execution robot device 3 will predict this erroneous data frame based on previously received data and replace the erroneous data with the predicted data. If the data is correct, the action information is extracted from the received data frame and processing begins. The received motion information includes position information, direction of motion, and speed of motion. The host of the execution robot device 3 processes this information: it is filtered by an extended Kalman filter and a low-pass filter, and then calculated using a corresponding nonlinear motion model to ultimately determine the rotation direction, speed, and distance of the robot's motors on each axis, controlling the movement of the corresponding mechanisms. The force data obtained by the execution robot device 3 is also processed using the same method as the main hand execution structure 11, and the resulting data frame is transmitted back to the main hand execution structure 11. After receiving the force data, the host of the main hand execution structure 11 first fuses it with the data obtained through a multimodal sensor fusion algorithm, and then transmits it to the main hand to feed the force back to the operator.

[0062] To ensure effective stopping of the motors, Hall effect sensors are used as limit switches to restrict movement when the motors are not in operation. Emergency braking of each axis motor is achieved by relays, with relay action and return times both less than 0.02s, thus improving robot safety.

[0063] Delay in remote control is unavoidable. To address the delay issue, the following strategies are used to achieve precise control even with delay: the control data from the master control terminal is compressed and encoded into data frames, and then quickly transmitted to the execution robot device 3 via the UDP protocol; the force data from the execution robot device 3 is processed in the same way as the control data from the master control terminal and then quickly transmitted to the master control terminal via the UDP protocol.

[0064] The entire transmission process is based on 5G technology. To achieve low latency, high reliability, and determinism in remote communication, this patent creates an independent virtual network for remote data transmission and divides the transmitted data into different standardized service levels, dynamically allocating resources. Transmission latency is reduced through short frame structures, unrestricted scheduling, and pre-scheduling technologies. Reliability of data transmission is ensured by creating redundant links. To avoid delays, a time synchronization module is included. This module is responsible for synchronizing the clocks of the master control device 1 and the execution robot device 3, enabling the execution robot device 3 to obtain the accurate time of the control signal through timestamps and maintain real-time updates on network latency and clock offset.

[0065] refer to Figure 11 The specific algorithm for the time synchronization module is as follows: To ensure time synchronization between the main hand control device 1 and the execution robot device 3, two problems must be solved: 1) Clock offset between the main hand control device 1 and the execution robot device 3. Each computing unit has its own internal clock. Upon power-on, the computer obtains a world time through the network. However, due to technical reasons, the world time obtained by each computer will have a certain deviation, which we denote as... Assume the network latency is a fixed value over a certain period of time, denoted as . Network latency and time deviation can be calculated by exchanging synchronization timestamps between the master control device 1 and the execution robot device 3.

[0066]

[0067]

[0068] Set up a second-order filter with a sampling rate of 0.1Hz and a cutoff frequency of 0.001Hz. The specific filtering formula is as follows:

[0069]

[0070] After filtering

[0071]

[0072] Achieve timestamp synchronization. A common method to ensure timestamp synchronization is used, achieving time synchronization through the equality of timestamps to avoid delays; ensures synchronized and precise control between the main hand execution structure 11 and the execution robot device 3, and guarantees the precision requirements of the fundus blood vessels.

[0073] In high-latency network environments during implementation, although the control signal can obtain an accurate timestamp through the time synchronization module, this control signal is already a control quantity from a certain time ago, and this delay may fluctuate significantly. Therefore, a more past control quantity is needed to predict the current control quantity. Assuming the current time is t, the received control value is actually from time t−d (i.e.,...). The problem needs to be solved by updating the predicted value at time t using the control value at time t−d.

[0074] Let the state of the most recent update be...

[0075] The state prediction equation before calibration at the current time is:

[0076]

[0077] The covariance prediction equation is:

[0078]

[0079] Map the measured delay value to the current time:

[0080]

[0081] The covariance across d steps is:

[0082]

[0083] Gain coefficient:

[0084]

[0085] Covariance update:

[0086]

[0087] Status Update:

[0088]

[0089] At this time This is the estimated value of the control signal at the current moment. This method...

[0090] In high-latency network environments, excessive delay in control inputs can lead to decreased robot motion stability. Therefore, a safety gain is added, which gradually decreases as the control signal delay increases.

[0091]

[0092] in To ensure a safe delay threshold, in practice we set the threshold to 500ms.

[0093] Besides avoiding operational delays, the procedure is also crucial because the injection distance into the retinal blood vessels is short, and force feedback occurs instantaneously. If force feedback is not timely and effective, it will significantly delay force feedback during the puncture process, thus hindering the effective completion of the puncture. Specifically, force prediction algorithms for environments with indeterminate delays address the issue of force feedback extension, requiring the force feedback from the robotic needle tip to the operator, improving the operator's feel and control. Due to the remote environment, the algorithm also needs to address latency issues.

[0094] The specific processing method is as follows: sensor force preprocessing, 300Hz sampling filtering:

[0095]

[0096] Based on a 10Hz cutoff frequency, the coefficients are set as follows:

[0097]

[0098]

[0099]

[0100]

[0101]

[0102] The frequency is reduced to 30Hz, and then every 3 data points are sent to the main unit via the network at a frequency of 10Hz.

[0103] Time synchronization module: Same as before.

[0104] Force prediction algorithm for indeterminate delay environments:

[0105] The main handpiece can receive sensor force data in groups of three at a time. , , , The time is 33.3ms. Let the current time be... Because the robot and the main device have already synchronized their times, therefore and They are all operating on the same clock. This means we need to use data from these three sensors to predict the force data at the current moment, in order to mitigate the effects of network latency fluctuations.

[0106] First, calculate the first derivative of the force.

[0107]

[0108] Find the second derivative of the force.

[0109]

[0110] set up Using second-order Taylor expansion to extrapolate the force at this moment

[0111]

[0112] This method exhibits good accuracy when the delay is small; however, a sudden increase in delay can lead to a significant amplification of the second-order terms. Dynamic weighting of the second-order terms is implemented to avoid inconsistencies between the force of the primary hand and the force of the puncture execution structure 31 caused by a sudden increase in delay. The specific formula is as follows.

[0113] The specific formula is as follows:

[0114]

[0115] Obtain sensor force The main hand feedback force needs to be designed next.

[0116] Distinguishing between effective contact force and interference force: Setting a force threshold , when At that time, it is determined that there is no contact, and the feedback force is applied. To avoid minor noises interfering with perception.

[0117] Increase the magnification factor to enhance force perception; the magnification ratio is [value missing].

[0118]

[0119] Adding damping term to improve feel, The damping coefficient is... The speed of the main hand movement.

[0120]

[0121] By amplifying force perception, the force received by the injection needle 314 of the puncture actuator 31 is increased, making it easier for the operator to perceive, thereby ensuring that the main hand actuator 11 receives more sensitive force.

[0122] The usage process of the equipment is as follows: First, use 5G equipment and 5G network to connect the main hand control device 1 and the execution robot device 3, and then assemble the main hand control device 1 and the execution robot device 3. The patient lies on the operating table, waiting for other surgical procedures to be completed. When the retinal vascular puncture and injection procedure is required, the execution robot device 3 is placed next to the operating table. The remote doctor holds the pen-shaped end effector and observes the video stream information sent back by the hemostasis robot device on the video display structure 13. The doctor operates the pen-shaped end effector according to the real-time video stream information. First, the left button of the foot switch structure 12 is pressed, which causes the pen-shaped end effector to control the SCARA mechanism 321 to move. The doctor determines whether the angle of the puncture execution structure 31 needs to be adjusted. When the angle needs to be adjusted, the right button of the foot switch structure 12 is pressed. The angle of the puncture execution structure 31 is adjusted, and the left button is switched again to adjust the position. This process is repeated until the accurate position and angle of the puncture execution structure 31 are found. Then, the pen-shaped end effector controls the puncture motor 311 of the puncture execution structure 31 to work. During the process, the real-time force feedback obtained by the Bragg grating is sent to the pen-shaped end effector. When the force feedback of successful puncture is felt, it proves that the injection needle 314 has entered the retinal blood vessel. Then release the pen-shaped end actuator. Once stopped, all motors stop, and the injection needle 314 stops at the injection position where it enters the blood vessel. Then, the microfluidic pump 315 slowly injects the drug into the blood vessel in the fundus.

[0123] The equipment demonstrated high operational stability and accuracy: the latency for two-way data transmission within Shanghai was less than 15ms; the video latency was less than 180ms; and the system enabled remote control between Beijing and Xi'an, Beijing and Guangzhou, and Beijing and Haikou. Under remote control within Shanghai, the robot was able to accurately puncture three points within 150 micrometers of the simulated artificial eyeball and accurately inject liquid. The fundus bulge of the simulated artificial eyeball could be observed under a microscope, and the fundus bulge indicated successful puncture and drug injection.

[0124] The above description is merely a specific embodiment of this application. Under the guidance of the above teachings, those skilled in the art can make other improvements or modifications based on the above embodiments. Those skilled in the art should understand that the above specific description is only to better explain the purpose of this application, and the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A remote surgical robot for retinal vascular puncture, comprising: The master control device includes a master execution structure, a foot pedal switching structure, and a video display structure; 5G communication module, including dedicated cloud network and data transmission terminal; The robotic device includes, The puncture actuation structure is used to position the injection needle and perform the puncture action; The motion control module includes two mechanisms: the SCARA mechanism and the RCM mechanism. The SCARA mechanism is controlled by four servo motors and includes a Z-axis movement mechanism and a horizontal movement structure. The first end of the horizontal movement structure is combined with the vertical movement structure, and the second end of the horizontal movement rod structure is connected to the RCM mechanism. The SCARA mechanism drives the RCM mechanism, which is equipped with an injection needle, to change its three-dimensional position, thereby determining the puncture position of the injection needle. After the injection needle position is determined, the position of the injection needle tip will not change until the puncture action occurs. The RCM mechanism has three motors: a tumbling motor, a pitching motor, and a needle rotation motor. The tumbling and pitching motors determine the needle puncture angle. The needle tip orientation is determined by a rotating structure that rotates along the longitudinal axis of the needle at the RCM's execution end. The needle rotation motor drives the rotating structure to adjust the tilted needle tip orientation to the predetermined puncture direction to coordinate with the puncture action. The needle is positioned at the RCM point. The video acquisition module is a video acquisition card that is mounted on an ophthalmic microscope; The video acquisition module collects and sends the collected video stream information of the injection needle position and angle in real time. Combined with the sent video stream information, the main hand actuator remotely controls the SCARA mechanism to determine the puncture position; controls the RCM mechanism to determine the puncture angle; controls the puncture actuator to perform the puncture action; and the foot pedal switching mechanism realizes the switching of the control of the SCARA mechanism and the RCM mechanism by the main hand actuator. The time synchronization module is responsible for synchronizing the clocks of the master control device and the execution robot device, enabling the execution robot device to obtain the accurate time of the control signal through the timestamp, and to keep up with real-time updates on network latency and clock offset. The force prediction algorithm for uncertain latency environments uses a time synchronization module to complete time synchronization, and then uses the force prediction algorithm for uncertain latency environments to predict the force data at the current moment in order to resist the impact of network latency fluctuations.

2. The remote surgical robot according to claim 1, characterized in that, The SCARA mechanism includes a horizontal movement structure and a vertical movement structure. The horizontal movement structure includes three servo motors: an A motor that rotates along the A-axis, a B motor that rotates along the B-axis, and a C motor that rotates along the C-axis. The A motor and the B motor are connected by a connecting plate one, and the B motor and the C motor are connected by a connecting plate two. The vertical movement structure includes a vertical base with a lifting track, a lifting structure connected to the C motor that moves along the lifting track, and a D motor that rotates along the D-axis that drives the lifting structure to move along the lifting track. The A motor is connected to the RCM mechanism mounting plate.

3. The remote surgical robot according to claim 2, characterized in that, The RCM mechanism includes a tumbling motor rotating along the E-axis mounted on a device plate, a parallelogram structure, a parallelogram structure rotating with the tumbling motor, a pitch guide rail, a moving block extending from the parallelogram structure and moving within the pitch guide rail, the moving block being hinged to the parallelogram structure, a lead screw mounted within the pitch guide rail, the moving block mounted on the lead screw, a pulley one mounted on the side of the lead screw near the tumbling motor, and a pitch motor mounted below the track on the side near the tumbling motor, with the F-axis as its axis of rotation, a pulley two extending from the pitch motor, and a belt connecting pulley one and pulley two; rotation of the pitch motor in different directions drives rotation of the lead screw in different directions, and rotation of the lead screw in different directions drives movement of the moving block in different directions, thus forming parallelogram structures with different pitch angles.

4. The remote surgical robot according to claim 3, characterized in that, A rotating structure is set at the end of the parallelogram structure, and a motor for rotating the injection needle is set inside the rotating structure. The axis of the motor for rotating the injection needle is the G axis. A puncture execution structure is set inside the rotating structure, wherein the needle tip of the puncture execution structure is set at the RCM point. The E axis and the G axis pass through the RCM point, and the E axis is parallel to the pitch guide rail.

5. The remote surgical robot according to claim 4, characterized in that, The puncture execution structure is equipped with a puncture motor, which rotates to drive the injection needle to perform the puncture action. The main hand execution structure controls the puncture path and sends the puncture path to the puncture execution structure via a 5G network, controlling the puncture motor to rotate and execute the puncture path.

6. The remote surgical robot according to claim 5, characterized in that, A motor control structure is set up to connect to the motor control signal. The motor control structure controls the rotation of each axis motor according to the motor signal, thereby controlling the changes in each position and angle.

7. The remote surgical robot according to claim 5, characterized in that, A micro-force measurement module is installed on the outside of the injection needle. The micro-force measurement module includes a Bragg grating. It measures the deformation data of the grating caused by the force deformation when the injection needle is punctured. The data is processed by a fiber optic grating demodulator to obtain the force result. The measured force result is sent to the cloud and then transmitted to the main hand control device. The main hand actuator senses the force situation when the injection needle is punctured in the form of force feedback to determine whether the injection needle has entered the blood vessel.

8. The remote surgical robot according to claim 4, characterized in that, The injection needle is connected to the syringe on the microfluidic pump via a soft tube.

9. The remote surgical robot according to claim 7, characterized in that, The lifting structure and microfluidic pump of the vertical moving structure of the SCARA mechanism are both fixedly mounted on the base, which is fixed to a platform that meets the height requirements.

10. The remote surgical robot according to claim 1, characterized in that, The cloud-based private network uses tunneling protocol technology for information encryption, access control, and network security protection between the master control device and the execution robot device. The data transmission terminal consists of two 5G CPE devices, located at the master control device and the execution robot device respectively. These devices are used to send control, feedback, and video stream information sent from both ends to the cloud-based private network, and also to read data sent by the other party from the cloud-based private network and transmit it to the other device.

Images