Remote surgical robot for fundus blood vessel puncture

Abstract

The remote surgical robot comprises a main hand control device, a 5G communication module and an execution robot device, the execution robot device is remotely controlled through a main hand, the execution robot device comprises an SCARA mechanism and an RCM mechanism, and the SCARA mechanism is used for finding the puncture position of an injection needle. The RCM mechanism is provided with three motors to control rolling, pitching and rotation of the injection needle face, the puncture angle of the injection needle is determined through rolling and pitching, rotation is achieved along the longitudinal axis of the injection needle to adjust the orientation of the needle point of the injection needle, and the main hand control device independently controls the SCARA mechanism or the RCM mechanism through pedal conversion. A puncture motor for controlling the injection needle to puncture is arranged on the end effector, a force feedback structure is arranged on the injection needle, and whether puncture succeeds or not is known by sensing the force change of the injection needle during puncture through the main hand control device. By means of the arrangement, feedback control over the puncture position, the puncture angle, the puncture orientation and the puncture force can be rapidly and effectively achieved in a remote mode.

Description

Technical Field

[0001] The present invention belongs to the technical field of ophthalmic surgical robots, in particular to the field of robots for fundus blood vessel puncture, and specifically to a remote surgical robot for fundus blood vessel puncture. Background Art

[0002] Blockage of retinal blood vessels is a serious ophthalmic emergency. Currently, the most common clinical approach is to inject thrombolytic drugs into the affected vessels to address the problem and clear the blockage. This procedure is a form of ophthalmic microsurgery, requiring the surgeon to perform delicate manipulations under a microscope using ophthalmic instruments. However, due to the small size of the eyeball and the delicate and fragile structure of the eyeball, injecting drugs to address blocked retinal blood vessels requires the surgeon to possess extremely high hand-eye coordination and the ability to perceive delicate manipulations to avoid surgical failure.

[0003] The most scarce resources for this type of surgery are highly skilled doctors and the ability to perform blood vessel puncture during drug injection, as the diameter of blood vessels on the retina of the fundus is approximately 80μm, while the average amplitude of hand tremors of general ophthalmologists reaches 156μm; therefore, if the doctor's skills are not high, the risk of puncture failure is extremely high.

[0004] Therefore, under the premise of such a shortage of highly skilled doctors, if patients are made to feel that the doctor's location is far away, the cost of medical treatment for patients will be greatly increased, and patients going to non-residential places for medical treatment will also affect their recovery. If scarce highly skilled doctors go to the patient's location to practice medicine, this cannot be achieved due to the scarcity of medical resources.

[0005] In order to solve the above problems, there is a need for a device that can complete the injection of drugs for blocking retinal blood vessels without patients and doctors having to leave their permanent work and living locations. Summary of the Invention

[0006] To address the aforementioned issues, this application discloses a remote surgical robot for fundus vascular puncture. The robot uses 5G communication technology to remotely control the puncture robot and perform the puncture. High-speed data transmission ensures delay-free, high-quality transmission of surgical images and immediate response to control commands. Furthermore, to facilitate the operator's operation, the robot's robotic arm is divided into two mechanisms: a SCARA mechanism and a RCM mechanism. The SCARA mechanism is controlled by four servo motors and comprises a vertical motion mechanism and a horizontal motion structure. The first end of the horizontal motion structure is coupled to the vertical motion structure, while the second end of the horizontal motion structure is connected to the RCM mechanism. The SCARA mechanism drives the three-dimensional position of the RCM mechanism, which houses an injection needle, to locate the puncture location. Once the needle tip is located, the needle position remains unchanged until the puncture occurs. The RCM mechanism incorporates three motors that control the needle's roll, pitch, and rotation. The roll and pitch mechanisms determine the needle's puncture angle, while a rotation mechanism, located at the actuator end of the RCM mechanism, rotates along the needle's longitudinal axis to determine the needle's tip's orientation. The two mechanisms effectively locate the puncture position and angle, stopping movement once found. This movement is controlled by the master-hand control module. Once the puncture position and angle are found, a puncture motor is installed on the end effector to control the movement of the injection needle along the puncture direction. A force feedback mechanism is also installed on the injection needle. The master-hand control device senses the change in force during the puncture to determine whether the puncture is successful. Furthermore, to enable step-by-step control of the SCARA and RCM mechanisms, a foot pedal is installed on the master-hand control device. This alternating mechanism controls the master-hand's control of the SCARA and RCM mechanisms. Separate control of the SCARA and RCM mechanisms maximizes convenience and speed in finding the position, and the choice of the SCARA and RCM mechanisms maximizes the simplicity of the master-hand operation. 5G technology transmits the motion paths of the SCARA and RCM mechanisms, as well as the puncture end effector, in real time, ensuring consistency between the operator's hand and eye.

[0007] The specific technical solutions are:

[0008] The present application also discloses a remote surgical robot for fundus vascular puncture, which includes a main hand control device, a 5G communication module and an execution robot device.

[0009] Among them, the master hand control device includes a master hand execution structure, a foot switch structure, and a video display structure;

[0010] The 5G communication module includes a dedicated cloud network and data transmission terminals;

[0011] The execution robot device includes a puncture execution structure, a motion control module, and a video acquisition module;

[0012] Among them, the puncture execution structure is used to set the injection needle and perform the puncture action;

[0013] The motion control module includes two mechanisms, the SCARA mechanism and the RCM mechanism. The SCARA mechanism is controlled by four servo motors and includes a Z-axis moving mechanism and a horizontal moving structure. The first end of the horizontal moving structure is combined with the vertical moving structure, and the second end of the horizontal moving rod structure is connected to the RCM mechanism. The SCARA mechanism drives the three-dimensional position change of the RCM mechanism equipped with an injection needle to determine the puncture position of the injection needle. After the position of the injection needle is determined, the position of the tip of the injection needle will no longer change from the time when the injection needle is positioned to before the puncture action occurs; the RCM mechanism has three motors, namely a roll motor, a pitch motor and a motor for rotating the injection needle; the roll motor and the pitch motor determine the puncture angle of the injection needle, and the direction of the needle tip is determined by setting a rotating structure at the execution end of the RCM to rotate along the longitudinal axis of the injection needle; the injection needle is set at the RCM point position.

[0014] The video acquisition module is a video capture card installed on the ophthalmic microscope. It collects video signals in real time during the operation and sends them to the cloud after processing by the control module.

[0015] The video acquisition module collects and sends back the collected video stream information of the injection needle position and angle in real time. Combined with the sent back video stream information, the master hand execution structure remotely controls the SCARA mechanism to determine the puncture position; controls the RCM mechanism to determine the puncture angle; and controls the puncture execution structure to perform the puncture action. The master hand switches the control of the SCARA mechanism and the RCM mechanism through the foot switching structure.

[0016] Furthermore, the SCARA mechanism includes horizontal and vertical movement structures. The horizontal movement structure includes three servo motors: motor A, which rotates along the A-axis; motor B, which rotates along the B-axis; and motor C, which rotates along the C-axis. Motors A and B are connected by connecting plate 1, and motor B and C are connected by connecting plate 2. The vertical movement structure includes a vertical base with a lifting track, a lifting structure connected to motor C, which moves along the lifting track, and motor D, which rotates along the D-axis. Motor D drives the lifting structure along the lifting track. Motor A is connected to the RCM mechanism setting plate. This setup ensures that horizontal movement is achieved using the three servo motors and the connecting plates between them, while vertical movement is achieved in conjunction with the vertical movement structure. When the main hand performs structural operations, horizontal and vertical position adjustments require only simple motion transmission.

[0017] Furthermore, the RCM mechanism includes a tumbling motor rotating along the E-axis arranged on the equipment plate, a parallelogram structure, a parallelogram structure rotating with the tumbling motor and a pitch guide track, a moving block extending from the parallelogram structure and moving in the pitch guide track, the moving block is hinged to the parallelogram structure, a lead screw is arranged in the pitch guide track, the moving block is arranged on the lead screw, a pulley 1 is arranged on the side of the lead screw close to the tumbling motor, and a pitch motor is arranged under the track on the side close to the tumbling motor, the rotating shaft is the F-axis, a pulley 2 extends from the pitch motor, and the pulley 1 and the pulley 2 are connected by a belt; the rotation of the pitch motor in different directions drives the rotation of the lead screw in different directions, and the different directions The rotation of the lead screw in the right direction drives the movement of the moving block in different directions, thereby forming a parallelogram structure with different pitch angles, a rotating structure is set at the end of the parallelogram structure, and a motor for rotating the injection needle is set in the rotating structure, and the axis of the motor for rotating the injection needle is the G axis; a puncture execution structure is set in the rotating structure, wherein the needle tip of the puncture execution structure is set on the RCM point; the E axis and the G axis pass through the RCM point, and the E axis is parallel to the pitch guide track; through this setting, the pitch motor realizes the pitch of the parallelogram structure with the RCM point as the center, and the rotation of the E axis and the G axis is realized without changing the position of the RCM point, thereby achieving accurate adjustment of the puncture angle after finding the puncture position.

[0018] Furthermore, a puncture execution structure is provided with a puncture motor, which rotates to drive the injection needle to perform the puncture action. The main hand execution structure controls the puncture path, sends the puncture path to the puncture execution structure through the 5G network, and controls the rotation of the puncture motor to execute the puncture path.

[0019] Furthermore, one of the biggest difficulties in injecting drugs into the fundus vessels is that even if the position and angle are accurately determined, there is no guarantee that the puncture will enter the vessel. This action requires the operator to sense the difference between puncturing the vessel and not entering the vessel. Therefore, it is necessary for the main hand controller to be able to receive the force conditions during the puncture action. Specifically, a micro-force measurement module is provided, which is a Bragg grating set on the outside of the injection needle. The force on the injection needle is measured in real time according to the deformation of the grating driven by the force deformation during the puncture action of the injection needle. The measured force result is processed by the fiber Bragg grating demodulator and finally sent to the cloud. It is transmitted to the main hand through the cloud. The main hand senses the force during the puncture action of the injection needle and understands whether the injection needle has penetrated the vessel. In specific use, when the injection is performed, the injection needle deforms, which drives the deformation of the optical fiber. This causes the grating period inside the optical fiber to change, which drives the change of the output wavelength. Based on the output wavelength, the wavelength offset inside the optical fiber is calculated, thereby obtaining the deformation of the needle tip and the magnitude of the force. The direction of the force is obtained by fusion of the data from the three optical fibers, achieving accurate force feedback.

[0020] Furthermore, the injection needle is connected to the syringe on the microfluidic pump through a soft tube, and micro-injection is performed through the microfluidic pump. When the injection needle punctures into the blood vessel, as long as the patient's position does not change, the stopped motor and various mechanisms can be used to ensure the precise and stable position of the injection needle. Therefore, there is no need to rely on manual injection. Controlling the speed of the microfluidic pump can ensure the safe completion of the injection of thrombolytic drugs and other related drugs into the fundus blood vessels. The pathway has been established, and only the type of drug needs to be changed at the injection pump end, which saves the difficulty of finding the blood vessel again and can complete the injection of multiple drugs very accurately.

[0021] Furthermore, the lifting structure of the vertical moving structure of the SCARA mechanism and the microfluidic pump are fixed to the base, and then the base is fixed to a high platform that meets the height requirements; through this setting, the structure can be effectively fixed to the bedside, and setting the SCARA mechanism and the microfluidic pump on a base at the same time can facilitate the integration and operation of the structure, and there is no need to set up an overly long soft tube, as long as the follow-up requirements are met.

[0022] Technical Effects

[0023] By utilizing a dedicated 5G network, real-time transmission of video and master-hand operation signals is ensured, effectively reducing latency. By dividing the motion control module into two mechanisms, the SCARA mechanism and the RCM mechanism, and independently controlling each mechanism through a foot-operated switching mechanism within the master-hand actuator control structure, the operator's difficulty in finding the puncture position is significantly reduced. Using the SCARA mechanism to find the puncture position and the RCM mechanism to find the puncture angle allows for more efficient and accurate positioning of the puncture position and angle. When the master-hand controls the SCARA and RCM mechanisms, the signals received are simple, linear motion signals, significantly reducing signal transmission latency and simplifying signal conversion between the software on both ends, further ensuring real-time signal transmission. Furthermore, the dual-mode SCARA and RCM mechanisms improve the accuracy of movement distance, ensuring effective puncture of fundus vessels. During operation, the foot-operated switching mechanism allows for repeated switching between the SCARA and RCM mechanisms to ensure optimal placement of the needle at the surgical site, allowing for multiple cuts to find the optimal position and angle.

[0024] A puncture execution structure is set in the rotating structure, wherein the needle tip of the puncture execution structure is set on the RCM point; the E-axis and the G-axis pass through the RCM point, and the setting of the E-axis parallel to the pitch guide track ensures that the position of the needle tip will not be changed regardless of pitch operation, flip operation or rotation operation.

[0025] The RCM mechanism is equipped with three motors, namely a tumbling motor, a pitching motor and an injection needle rotation motor. The motor setting can realize the flipping, pitching and rotation of the injection needle, and can realize the disadvantages of the injection needle angle. In particular, the injection needle rotation motor controls the rotation of the injection needle to adjust the direction of the inclined injection needle tip to a suitable angle, so that it achieves an optimal puncture direction and ensures the success rate of puncturing the fundus blood vessels. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the overall process structure of the robot of the present invention;

[0027] Figure 2 This is a schematic diagram of the overall structure of the motion control module of the present invention;

[0028] Figure 3 Schematic diagram of the SCARA structure of the present invention;

[0029] Figure 4 This is a schematic structural diagram of the vertical seat portion of the SCARA mechanism structure of the present invention;

[0030] Figure 5 This is a schematic diagram of the front and side view of the RCM mechanism structure of the present invention;

[0031] Figure 6 Schematic diagram of the side view of the RCM mechanism structure of the present invention;

[0032] Figure 7 This is a schematic structural diagram of a puncture execution structure with a motor for rotating an injection needle according to the present invention;

[0033] Figure 8 This is a schematic structural diagram of the puncture execution structure of the present invention;

[0034] Figure 9 This is a schematic structural diagram of the master hand execution structure of the present invention;

[0035] Figure 10 This is a schematic diagram of the structure of the robot device operating process of the present invention;

[0036] Figure 11 This is a flow chart of signal transmission and corresponding time between the master hand control device and the execution robot of the present invention;

[0037] Description of main reference numerals

[0038] 1. Master hand control device; 11. Master hand execution structure; 12. Foot switch structure; 13. Video display structure;

[0039] 2. 5G communication module; 21. Cloud dedicated network; 22. Data transmission terminal; 221. CPE equipment;

[0040] 3. Execution robot device; 31. Puncture execution structure; 311. Puncture motor; 312. Fiber Bragg grating (FBG); 313. FBG demodulator; 314. Injection needle; 315. Microfluidic pump; 316. Soft tube; 32. Motion control module; 321. SCARA mechanism; 3211. Motor A; 3212. Motor B; 3213. Motor C; 3214. Motor D; 3215. Connecting plate 1; 3216. Connecting plate 2; 3217. Vertical seat; 3218. Lift Lowering structure; 3219, lifting rail; 322, RCM mechanism; 3221, flip motor; 3222, pitch motor; 32221, pulley 1; 32222, pulley 2; 32223, endless belt; 3223, needle rotation motor; 3224, setting plate; 32251, parallelogram structure; 32252, moving block; 32253, pitch guide rail; 32254, moving block; 32255, lead screw; 33, video acquisition module;

[0041] 4. Motor control structure. DETAILED DESCRIPTION

[0042] To make the purpose, technical solutions, and advantages of this application more clear, the following will provide a clear and complete description of the technical solutions of this application in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] In this document, “illustrative” means “serving as an example, instance or illustration”, and any diagram or implementation described in this document as “illustrative” should not be interpreted as a more preferred or more advantageous technical solution.

[0044] To simplify the drawings, only the parts related to the present application are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, to simplify the drawings and facilitate understanding, in some figures, only one of the components with the same structure or function is schematically shown or labeled.

[0045] In this document, unless otherwise specified or limited, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance. Unless otherwise specified or explained, the term "plurality" refers to two or more. The terms "connected" and "fixed" should be understood broadly. For example, "connected" can mean a fixed connection, a detachable connection, an integral connection, or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0046] refer to Figure 1 A remote surgical robot for fundus blood vessel puncture includes a main hand control device 1, a 5G communication module and an execution robot device 3.

[0047] The master-hand control device 1 includes a master-hand execution structure 11 , a foot-operated switching structure 12 , and a video display structure 13 .

[0048] The 5G communication module includes a dedicated cloud network 21 and a data transmission terminal 22. The dedicated cloud network 21 uses tunneling protocol technology to implement information encryption, access control, and network security between the master hand control unit 1 and the execution robot unit 3. The data transmission terminal 22 consists of two 5G CPE devices 221, located at the master hand control unit 1 and the execution robot unit 3, respectively. These devices can transmit control, feedback, and video stream information sent by each end to the dedicated cloud network 21. They can also read data sent by the other end from the dedicated cloud network 21 and transmit it to the other end. Two-way communication via the 5G CPE devices 221 leverages the high bandwidth and low latency of 5G to enable remote surgical operations. The 5G CPE devices 221 can achieve a transmission rate of up to 1167 Mbps and feature four built-in signal amplifiers, significantly enhancing wireless coverage and signal strength while ensuring a secure and stable network connection. Two-way communication via the two 5G CPE devices 221 prioritizes TCP as the transmission protocol for high-definition, real-time transmission of surgical footage from the robot to the master hand control unit.

[0049] The execution robot device 3 includes a puncture execution structure 31, a motion control module 32, and a video acquisition module 33;

[0050] refer to Figure 7 and Figure 8The puncture execution structure 31 is used to set the injection needle 314 and perform the puncture action. The puncture execution structure 31 is equipped with a puncture motor 311, which rotates to drive the injection needle 314 to perform the puncture action. The master hand execution structure 11 controls the puncture path, sending the puncture path to the puncture execution structure 31 in real time via the 5G network, controlling the rotation of the puncture motor 311 to execute the puncture path. More specifically, the puncture path signal is converted into information about the speed and number of revolutions of the puncture motor 311. This conversion method enables the master hand to control the puncture path. The injection needle 314 is equipped with a Bragg grating for real-time force measurement. Three fiber gratings 312 are attached to the upper part of the injection needle 314 at a 120° interval. The fiber grating 312 contains a phase grating, which forms a narrowband transmission filter within the fiber core. When a beam of light passes through the grating, the wavelengths that meet the requirements are reflected, while the remaining wavelengths are transmitted through the grating. During injection, the needle 314 deforms, causing the optical fiber to deform as well. This causes a change in the grating period within the optical fiber, which in turn causes a change in the output wavelength. Based on the output wavelength, the offset of the optical fiber's internal wavelength is calculated, thereby obtaining the needle tip's deformation, and thus the force magnitude. The force direction is then determined by fusing data from the three optical fibers, achieving precise force feedback. The micro-force measurement module includes this force detection component and an instrument that receives the optical fiber output signal and calculates the force magnitude. The Bragg grating measurement data is processed using a high-precision, high-resolution grating demodulator 313. This grating demodulator 313 is used for signal demodulation and sensor data acquisition for various types of grating sensors, with an optional scanning frequency between 1 and 1000 Hz, ensuring accurate micro-force data. The received real-time force is then transmitted back to the master hand actuator 11 via the 5G network. Force feedback allows the operator to understand the force applied during the puncture process, ensuring successful puncture.

[0051] refer to Figure 8 The injection needle 314 is connected to the syringe on the microfluidic pump 315 through a soft tube 316, and microinjection is performed through the microfluidic pump 315. Once the injection needle 314 punctures into the blood vessel, as long as the patient's position does not change, the stopped motor and various mechanisms can be used to ensure the precise and stable position of the injection needle 314. Therefore, there is no need to rely on manual injection. Controlling the speed of the microfluidic pump 315 can ensure the safe completion of the injection of thrombolytic drugs and other related drugs into the fundus blood vessels. This pathway has been established, and only the type of drug can be changed at the injection pump end, saving the difficulty of finding the blood vessel again, and can very accurately complete the injection of multiple drugs. The microfluidic pump 315 can achieve an injection volume lower limit of 1pl and an injection flow rate of 0.05nl / min to 520ul / min. This flow control can meet the demand for microinjection of the fundus.

[0052] refer to Figure 2-7; The motion control module 32 includes two mechanisms, a SCARA mechanism 321 and an RCM mechanism 322. The SCARA mechanism 321 is controlled by four servo motors, and includes a Z-axis moving mechanism and a horizontal moving structure. The first end of the horizontal moving structure is combined with the vertical moving structure, and the second end of the horizontal moving rod structure is connected to the RCM mechanism 322. The SCARA mechanism 321 drives the three-dimensional position change of the RCM mechanism 322 equipped with the injection needle 314 to perform the puncture position determination of the injection needle 314. After the position of the injection needle 314 is determined, the position of the tip of the injection needle 314 no longer changes from the position before the puncture action occurs. The RCM mechanism 322 has three motors, namely a roll motor, a pitch motor 3222 and a motor 3223 for rotating the injection needle; the roll motor and the pitch motor 3222 determine the puncture angle of the injection needle 314, and the direction of the needle tip of the injection needle 314 is determined by setting a rotating structure at the execution end of the RCM to rotate along the longitudinal axis of the injection needle 314; the injection needle 314 is set at the RCM point position.

[0053] refer to Figure 3 and Figure 4 ; The SCARA mechanism 321 includes a horizontal moving structure and a vertical moving structure. The horizontal moving structure includes three servo motors, namely, an A motor 3211 rotating along the A axis, a B motor 3212 rotating along the B axis, and a C motor 3213 rotating along the C axis. The A motor 3211 and the B motor 3212 are connected by a connecting plate 1 3215, and the B motor 3212 and the C motor 3213 are connected by a connecting plate 2 3216; the vertical moving structure includes a vertical seat 3217 with a lifting rail 3219, a lifting structure 3218 connected to the C motor 3213, the lifting structure 3218 moves along the lifting rail 3219, and a D motor 3214 rotating along the D axis is also provided, and the D motor 3214 drives the lifting structure 3218 to move along the lifting rail 3219; the A motor 3211 is connected to the RCM mechanism 322 setting plate 3224. This setup ensures horizontal movement using three servo motors and the connecting plates between them, while also enabling vertical movement in conjunction with the vertical motion structure. When operating Structure 11 via the master hand, simple motion transmission is required to adjust both horizontal and vertical positions. The four servo motors in the AD Motor 3214 precisely control the position, speed, and acceleration of the motors through internal encoders and feedback devices. Braking devices are also included for additional safety.

[0054] refer to Figure 5-Figure 7The RCM mechanism 322 includes a roll motor that rotates along the E axis, a parallelogram structure 32251, a parallelogram structure that rotates with the roll motor, and a pitch guide track 32253 provided on the roll motor, the parallelogram structure 32251 extending out of a moving block 3225432252 that moves within the pitch guide track 32253, the moving block 3225432252 being hinged to the parallelogram structure 32251, and the pitch guide track A lead screw 32255 is provided in 32253, and a moving block 32254 32252 is provided on the lead screw 32255. A pulley 1 32221 is provided on the side of the lead screw 32255 close to the rolling motor, and a pitch motor 3222 is provided under the track on the side close to the rolling motor. The rotating shaft is the F axis, and a pulley 2 32222 extends from the pitch motor 3222. The pulley 1 32221 and the pulley 2 32222 are connected by an annular belt 32223; The rotation of the motor 3222 in different directions drives the screw 32255 to rotate in different directions. The rotation of the screw 32255 in different directions drives the moving block 3225432252 to move in different directions, thereby forming a parallelogram structure 32251 with different pitch angles. A rotating structure is fixedly set at the end of the parallelogram structure 32251, and a motor 3223 for rotating the injection needle is set in the rotating structure, and the axis of the motor 3223 for rotating the injection needle is the G axis; a puncture execution structure 31 is set in the rotating structure, wherein the needle tip of the puncture execution structure 31 is set on the RCM point; the E axis and the G axis pass through the RCM point, and the E axis is parallel to the pitch guide track 32253; through this setting, the pitch motor 3222 realizes the pitch of the parallelogram structure 32251 with the RCM point as the center, and the rotation of the E axis and the G axis realizes the purpose of not changing the position of the RCM point, thereby achieving accurate adjustment of the puncture angle after finding the puncture position. The RCM mechanism 322 allows for adjustment of not only the pitch angle but also the roll angle. Most importantly, the needle rotation motor 3223 allows for adjustment of the needle tip orientation of the needle 314, ensuring optimal puncture angle and needle tip orientation, maximizing the puncture success rate. Each motor is controlled by its speed and direction, simplifying signal transmission. The roll motor is a servo motor with a maximum speed of 6000 rpm, enabling high-precision positioning and exhibiting high efficiency and low noise. The pitch motor 3222 and needle rotation motor 3223 are brushed motors, offering a lightweight, compact system while still maintaining high power, achieving an efficiency of nearly 90%, low energy consumption, and being silent.

[0055] A motor control structure 4 connected to the motor control signal is provided. The motor control structure 4 controls the rotation of each axis motor according to the motor signal, thereby controlling the change of each position and angle.

[0056] refer to Figure 10The master-hand actuator 11 can control only one of the RCM and SCARA mechanisms at a time, with this switching being controlled by the foot-operated switching mechanism 12: Pressing the left button causes the master-hand actuator 11 to control the SCARA mechanism 321; pressing the right button causes the master-hand actuator 11 to control the RCM mechanism. The motors in both the RCM and SCARA mechanisms 321 are connected to an 8-axis motor control mechanism 4. Control data from the master-hand actuator 11 is fed into a control card, which then controls the corresponding motors based on the input data. When the operator presses a button on the master-hand actuator 11, it begins recording motion data. While the operator is holding down the button, the master-hand actuator 11 moves, recording hand displacement, velocity, angle, and angular velocity in real time. This data is sent to the host computer controlling the master-hand actuator 11. After simple processing, it is sent to the execution robot 3 via the UDP protocol. The host computer on the execution robot 3 further processes this data: first, it extracts the displacement, velocity, and angle data collected by the master-hand actuator 11. After filtering and debouncing, the data instructions are converted into the robot's joint angles and end-user pose. Finally, an inverse kinematics algorithm is used to calculate motor control signals, such as the rotation direction, speed, and distance of each axis motor in the RCM or SCARA mechanism 321. These motor control signals are then transmitted to the motor control structure 4, which controls the movement of each axis motor based on these signals. When the operator releases the button of the master hand execution structure 11, no information will be transmitted to the motor no matter how it moves, and the motor will be fixed at the position reached by the last movement. The displacement and speed of the operator's hand are converted into the rotation information of the motor, thereby realizing the control of the 8-axis motion mechanism by the master hand execution structure 11. During the actual operation, the operator first steps on the left button of the foot pedal and uses the master hand execution structure 11 to control the SCARA mechanism 321. At this time, any motor in the RCM mechanism 322 will not move. The operator controls the injection needle 314 head to move above the injection target. When the injection needle 314 head has reached the target position, the operator can step on the right button of the foot pedal and use the master hand execution structure 11 to control the RCM mechanism 322. At this time, the motors of each axis of the SCARA mechanism 321 will not move, and no matter how the RCM mechanism 322 moves, the position of the needle will not change. At this time, the operator can control the RCM mechanism 322 to change the direction and angle of needle insertion. When all positions, directions, and angles are determined, the operator locks the motors of each axis of the RCM mechanism 322 through software, leaving only the motor that controls the movement of the injection needle 314 head, and controls the motor to move the needle up and down to complete the injection task.Of course, the need for the injection needle 314 to enter the surgical position must be considered during the specific adjustment process. This requires repeated switching of the pedal switching structure 12 and multiple adjustments to the control of the RCM or SCARA mechanism 321 to ensure that the angle and position are appropriate.

[0057] The video capture module 33 is a USB video capture card installed on the ophthalmic microscope. It collects the video stream captured by the ophthalmic microscope in real time during surgery, processes it, and then transmits it to the cloud via the control module. Data transmission via USB improves data transmission efficiency and further reduces latency. The video capture module 33's camera has a maximum resolution of 2160*3840 and features dual built-in microphones. Connecting to the video capture card via USB enables high-definition image capture and high-speed transmission.

[0058] The video capture module 33 collects and sends back real-time video stream information on the position and angle of the injection needle 314. Combined with this video stream information, the master-hand execution structure 11 remotely controls the SCARA mechanism 321 to determine the puncture position, the RCM mechanism 322 to determine the puncture angle, and the puncture execution structure 31 to perform the puncture. The master-hand switches control of the SCARA mechanism 321 and the RCM mechanism 322 via the foot-operated switching structure 12. This real-time video stream ensures consistent hand-eye coordination.

[0059] The lifting structure 3218 and the microfluidic pump 315 of the vertical moving structure of the SCARA mechanism 321 are fixed to the base, and then the base is fixed to a high platform that meets the height requirements, the high platform is not shown; through this setting, the structure can be effectively fixed to the bedside, and setting the SCARA mechanism 321 and the microfluidic pump 315 on a base at the same time can facilitate the integration and operation of the structure, and there is no need to set an overly long soft tube 316, as long as the follow-up requirements are met.

[0060] refer to Figure 9The haptic device in the master hand actuator structure 11 uses a high-precision pen-shaped end effector, which provides highly accurate position capture and maintains precise gravity compensation in translation and positioning. The haptic device renders high contact forces at a 4 kHz rate and combines passive and actuated components to enhance tactile transparency. The master hand actuator structure 11 can also use the Omega.6 haptic feedback device. When the foot-operated switching structure 12 switches to control the operation of the SCARA mechanism 321, the pen-shaped end effector is controlled to perform forward and backward movement, left and right movement, and up and down movement, and the SCARA mechanism 321 is controlled to perform forward and backward, left and right movement, and up and down movement. However, because the range of activity of the fundus blood vessels is small, the complete stroke of controlling a single direction only reaches a set distance of movement of the SCARA mechanism 321. The position is continuously adjusted through repeated single-direction movement. For example, a complete front-to-back movement of the pen-shaped end effector controls the SCARA mechanism 321 to move forward by 1mm; a back-to-front movement controls the SCARA mechanism 321 to move backward by 1mm. The movement distance of a single main hand execution structure 11 in one direction is positively correlated with the movement distance of the SCARA mechanism 321. Through repeated fine-tuning, the SCARA mechanism 321 moves to the appropriate position with the injection needle 314.

[0061] To ensure the accuracy of transmitted data, various data processing methods are used. After receiving the operator's motion data, the master-hand execution structure 11 processes the data accordingly: first, the motion information is extracted and then placed into the corresponding position in a pre-established communication protocol. This protocol contains the sequence number of the transmitted data and the error checksum of each frame. The master-hand execution structure 11 then sends the frame data containing the position information, sequence number, and error checksum to the execution robot 3. If various conditions occur during the transmission process and cause the data frame to contain errors or transmission failures, the execution robot 3 will detect a mismatch in the sequence number or an error checksum. After receiving the data, the execution robot 3's control structure first checks whether the sequence number and error checksum match. If not, it indicates a data frame error. The execution robot 3 predicts the error data in this frame based on previously received data and replaces the error data with the predicted data. If the data is correct, the motion information is extracted from the received data frame and processing begins. The received motion information includes position information, direction of movement, and speed. The mainframe of the robot execution device 3 processes this information: it is filtered through an extended Kalman filter and a low-pass filter, and calculated using the corresponding nonlinear motion model. Ultimately, it is converted into the rotation direction, speed, and distance of the robot's motors on each axis, controlling the movement of the corresponding robot mechanisms. The force data obtained by the robot execution device 3 is also processed using the same method as the master hand execution structure 11, and the resulting data frame is transmitted back to the master hand execution structure 11. After receiving the force data, the mainframe of the master hand execution structure 11 first fuses it with the data obtained through the multimodal sensor fusion algorithm, and then transmits it to the master hand to provide force feedback to the operator.

[0062] To ensure the effective stopping of the motor, Hall sensors are used as limit switches to restrict the movement of the motor when it is not working. The emergency stop of the motor of each axis is achieved by a relay. The relay action and return time are both less than 0.02s, which improves the safety of the robot.

[0063] Delays in remote control are unavoidable. To address the delay issue, the following strategies are used to achieve precise control in the presence of delays: the control data from the master-hand control end is compressed and encoded to form data frames, which are then quickly transmitted to the execution robot device 3 via the UDP protocol; the force data of the execution robot device 3 is processed in the same way as the control data from the master-hand control end and then quickly transmitted to the master-hand control end via the UDP protocol.

[0064] The entire transmission process is implemented based on 5G technology. In order to achieve low latency, high reliability and certainty in remote communications, this patent creates an independent virtual network for remote data transmission, divides the transmitted data into different standardized service levels, and dynamically allocates resources; reduces transmission delays through short frame structures, weight-free scheduling, pre-scheduling and other technologies; and ensures the reliability of data transmission by creating redundant links. In order to avoid delays, a time synchronization module is set up, which is responsible for synchronizing the clocks of the main hand control device 1 and the execution robot device 3, so that the execution robot device 3 can obtain the accurate time of the control signal through the timestamp and maintain real-time updates of network delays and clock offsets.

[0065] refer to Figure 11 The specific algorithm of the time synchronization module is to synchronize the time between the master hand control device 1 and the execution robot device 3. Two problems must be solved. 1) The clock offset between the master hand control device 1 and the execution robot device 3. Each computing unit has its own internal clock. When it is turned on, the computer will obtain a world time through the network. However, due to technical reasons, the world time obtained by each computer will have a certain deviation. This deviation is recorded as Assume that the network delay is a fixed value within a certain period of time, recorded as The network delay and time deviation can be calculated by exchanging synchronization timestamps between the master hand control device 1 and the execution robot device 3.

[0066]

[0067]

[0068] Set a 2nd order filter, sampling rate 0.1Hz, cutoff frequency: 0.001Hz, the specific filtering formula is as follows:

[0069]

[0070] After filtering

[0071]

[0072] Achieve timestamp synchronization. A public method for ensuring timestamp synchronization achieves time synchronization through the equality of timestamps to avoid delays; ensure the master hand execution structure 11 and the execution robot device 3 are synchronously and accurately controlled to ensure the accuracy requirements of the fundus blood vessels.

[0073] In a high-latency network environment, the control signal can obtain an accurate timestamp thanks to the time synchronization module, but the control signal is already a control value from a certain time ago, and this delay may fluctuate greatly. Therefore, it is necessary to use control values ​​from the past to predict the current control value. Assuming that the current time is t, the control value received is actually at time t-d (i.e. ), it is necessary to solve the problem of updating the predicted value at time t with the control value at time t−d.

[0074] Assume the last updated status is

[0075] Then the state prediction equation before calibration at the current moment is:

[0076]

[0077] The covariance prediction equation is:

[0078]

[0079] Map the measured value of the delay to the current moment:

[0080]

[0081] The covariance across d steps is:

[0082]

[0083] Gain factor:

[0084]

[0085] Covariance update:

[0086]

[0087] Status Update:

[0088]

[0089] At this time This is the estimated value of the control signal at the current moment.

[0090] In a high-latency network environment, if the control delay is too large, it may cause the robot's motion stability to decrease. Therefore, a safety gain is added, which will gradually decay as the control signal delays.

[0091]

[0092] in As a safety delay threshold, in practice, we design the threshold to be 500ms.

[0093] In addition to avoiding delays during the operation, the injection distance of the fundus blood vessels is small, and force feedback occurs instantaneously. If force feedback is not timely and effective, it will greatly delay the force feedback during the puncture process, and the puncture action will not be completed effectively. Specifically for the force prediction algorithm in the uncertain delay environment to solve the extension problem of force feedback, it is necessary to feedback the force of the robot needle tip to the main operator to improve the operator's feel and control ability. Due to the remote environment, the algorithm also needs to deal with the problem of delay.

[0094] The specific processing method is: sensor force pre-processing, 300Hz sampling filtering:

[0095]

[0096] According to the 10Hz cutoff frequency, the coefficients are set as:

[0097]

[0098]

[0099]

[0100]

[0101]

[0102] The frequency is reduced to 30Hz, and then every 3 data are grouped together and sent to the master at a frequency of 10Hz.

[0103] Time synchronization module: Same as before

[0104] Force prediction algorithm for an indefinite delay environment:

[0105] The master hand can receive sensor force data in groups of 3 at a time. , , , is 33.3ms, and the current time is , because the robot side and the master side have completed time synchronization, and The three sensors are clocked at the same time. This means that the force data at the current moment needs to be predicted using the data from these three sensors to mitigate the impact of network latency fluctuations.

[0106] First, find the first derivative of the force

[0107]

[0108] Then find the second derivative of the output force

[0109]

[0110] set up , use the second-order Taylor expansion to extrapolate the force at this moment

[0111]

[0112] This method has good accuracy when the delay is small, but a sudden increase in delay can significantly amplify the second-order term. Dynamic second-order term weighting is set to avoid inconsistencies between the primary hand force and the force of the puncture execution structure 31 caused by a sudden increase in delay. The specific formula is:

[0113] The specific formula is:

[0114]

[0115] Get sensor force Then you need to design the main hand feedback force

[0116] Distinguishing between effective contact forces and interference forces: Setting force thresholds , when When it is determined that there is no contact, the feedback force , to avoid small noises interfering with perception.

[0117] Add amplification items to expand force perception, the amplification ratio is

[0118]

[0119] Add damping items to improve the feel, is the damping coefficient, The main hand movement speed.

[0120]

[0121] By expanding the force perception, the force applied to the injection needle 314 of the puncture execution structure 31 is increased, making it easier for the operator to perceive, thereby ensuring that the main hand execution structure 11 receives a more sensitive force.

[0122] The usage process of the device is as follows: first, use 5G equipment and 5G network to connect the main hand control device 1 and the execution robot device 3, and assemble the main hand control device 1 and the execution robot device 3. The patient lies on the operating table, waiting for other surgical procedures to be completed. When the fundus vascular puncture and injection step is required, the execution robot device 3 is placed next to the operating table. The remote doctor holds the pen-shaped end effector and observes the video stream information sent back by the hemostasis robot device on the video display structure 13. He then operates the pen-shaped end effector based on the real-time video stream information. First, the left button of the foot-operated switching structure 12 is pressed, causing the pen-shaped end effector to control the movement of the SCARA mechanism 321. The doctor then determines whether the angle of the puncture execution structure 31 needs to be adjusted. If the angle needs to be adjusted, the right button of the foot-operated switching structure 12 is pressed. The angle of the puncture execution structure 31 is adjusted, and the left button is switched again to adjust the position. This process is repeated until the accurate position and angle of the puncture execution structure 31 are found. The pen-shaped end effector then controls the puncture motor 311 of the puncture execution structure 31. During this process, the real-time force feedback obtained by the Bragg grating is provided to the pen-shaped end effector. When the force feedback is felt, it proves that the injection needle 314 has entered the fundus vascular. Then, the pen-shaped end actuator is released. Once all motors stop, the injection needle 314 stops at the injection position where it enters the blood vessel. The microfluidic pump 315 is then used to slowly inject the drug into the fundus blood vessels.

[0123] Equipment usage effect: The above-mentioned equipment shows high operational stability and accuracy: the two-way data transmission delay in Shanghai is less than 15ms; the video delay is less than 180ms; and the system realizes remote control between Beijing-Xi'an, Beijing-Guangzhou, and Beijing-Haikou; under remote control in Shanghai, the robot can accurately puncture three points within 150 microns of the simulated fake eyeball and accurately inject liquid. The fundus bulge of the simulated fake eyeball can be observed through a microscope, and the fundus bulge indicates the success of puncture and drug injection.

[0124] The above description is merely a specific embodiment of the present application. Based on the above teachings of the present application, those skilled in the art may make other improvements or modifications based on the above embodiments. Those skilled in the art should understand that the above description is only a better explanation of the purpose of the present application, and the scope of protection of the present application shall be based on the scope of protection of the claims.

Claims

1. A remote surgical robot for fundus vascular puncture, comprising A master hand control device, including a master hand execution structure, a foot switch structure, and a video display structure; 5G communication module, including cloud-based dedicated network and data transmission terminal; The execution robot device includes: The puncture execution structure is used to set the injection needle and perform the puncture action; The motion control module includes two mechanisms, a SCARA mechanism and an RCM mechanism. The SCARA mechanism is controlled by four servo motors and includes a Z-axis moving mechanism and a horizontal moving structure. The first end of the horizontal moving structure is combined with the vertical moving structure, and the second end of the horizontal moving rod structure is connected to the RCM mechanism. The SCARA mechanism drives the three-dimensional position change of the RCM mechanism equipped with an injection needle to determine the puncture position of the injection needle. After the injection needle position is determined, the position of the injection needle tip does not change until the puncture action occurs. The RCM mechanism has three motors: a roll motor, a pitch motor, and a needle rotation motor. The roll and pitch motors determine the needle's puncture angle, and the needle tip's orientation is determined by a rotating mechanism at the RCM's actuator end that rotates along the needle's longitudinal axis. The needle is positioned at the RCM point. The video acquisition module is a video acquisition card arranged on the ophthalmic microscope; The video acquisition module collects and sends the collected video stream information of the injection needle position and angle in real time. Combined with the sent back video stream information, the master-hand execution structure remotely controls the SCARA mechanism to determine the puncture position; controls the RCM mechanism to determine the puncture angle; controls the puncture execution structure to perform the puncture action; and the foot-operated switching structure realizes the switching of the control of the SCARA mechanism and the RCM mechanism by the master-hand execution structure.

2. The remote surgical robot according to claim 1, characterized in that: The SCARA mechanism includes a horizontal moving structure and a vertical moving structure. The horizontal moving structure includes three servo motors, namely motor A rotating along the A axis, motor B rotating along the B axis, and motor C rotating along the C axis. Motor A and motor B are connected by connecting plate 1, and motor B and motor C are connected by connecting plate 2; the vertical moving structure includes a vertical seat with a lifting track, a lifting structure connected to motor C, the lifting structure moves along the lifting track, and motor D rotating along the D axis is also provided, and motor D drives the lifting structure to move along the lifting track; motor A is connected to the RCM mechanism setting plate.

3. The remote surgical robot according to claim 2, characterized in that: The RCM mechanism includes a tumbling motor arranged on the equipment plate and rotating along the E-axis, a parallelogram structure, a parallelogram structure arranged on the tumbling motor and rotating with the tumbling motor, and a pitch guide track. The parallelogram structure extends out of a moving block that moves in the pitch guide track. The moving block is hinged to the parallelogram structure. A lead screw is arranged in the pitch guide track. The moving block is arranged on the lead screw. A pulley 1 is arranged on the side of the lead screw close to the tumbling motor, and a pitch motor is arranged under the track on the side close to the tumbling motor. The rotating shaft is the F-axis. Pulley 2 extends from the pitch motor, and pulley 1 and pulley 2 are connected by a belt. The rotation of the pitch motor in different directions drives the rotation of the lead screw in different directions, and the rotation of the lead screw in different directions drives the movement of the moving block in different directions, thereby forming a parallelogram structure with different pitch angles.

4. The remote surgical robot according to claim 3, characterized in that: A rotating structure is set at the end of the parallelogram structure, and a motor for rotating the injection needle is set inside the rotating structure, and the axis of the motor for rotating the injection needle is the G axis; a puncture execution structure is set inside the rotating structure, wherein the needle tip of the puncture execution structure is set on the RCM point; the E axis and the G axis pass through the RCM point, and the E axis is parallel to the pitch guide track.

5. The remote surgical robot according to claim 4, characterized in that: A puncture execution structure is equipped with a puncture motor, which rotates to drive the injection needle to perform the puncture action. The main hand execution structure controls the puncture path, sends the puncture path to the puncture execution structure through the 5G network, and controls the rotation of the puncture motor to execute the puncture path.

6. The remote surgical robot according to claim 5, characterized in that: A motor control structure connected to a motor control signal is provided. The motor control structure controls the rotation of each axis motor according to the motor signal, thereby controlling changes in each position and angle.

7. The remote surgical robot according to claim 5, characterized in that: A micro-force measurement module is set on the outside of the injection needle. The micro-force measurement module includes a Bragg grating. It measures and obtains the deformation data of the grating driven by the force deformation when the injection needle performs a puncture action. The force result is processed by a fiber optic Bragg grating demodulator, and the measured force result is sent to the cloud. It is then transmitted to the main hand control device through the cloud. The main hand execution structure senses the force situation during the puncture action of the injection needle in the form of force feedback to determine whether the injection needle has penetrated into the blood vessel.

8. The remote surgical robot according to claim 4, characterized in that: The injection needle is connected to the syringe on the microfluidic pump through a soft tube.

9. The remote surgical robot according to claim 7, characterized in that: The lifting structure and the microfluidic pump of the vertical moving structure of the SCARA mechanism are fixed to a base, and the base is fixed to a high platform that meets the height requirements.

10. The remote surgical robot according to claim 1, characterized in that: The cloud-based private network uses tunneling protocol technology for information encryption, access control, and network security protection between the master hand control device and the execution robot device. The data transmission terminal consists of two 5G CPE devices, one located at the master hand control device and the other at the execution robot device. They are used to send control, feedback, and video stream information sent by both ends to the cloud-based private network, and also read data sent by the other party from the cloud-based private network and transmit it to the other party's device at the same time.

Images