Remote interactive ultrasound-guided puncture system and method with master-end force feedback

Abstract

The present invention relates to a remote interactive ultrasound-guided puncture system and method with master-end force feedback, comprising a prosthesis, a master-end operating arm, a simulated operating table, a slave-end robot, and a display unit; the master-end operating arm is provided with a gripping portion, and the gripping portion contacts the prosthesis to realize simulated operation; the master-end operating arm is provided with an encoder, a distance measuring sensor, an inclination sensor, and a master-end multi-dimensional force sensor, which can obtain the position, posture information, and force information of the gripping portion; the simulated operating table comprises a table top and a driving mechanism; the slave-end robot is provided with a slave-end multi-dimensional force sensor for real-time detection of the fitting force; the slave end is interconnected and communicated with the master end, and operates in real time according to the position, posture, and force information of the gripping portion; the display unit displays the master-end posture information, multi-dimensional force information, and the slave-end on-site audio and video information, medical images, and slave-end robot force signals in real time. The present invention realizes real-time interaction, feedback, and verification between the master and the slave, and the simulation scene is more realistic and the medical operation is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of telemedicine, and in particular to a remote interactive ultrasound-guided puncture system and method with master-end force feedback. Background Art

[0002] With the improvement of medical technology, remote interactive medical care has been widely used. At present, the existing remote methods are mostly experts guiding remote medical operations through video and audio. However, medical operations or medical examinations require very high professional skills of doctors, such as ultrasound examination, puncture, ultrasound-guided puncture, etc. Due to the high concentration of medical resources in my country, experts with such skills are mostly distributed in first-tier cities such as Beijing, Shanghai, Guangzhou and Shenzhen. Doctors in the majority of grassroots medical units rarely have high medical skills. This is bound to affect the timely treatment of local patients.

[0003] As an improvement:

[0004] One of the existing technologies is to collect the hand posture signals of the expert end through visual recognition to control the operation of the slave robot. However, this method is easily blocked and easily affected by light sources, has high requirements on the algorithm, and has poor interaction accuracy and real-time performance.

[0005] One of the existing technologies is that the main end uses commercial forcedimension, haptic and other commercial main hands. The haptic workspace is small and cannot meet the requirements of B-ultrasound detection range. Forcedimension is expensive, and because there are driving components such as motors inside, it is relatively laborious to drag.

[0006] One of the existing technologies is that the main end uses an analog B-ultrasound probe + touch panel to detect the position (X, Y direction). Since the touch panel is flat, it cannot simulate the real feel of a doctor's examination (human body curve, softness and hardness, etc.), and cannot realize side examination of the human body. Summary of the invention

[0007] The technical problem to be solved by the present invention is to solve the problems existing in the prior art. The present invention relates to a remote interactive ultrasound-guided puncture system and method with master-end force feedback, which realizes real-time master-slave interaction, feedback and verification to ensure that the remote end can simulate real scenes to achieve accurate medical operations.

[0008] The object of the present invention is achieved in that:

[0009] A remote interactive ultrasound-guided puncture system with master-end force feedback, comprising:

[0010] A prosthesis, wherein the prosthesis is used to simulate a real human body;

[0011] The main end operating arm has a gripping part operated by the main end doctor, and the gripping part contacts the prosthesis to realize the simulated operation; the main end operating arm is provided with an encoder, a distance measuring sensor, an inclination sensor and a main end multi-dimensional force sensor, and the position and posture information of the gripping part are obtained based on the data monitored by the encoder, the distance measuring sensor and the inclination sensor; the main end multi-dimensional force sensor is used to detect the force information of the gripping part in real time;

[0012] A simulated operating table, the simulated operating table comprising a table top and a driving mechanism, the table top is used to support the prosthesis, and the driving mechanism is used to drive the table top to move within a spatial range;

[0013] The slave robot is provided with a slave multi-dimensional force sensor for real-time detection of the bonding force; the slave robot is interconnected and communicated with the master operating arm, and moves in real time according to the position, posture and force information of the gripping part;

[0014] The display unit displays in real time the master end posture information, the master end multi-dimensional force information, the slave end on-site audio and video information, the slave end medical image and the slave end robot force signal.

[0015] The slave end of the present invention moves in real time according to the position, posture and force of the master end, and feeds back the patient's medical image and force. The master end adjusts the position, posture and force in real time according to the medical image and force information of the slave end. The slave end moves in real time according to the adjustment of the master end and feeds back the signal. A closed loop is formed between the master and the slave to ensure that the doctor's operation can simulate the real scene to achieve accurate medical operation.

[0016] The prosthesis here can be the entire human body model or a part of the human body model, and can also be customized according to the patient's body characteristics before surgery.

[0017] The display unit can also display the live audio and video information of the slave end, the master end posture and three-dimensional force signal.

[0018] In a preferred embodiment of the present invention, the driving mechanism comprises a lifting driving mechanism, a top end of the lifting driving mechanism is connected to the table top, and a bottom end of the lifting driving mechanism is fixed to the base.

[0019] During the simulated operation, the prosthesis can apply feedback force to the gripping part under the driving action of the driving mechanism. The doctor at the main end can feel the magnitude of the prosthesis's feedback force, thereby enhancing the realism of the operation.

[0020] In a preferred embodiment of the present invention, there are multiple lifting drive mechanisms, the multiple lifting drive mechanisms are evenly arranged, and the top ends of the lifting drive mechanisms are rotatably connected to the table top.

[0021] Each lifting drive mechanism can independently control the lifting and lowering, and the lifting and lowering and tilting state adjustment of the tabletop can be driven by controlling the lifting and lowering distances of the lifting drive mechanisms at different positions.

[0022] In a preferred embodiment of the present invention, the driving mechanism further comprises a rotating mechanism, and the rotating mechanism is used to drive the table to rotate.

[0023] The cooperation between the lifting drive mechanism and the rotating mechanism can enable the table top to perform composite operations such as lifting, rotating, and tilting state adjustment.

[0024] In a preferred embodiment of the present invention, the lifting drive mechanism is arranged above the rotating mechanism through a mounting plate, the top end of the rotating mechanism is fixedly connected to the lower surface of the mounting plate, and the bottom end of the lifting drive mechanism is fixedly arranged on the upper surface of the mounting plate.

[0025] When the angle of the prosthesis needs to be adjusted, the rotating mechanism is controlled to rotate a certain angle, and the lifting drive mechanism is coordinated to rise and fall, so that the spatial position and posture of the prosthesis can be adjusted quickly and conveniently, which is convenient for the doctor to operate.

[0026] In a preferred embodiment of the present invention, the table top includes a first layer and a second layer arranged in parallel, the first layer and the second layer are connected via the rotating mechanism, and the lower surface of the second layer is connected to the top end of the lifting drive mechanism.

[0027] The rotating mechanism drives the first layer of the table to rotate, thereby rotating the prosthesis. At the same time, the lifting drive mechanism controls the lifting and lowering of the second layer to adjust the spatial position and posture of the prosthesis. The structure is simple and easy to operate.

[0028] In a preferred embodiment of the present invention, the lifting drive mechanism is driven by a hydraulic cylinder or a motor.

[0029] The use of hydraulic drive or private motor drive makes the lifting of the prosthesis more stable.

[0030] In a preferred embodiment of the present invention, the main-end operating arm includes a base, an upper arm, a lower arm and a manual linkage part, the base is rotationally connected to the upper arm, the upper arm is rotationally connected to the lower arm, and the manual linkage part is connected to the lower arm through a universal joint; a first encoder is provided at the rotating shaft of the upper arm, a second encoder is provided at the connection between the upper arm and the lower arm, and the three-axis inclination sensor and the main-end multi-dimensional force sensor are provided on the manual linkage part.

[0031] The upper arm and the lower arm make the plane movement more flexible and free, with a large reachable area, a light and compact structure, greater flexibility in horizontal movement, large workspace utilization, fewer parts, low manufacturing cost, and easy disassembly and maintenance. Two rotating joints are used to achieve rapid positioning in the XY plane, one moving joint is used to extend and retract in the Z direction, and a universal joint structure is used to achieve rotational movement of the end in three directions. The manual linkage part can achieve flexible movement of up, down, left, right, and angle in another dimension. Through the mutual cooperation of the three, any medical action can be achieved in the smallest possible space, which can meet the coverage of the detection area.

[0032] In a preferred embodiment of the present invention, the main end operating arm further includes a lifting mechanism, and the manual linkage part is connected to the small arm through the lifting mechanism.

[0033] The lifting mechanism allows for more flexible movement on concave and convex surfaces, enhancing the operating realism of the doctor at the main end.

[0034] In a preferred embodiment of the present invention, the ranging sensor is provided on the lifting mechanism; the master-end computing unit obtains first position information according to the first encoder, obtains second position information according to the second encoder, and obtains first spatial position information by combining the first position information and the second position information; obtains second spatial position information according to the inclination sensor and the distance sensor, and obtains force information according to the master-end multi-dimensional force sensor; the slave-end robot performs actions according to the posture information calculated by the slave-end computing unit, and the slave-end computing unit corrects the posture in real time according to the forces of the master and slave ends.

[0035] The slave-end robot arm can completely follow the master-end position, posture, and force and perform positioning actions with high accuracy and reliability. At the same time, the slave-end on-site audio and video, medical images, and force information are fed back to the display unit. The master-end doctor can comprehensively correct the position and action based on the medical images, force and other information fed back from the slave, thus realizing a closed loop between the master and slave ends.

[0036] In a preferred embodiment of the present invention, the slave robot is installed on the slave bracket in an inverted manner.

[0037] The inverted installation method can effectively utilize the arm span, so that the end of the slave robot can reach various parts of the patient's body to the maximum extent, avoid the impact of the robotic arm elbow joint on the human body caused by the inverse solution of the series structure, and improve the safety of the system.

[0038] In a preferred embodiment of the present invention, a distance measuring sensor is installed at the end of the slave robot; the distance measuring sensor obtains three-dimensional surface information of the patient's body surface by grid scanning in an initial state.

[0039] The ranging sensor may be a laser ranging sensor or an ultrasonic ranging sensor. In the initial state, the ranging sensor obtains rough three-dimensional surface information of the patient's body surface by moving the robot in a plane grid (grid spacing 1.5 cm) at a set absolute safety height (about 20 cm from the patient's body).

[0040] In a preferred embodiment of the present invention, an axial drive motor is installed at the end of the slave robot, and the axial drive motor drives the end-holding medical device to be axially close to the patient's body to meet the required pressure.

[0041] The medical device is installed at the end of the slave robot through a driving motor. The axial motor can control the end medical device to move axially, eliminating the six-axis inverse analysis and linkage process of the slave robot arm, so that the end medical device can get closer to the human body more quickly, and the timeliness of the slave response is improved; at the same time, due to the presence of the ranging sensor, the situation that the medical device is not close to the human body or exerts too much pressure on the human body due to the unknown human body shape and surface features and the inherent overshoot or slow response defects of the control algorithm is avoided.

[0042] In a preferred embodiment of the present invention, adjustable damping is provided on the lifting mechanism, at the rotation point between the base and the upper arm, and at the rotation point between the upper arm and the lower arm.

[0043] The damping force of the movement can be adjusted arbitrarily according to the doctor's operating habits to enhance the realism of the operation.

[0044] The present invention also relates to a remote master-slave interactive ultrasound-guided puncture method, based on the above-mentioned remote interactive ultrasound-guided puncture system with master-end force feedback, comprising the following steps:

[0045] 1) Before the operation, the slave robot measures the human body shape and obtains the three-dimensional shape information of the patient's surgical area.

[0046] 2) The doctor operates the master-end manipulator to move on the prosthesis, and the master-end computing unit collects the signals of the position, posture, and force of the master-end manipulator calculated by the master-end multi-sensor, processes them, and then feeds them back to the slave-end robot;

[0047] 3) The slave robot performs actions according to the received posture signal, and corrects the posture according to the previous three-dimensional data of the profile and the real-time force signal of the slave until the force magnitude of the slave is consistent with that of the master;

[0048] 4) The doctor at the master end adjusts the operation and force according to the slave end medical image and the slave end three-dimensional force data displayed on the display unit;

[0049] 5) Repeat steps 2)-4) until a suitable positioning point is reached.

[0050] A preferred embodiment of the present invention comprises two sets of master-end operating arms and slave-end robots; the doctor uses one hand to locate the lesion in real time, and uses the other hand to perform medical surgery or surgical positioning in real time.

[0051] The present invention has at least the following beneficial effects:

[0052] 1) In the present invention, the slave end moves in real time according to the position, angle and force of the master end, and feeds back the patient's medical image and force. The master end adjusts the movement and force in real time according to the medical image and force of the slave end. The slave end also moves in real time according to the adjustment of the master end and feeds back the signal. The master and the slave provide mutual feedback and verification to ensure that the doctor's operation can simulate the real scene to achieve accurate medical operation.

[0053] 2) The main end is equipped with an arm and a forearm to make planar movements more flexible and free, with a large horizontal reachable area to meet detection requirements, small dragging resistance, and flexible operation. The end can move arbitrarily in the vertical direction and can reach any point in the workspace in a variety of postures. Through the mutual cooperation of this structure, any medical action can be achieved in the smallest possible space.

[0054] 3) The lifting mechanism allows for more flexible movement on concave and convex surfaces, and combined with customizable prostheses, the operating realism of the primary doctor is enhanced.

[0055] 4) Through position, distance, angle, force, etc., the slave robot arm can completely follow the master position, posture, force, and perform positioning actions with high accuracy and reliability.

[0056] 5) In the initial state, the ranging sensor obtains rough three-dimensional surface information of the patient's body surface by moving the robot in a plane grid (grid spacing 1.5 cm) at a set absolute safety height (about 20 cm away from the patient's body); the system response speed can be improved.

[0057] 6) The axial drive motor can drive the terminal medical device to move axially close to the patient's body. The axial motor can control the terminal medical device to move axially, eliminating the six-axis reverse analysis and linkage process of the slave robot arm, so that the terminal medical device can be closer to the human body more quickly, and the timeliness of the slave response is improved; at the same time, due to the presence of the ranging sensor, it avoids the situation where the medical device is not close to the human body or puts too much pressure on the human body due to the inherent overshoot or slow response defects of the control algorithm.

[0058] 7) The damping force of the up and down movement can be adjusted arbitrarily according to the doctor's operating habits to enhance the realism of the operation.

[0059] 8) Through the multi-dimensional force sensors and force control algorithms of the master and slave robots, the force is guaranteed to be safe, within a reasonable range, and controllable by the master. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0061] Figure 2 This is a schematic diagram of the overall structure of the main end operating arm of the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of the manual control linkage unit;

[0063] Figure 4-5 The overall structural diagram of the slave robot of the present invention;

[0064] Figure 6 A schematic diagram of the signal principle of ultrasound-guided puncture in the present invention;

[0065] Figure 7 This is the coordinate analysis model diagram of the main end robot arm;

[0066] Figure 8 The structure of the simulated operating table of the present invention is shown in FIG. Figure 1 ;

[0067] Fig. 9 The structure of the simulated operating table of the present invention is shown in FIG. Figure 2 ;

[0068] Fig.10 It is a schematic diagram of the interference structure between the slave robot and the patient;

[0069] Figure 11-12 It is a schematic diagram of the dual system structure;

[0070] Fig.13 It is a control flow diagram of the present invention.

[0071] 1-vacuum suction cup; 2-base; 3-upper arm rotation axis; 4-lower arm rotation axis; 5-wiring tube; 6-pull rope displacement sensor; 7-lifting mechanism; 8-lifting damping adjustment knob; 9-universal joint; 9-1-T-pin shaft; 9-2-first swing member; 9-3-second swing member; 10-three-axis tilt sensor; 11-grip; 12-six-dimensional force sensor; 13-chassis; 14-slave robot; 15-six-dimensional force sensor; 16-clamp; 17-B-ultrasound probe; 18-patient; 19-prosthesis; 20-axial drive motor; 21-distance sensor; 22-table; 221-first layer; 222-second layer; 23-lifting drive mechanism; 24-rotation mechanism. DETAILED DESCRIPTION

[0072] The present invention will be further described below in conjunction with the accompanying drawings.

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] To facilitate understanding of the embodiments of the present application, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present application.

[0075] Embodiment 1:

[0076] like Figure 1 As shown, a remote interactive ultrasound-guided puncture system with master-end force feedback comprises:

[0077] A prosthesis 19, wherein the prosthesis 19 is used to simulate a real human body;

[0078] The main end operating arm has a gripping portion 11 for operation by the main end doctor; the gripping portion 11 contacts the prosthesis 19 to realize simulated operation; the main end operating arm is provided with an encoder, a distance measuring sensor 6, a three-axis tilt sensor 10 and a main end multi-dimensional force sensor 12; based on the data monitored by the encoder, the distance measuring sensor and the tilt sensor, the position and posture information of the gripping portion are obtained;

[0079] The main-end multi-dimensional force sensor is used to detect the force information of the gripping part in real time, that is, the main-end multi-dimensional force sensor 12 detects the force applied by the doctor in real time;

[0080] A simulated operating table, the simulated operating table comprising a table top 22 and a driving mechanism, the prosthesis 19 is placed on the table top 22, and the driving mechanism is used to drive the table top 22 to move within a spatial range;

[0081] A slave robot 14, wherein the slave robot 14 is provided with a slave multi-dimensional force sensor, and the slave multi-dimensional force sensor is used to detect the fit force between the device and the human body in real time; the slave robot 14 is interconnected and communicated with the master operating arm, and the position, posture and force information of the master gripping part are transmitted to the slave robot 14 in real time, and the slave computing unit processes the position, posture and force information of the gripping part, and the slave robot 14 moves in real time according to the signal processed by the slave computing unit;

[0082] The display unit displays the master end posture information, the master end multi-dimensional force information, the slave end on-site audio and video information, the slave end medical image and the slave end robot force signal in real time.

[0083] The slave end of the present invention moves in real time according to the position, posture and downward pressure of the master end, and feeds back the patient's medical image and force. The master end adjusts the position, posture and downward pressure in real time according to the medical image and force of the slave end. The slave end moves in real time according to the adjustment of the master end and feeds back the signal. A closed loop is formed between the master and the slave to ensure that the doctor's operation can simulate the real scene to achieve accurate medical operation.

[0084] The prosthesis here can be the entire human body model, or a part of the human body model, or it can be customized according to the patient's body characteristics before surgery.

[0085] Specifically, the master-side expert simulates medical operations on the virtual human body model 19 by operating the master-side passive robotic arm. The posture information of the gripping part of the master-side passive robotic arm is processed by the computing unit and then transmitted to the slave-side computing unit via the server. The slave-side enables the remote collaborative robotic arm to perform actions on the patient 18 based on the posture information corrected by the slave-side computing unit.

[0086] Specifically, the master and slave ends are connected through the 5G low-latency communication network, ensuring high-speed, real-time transmission of control signals, audio, video, images and other information, solving the problem of asynchronous actions between the master and slave ends due to long physical distances.

[0087] In order to obtain posture and mechanical information more accurately, the main-end multi-dimensional force sensor here uses a six-dimensional force sensor, which can collect the magnitude and direction of the resultant force.

[0088] Preferably, see Figure 2-3 The main end operating arm includes a base 2, an upper arm, a lower arm and a manual linkage part. The base 2 is rotatably connected to the upper arm, and the upper arm is rotatably connected to the lower arm. The manual linkage part is connected to the lower arm through a universal joint. A first encoder is arranged at the upper arm rotation shaft 3, and a second encoder is arranged at the connection between the upper arm and the lower arm, that is, the lower arm rotation shaft 4, which is not shown in the figure. A damping adjustment device is also arranged at the rotation shafts 3 and 4 to adapt to the hand feel of different experts. A three-axis inclination sensor 10 is arranged on the manual linkage part, and a six-dimensional force sensor 12 is arranged at the lower end of the manual linkage part.

[0089] The two rotating joints are respectively equipped with an absolute encoder to detect the rotation angle of the upper arm and the lower arm; the two rotating shafts 3 and 4 are arranged in parallel, and the axis of the upper arm rotating shaft 3 and the axis of the lower arm rotating shaft 4 are perpendicular to the installation plane. According to the length of the upper arm L1, the length of the lower arm L2, and the angles θ1 and θ2 of the rotation of the upper arm and the lower arm, the x and y coordinates of point A can be calculated, as shown in Figure 7 As shown, point A is located at the end of the forearm.

[0090] The upper arm and the lower arm make the planar movements more flexible and free, and the reachable area is large; the hand-controlled linkage part can change the vertical height and posture for flexible movements, and through the mutual cooperation of the mechanisms, any medical action can be achieved in the smallest possible space.

[0091] Specifically, the main end operating arm has a base, on which a vacuum suction cup is arranged; and the base 2 is installed on the base.

[0092] The base supports the main end, and a base 2 is provided on the base. The base 2 is fixed on the base, and a vacuum suction cup 1 is provided below the base 2 for fixed connection with the workbench installation plane. The base can be easily moved and maintained, and can be flexibly adjusted according to changes in the operating room. One end of the base 2 is fixedly connected to the vacuum suction cup 1, and the other end is connected to the upper arm through a rotating joint, and the upper arm and the forearm are connected through a rotating joint.

[0093] The vacuum suction cup 1 is a mechanical suction cup, which can achieve adsorption and suction through manual operation. It is easy to move and flexible to operate. The position of the main end operating arm can be adjusted according to the doctor's on-site situation.

[0094] The upper arm realizes a relatively large swing, and the lower arm realizes a secondary small swing. Through the setting of the upper arm and the lower arm, the size is smaller while realizing the same swing distance.

[0095] Preferably, the main end operating arm further includes a lifting mechanism 7, and the manual linkage part is connected to the forearm via the lifting mechanism 7.

[0096] The lifting mechanism 7 can move more flexibly on the concave and convex surface, thereby improving the operating realism of the doctor at the main end.

[0097] Preferably, a ranging sensor 6 is provided on the lifting mechanism 7; the server obtains first position information according to the first encoder, obtains second position information according to the second encoder, and obtains first spatial position information by combining the first position information and the second position information; the second spatial position information is obtained according to the inclination sensor and the distance sensor, and the force information is obtained according to the six-dimensional force sensor of the master end; the slave robot performs actions according to the posture information calculated by the server, and the master end corrects the position instruction according to the force information fed back by the slave end.

[0098] Through position, distance, angle, force, etc., the slave robot arm can completely follow the master position, posture, and force and perform positioning actions with high accuracy and reliability.

[0099] The master computing unit collects and processes the master's posture signal and force signal, and the processed signal is transmitted to the slave via the server.

[0100] The slave computing unit receives the force signals from the master computing unit and the slave, and transmits the processed signals to the slave;

[0101] Display unit: receives medical images from the slave end; force signals from the master and slave ends, posture signals from the master end, and audio and video information from the slave end.

[0102] The master and slave terminals are connected to the computing unit through a network. The master terminal sends signals to the master terminal computing unit, and the slave terminal sends signals to the slave terminal computing unit. The slave terminal receives the posture signal processed by the slave terminal computing unit. Fig.13 .

[0103] Preferably, the universal joint 9 includes a T-pin shaft 9-1, a first swinging member 9-2 is hinged at the lower end of the T-pin shaft 9-1, and a second swinging member 9-3 is hinged at the lower end of the first swinging member 9-2; the T-pin shaft 9-1 is rotatably installed in the opening at the top of the forearm.

[0104] By means of a pin shaft and an articulated swinging member, the ball articulation can have a wider rotation angle range, and any medical action can be achieved in the smallest possible space.

[0105] In a preferred embodiment of the present invention, adjustable damping is provided on the lifting mechanism, at the rotation point between the base and the upper arm, and at the rotation point between the upper arm and the lower arm, and corresponding damping adjustment knobs 8 can also be provided at corresponding positions.

[0106] The damping force of the movement can be adjusted arbitrarily according to the doctor's operating habits to enhance the realism of the operation.

[0107] The manual linkage component is manually controlled by the operator, and the expert doctor manually controls the gripping part 11 to move up and down and rotate in all directions in space. The manual linkage component has a simple structure and is easy to operate, which is highly compatible with the operation of the expert in the actual surgical process.

[0108] Specifically, the lifting mechanism 7 is slidably arranged on the forearm. Preferably, the lifting mechanism 7 is connected to the forearm through a slide rail. The lifting mechanism 7 is manually driven by an operator to move up and down along an axis parallel to the rotating shafts 3 and 4. Furthermore, the lifting mechanism is equipped with a lifting adjustment damping device, and the two rotating shafts 3 and 4 are equipped with a rotation adjustment damping device. By setting the damping device, the damping during dragging can be changed to adapt to the hand feeling of different experts.

[0109] The angle adjustment mechanism 9 is composed of two universal joints. The first universal joint cooperates with the second universal joint to realize rotation in three directions of x, y and z. Specifically, the first universal joint is connected to the lifting device through a bearing, and the second universal joint is connected to the gripping part 11 through a second connecting rod. The second connecting rod is provided with an inclination sensor mounting part.

[0110] Among them, the distance measuring sensor is a pull-wire displacement sensor 6, which can more easily detect the change of position. The pull-wire displacement sensor 6 is installed on the upper part of the forearm. The pull-wire displacement sensor 6 includes a pull-wire, a bobbin and an adjustable resistor. The pull-wire is wound on a precisely machined straight cylindrical bobbin. The moving end of the pull-wire is connected to the lifting mechanism. The movement of the pull-wire along the moving direction of the lifting mechanism can change the resistance of the resistor, and then convert the mechanical movement of the pull-wire into an electrical signal that can be measured and recorded, and obtain the movement information of the lifting mechanism. The pull-wire displacement sensor 6 has a small installation size, a compact structure, a large measuring stroke and high precision.

[0111] In this embodiment, the driving mechanism is used to drive the table 22 to move within a spatial range. That is, the driving mechanism drives the table 22 to move, thereby realizing the movement of the prosthesis 19 within a spatial range. The movement of the prosthesis 19 includes lifting, rotating, tilting posture adjustment and other actions.

[0112] The doctor at the main end holds the gripping portion 11 in contact with the prosthesis 19 to perform simulated operation. During the simulated operation, the driving mechanism drives the table 22 to move the prosthesis 19 within a spatial range, such as lifting and lowering. The multi-dimensional force sensor 12 at the main end detects the contact force between the gripping portion 11 and the prosthesis 19 in real time. During the simulated operation, the prosthesis 19 can apply feedback force to the gripping portion 11 under the driving action of the driving mechanism, so the doctor at the main end can feel the magnitude of the feedback force of the prosthesis 19, thereby improving the realism of the operation.

[0113] In a preferred implementation manner of this embodiment, the driving mechanism includes a lifting driving mechanism 23, the top end of the lifting driving mechanism 23 is fixedly connected to the table top 22, the number of the lifting driving mechanism 23 is one or more, and the lifting driving mechanism 23 is driven by a hydraulic cylinder or a motor.

[0114] If a lifting drive mechanism 23 is provided, the driving end of the lifting drive mechanism 23 is provided at the center of the lower surface of the table 22 . The driving mechanism of this structure can only be used for the lifting and lowering of the table 22 .

[0115] If multiple lifting drive mechanisms 23 are set, the multiple lifting drive mechanisms 23 are symmetrically arranged, and the driving ends of the multiple lifting drive mechanisms 23 are evenly distributed on the lower surface of the table 22 and are rotatably connected to the lower surface of the table 22. Each lifting drive mechanism 23 can independently control the lifting and lowering, and the lifting and lowering of the table 22 can be driven by controlling the lifting distance of the lifting drive mechanisms 23 at different positions.

[0116] In a preferred implementation manner of this embodiment, the driving mechanism may further include a rotating mechanism 24, and the rotating mechanism 24 is used to drive the table top 22 to rotate. That is to say, the driving mechanism includes a lifting driving mechanism 23 and a rotating mechanism 24. The lifting driving mechanism 23 cooperates with the rotating mechanism 24 to enable the table top 22 to perform complex operations such as lifting, rotating, and adjusting the tilt state.

[0117] like Figure 8 As shown in the first structure of the driving mechanism, the lifting driving mechanism 23 is arranged above the rotating mechanism 24 through the mounting plate, the top of the rotating mechanism 24 is fixedly connected to the lower surface of the mounting plate, the bottom of the lifting driving mechanism 23 is fixedly arranged on the upper surface of the mounting plate, and the top of the lifting driving mechanism 23 is connected to the lower surface of the table 22. When the angle of the prosthesis 19 needs to be adjusted, the rotating mechanism 24 is controlled to rotate a certain angle, and the lifting driving mechanism 23 is coordinated to rise and fall, so that the spatial position and posture of the prosthesis 19 can be adjusted conveniently and quickly, which is convenient for doctors to operate.

[0118] like Fig. 9 As shown, in the second structure of the driving mechanism, the table 22 is a double-layer structure, including a first layer 221 and a second layer 222 arranged in parallel, the first layer 221 and the second layer 222 are connected by a rotating mechanism 24, the double-layer table 22 is located above the lifting driving mechanism 23, and the top of the lifting driving mechanism 23 is connected to the bottom surface of the second layer 222. Specifically, the first layer 221 is located above the second layer 222, the first layer 221 and the second layer 222 are arranged in parallel, the prosthesis 19 is directly placed on the upper surface of the first layer 221, the second layer 222 is connected to the lifting driving mechanism 23, and the rotating driving end of the rotating mechanism 24 is connected to the bottom surface of the first layer 221, which can drive the first layer 221 of the table 22 to rotate, thereby rotating the prosthesis 19, and at the same time, the lifting driving mechanism 23 controls the lifting and lowering of the second layer 222 to adjust the spatial position and posture of the prosthesis 19.

[0119] By setting up a driving mechanism at the main end, not only can the spatial position and posture of the prosthesis 19 be adjusted conveniently and quickly, which is convenient for the main-end doctor to operate, but also feedback force can be applied to the main-end doctor through lifting and lowering during the simulated operation, so that the main-end doctor can feel the size of the feedback force of the prosthesis 19, thereby enhancing the main-end doctor's sense of reality in operation.

[0120] In a preferred implementation manner of this embodiment, the simulated operating table also includes a base, and the driving mechanism is arranged on the base. The base is provided with rollers to facilitate the horizontal movement of the simulated operating table, and the rollers are also equipped with a brake structure. When the simulated operating table moves to a specified position, the brake mechanism is used to lock the rollers to prevent the simulated operating table from moving during the operation.

[0121] In a preferred implementation manner of this embodiment, the upper arm and the lower arm are made of high-strength aluminum alloy material.

[0122] In this embodiment, ultrasonic detection is taken as an example, but the technology involved in the present invention is not limited to ultrasonic examination, and any operation that can use the technology of the present invention to achieve remote medical treatment is applicable.

[0123] The slave end adopts a collaborative robotic arm, which can be built by a slave end robot. The end of the slave end robot is provided with a clamp 16 and a six-dimensional force sensor 15. The clamp 16 is used to hold the B-ultrasound probe 17, and the six-dimensional force sensor 15 is used to monitor the magnitude of the force applied by the B-ultrasound probe of the slave end on the patient 18. The distance sensor 21 is used to measure the three-dimensional surface information of the patient's surface in the initial state, and the axial drive motor 20 is used to control the movement of the B-ultrasound probe in the axial direction; specifically, the end of the slave end robot is equipped with a distance sensor 21 and an axial drive motor 20, and the medical device is installed on the end of the slave end robot through the axial drive motor 20; the distance sensor 21 obtains the three-dimensional information of the patient's body surface in real time, and the axial drive motor 20 drives the B-ultrasound probe 17 to approach the patient's body along the axial direction according to the control instruction.

[0124] The end of the slave robot is provided with a distance sensor 21 and an axial drive motor 20. The distance sensor 21 can obtain the three-dimensional surface information of the human body at the initial position. Due to the inherent overshoot or slow response defects of the control algorithm, the medical device is not close to the human body or the pressure on the human body is too large; the axial drive motor 20 can control the B-ultrasound probe 17 to be close to the patient's body surface along the axial direction, and the axial motion motor can control the medical device to move along the axial joint, eliminating the six-axis reverse analysis and linkage process of the slave robot arm, so that the end medical device can be closer to the human body more quickly, and the timeliness of the slave response is improved.

[0125] See also Figure 5 The axial drive motor 20 can drive the B-ultrasound probe 17 to move axially.

[0126] like Figures 4 to 5 As shown, the fixed installation end of the slave robot is provided with a chassis 13, and the slave robot 14 is installed in an inverted manner, that is, the chassis 13 of the slave robot is installed on the operating table, and the end of the slave robot extends downward to the body of the patient 18. The inverted installation method can effectively utilize the arm span, so that the end of the slave robot can reach various parts of the patient 18 to the maximum extent, and avoid the impact of the elbow joint of the robot arm on the human body caused by the inverse solution of the series structure. Fig.10 As shown, improving system security.

[0127] Embodiment 2:

[0128] The present invention also relates to a remote master-slave interactive ultrasound-guided puncture method, which uses the remote interactive ultrasound-guided puncture system with master-end force feedback of the first embodiment, and comprises the following steps:

[0129] 1) Before the operation, the slave robot measures the human body surface to obtain the three-dimensional surface information of the patient's surgical area;

[0130] 2) The doctor at the master end controls the driving mechanism to adjust the spatial position and posture of the prosthesis 19 according to the operation needs. The doctor operates the master end operating arm to move on the prosthesis 19. During the simulation operation, the driving mechanism controls the lifting and lowering of the table surface to achieve the lifting and lowering of the prosthesis 19. The doctor at the master end can feel the magnitude of the feedback force of the prosthesis 19. The master end computing unit collects the signals of the position, posture and force of the master end operating arm and feeds them back to the slave end robot after computing and processing;

[0131] 3) The slave robot performs actions according to the received posture signals until the force on the slave robot is consistent with that on the master robot and the robot stops moving;

[0132] 4) The doctor at the master end adjusts the operation and force according to the medical images and force data fed back by the slave end displayed on the display unit;

[0133] 5) Repeat steps 2)-4) until a suitable positioning point is reached.

[0134] Preferably, if Figure 11-12 As shown, it includes two sets of master-end operating arms and slave-end robots; the doctor uses one hand to locate the lesion in real time, and uses the other hand to perform medical surgery or surgical positioning in real time.

[0135] Taking ultrasound-guided puncture surgery as an example, the present invention works in the following manner:

[0136] Before the operation, the range-finding sensor 21 of the slave robot measures the human body surface to obtain the three-dimensional surface information of the patient's surgical area;

[0137] The master end robot arm is pre-fixed on the master end workbench by using a vacuum suction cup 1, and the master end workbench can be moved to suit different working locations of the master end doctor. The slave end robot arm is pre-installed upside down on the mounting frame, and a roller mechanism is set at the bottom of the mounting frame. The slave end doctor pushes the slave end robot arm to the surgical area, locks the roller mechanism of the mounting frame, and adjusts the posture of the slave end so that the end of the slave end robot arm extends downward to the patient's 18 body part.

[0138] The master-side doctor controls the driving mechanism to adjust the spatial position and posture of the prosthesis 19 according to the operational needs. The master-side expert doctor holds the gripping part 11 and simulates the scanning and movement of the B-ultrasound probe on the virtual human body model 19. The master-side doctor can feel the feedback force of the prosthesis 19. The sensor component monitors the sensor information of the gripping part 11 in real time, obtains the master-side signal, and transmits the master-side signal to the computing unit through the 5G communication network. The computing unit calculates and calibrates the received master-side signal to obtain the spatial posture information of the gripping part 11, and the computing unit transmits the spatial posture information to the slave end.

[0139] Specifically, the main-end expert doctor holds the gripping part 11 with one hand, driving the upper arm rotation axis 3 and the lower arm rotation axis 4 to realize the change of the spatial position of the gripping part 11. The main-end doctor operates the gripping part 11 to drive the lifting mechanism 7 to move up and down, and the pull rope displacement sensor 6 monitors the moving distance signal of the gripping part 11 in the Z direction in real time. The lifting damping adjustment device 8 can be adjusted according to the hand feel of the main-end expert doctor to meet the precise adjustment requirements of different experts; the gripping part 11 can realize changes in various angles under the action of the universal joint; a tilt sensor 10 is connected under the universal joint mechanism 9 to detect the tilt information of the gripping part 11; the six-dimensional force sensor 12 on the gripping part 11 can monitor the contact force and direction information between the gripping part 11 and the human body model 19 in real time, thereby simulating the scanning of the B-ultrasound probe on the virtual human body model 19. This process is completely completed by the main-end doctor manually driving the gripping part 11, and in the simulation operation process, the driving mechanism controls the lifting and lowering of the console, thereby realizing the lifting and lowering of the prosthesis 19, and the main-end doctor can feel the magnitude of the feedback force of the prosthesis 19. The holding part 11 can reach various required positions, and the sensor component can accurately obtain the posture and position information (x, y, z, α, β, γ), which can be displayed in real time on the display module and uploaded to the computing unit. After processing by the computing unit, it is transmitted to the slave end through the 5G low-latency communication network.

[0140] In this embodiment, the X and Y coordinates of the grip 11 can be obtained by the length and angle of each mechanism (the encoder measures the joint angle). The moving distance of the grip 11 in the Z direction is obtained by the pull rope displacement sensor 6, and the inclination sensor 10 can obtain the rotation angle of the grip 11 around the three axes of X, Y, and Z. Combined with the length parameters of each mechanism, the position and posture of the end of the grip 11 in contact with the virtual human model 19 can be obtained. Through the six-dimensional force sensor 12, the master doctor can know the magnitude of the force applied by the grip 11 on the virtual human model 19. The force applied by the operating grip 11 can be adjusted at any time according to clinical experience. The posture information is directly sent to the slave robot through the network, and the slave robot adjusts the posture of the B-ultrasound probe according to the posture signal. The position information needs to be corrected by the impedance control algorithm based on the real-time fit between the B-ultrasound probe and the patient by the PC, and the master-side force applied is corrected by the impedance control algorithm, and the PC sends the corrected position parameters to the slave robot. The above-mentioned slave end acts according to the signal transmitted from the master end, wherein the inclination parameter is directly sent to the robot, and the position parameter is based on the real-time fitting force between the B-ultrasound probe and the patient. The master end applies force to correct the position parameter.

[0141] Based on the received signal from the master end, the slave end mechanical arm drives the B-ultrasound probe 17 to approach the patient 18 body to start detection; the slave end sensor component transmits the collected slave end signal to the computing unit in real time, and after the slave end signal is processed by the computing unit, the force information fed back by the slave end and the B-ultrasound image are displayed on the master end display; the master end doctor determines the location of the puncture point based on the clear B-ultrasound image on the master end display, and transmits the signal of the located puncture point to the slave end, and the slave end is locked at this position to complete the positioning of the puncture.

[0142] Specifically, when the master-end signal is transmitted to the slave-end, the slave-end collaborative robot will automatically move the B-ultrasound probe 17 close to the patient 18's body according to the signal, and start detection on the patient 18's body according to the master-end operation. The six-dimensional force sensor at the end of the slave-end collaborative robot will collect the force information collected during detection and the B-ultrasound image detected by the B-ultrasound, which can be transmitted to the computing unit in real time. The computing unit processes the received information, and the master-end doctor can see the slave-end force information through the master-end display, and can also see whether the B-ultrasound image is clear or not. The signal principle is as follows Figure 6 The expert doctor at the master end can accurately locate the puncture point based on the clear B-ultrasound image and transmit the signal of this location point to the slave end, which will lock on this position to complete the puncture positioning.

[0143] In this embodiment, the slave end operating force refers to the contact force between the B-ultrasound probe 17 and the patient's body. The magnitude of the slave end operating force varies with the master end operating force. In order to prevent the slave end operating force from being too small, resulting in unclear B-ultrasound images, and the slave end operating force from being too large, causing harm to the patient, the master end operating force is required to be within a suitable range. Under normal circumstances, there is a suitable threshold range Fmin-Fmax for the slave end operating force. That is, when the slave end operating force is within the range of Fmin-Fmax, the B-ultrasound image is clear and the patient does not feel uncomfortable; when the slave end operating force is less than Fmin, the B-ultrasound image is unclear; when the slave end operating force is greater than Fmax, the slave end operating force is too large and the patient will feel uncomfortable. The master end can control the size of the slave end contact force within this range. Within this range, the slave end has the same amount of contact force as the master end applies. When the master end force is lower than the threshold, the slave end contact force is Fmin. When the master end force is greater than the threshold, the slave end force is Fmax. According to the required force of the slave end, the position is corrected according to the impedance control algorithm.

[0144] F real is the real-time adhesion force from the end, F master Apply force F to the main end desired Expected fit

[0145]

[0146] ΔF=F real -F desired

[0147] Δd is the axial increment, which can be changed by changing the end position of the robot;

[0148] Correct position based on impedance control.

[0149] In this embodiment, the six-dimensional force sensor at the end of the collaborative robot is used to adopt flexible force control technology to achieve constant force fitting of different human body shapes, and there is no impact force at the moment of contact, which can ensure the safety of people with different human shapes, fatness and thinness (softness and hardness).

[0150] The doctor at the end completes the puncture operation based on the locked positioning puncture point.

[0151] When the primary doctor completes the puncture positioning, the positioning puncture point information will be displayed on the secondary monitor. The secondary doctor completes the puncture operation based on the locked positioning puncture point. During the puncture process, the puncture image is transmitted to the primary monitor through the 5G network. The primary doctor can observe the puncture operation of the secondary doctor in real time and provide real-time guidance to ensure the smooth progress of the puncture operation.

[0152] Compared with the prior art, the method of this embodiment has at least the following beneficial effects:

[0153] 1. The operation has high precision and good stability. It adopts remote interactive puncture positioning surgery and uses 5G low-latency data transmission to ensure the safety and reliability of remote-guided puncture surgery, which can meet the needs of puncture guidance in areas with poor medical conditions.

[0154] 2. The main end robot arm is completely manually driven, which is more flexible to operate, can maximize the actual surgical operation, and has lower costs. It covers the surgical area and has a large working space.

[0155] 3. The master end uses a six-dimensional force sensor to collect the full force information of the grip in three-dimensional space in real time, and transmits the appropriate force information to the slave end after calculation, ensuring that the slave end force control is controlled by the master end and is within an appropriate range. This can not only ensure the safety of the operation but also ensure the clarity of the B-ultrasound image acquisition, thereby improving the operability of the puncture system.

[0156] 4. Based on the six-dimensional force sensor at the end of the collaborative robot, the flexible force control technology is adopted to achieve constant force fitting of different human body shapes, and there is no impact force at the moment of contact, which can ensure the safety of different human body shapes.

[0157] 5. The collaborative robot at the slave end is installed in an inverted manner, which can effectively utilize the arm span and maximize the reach of the end of the collaborative robot to various parts of the patient's body, avoiding the impact of the robotic arm on the human body that may be caused by the inverse solution of the serial structure, improving the safety of the system, and ensuring safety without sacrificing the working space.

[0158] 6. The master and slave ends can be in many-to-many mode, that is, one master end can control multiple slave ends respectively, and one slave end can be controlled by different master ends. In this way, the same doctor can interact with slave ends in different regions. At the same time, slave ends in the same region can also accept telemedicine operations from different doctors. However, in the working mode after telemedicine pairing is realized, it can only be one-to-one.

[0159] 7. In the initial state, the ranging sensor obtains rough three-dimensional surface information of the patient's body surface by moving the robot in a plane grid (grid spacing 1.5 cm) at a set absolute safety height (about 20 cm away from the patient's body); the system response speed can be improved.

[0160] 8. The axial drive motor can drive the terminal medical device to move axially close to the patient's body. The axial motor can control the terminal medical device to move axially, eliminating the six-axis reverse analysis and linkage process of the slave robot arm, so that the terminal medical device can be closer to the human body more quickly and improve the timeliness of the slave response; at the same time, due to the presence of the ranging sensor, it avoids the situation where the medical device is not close to the human body or puts too much pressure on the human body due to the inherent overshoot or slow response defects of the control algorithm.

[0161] The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that these embodiments can be modified and improved without departing from the principle and spirit of the present invention whose scope is defined by the claims and their equivalents, and these modifications and improvements should also be within the protection scope of the present invention.

Claims

1. A remote interactive ultrasound-guided puncture system with master-end force feedback, characterized in that: include: A prosthesis, wherein the prosthesis is used to simulate a real human body; The main end operating arm has a gripping part operated by the main end doctor, and the gripping part contacts the prosthesis to realize the simulated operation; the main end operating arm is provided with an encoder, a distance measuring sensor, an inclination sensor and a main end multi-dimensional force sensor, and the position and posture information of the gripping part are obtained based on the data monitored by the encoder, the distance measuring sensor and the inclination sensor; The master-end multi-dimensional force sensor is used to detect the force information of the gripping part in real time; A simulated operating table, the simulated operating table comprising a table top and a driving mechanism, the table top is used to support the prosthesis, and the driving mechanism is used to drive the table top to move within a spatial range; The prosthesis can be raised and lowered, rotated, tilted and adjusted under the driving action of the driving mechanism, and a feedback force can be applied to the gripping part, and the doctor at the main end can feel the magnitude of the feedback force of the prosthesis; The slave robot is provided with a slave multi-dimensional force sensor for real-time detection of the bonding force; the slave robot is interconnected and communicated with the master operating arm, and moves in real time according to the position, posture and force information of the gripping part; A display unit, which displays in real time the master end posture information, the master end multi-dimensional force information, the slave end on-site audio and video information, the slave end medical image and the slave end robot force signal; The magnitude of the slave end operating force varies with the master end operating force. When the slave end operating force is within the appropriate threshold range Fmin-Fmax, the slave end operating force is equal to the master end operating force; when the master end operating force is lower than the threshold, the slave end bonding force is Fmin; when the master end operating force is greater than the threshold, the slave end bonding force is Fmax; The main-end operating arm includes a base, an upper arm, a lower arm and a manual linkage part, the base is rotatably connected to the upper arm, the upper arm is rotatably connected to the lower arm, and the manual linkage part is connected to the lower arm through a universal joint; a first encoder is provided at the rotating shaft of the upper arm, a second encoder is provided at the connection between the upper arm and the lower arm, and the inclination sensor and the main-end multi-dimensional force sensor are provided on the manual linkage part; the main-end operating arm also includes a lifting mechanism, and the manual linkage part is connected to the lower arm through the lifting mechanism; adjustable damping is provided on the lifting mechanism, at the rotation point between the base and the upper arm, and at the rotation point between the upper arm and the lower arm.

2. The remote interactive ultrasound-guided puncture system with master-end force feedback according to claim 1, characterized in that: The driving mechanism comprises a lifting driving mechanism, the top end of the lifting driving mechanism is connected to the table top, and the bottom end of the lifting driving mechanism is fixed on the base.

3. The remote interactive ultrasound-guided puncture system with master-end force feedback according to claim 2, characterized in that: There are multiple lifting drive mechanisms, which are evenly arranged, and the top ends of the lifting drive mechanisms are rotatably connected to the table top.

4. The remote interactive ultrasound-guided puncture system with master-end force feedback according to claim 2 or 3, characterized in that: The driving mechanism further comprises a rotating mechanism, and the rotating mechanism is used for driving the table to rotate.

5. The remote interactive ultrasound-guided puncture system with master-end force feedback according to claim 4, characterized in that: The lifting drive mechanism is arranged above the rotating mechanism through a mounting plate, the top end of the rotating mechanism is fixedly connected to the lower surface of the mounting plate, and the bottom end of the lifting drive mechanism is fixedly arranged on the upper surface of the mounting plate.

6. The remote interactive ultrasound-guided puncture system with master-end force feedback according to claim 4, characterized in that: The table top includes a first layer and a second layer which are arranged in parallel, the first layer and the second layer are connected via the rotating mechanism, and the lower surface of the second layer is connected to the top end of the lifting drive mechanism.

7. The remote interactive ultrasound-guided puncture system with master-end force feedback according to any one of claims 2-3, 5-6, characterized in that: The lifting drive mechanism is driven by a hydraulic cylinder or a motor.

Images