Remote operating system
Through the microphone, detecting the sound when the surgical instrument is contacted and transmitting tactile information, the problem that the surgical robot control system is difficult to perceive the tiny collision and contact of the object is achieved, and the precise perception of the object's hardness and surface state is achieved, and the accuracy and safety of surgical operations are improved. sex.
Patent Information
- Application Number
- CN202280094196.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-28
- Publication Date
- 2024-11-12
AI Technical Summary
When existing surgical robot manipulation systems transmit force perception, it is difficult to capture tiny collisions and contacts of objects, resulting in the inability to accurately sense the hardness and surface state of objects.
A microphone is used to detect the sound when the object is in contact with the surgical instrument, and the tactile information is transmitted through the calculation unit and the tactile prompt unit is used to transmit the information to the operator, so as to realize the perception of the surface state of the object.
The operator can more accurately sense the hardness and surface state of the object, and improve the accuracy and safety of surgical operations.
Smart Images

Figure CN118946328A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a remote operation system. Background Art
[0002] In the current medical field, surgical robots (manipulators) are becoming popular for the purpose of reducing the burden on surgical operators during surgery. For example, there is a minimally invasive computer-assisted remote-operated surgery system, in which the surgical operator controls one or more main input devices, thereby performing surgery on the patient through the movement of the corresponding remote-operated tool (see Patent Document 1).
[0003] On the other hand, when using these surgical robots, the surgeon needs to rely only on visual information to perform the surgery. Therefore, there may be an impact caused by not having the finger sensation that could be obtained. For example, the patient's organs may be compressed without noticing that the forceps collide with the organs. Therefore, when using them, it is necessary to perform the surgery based on this after sufficient training. As a solution to this, it is required to develop a function that prompts the surgeon's force and touch sense.
[0004] For example, Patent Document 2 discloses a manipulation system that controls the automatic operation of a slave manipulation device that imitates the manual operation of a master manipulation device, wherein the slave manipulation device mainly uses force control based on a pneumatic drive system. The manipulation system has a force sense prompting function for the operator.
[0005] Prior art literature
[0006] Patent Literature
[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2022-48173
[0008] Patent document 2: International Publication No. 08 / 108289. Summary of the invention
[0009] Problems to be solved by the invention
[0010] However, the force sense that the manipulation system provides to the operator requires a relatively large reaction force from the object, and is not easy to respond to small collisions and contacts with the object. Therefore, it is difficult to capture the hardness and surface state of the object surface as a force sense.
[0011] The present invention has been made in view of such circumstances, and one exemplary object of the present invention is to provide a new technology that enables an operator to perceive the state of a surface of an object touched by a device.
[0012] Solutions for solving problems
[0013] In order to solve the above problems, a remote operation system according to one aspect of the present invention comprises: an operation device for remotely operating a device acting on an object; a microphone for detecting a sound when the object and the device come into contact; a tactile prompting unit provided in the operation device for prompting a tactile sensation to an operator; and a computing unit for calculating an action corresponding to the tactile sensation prompted by the tactile prompting unit. The computing unit calculates the action based on the sound.
[0014] According to this aspect, when the operating device is operated, information about the object that cannot be fully transmitted by force sense can be transmitted to the operator as a sense of touch. Here, the sense of touch refers to, for example, a feeling that can transmit information about the hardness and surface state (concavity and convexity) of the object surface when touching the object. In addition, the object refers to, for example, an object grasped or cut by a device, or a part of the human body treated with a surgical instrument during surgery.
[0015] Alternatively, the tactile feedback unit may include a driver that is driven based on the sound detected by the microphone. Thus, the feeling of the device touching the object can be indirectly transmitted to the operator as vibration by remote operation of the operating device.
[0016] The actuator may be a voice coil motor that can vibrate at least in the range of 200 Hz to 400 Hz. This allows the operator to sense the hardness of the object based on the frequency of the vibration.
[0017] Alternatively, the device may further include a surgical instrument for surgery. Alternatively, the device may be characterized in that the microphone is disposed near a position where the object contacts the surgical instrument. Thus, the sound when the object contacts the surgical instrument can be detected with good accuracy.
[0018] The microphone can collect sounds in the range of at least 50 Hz to 1000 Hz, thereby being able to collect sounds of frequencies that may be generated when an object comes into contact with the device.
[0019] Alternatively, the surgical instrument may be forceps having a shaft. Alternatively, the microphone may be provided in a main body that holds the forceps in a detachable manner, and the microphone may detect sound transmitted through the forceps.
[0020] A partition may be provided to separate the main body in the unclean area from the tweezers in the clean area, and a conducting member that conducts sound more easily than air may be sandwiched between the partition and the tweezers.
[0021] Alternatively, the surgical instrument is a pair of forceps having a shaft. Alternatively, the surgical instrument may include an adhesive for fixing the microphone to the shaft. In addition, the microphone may be covered with an adhesive so that the sound collecting portion is not exposed. Thus, external sounds including ambient sounds are not easily collected by the sound collecting portion, and solid-conducted sounds transmitted through the shaft itself can be collected preferentially.
[0022] Alternatively, polyurethane rubber may be sandwiched between the microphone and the shaft, thereby cutting off external sound and collecting solid-conducted sound with high accuracy.
[0023] Furthermore, arbitrary combinations of the above-mentioned constituent elements and embodiments in which the present invention is expressed in the form of methods, apparatuses, systems, etc. may also be additional modes of the present invention.
[0024] Effects of the Invention
[0025] According to the present invention, it is possible to make the operator feel the state of the surface of the object touched by the device. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is a diagram showing a schematic configuration of a remote operation system according to the present embodiment.
[0027] Figure 2 This is a diagram that enlarges the area around the connection between the microphone and the tweezers.
[0028] Figure 3 It is a side view showing a schematic structure of the operating device according to the present embodiment.
[0029] Figure 4 It is a plan view showing a schematic structure of the operating device according to the present embodiment.
[0030] Figure 5 It is a side view for explaining the schematic structure of the gimbal portion included in the operating device according to the present embodiment.
[0031] Figure 6 It is a plan view for explaining a schematic structure of a gimbal portion included in the operating device according to the present embodiment.
[0032] Figure 7 It is a side view showing a schematic structure of a slave device according to a modification.
[0033] Figure 8 This is the E area of the slave device involved in the modification (refer to Figure 7 ) is an enlarged cross-sectional view.
[0034] Fig. 9 It is the E area of the slave involved in other modified examples (refer to Figure 7 ) is an enlarged cross-sectional view. DETAILED DESCRIPTION
[0035] Hereinafter, the embodiments of the present invention will be described with reference to the accompanying drawings. The same or equivalent structural elements, components, and processes shown in the respective drawings are denoted by the same reference numerals, and repeated descriptions are appropriately omitted. In addition, the embodiments are merely illustrative and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily intended to limit the essence of the invention.
[0036] The remote operation system involved in this embodiment is used to operate the master-side manipulator in a master-slave surgical assistance robot. As a surgical assistance robot, for example, a robot that operates forceps used in endoscopic surgery can be cited. Moreover, the use of the remote operation system is not limited to surgical assistance robots. For example, it can also be used as a system for remotely operating robots used in logistics factories and manufacturing factories. It is particularly suitable as a remote operation system for remotely executing processes that require long-term delicate work via robots.
[0037] Furthermore, remote operation includes not only the case where the operator is physically far from the object of operation, but also the case where the host operated by the operator and the slave as the object of operation are mechanically separated. In this case, even if the host and the slave are arranged close to each other, it is definitely remote operation.
[0038] Figure 1 1 is a diagram showing a schematic structure of a remote operation system according to the present embodiment. The remote operation system 100 includes: an operation device 10, which is a host for remotely operating a device acting on an object O; a slave 52, which includes a mechanism for moving or driving the device, i.e., forceps 50, according to the operation of the operation device 10; a microphone 54, which detects the sound when the object O contacts the surgical instrument, i.e., forceps 50; a tactile prompting unit 56, which is provided on the operation device 10 and prompts the operator with tactile sensation; and a computing unit 58, which calculates an action corresponding to the tactile sensation prompted by the tactile prompting unit 56.
[0039] The calculation unit 58 calculates the above-mentioned operation based on the sound detected by the microphone 54. The microphone 54 is provided near the contact position Q where the object O and the tweezers 50 come into contact. Thus, the sound when the object O and the tweezers 50 come into contact can be detected with high accuracy.
[0040] Figure 2 This is a magnified schematic diagram of the area near the joint between the microphone and the tweezers. Figure 1 As shown, the sound S when the object O contacts (collides with) the tweezers 50 includes: air-conducted sound So conducted in the air outside the tweezers 50; and solid-conducted sound Si conducted in the shaft 50a of the tweezers 50 itself.
[0041] The inventors have conducted in-depth research on a preferred structure that removes air-conducted sound So, which is an environmental sound, as much as possible and can selectively collect solid-conducted sound Si, which is used to better reflect the properties (hardness, surface state) of the object O that is collided. Figure 2 As shown, the microphone 54 according to the present embodiment is configured such that the sensor portion 54 a as a sound collecting portion does not pick up the air-conducted sound So as much as possible, and is fixed to the outer periphery of the base portion of the shaft 50 a of the tweezers 50 .
[0042] The recessed portion in which the sensing portion 54a is provided may be filled with a filling material so that the air-conducted sound So does not directly reach the sensing portion 54a. In this embodiment, an adhesive 66 is used as the filling material. In addition, a conductive member 68 is arranged between the microphone 54 and the shaft 50a, and the conductive member 68 is used to easily transmit the solid-conducted sound Si conducted from the shaft 50a itself to the sensing portion 54a. The conductive member 68 and the microphone 54 are bonded by the adhesive 66, and the conductive member 68 and the shaft 50a are bonded by the adhesive 70.
[0043] The microphone 54 involved in this embodiment uses a MEMS microphone (ICS-40180 manufactured by Invensense) that can be mounted on thin-diameter tweezers. The frequency band that the MEMS microphone can collect is 60Hz~20kHz, and the supply voltage is 1.5V~3.6V. Adhesive 66 and adhesive 70 are hot-melt adhesives with a thickness of about 0.5mm~1mm. Other materials for adhesives include epoxy resin. The conductive component 68 is polyurethane rubber with a hardness of A70 and a thickness of about 5mm±2mm. In addition, the conductive component 68 can be made of a material that conducts sound more easily than air, and fluororubber can be mentioned as other materials. In addition, if the adhesive can prevent the two components from separating from each other, the material and shape are not particularly limited. Alternatively, it is also possible to use screws or springs instead of adhesives to press and fix the two components to each other.
[0044] In addition, the microphone 54 is arranged at the base of the shaft 50a which is sufficiently far away from the front end of the forceps. This place is not easily affected by the driving sound of the internal rope caused by moving the operating part of the front end of the forceps. Therefore, the solid-conducted sound Si can be detected with good accuracy.
[0045] The remote operation system 100 further includes: a slave-side control unit 60, which controls the rotation and bending movements of the various parts of the manipulator of the slave 52; and a master-side control unit 62, which is linked with the slave-side control unit 60. In addition to the functions of the operation unit 58, the master-side control unit 62 controls the movement of the slave 52 based on the movement information of each part of the operating device 10. In addition, the signal processing unit 64 transforms the information of the sound detected by the microphone 54 so that the subsequent operation unit 58 can use it. Specifically, the signal processing unit 64 performs filtering based on a bandpass filter or attenuates noise based on a dead zone.
[0046] The signal processed by the signal processing unit 64 is emphasized in the operation unit 58 at a specific wavelength that affects the sense of touch, and is supplied to the tactile prompting unit 56. The tactile prompting unit 56 has a driver that is driven based on the sound S detected by the microphone 54. Thus, when the operation is performed by the operating device, information about the object that cannot be fully transmitted by force sense can be transmitted to the operator as a sense of touch. Here, the sense of touch refers to, for example, the feeling that can transmit information about the hardness and surface state (concavities and convexities) of the surface of the object when it touches the object O. In addition, the object O refers to, for example, an object grasped or cut by a device, or a part of the human body treated with surgical instruments during surgery.
[0047] The actuator involved in this embodiment is a voice coil motor that can vibrate at least in the range of 200 Hz to 400 Hz. Vibration stimulation in the range of 200 Hz to 400 Hz is suitable for allowing the operator to sense the hardness of the object O. Thus, by remote operation of the operating device 10, the feeling when the tweezers 50 touch the object O can be indirectly transmitted to the operator as a vibration. In addition, the operator can sense the hardness of the object based on the frequency of the vibration. It is also possible to use a piezoelectric element as a driver.
[0048] The microphone 54 involved in this embodiment can collect sounds in the range of at least 50 Hz to 1000 Hz. Thus, it is possible to collect sounds of frequencies related to hardness that may be generated when the object O contacts the tweezers 50. Moreover, as a preferred example of the microphone 54, a MEMS microphone can be cited. Alternatively, a piezoelectric vibration sensor or an acceleration sensor can be used as a detection unit for detecting the vibration of the sound. Alternatively, a plurality of microphones and sensors can be combined as a detection unit.
[0049] The mechanism of the operating device 10 involved in this embodiment is not particularly limited, but preferably has a multi-degree-of-freedom joint that can be used as a tactile device. Similarly, it is preferred that the mechanism of the slave 52 is a multi-degree-of-freedom mechanical arm that changes the position and posture of the tweezers 50. Alternatively, as the mechanism of the operating device 10 and the slave 52, a pneumatic drive system that can transmit force sensation can also be used.
[0050] Figure 3 It is a side view showing a schematic structure of the operating device according to the present embodiment. Figure 4 : is a top view showing the schematic structure of the operating device involved in this embodiment. The operating device 10 is used when performing remote operation, and the operating device 10 includes: a handle portion 12 mainly held by the palm of the operator; and a clamping portion 14 mainly pinched by the fingers of the operator. The handle portion 12 is provided with a driver (vibration source) as the tactile prompting portion 56 in the area where the palm of the operator contacts. The clamping portion 14 is configured to rotate relative to the handle portion 12 (in the direction of the arrow symbol R1) and slide relative to the handle portion 12 (in the direction of the arrow symbol S1). It is also possible that the tactile prompting portion 56 is provided on a part of the clamping portion 14.
[0051] Thus, the entire operating device 10 can be moved as a power grip type, and the clamping portion 14 can be rotated or slid while the handle portion 12 is gripped by the palm. Therefore, the functions of both the power grip type and the clamp grip type can be combined, the operator's operating burden can be reduced, and delicate work can be performed.
[0052] The handle portion 12 includes a cylindrical first holding portion 12a directly held by the palm, and a rotating portion 12b that can rotate relative to the first holding portion 12a. A cylindrical hole is provided at the center of the first holding portion 12a, and the cylindrical rotating portion 12b is rotatably received in the hole. The clamping portion 14 includes a second holding portion 14a directly held by the fingers, and a guide portion 14b that guides the second holding portion 14a to separate from or approach the handle portion 12.
[0053] Thus, the rotating portion 12b can be rotated while the first holding portion 12a of the handle 12 is fixed by the palm. Also, the second holding portion 14a can be moved away from or closer to the handle 12 while pinching the second holding portion 14a with the fingers.
[0054] The rotating portion 12b is a rotating shaft provided inside the first retaining portion 12a. The guide portion 14b is coupled to and fixed to the rotating shaft serving as the rotating portion 12b. In addition, the guide portion 14b has a linear guide rail 14c extending in a direction intersecting the axis A of the rotating shaft. Thus, the second retaining portion 14a can be moved in a predetermined direction (direction of the arrow symbol S1) relative to the handle portion 12. Moreover, in the operating device 10 involved in the present embodiment, the angle formed by the axis A and the extension direction of the guide rail 14c is recorded as 90°, but the angle may not be 90°, and may be set within the range of 90°±45°, may be set within the range of 90°±30°, or may be set within the range of 90°±15°.
[0055] The second holding portion 14a is guided by the guide rail 14c so as to be movable within ±5 mm or more relative to the rotation axis about the reference point P. In other words, the second holding portion 14a is configured to be movable within a range of at least 10 mm along the guide rail 14c.
[0056] Figure 5 It is a side view for explaining the schematic structure of the gimbal portion included in the operating device according to the present embodiment. Figure 6 It is a plan view for explaining a schematic structure of a gimbal portion included in the operating device according to the present embodiment.
[0057] The second holding portion 14a of the operation device 10 is movably supported by the gimbal portion 20. The gimbal portion 20 is configured to be rotatable around three mutually intersecting (orthogonal) rotation axes. The gimbal portion 20 is rotatable around the X-axis Ax, the Y-axis Ay, and the Z-axis Az.
[0058] Specifically, the end of the second holding portion 14a in the Z-axis direction is connected to the Z-axis portion 22, and the Z-axis portion 22 is rotatably held by the first arm 24 with respect to the L-shaped first arm 24. In addition, the first arm 24 holds the Y-axis portion 26 in a rotatable manner, and the first arm 24 is rotatably provided on the second arm 28 connected to the Y-axis portion 26. The second arm 28 is L-shaped, and the X-axis portion 30 is rotatably held at the end of the second arm 28 on the opposite side of the end holding the Y-axis portion 26. The X-axis portion 30 is connected to a translation mechanism such as an XYZ stage and an XYZ connection mechanism for three-dimensional movement.
[0059] Thus, the movement information of the operating device 10 can be used to control the position and posture of the manipulator of the slave machine via the gimbal portion 20 and the translation mechanism. For example, in the case where a surgical assistance robot capable of operating forceps is a slave machine, the operator moves the operating device 10 as a whole in a desired direction or rotates it in a desired direction while holding the handle portion 12. The movement of the operating device 10 is detected by a sensor (not shown) that obtains information (movement information) on the rotation angles of the Z-axis portion 22, the Y-axis portion 26, and the X-axis portion 30. The movement information detected by the sensor is transmitted to the surgical assistance robot. The surgical assistance robot controls the movement of the position of the forceps based on the transmitted movement information.
[0060] The second holding portion 14a is configured to have a point ( Figure 5 , Figure 6 C) shown in the figure is used as a reference to slide relative to the handle 12. Thereby, the movement of a specific position of the robot hand of the slave can correspond to the sliding of the handle 12.
[0061] have Figures 3 to 6 The remote operation system 100 of the operating device 10 shown in the figure has the following effects.
[0062] (1) Enable the operator to sense the collision (feel) and the state of the object surface.
[0063] (2) Even when the force sense is approximately zero (buried by error), the operator can sense the contact or its change.
[0064] (3) By combining not only a tactile sense transmission mechanism but also a force sense transmission mechanism, the operator's perception accuracy of the texture and hardness of the operation object can be increased.
[0065] (4) The system event information (alarm, operation guidance) can be presented as a tactile prompt without interfering with operation or vision.
[0066] (Variation Example)
[0067] In the above-mentioned embodiment, the case where the microphone is attached to the shaft via another member is described. However, in the following modified example, a structure in which the microphone is provided on the main body side of the slave device to which the tweezers can be attached and detached will be described.
[0068] Figure 7 : is a side view showing the schematic structure of the slave involved in the modification. The slave 72 includes: forceps 50; a main body 74, which holds the forceps 50 in a manner that allows the forceps 50 to be attached and detached. A partition 76 is provided between the main body 74 and the forceps 50 to separate the unclean area DA from the clean area CA. In addition, a sterilization cover 78 is provided in a manner that surrounds the unclean area DA on the main body 74 side from the partition 76. A microphone 80 is provided in a part of the main body 74 to detect the sound transmitted through the forceps 50 and the partition 76.
[0069] Figure 8 This is the E area of the slave device involved in the modification (see Figure 7 ). Figure 8 , it is shown that the sound during contact is transmitted as solid-conducted sound Si in the tweezers 50 and detected by the microphone 80. The solid-conducted sound Si is conducted in the solid parts of the shaft 50a and the housing 50b, and reaches the microphone 80 via the conductive member 68 and the partition 76. Thus, even the microphone 80 in the unclean area DA can detect the contact sound between the tweezers 50 and the object O.
[0070] Fig. 9 It is the E area of the slave involved in other modified examples (refer to Figure 7 ). Fig. 9, it is shown that the sound during contact is transmitted as air-conducted sound So in the tweezers 50 and detected by the microphone 80. The air-conducted sound So is conducted in the internal space of the shaft 50a and the housing 50b, and reaches the microphone 80 via the through path 82 inside the partition 76. Thus, even the microphone 80 in the unclean area DA can detect the contact sound between the tweezers 50 and the object O.
[0071] The present invention has been described above with reference to the above-mentioned embodiments, but the present invention is not limited to the above-mentioned embodiments, and even if the structures of the embodiments are appropriately combined or replaced, they also belong to the present invention. In addition, based on the knowledge of those skilled in the art, the combinations and processing sequences in the embodiments can be appropriately combined and replaced, or the embodiments can be deformed by various design changes, and the embodiments obtained by such deformation also belong to the scope of the present invention.
[0072] Description of Reference Numerals
[0073] 10: operating device; 12: handle; 14: clamping part; 50: tweezers; 52: slave; 54: microphone; 56: tactile prompting part; 58: operation part; 60: slave-side control part; 62: master-side control part; 64: signal processing part; 66: adhesive; 68: conductive component; 70: adhesive; 100: remote operation system.
[0074] Industrial Applicability
[0075] The present invention can be used for remote operation systems such as assembly or sorting of products, surgery, etc.
Claims
1. A remote operating system, characterized in that: have: operating device, which remotely operates a device acting on an object; a microphone that detects sound when the object comes into contact with the device; A tactile prompting unit, which is provided on the operating device and provides a tactile prompt to the operator; as well as a calculation unit that calculates an action corresponding to the tactile sensation prompted by the tactile sensation prompting unit, The calculation unit calculates the action based on the sound.
2. The remote operation system according to claim 1, characterized in that: The tactile feedback unit includes a driver that is driven based on the sound detected by the microphone.
3. The remote operation system according to claim 2, characterized in that: The driver is a voice coil motor capable of vibrating at least in the range of 200 Hz to 400 Hz.
4. The remote operation system according to any one of claims 1 to 3, characterized in that: The equipment also includes surgical instruments used for surgery. The microphone is disposed near a location where the object contacts the surgical instrument.
5. The remote operation system according to claim 4, characterized in that: The microphone is capable of collecting sounds in a range of at least 50 Hz to 1000 Hz.
6. The remote operation system according to claim 4, characterized in that: The surgical instrument is a forceps having a shaft, The microphone is provided on a main body that holds the tweezers in a detachable manner, and the microphone detects sound transmitted through the tweezers.
7. The remote operation system according to claim 6, characterized in that: A separator is also provided, the separator separating the main body in the unclean area from the tweezers in the clean area, A conductive member that conducts sound more easily than air is interposed between the partition and the tweezers.
