Control system, control device, control method and program

Abstract

With a view to transmit, to a master device in an easily perceivable manner, a tactile force inputted to a slave device from outside, this control system (1) comprises a master device (10), a slave device (20), and a control device (30). The control device is equipped with a tactile force transmission unit (353) and a mode setting unit (351). The tactile force transmission unit is configured to control the transmission of a tactile force in the master device and the slave device, and the mode setting unit (351) is configured to vary the amplification ratio of the force transmitted to the master device from the slave device in a specific segment where movement is made by a mover of the slave device, on the basis of a physical quantity concerning the mover of the slave device.

Description

[Technical Field]

[0001] The present invention relates to a control system, a control device, a control method, and a program. [Background technology]

[0002] Conventionally, in master-slave manipulators, a configuration is known in which bilateral control is performed, such as force feedback control, which applies an operating reaction force to the operating tool on the master device side according to the operating load on the slave device side (see, for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 64-34686 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, when performing bilateral control in a master-slave system, the force detected by the slave device may be attenuated compared to the force actually input to the slave device. For example, if the slave device is a device that uses a flexible member such as a wire as its moving element, the force applied in the thrust direction may be weakened and detected due to the bending of the moving element of the slave device. In this case, the force and tactile feedback input to the slave device may not be accurately fed back to the master device, potentially preventing the master device operator from performing the operation correctly. The objective of this invention is to transmit force and tactile feedback input from an external source to a master device in a way that makes it easier for the slave device to perceive it. [Means for solving the problem]

[0005] To solve the above problems, a control system according to one aspect of the present invention is: A control system including a master device that receives operator input and a slave device that operates in response to the operation input to the master device, Control means for controlling the transmission of force and tactile sensations in the master device and the slave device, An amplification factor setting means for changing the amplification factor of the force transmitted from the slave device to the master device in a specific section in which the mobile element of the slave device moves, based on a physical quantity in the mobile element of the slave device, It is characterized by having the following features. [Effects of the Invention]

[0006] According to the present invention, it becomes possible to transmit force and tactile sensations input from an external source to a slave device to a master device in a way that makes them easier to perceive. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing the overall configuration of control system 1 according to one embodiment of the present invention. [Figure 2] This is a schematic diagram illustrating the basic principle of force-tactile feedback control performed by the control device 30. [Figure 3] This is a block diagram showing the hardware configuration of the control system in control system 1. [Figure 4] This is a schematic diagram showing the hardware configuration of the information processing device that constitutes the control device 30. [Figure 5] This is a block diagram showing the functional configuration of control system 1. [Figure 6] This is a flowchart illustrating the flow of force-tactile feedback processing performed by the control device 30. [Figure 7] This is a schematic diagram showing the time variation of the magnitude of the force fed back from the slave device 20 to the master device 10, assuming a constant amplification factor. [Figure 8] This schematic diagram shows the time evolution of the magnitude of the force fed back from the slave device 20 to the master device 10 when the amplification factor is increased compared to the example shown in Figure 7. [Figure 9] It is a schematic diagram showing the time change of the magnitude of the force fed back from the slave device 20 to the master device 10 when the force amplification factor is increased during the period P3. [Figure 10] It is a schematic diagram showing the configuration of the control system 1 that performs force feedback after an operator manually inserts the catheter of the slave device 20. [Embodiment for Carrying Out the Invention]

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0009] [First Embodiment] [Configuration] FIG. 1 is a schematic diagram showing the overall configuration of a control system 1 according to an embodiment of the present invention. As shown in FIG. 1, the control system 1 according to the present embodiment is configured as a master-slave system including a mechanically separated master device 10 and slave device 20. As an example, in the control system 1 of the present embodiment, the master device 10 constitutes a manipulator operated by a user, and the slave device 20 constitutes a catheter system provided with an end effector inserted into a subject. In FIG. 1, the control system 1 includes a master device 10, a slave device 20, and a control device 30, and the master device 10, the slave device 20, and the control device 30 are configured to be able to communicate wired or wirelessly via a network 40. Note that the control system 1 can appropriately include a display L and a plurality of cameras C. As the camera C, various imaging devices such as a video camera that photographs the appearance of the subject into which the slave device 20 is inserted, or an X-ray camera that photographs the inside of the subject by X-rays can be used. Also, a plurality of displays L that photograph various images with a plurality of cameras C and display these images can be provided.

[0010] The master device 10 accepts operations similar to those performed on conventional mechanically constructed catheters and detects the position of the movable part (such as the movable member of a manipulator) that moves as a result of the input operation. The master device 10 transmits information representing the detected position of the movable part to the control device 30. The master device 10 also outputs a reaction force via an actuator in accordance with the instructions of the control device 30 in response to the input operation. Specifically, the master device 10 receives operations to advance or retract the catheter (for example, operations to insert it into a blood vessel or operations to make slight movements to detect force and tactile sensation near a lesion), operations to rotate the catheter around its axis (for example, operations to change the orientation of the end effector), and operations to operate the end effector (for example, operations to expand or deflate the end effector if it is a balloon, or operations to open or close the end effector if it is forceps, etc.), applies a reaction force to these operations, and transmits information representing the position of the movable part moved by each operation to the control device 30.

[0011] The slave device 20 performs an operation corresponding to the input to the master device 10 by driving the actuator according to the instructions of the control device 30, and detects the position of the movable part (movable element of the actuator or catheter moved by the actuator, etc.) that moves as a result of the operation. When the slave device 20 operates, various external forces are input to the slave device 20 from the environment. As a result, the position of the movable part in the slave device 20 indicates the result of various external forces acting on the output of the actuator. The slave device 20 then transmits information representing the detected position of the movable part to the control device 30. Here, the various external forces input to the slave device 20 from the environment include pressing force in a direction intersecting the thrust direction that bends the catheter, and thrust-direction resistance force that the catheter receives from the blood vessel. Therefore, if the thrust-direction force in the slave device 20 is simply detected (or calculated) and fed back to the master device 10, the external force input to the slave device 20 will be an unintendedly large resistance force, and the desired palpable information cannot be transmitted. Therefore, in this embodiment, as will be described later, the thrust force detected in the slave device 20 is amplified at a predetermined amplification factor and transmitted to the master device 10, making it easier for the user to perceive the force.

[0012] The control device 30 is composed of an information processing device such as a PC (Personal Computer) or a server computer, and controls the master device 10, the slave device 20, the display L, and the camera C. For example, the control device 30 acquires the position of the movable parts of the master device 10 and the slave device 20 (such as the rotation angle of the actuator detected by a rotary encoder or the forward and backward position of the movable part detected by a linear encoder), and performs control to transmit force and tactile sensation between the master device 10 and the slave device 20.

[0013] In this embodiment, when the control device 30 operates the master device 10 and the slave device 20 as a master-slave system, it performs a coordinate transformation (transformation by transformation matrix) on real-space parameters (input vectors) calculated based on information representing the position of the movable part (information representing the position of the movable element of the actuator or the position of the member moved by the actuator) into a virtual space in which position and force can be handled independently. That is, the input vector is transformed from a real space in an oblique coordinate system in which position and force are related to each other, to a virtual space in an orthogonal coordinate system in which position and force are independent of each other. The parameters calculated by the coordinate transformation represent the position and force state values ​​corresponding to the input vector in the virtual space. Then, in the virtual space after the coordinate transformation, the control device 30 performs calculations to make the position and force state values ​​calculated from the input vector follow target values ​​for position and force, respectively, for controlling position and force (in this case, transmitting force and tactile sensation), and performs an inverse transformation (transformation by the inverse matrix of the transformation matrix) to return the calculation result to the real space. Furthermore, the control device 30 realizes a master-slave system that transmits force and tactile sensations between the master device 10 and the slave device 20 by driving each actuator based on real-space parameters (such as current command values) obtained by inverse conversion.

[0014] Furthermore, in this embodiment, the control device 30 amplifies the force transmitted from the slave device 20 to the master device 10 when transmitting force and tactile sensations between the master device 10 and the slave device 20, enabling the operator of the master device 10 to more easily perceive the force. Here, if the force transmitted from the slave device 20 to the master device 10 is amplified (i.e., enlarged) by an amplification factor greater than 1, the force transmitted from the master device 10 to the slave device 20 may be amplified (i.e., reduced) by an amplification factor less than 1. This can suppress control divergence. Also, in this embodiment, when transmitting force and tactile sensations between the master device 10 and the slave device 20, positional information can be transmitted either amplified or directly without amplification.

[0015] Furthermore, since position and velocity (or acceleration) or angle and angular velocity (or angular acceleration) are parameters that can be substituted using calculus, when performing operations related to position or angle, it is possible to appropriately substitute them with velocity or angular velocity, etc.

[0016] Figure 2 is a schematic diagram illustrating the basic principle of force-tactile transmission control performed by the control device 30. The basic principle shown in Figure 2 involves taking information representing the position of the movable part (the current position of the movable part) as input and performing calculations in at least one of the domains of velocity or force to determine the operation of the actuator. In other words, the basic principle of the present invention is expressed as a control law that includes a controlled system S, a function-specific force-velocity assignment conversion block FT, at least one of an ideal force source block FC or an ideal velocity source block PC, and an inverse conversion block IFT.

[0017] The controlled system S is a master device 10 or slave device 20 equipped with actuators, and controls the actuators based on acceleration, etc. As mentioned above, acceleration, velocity, and position are physical quantities that can be converted to one another by calculus, so control may be performed using any of acceleration, velocity, or position. Here, the control law will be expressed mainly using velocity calculated from position.

[0018] The Functional Force / Velocity Assignment Conversion Block FT is a block that defines the conversion of control energy to velocity and force domains set according to the function of the controlled system S. Specifically, the Functional Force / Velocity Assignment Conversion Block FT defines a coordinate transformation that takes a reference value (reference value) for the function of the controlled system S and the current position of the movable part as inputs. This coordinate transformation generally converts an input vector with the reference value and current velocity as elements into an output vector consisting of velocity for calculating the velocity control target value, and also converts an input vector with the reference value and current force as elements into an output vector consisting of force for calculating the force control target value. Specifically, the coordinate transformation in the Functional Force / Velocity Assignment Conversion Block FT is expressed in general terms as shown in the following equations (1) and (2).

[0019]

number

[0020] However, in equation (1), x'1~x' n (where n is an integer greater than or equal to 1) is the velocity vector used to derive the velocity state value, and x' a ~x' m (m is an integer greater than or equal to 1) is a vector whose elements are the reference value and the velocity based on the action of the actuator (the velocity of the actuator's movable element or the velocity of the member moved by the actuator), h 1a ~h nm is an element of the transformation matrix representing the function. Also, in equation (2), f''1~f'' n (where n is an integer greater than or equal to 1) is the force vector used to derive the force state value, and f'' a ~f'' m (where m is an integer greater than or equal to 1) is a vector whose elements are the reference value and the force based on the action of the actuator (the force of the actuator's movable element or the force of the member moved by the actuator).

[0021] By setting the coordinate transformation in the function-specific force / velocity assignment conversion block FT according to the function to be implemented, various operations and scaling can be performed. In other words, the basic principle of the present invention is that, in the function-specific force / velocity assignment conversion block FT, the variables of an individual actuator (variables in real space) are "converted" into a group of system-wide variables (variables in virtual space) that represent the function to be realized, and control energy is assigned to the control energy of velocity and the control energy of force. In other words, the basic principle of the present invention converts from a coordinate space in which velocity and force are related to each other to a coordinate space in which velocity and force are independent of each other, and then performs calculations related to the control of velocity and force. Therefore, compared to the case in which control is performed using the variables of an individual actuator (variables in real space), it is possible to independently assign the control energy of velocity and the control energy of force.

[0022] The Ideal Force Source Block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the Functional Force-Velocity Assignment Conversion Block FT. In the Ideal Force Source Block FC, a target value for force is set when performing calculations based on the coordinate transformation defined by the Functional Force-Velocity Assignment Conversion Block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, the target value can be set to zero, or when scaling can be set to a value that is enlarged or reduced in information indicating the function to be realized.

[0023] The Ideal Velocity Source Block PC is a block that performs calculations in the velocity domain according to the coordinate transformation defined by the Functional Force-Velocity Assignment Conversion Block FT. The Ideal Velocity Source Block PC has a target value for velocity set when performing calculations based on the coordinate transformation defined by the Functional Force-Velocity Assignment Conversion Block FT. This target value is set as a fixed or variable value depending on the function to be implemented. For example, when implementing a function similar to that indicated by a reference value, the target value can be set to zero, or when scaling is performed, a value that enlarges or reduces the information indicating the function to be implemented can be set.

[0024] The inverse conversion block IFT is a block that converts the values in the speed and force domains into the values in the domain of the input to the controlled system S (such as voltage value, current value, etc.). According to such a basic principle, when the position information in the actuator of the controlled system S is input to the function-specific force-velocity allocation conversion block FT, the control rules for the position and force domains corresponding to the function are applied in the function-specific force-velocity allocation conversion block FT using the speed and force information obtained based on the position information. Then, in the ideal force source block FC, the force calculation corresponding to the function is performed, in the ideal speed source block PC, the speed calculation corresponding to the function is performed, and the control energy is distributed to the force and speed respectively.

[0025] The calculation results in the ideal force source block FC and the ideal speed source block PC become the information indicating the control target of the controlled system S, and these calculation results are used as the input values of the actuator in the inverse conversion block IFT and input to the controlled system S. As a result, the actuator of the controlled system S executes an operation according to the function defined by the function-specific force-velocity allocation conversion block FT, and the operation of the target device is realized.

[0026] When a force feedback function involving scaling (amplification of force or position) is realized, the coordinate transformation in the function-specific force-velocity allocation conversion block FT in FIG. 2 is expressed by the following equations (3) and (4).

[0027] [[ID=I5]]

Equation

[0028] However, in Equation (3), x’ p is the speed for deriving the state value of speed, x’ f is the speed related to the state value of force. Also, x’ m is the speed of the reference value (input from the master device 10) (the differential value of the current position of the master device 10), x’ sThe current speed of the slave device 20 (current It is the derivative of position. Also, in equation (4), f p This is a force related to the velocity state, f f This is a force used to derive the state value of the force. Also, f m This is the reference value (input from master device 10). ) force, f s This is the current power of the slave device 20.

[0029] When the coordinate transformations shown in equations (3) and (4) are applied, the position of the slave device 20 is multiplied by α (where α is a positive number), and the force of the slave device 20 is multiplied by β (where β is a positive number), and these are transmitted to the master device 10. If only the force transmitted from the slave device 20 to the master device 10 is to be amplified, then α = 1 and the value of β can be set according to the purpose.

[0030] [Hardware configuration] Next, the hardware configuration of the control system in control system 1 will be described. Figure 3 is a block diagram showing the hardware configuration of the control system in control system 1. As shown in Figure 3, the control system 1 comprises, as a hardware configuration of the control system, a control device 30 composed of an information processing device such as a PC or server computer, a control unit 101 of the master device 10, a communication unit 102, an insertion actuator 103, a detection actuator 104, a rotation actuator 105, an operation actuator 106, linear encoders 107, 108, rotary encoders 109, 110, and drivers 111 to 114, a control unit 201 of the slave device 20, a communication unit 202, an insertion actuator 203, a detection actuator 204, a rotation actuator 205, an operation actuator 206, linear encoders 207, 208, rotary encoders 209, 210, drivers 211 to 214, a display L, and a camera C.

[0031] The control unit 101 of the master device 10 is composed of a microcomputer equipped with a processor and memory, and controls the operation of the master device 10. For example, the control unit 101 controls the driving of the insertion actuator 103, detection actuator 104, rotation actuator 105, and operation actuator 106 of the master device 10 according to control parameters transmitted from the control device 30. The communication unit 102 controls the communication that the master device 10 makes with other devices via the network 40.

[0032] The insertion actuator 103 is configured, for example, by a linear motor and applies a reaction force to the operator's input to the master device 10, which moves the catheter forward and backward to insert it into the blood vessel, according to the instructions of the control unit 101. The detection actuator 104 is configured, for example, by a voice coil motor and applies a reaction force to the operation of advancing and retracting the catheter near the lesion for treatment, which is input by the operator to the master device 10 according to the instructions of the control unit 101. In this embodiment, the insertion actuator 103 has a longer stroke than the detection actuator 104, while the detection actuator 104 is capable of more precise position and force control than the insertion actuator 103. The rotary actuator 105 is configured, for example, by a rotary motor and applies a reaction force to the operator's operation of rotating the master device 10 around a rotation axis along the forward and backward direction, in accordance with the instructions of the control unit 101. The operating actuator 106 is configured, for example, by a rotary motor, and applies a reaction force to the operation input by the operator to the lever (gripping part) or the like for operating the end effector, in accordance with the instructions of the control unit 101.

[0033] The linear encoder 107 detects the position of the movable element of the insertion actuator 103 (the forward and backward position on the linear axis). The linear encoder 108 detects the position of the movable element of the detection actuator 104 (the forward and backward position on the linear axis). The rotary encoder 109 detects the position (rotation angle) of the movable element of the rotary actuator 105. The rotary encoder 110 detects the position (rotation angle) of the movable element of the operating actuator 106.

[0034] The driver 111 outputs a drive current to the insertion actuator 103 according to the instructions of the control unit 101. The driver 112 outputs a drive current to the detection actuator 104 according to the instructions of the control unit 101. The driver 113 outputs a drive current to the rotary actuator 105 according to the instructions of the control unit 101. The driver 114 outputs a drive current to the operating actuator 106 according to the instructions of the control unit 101.

[0035] The control unit 201 of the slave device 20 is composed of a microcomputer equipped with a processor and memory, and controls the operation of the slave device 20. For example, the control unit 201 controls the driving of the insertion actuator 203, detection actuator 204, rotation actuator 205, and operation actuator 206 of the slave device 20 according to control parameters transmitted from the control device 30. The communication unit 202 controls the communication that the slave device 20 makes with other devices via the network 40.

[0036] The insertion actuator 203 is configured, for example, by a linear motor, and moves the catheter of the slave device 20 forward and backward in response to the operation input by the operator to the master device 10 to move the catheter forward and backward in order to insert the catheter into the blood vessel, according to the instructions of the control unit 201. The detection actuator 204 is configured, for example, as a voice coil motor, and, in accordance with instructions from the control unit 201, moves the catheter of the slave device 20 forward and backward in response to an operation input by the operator to the master device 10 to advance and retract the catheter for treatment near the lesion. In this embodiment, the insertion actuator 203 has a longer stroke than the detection actuator 204, while the detection actuator 204 is capable of more precise position and force control than the insertion actuator 203. The rotary actuator 205 is configured, for example, by a rotary motor, and rotates the catheter of the slave device 20 around a rotation axis along the forward and backward direction in accordance with the operation input by the operator to the master device 10, according to the instructions of the control unit 201. The operating actuator 206 is configured, for example, by a rotary motor, and operates the end effector (such as expansion, contraction, or opening / closing) according to the instructions of the control unit 201 and the operations input by the operator to the master device 10.

[0037] The linear encoder 207 detects the position of the movable element of the insertion actuator 203 (the forward and backward position on the linear axis). The linear encoder 208 detects the position of the movable element of the detection actuator 204 (the forward and backward position on the linear axis). The rotary encoder 209 detects the position (rotation angle) of the movable element of the rotary actuator 205. The rotary encoder 210 detects the position (rotation angle) of the movable element of the operating actuator 206.

[0038] The driver 211 outputs a drive current to the insertion actuator 203 according to the instructions of the control unit 201. The driver 212 outputs a drive current to the detection actuator 204 according to the instructions of the control unit 201. The driver 213 outputs a drive current to the rotary actuator 205 according to the instructions of the control unit 201. The driver 214 outputs a drive current to the operating actuator 206 according to the instructions of the control unit 201.

[0039] Display L is installed in a location where the operator of the master device 10 can see the screen, and displays images (such as visible light images or X-ray images of the subject captured by camera C) that are instructed to be displayed by the control device 30. Camera C is positioned in a location where it can photograph the subject into which the slave device 20 is inserting the catheter. Camera C captures an image of the subject (visible light image or X-ray image, etc.) and transmits the captured image to the control device 30.

[0040] Figure 4 is a schematic diagram showing the hardware configuration of the information processing device that constitutes the control device 30. As shown in Figure 4, the control device 30 is a CPU (Central Processing Unit). It comprises a Unit 311, a ROM (Read Only Memory) 312, a RAM (Random Access Memory) 313, a bus 314, an input unit 315, an output unit 316, a storage unit 317, a communication unit 318, and a drive 319.

[0041] The CPU 311 executes various processes according to the program recorded in the ROM 312 or the program loaded into the RAM 313 from the storage unit 317. RAM313 also stores data and other information necessary for the CPU311 to perform various processes.

[0042] The CPU 311, ROM 312, and RAM 313 are interconnected via a bus 314. The input unit 315, output unit 316, storage unit 317, communication unit 318, and drive 319 are connected to the bus 314.

[0043] The input unit 315 consists of various buttons and other components, and inputs various types of information according to the instructions given. The output unit 316 consists of a display, speakers, etc., and outputs images and sound. Furthermore, if the control device 30 is configured as a smartphone or tablet terminal, the input unit 315 and the output unit 316 display may be placed on top of each other to form a touch panel. The memory unit 317 consists of a hard disk or DRAM (Dynamic Random Access Memory), and stores various types of data managed by each server. The communication unit 318 controls the communication that the control device 30 makes with other devices via the network.

[0044] A removable media 331, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately mounted in the drive 319. Programs read from the removable media 331 by the drive 319 are installed in the storage unit 317 as needed.

[0045] [Functional configuration] Next, the functional configuration of the control system 1 will be described. Figure 5 is a block diagram showing the functional configuration of the control system 1. As shown in Figure 5, in the control system 1, the control device 30 performs various processes, causing the CPU 311 to function as follows: mode setting unit 351, sensor information acquisition unit 352, force / tactile sensation transmission unit 353, and physical quantity acquisition unit 354. In addition, a control parameter storage unit 371 is formed in the storage unit 317. The control parameter storage unit 371 stores control parameters acquired in chronological order during the control of force and tactile sensation transmission between the master device 10 and the slave device 20 by the control device 30. In this embodiment, the information stored as control parameters can be various parameters acquired in the force and tactile sensation transmission control, and can include various information that allows for the reproduction of the force and tactile sensation transmission control. For example, sensor information acquired by the master device 10 and the slave device 20, state values ​​obtained by coordinate transformation of this sensor information, current command values ​​to each actuator, or various setting values ​​set in the control device 30 for force and tactile sensation transmission control can be stored as control parameters.

[0046] The mode setting unit 351 determines whether the physical quantity of the catheter acquired by the physical quantity acquisition unit 354 matches the conditions set for switching between insertion mode and detection mode (hereinafter referred to as "mode switching conditions"), and switches between insertion mode and detection mode if it determines that the mode switching conditions are met. The "insertion mode" is a mode in which the external force (resistance force) detected in the slave device 20 is amplified by a first amplification factor and transmitted to the master device 10 as force-tactile sensation. The "insertion mode" is a mode that is set, for example, when an operator inserts a catheter into a subject and the tip of the catheter reaches the vicinity of the lesion. In this embodiment, in the "insertion mode," the force detected in the slave device 20 is amplified by a first amplification factor, which is a relatively small amplification factor, and transmitted to the master device 10. The first amplification factor can be set based on measured values, statistical values, or estimated values ​​obtained by simulation, for example, depending on the catheter configuration, the physical characteristics of the subject (blood vessel diameter, etc.), the length of catheter insertion, etc.

[0047] The "detection mode" is a mode in which the external force (resistance force) detected in the slave device 20 is amplified by a second amplification factor greater than the first amplification factor and transmitted to the master device 10 as force and tactile sensation. The "detection mode" is a mode that is set, for example, after the operator inserts a catheter into the subject and the tip of the catheter reaches the vicinity of the lesion. In this embodiment, in the "detection mode", the force detected in the slave device 20 is amplified by a second amplification factor which is a relatively large amplification factor and transmitted to the master device 10. Therefore, in "detection mode" compared to "insertion mode," the operator can more sensitively perceive the external force input to the slave device 20. In the following, sections where it is deemed appropriate to transmit force and tactile sensations from the catheter sensitively near the lesion will be referred to as "specific sections" as appropriate.

[0048] Here, the mode switching conditions can be set, for example, by the user manually instructing a mode switch, or by the physical quantity used to determine the mode switching condition meeting a set condition (such as the resistance force input to the catheter changing by more than a threshold, or the distance between the catheter tip and the lesion becoming within a threshold). For example, in insertion mode, if it is determined that the distance between the catheter tip of the slave device 20 and the lesion is within a threshold, the system can switch to detection mode. Also, in detection mode, if it is determined that the distance between the catheter tip of the slave device 20 and the lesion is greater than or equal to a threshold, the system can switch to insertion mode.

[0049] The sensor information acquisition unit 352 acquires information (sensor information) detected by various sensors installed in the master device 10 and the slave device 20. For example, the sensor information acquisition unit 302 acquires information indicating the position (advance / retraction position or rotation angle) of the movable element of each actuator detected by the linear encoders 107, 108, 207, 208 and the rotary encoders 109, 110, 209, 210. The sensor information acquisition unit 352 also stores the acquired sensor information as time-series data in the control parameter storage unit 371.

[0050] The force-tactile transmission unit 353 controls the transmission of force-tactile information between the master device 10 and the slave device 20 according to the control algorithm shown in Figure 2. For example, in the force-tactile transmission process, the force-tactile transmission unit 353 performs control to transmit force-tactile information between actuators for corresponding operations of the master device 10 and the slave device 20. At this time, the force-tactile transmission unit 353 amplifies the external force detected in the slave device 20 at the amplification factor set by the mode setting unit 351 and transmits it to the master device 10. As described above, information related to position (position or velocity, etc.) can be transmitted from the slave device 20 to the master device 10 either amplified or directly without amplification.

[0051] The physical quantity acquisition unit 354 acquires physical quantities (such as the resistance force input to the catheter or the distance between the catheter tip and the lesion) for determining the conditions related to the mode in force-tactile transmission processing (mode switching conditions). For example, the physical quantity acquisition unit 354 reads the time-series control parameters stored in the control parameter storage unit 371 and calculates the resistance force to be input externally to the catheter of the slave device 20. When the catheter is inserted into a subject, the resistance force changes depending on whether it is advancing through an artery or has reached a lesion site such as a narrowed heart. By calculating the resistance force to be input externally to the catheter of the slave device 20, the physical quantity acquisition unit 354 can determine the insertion state of the catheter. The physical quantity acquisition unit 354 can acquire instantaneous values ​​as physical quantities, or values ​​obtained by arithmetic calculations such as moving averages. Furthermore, the physical quantity acquisition unit 354 may apply filtering (band-limiting filter) to the waveform of the instantaneous value of the physical quantity before acquiring the physical quantity for determining the mode switching condition.

[0052] Furthermore, the physical quantity acquisition unit 354 calculates the distance between the catheter tip and the lesion based on the image captured by camera C. By calculating the distance between the catheter tip and the lesion from the image captured by camera C, the approach of the catheter to the lesion can be determined using the same criteria as when a human visually judges. When acquiring the distance between the catheter tip and the lesion, various sensors can be used. For example, a magnetic detection marker may be attached to the tip of the catheter, and the position of the catheter may be detected from outside the subject using a magnetic sensor to acquire the distance to the lesion. Alternatively, a sensor for detecting the position of the catheter tip may be pre-installed inside the subject, and the position of the catheter tip may be detected using this sensor to acquire the distance to the lesion.

[0053] [Setting the amplification factor] As described above, in the present embodiment, in a position (specific section) close to the target site (lesion) to be treated with the catheter, the detection mode is set, and the force detected by the slave device 20 is amplified at a second amplification factor and transmitted to the master device 10. Further, in a position away from the target site (lesion) to be treated with the catheter, the insertion mode is set, and the force detected by the slave device 20 is amplified at a first amplification factor and transmitted to the master device 10. The second amplification factor set in the detection mode can be set based on a boundary value (threshold value) at which it becomes easy for a human to perceive a change in force (force sense). As an example, when the first amplification factor in the insertion mode is k1 and the second amplification factor in the detection mode is k2, in order to make the change rate D of the force amplified at the first amplification factor k1 equal to the threshold value Dth regarding the change rate of the force, k2 may be set such that Dth / D = k2 / k1.

[0054] That is, in a state where the force detected by the slave device 20 is amplified at the first amplification factor k of the insertion mode, in an environment where the maximum value of the change rate of the force input to the catheter in the vicinity of the lesion (specific section) is Dmax (< Dth), by setting the second amplification factor k2 = (Dth / Dmax)·k1 of the detection mode, the operator can sense a change in force greater than or equal to the threshold value Dth regarding the change rate of the force. In order to set the second amplification factor k2 in this way, for example, in a state where the insertion mode is set, it is useful to advance the catheter in advance to the vicinity of the lesion (specific section) and detect the change rate of the force. Alternatively, the second amplification factor k2 may be set based on an estimated value obtained by a statistical value or simulation.

[0055] [Operation] Next, the operation of the control system 1 will be described.

[0056] [Force sense transmission process] FIG. 6 is a flowchart for explaining the flow of the force sense transmission process executed by the control device 30. The force-tactile feedback processing is initiated in response to an instruction to perform the force-tactile feedback processing via the input unit 315 or the communication unit 318. In this embodiment, when the force-tactile feedback processing is initiated, it is initiated when the tip of the catheter has been inserted into the subject by a predetermined distance (for example, approximately 1 to 10 cm) by an operator assisting the operation of the slave device 20, or by remote operation from the master device 10. This prevents the control of the control device 30 from becoming unstable when there are large changes in external force during the initial stages of insertion.

[0057] In step S1, the mode setting unit 351 sets the mode for transmitting force tactile sensation to insertion mode. As a result, the amplification factor of the force fed back from the slave device 20 to the master device 10 is set to the first amplification factor. In step S2, the sensor information acquisition unit 352 acquires information (sensor information) detected by various sensors installed on the master device 10 and the slave device 20. The sensor information acquired in step S2 is stored as time-series data in the control parameter storage unit 371. In step S3, the force-tactile transmission unit 353 amplifies the external force detected in the slave device 20 at a first amplification factor and performs force-tactile transmission control based on the sensor information. In step S4, the physical quantity acquisition unit 354 acquires physical quantities (such as the resistance force input to the catheter or the distance between the tip of the catheter and the lesion) for determining the mode switching conditions.

[0058] In step S5, the mode setting unit 351 determines whether the acquired physical quantity matches the mode switching conditions set for switching between insertion mode and detection mode. Specifically, the mode setting unit 351 determines whether the mode has been manually switched by user operation, or whether the physical quantity used to determine the mode switching conditions matches the set conditions (e.g., whether the resistance force input to the catheter has changed by more than a threshold, or whether the distance between the tip of the catheter and the lesion has fallen within a threshold). If the acquired physical quantity does not match the mode switching conditions set for switching between insertion mode and detection mode, the result is determined to be NO in step S5, and the process proceeds to step S7. On the other hand, if the acquired physical quantity matches the mode switching conditions set to switch between insertion mode and detection mode, the result is determined to be YES in step S5, and the process proceeds to step S6.

[0059] In step S6, the mode setting unit 351 switches between insertion mode and detection mode. That is, if the mode setting unit 351 is set to insertion mode, it switches to detection mode, and if it is set to detection mode, it switches back to insertion mode. In step S7, the force-tactile sensation transmission unit 353 determines whether or not the termination of the force-tactile sensation transmission process has been instructed. If the termination of force-tactile feedback processing is not instructed, the result in step S7 is determined to be NO, and the process proceeds to step S2. On the other hand, if the force-tactile feedback processing is instructed to end, the result in step S7 is determined to be YES, and the force-tactile feedback control processing ends.

[0060] [Verification of effectiveness] Figure 7 is a schematic diagram showing the time evolution of the magnitude of the force fed back from the slave device 20 to the master device 10 when the amplification factor is kept constant. Figure 7 shows the magnitude of the force when the operator operates the master device 10 and inserts the catheter of the slave device 20 into the subject, with the amplification factor kept constant and the force being fed back to the master device 10. In Figure 7, the amplification factor for amplifying the external force (resistance force) detected in the slave device 20 is assumed to be k1 (constant). Also, in Figure 7, the horizontal axis represents time, and the vertical axis represents the magnitude of the force fed back to the operator.

[0061] During period P1, as the operation begins, the catheter starts to move, and the external force (resistance) detected by the slave device 20 increases. During period P2, the catheter of the slave device 20 is moving through a steady environment within the subject (such as an artery), and the resistance force, including kinetic friction from the inner wall of the blood vessel, is of a nearly constant magnitude F. b It operates in this manner. During period P2, the force fed back from the slave device 20 to the master device 10 is F b It is represented as ×k1. During period P3, acceleration occurs (resistance increases) as the catheter of the slave device 20 comes into contact with lesions, etc., within the subject. During period P3, the maximum force input to the slave device 20 is F p Therefore, the maximum value of the force fed back from the slave device 20 to the master device 10 is F p It is represented as ×k1. In the example shown in Figure 7, during period P3, the maximum value of the rate of change D of the force fed back to the master device 10 is smaller than the threshold Dth related to the rate of change of force. In this case, the user may have difficulty perceiving the change in force (force-tactile feedback) fed back from the slave device 20 to the master device 10, and may not be able to perform the operation properly.

[0062] Figure 8 is a schematic diagram showing the time change in the magnitude of the force fed back from the slave device 20 to the master device 10 when the amplification factor is increased compared to the example shown in Figure 7. In Figure 8, the magnitude of the force when the operator operates the master device 10 and inserts the catheter of the slave device 20 into the subject, with the amplification factor set to k2, which is greater than the amplification factor k1 in Figure 7, and force is fed back to the master device 10. For reference, the magnitude of the force when amplified at amplification factor k1 is shown by the dashed line in Figure 8. Also, in Figure 8, the horizontal axis represents time, and the vertical axis represents the magnitude of the force fed back to the operator.

[0063] During period P1, as the operation begins, the catheter starts to move, and the external force (resistance) detected by the slave device 20 increases. During period P2, the catheter of the slave device 20 is moving through a steady environment within the subject (such as an artery), and resistance forces, including kinetic friction from the inner wall of the blood vessel, are acting at a nearly constant magnitude. During period P2, the force fed back from the slave device 20 to the master device 10 is F b It is represented as ×k². During period P3, acceleration occurs (resistance increases) due to the catheter of the slave device 20 coming into contact with lesions or other objects within the subject.

[0064] In the example shown in Figure 8, during period P3, the maximum value of the rate of change D of the force fed back to the master device 10 is equal to the threshold Dth related to the rate of change of force. In this case, the user can easily perceive the change in force (force-tactile feedback) fed back from the slave device 20 to the master device 10, and perform appropriate operations. In the control system 1 of this embodiment, the force fed back from the slave device 20 to the master device 10 is amplified by an amplification factor k1 until period P2, and in period P3, the force fed back from the slave device 20 to the master device 10 is controlled to increase by an amplification factor k2. Figure 9 is a schematic diagram showing the time evolution of the magnitude of the force fed back from the slave device 20 to the master device 10 when the force amplification factor is increased during period P3. In Figure 9, for reference, the magnitude of the force when amplified at amplification factor k1 is shown by a dashed line. Also, in Figure 9, the horizontal axis represents time, and the vertical axis represents the magnitude of the force fed back to the operator. As shown in Figure 9, in this embodiment, during period P2, the system is set to insertion mode and force is fed back from the slave device 20 to the master device 10 with a first amplification factor k1. During period P3, the system is set to detection mode and force is fed back from the slave device 20 to the master device 10 with a second amplification factor k2. This allows for the suppression of resistance felt during catheter insertion, while easily perceiving changes in force in specific sections (near the lesion) and enabling appropriate operation.

[0065] [Example 1] In the above-described embodiment, an example was given of a configuration in which the actuator operates the catheter from insertion until it reaches the lesion, but the invention is not limited to this. For example, the catheter may be manually inserted to the vicinity of the lesion, and force may be fed back from the slave device 20 to the master device 10 in a specific section near the lesion. In this case, the operator manually inserts the catheter of the slave device 20 to the vicinity of the lesion, and in the vicinity of the lesion, force is initially fed back at a first amplification factor, and then switched to a second amplification factor at a position (specific section) closer to the lesion.

[0066] Figure 10 is a schematic diagram showing the configuration of the control system 1, which provides force feedback after the operator manually inserts the catheter of the slave device 20. As shown in Figure 10, in this modified example, the control system 1 is equipped with an operating lever (grasping part) etc. on the catheter of the slave device 20, allowing for manual operation by an operator. Furthermore, the control system 1 of this modified example includes only the detection actuators 104 and 204 among the linear actuators of the control system 1 of the first embodiment shown in Figure 1, and does not include the insertion actuators 103 and 203. When an operator manually inserts the catheter, the slave device 20 releases the catheter from the movement control of the detection actuator 204 and the rotation actuator 205, allowing it to be operated in the same way as a conventional catheter. At this point, the operator inserts the catheter to a position just before the lesion, and this state is considered the initial state, from which force-tactile sensation transmission processing begins.

[0067] When force-tactile feedback processing is initiated, the catheter is held for movement control by the detection actuator 204 and the rotation actuator 205. In response to operations on the master device 10, the slave device 20 moves the catheter, and the external force applied to the catheter is fed back from the slave device 20 to the master device 10. In the force-tactile transmission process of this modified example, similar to the first embodiment, the system is initially set to insertion mode, and the external force input to the slave device 20 is amplified at a first amplification factor and transmitted to the master device 10. Then, if the physical quantity used to determine the mode switching condition is determined to match the mode switching condition set to switch between insertion mode and detection mode, the device is set to detection mode, and the external force input to the slave device 20 is amplified at a second amplification factor and transmitted to the master device 10. In this modified example, since the distance the catheter is moved by the actuator is relatively short, it is sufficient to use an actuator with a short stroke, such as a voice coil motor.

[0068] Thus, even when inserting a catheter into a subject using the control system 1 of this modified example, similar to the control system 1 of the first embodiment, the amplification factor of the force fed back from the slave device 20 to the master device 10 can be switched from the first amplification factor to the second amplification factor near the lesion. Therefore, force and tactile sensations input from an external source to the slave device 20 can be transmitted to the master device 10 in a way that makes them easier to perceive.

[0069] [Other variations] In the above-described embodiment, the force in the thrust direction (advancing / retracting direction) of the catheter was described as being transmitted via force-tactile feedback between the master device 10 and the slave device 20, but this is not limited to this. For example, the force-tactile feedback may be transmitted between the master device 10 and the slave device 20 as rotation around a rotation axis along the advancing / retracting direction, or as force related to the operation of the end effector. In this case, the force-tactile feedback may be performed by switching to increase the amplification factor of the force fed back from the slave device 20 to the master device 10 in a specific section.

[0070] Furthermore, although the above-described embodiment explained that the amplification factor of the force fed back from the slave device 20 to the master device 10 is set in two stages, a first amplification factor and a second amplification factor, it is not limited to this. That is, more than two types of modes, such as insertion mode and detection mode, may be set, and three or more amplification factors corresponding to these modes may be set to feed back force from the slave device 20 to the master device 10.

[0071] Furthermore, in the above-described embodiment, the second amplification factor in a specific section may be set as a function that reflects the change in the resistance force input to the catheter. This allows the force fed back from the slave device 20 to the master device 10 to be amplified by adaptively reflecting the change in the resistance force input to the catheter.

[0072] Furthermore, in the embodiments described above, the second amplification factor in a particular section was set so that the rate of change of force is greater than or equal to a set threshold, but this is not limited to this. That is, the second amplification factor in a particular section may be set so that the amount of change of force is greater than or equal to a set threshold.

[0073] Furthermore, while the above-described embodiment explained the case where force and tactile sensation are transmitted by associating the actuators provided in the master device 10 with the actuators provided in the slave device 20 on a one-to-one basis, the invention is not limited to this. That is, it is possible to transmit force and tactile sensation by associating multiple actuators of the master device 10 with one actuator of the slave device 20, or by associating one actuator of the master device 10 with multiple actuators of the slave device 20. It is also possible to transmit force and tactile sensation by associating multiple actuators of the master device 10 with multiple actuators of the slave device 20. As an example, it is possible to transmit force and tactile sensation by associating the insertion actuator 203 and detection actuator 204 of the slave device 20 shown in Figure 3 with the insertion actuator 103 of the master device 10. In this case, it is not necessary to provide the detection actuator 104 of the master device 10, which can be used to reduce costs and lighten the weight of the device.

[0074] Furthermore, in the above-described embodiment, an example configuration was given in which an insertion actuator 203 and a detection actuator 204 are provided as actuators for advancing and retracting the catheter of the slave device 20, but the configuration is not limited to this. That is, any actuator that satisfies the required performance in terms of stroke and accuracy of movement may be used to advance and retract the catheter of the slave device 20 with a single actuator.

[0075] As described above, the control system 1 according to this embodiment comprises a master device 10, a slave device 20, and a control device 30. The control device 30 also comprises a force-feedback unit 353 and a mode setting unit 351. The force-tactile transmission unit 353 controls the transmission of force-tactile signals between the master device 10 and the slave device 20. The mode setting unit 351 changes the amplification factor of the force transmitted from the slave device 20 to the master device 10 in a specific section in which the mobile of the slave device 20 moves, based on a physical quantity in the mobile of the slave device 20. This makes it possible to realize a control system that can easily perceive changes in force in a specific area (for example, near a lesion) and perform appropriate operations.

[0076] A specific interval is defined by the distance from the target to which the mobile element of the slave device 20 is to be reached. The mode setting unit 351 changes the amplification factor when the distance from the target as a physical quantity falls within a set threshold. This allows the approach of the slave device 20's moving element to the target to be determined using the same criteria as a human visually, and an appropriate amplification factor to be set.

[0077] A specific section is defined by the change in the external force applied along the path to the target to which the mobile element of the slave device 20 is to reach. The mode setting unit 351 changes the amplification factor when the change in the external force as a physical quantity exceeds a set threshold. This allows the slave device 20 to determine when the mobile is approaching its target based on changes in the environment along the mobile's path, and to set an appropriate amplification factor.

[0078] The mode setting unit 351 sets the amplification factor so that the rate of change of the force transmitted from the slave device 20 to the master device 10 in a specific section is greater than or equal to a set threshold. This allows the amplification factor to be set based on a threshold value that makes it easier for humans to perceive changes in force (force-tactile sensation) in terms of the rate of change of force.

[0079] The mode setting unit 351 sets the amplification factor so that the amount of change in force transmitted from the slave device 20 to the master device 10 in a specific section is greater than or equal to a set threshold. This allows the amplification factor to be set based on a threshold value that makes it easier for humans to perceive changes in force (force-tactile sensation), in terms of the amount of force change.

[0080] Furthermore, the present invention is not limited to the embodiments described above, and any modifications, improvements, etc., that can achieve the objectives of the present invention are included within the scope of the present invention. For example, the present invention can be implemented not only as the control system 1 in the above-described embodiment, but also as a control device that controls the control system 1, a control method composed of each step performed in the control system 1, or a program executed by a processor to realize the functions of the control system 1. Furthermore, although the above-described embodiment explained an example in which the control device 30 is implemented as an independent device, the functions of the control device 30 can be implemented in either the control unit 101 of the master device 10 or the control unit 201 of the slave device 20, or they can be distributed and implemented in both.

[0081] Furthermore, the processing in the above-described embodiment can be performed by either hardware or software. In other words, the control system 1 only needs to be equipped with a function that can perform the above-mentioned processing, and the functional configuration and hardware configuration used to realize this function are not limited to the examples given above. When the above process is performed by software, the programs that make up that software are installed on the computer from a network or storage medium.

[0082] The storage medium for storing programs consists of removable media distributed separately from the main unit, or storage media pre-installed in the main unit. Removable media consists of, for example, semiconductor memory, magnetic disks, optical disks, or magneto-optical disks. Optical disks consist of, for example, CD-ROM (Compact Disk-Read Only Memory), DVD (Digital Versatile Disk), Blu-ray® Disc, etc. Magneto-optical disks consist of, for example, MD (Mini-Disk). Furthermore, storage media pre-installed in the main unit consists of, for example, ROM (Read Only Memory), hard disks, or semiconductor memory on which programs are stored.

[0083] The above embodiments are merely examples of how the present invention can be applied and do not limit the technical scope of the present invention. That is, the present invention can be modified in various ways, such as by omitting or substituting, without departing from the spirit of the invention, and various embodiments other than those described above are possible. The various embodiments that the present invention can take and their variations are included within the scope of the invention described in the claims and its equivalents. [Explanation of Symbols]

[0084] 1 Control system, 10 Master device, 20 Slave device, 30 Control device, 40 Network, L Display, C Camera, FT Function-specific force / velocity assignment conversion block, FC Ideal force source block, PC Ideal velocity (position) source block, IFT Inverse conversion block, S Controlled system, 101,201 Control unit, 102,202 Communication unit, 103,203 Insertion actuator, 104,204 Detection actuator, 105,205 Rotation actuator, 106,206 Operation actuator, 107,108,207,208 Linear encoder, 109,110,209,210 Rotary encoder, 111~114,211~214 Driver, 311 Processor, 312 ROM, 313 RAM, 314 Bus, 315 Input section, 316 Output section, 317 Memory unit, 318 Communication unit, 319 Drive, 331 Removable media, 351 Mode setting unit, 352 Sensor information acquisition unit, 353 Force and tactile sensation transmission unit, 354 Physical quantity acquisition unit, 371 Control parameter storage unit

Claims

1. A control system including a master device that receives operator input and a slave device that operates in response to the operation input to the master device, Control means for controlling the transmission of force and tactile sensations in the master device and the slave device, An amplification factor setting means for changing the amplification factor of the force that is fed back to the master device as a force corresponding to the force sensation transmitted from the slave device to the master device, based on a physical quantity in the moving element of the slave device, in a specific section in which the moving element of the slave device moves, Equipped with, The aforementioned specific section is a section determined by changes in external forces input to the slave device from the environment along the path to the target to which the mobile element of the slave device is to reach. The amplification factor setting means is a control system that increases the amplification factor when the change in the external force as a physical quantity exceeds a set threshold.

2. The control system according to claim 1, characterized in that the amplification factor setting means sets the amplification factor such that the rate of change of the force transmitted from the slave device to the master device in the specific section is greater than or equal to a set threshold.

3. The control system according to claim 1, characterized in that the amplification factor setting means sets the amplification factor such that the amount of change in the force transmitted from the slave device to the master device in the specific interval is greater than or equal to a set threshold.

4. A control device for controlling the transmission of force and tactile sensations between a master device that receives operator input and a slave device that operates in response to the input to the master device, The system includes an amplification factor setting means that, based on a physical quantity in the moving element of the slave device, changes the amplification factor of the force that is fed back to the master device as a force corresponding to the force sensation transmitted from the slave device to the master device in a specific section in which the moving element of the slave device moves, The aforementioned specific section is a section determined by changes in external forces input to the slave device from the environment along the path to the target to which the mobile element of the slave device is to reach. The amplification factor setting means is a control device that increases the amplification factor when the change in the external force as a physical quantity exceeds a set threshold.

5. A control method performed by a control system including a master device that receives operator inputs and a slave device that operates in response to the operations input to the master device, A control step for controlling the transmission of force and tactile sensation in the master device and the slave device, An amplification factor setting step of changing the amplification factor of the force that is fed back to the master device as a force corresponding to the force sensation transmitted from the slave device to the master device, based on a physical quantity in the moving element of the slave device, in a specific section in which the moving element of the slave device moves, Includes, The aforementioned specific section is a section determined by changes in external forces input to the slave device from the environment along the path to the target to which the mobile element of the slave device is to reach. A control method comprising increasing the amplification factor in the amplification factor setting step when the change in the external force as a physical quantity exceeds a set threshold.

6. A computer controls the transmission of force and tactile sensations between a master device that receives operator input and a slave device that operates in response to the input to the master device. Based on the physical quantities in the moving element of the slave device, an amplification factor setting function is realized that changes the amplification factor of the force that is fed back to the master device as a force corresponding to the force sensation transmitted from the slave device to the master device in a specific section in which the moving element of the slave device moves. The aforementioned specific section is a section determined by changes in external forces input to the slave device from the environment along the path to the target to which the mobile element of the slave device is to reach. A program that, as the amplification factor setting function, increases the amplification factor when the change in the external force as a physical quantity exceeds a set threshold.

Images